'Paradigm' shift in data analysis promises change

The Big Data Industry Summit 2016 was held in Guiyang, Guizhou province, on May 25, 2016. [Photo/Xinhua]

Statistics experts in hot demand as research into AI gathers pace in China, elsewhere

A special statistical modeling competition was going on in a well-decorated office in northwestern Beijing. The participants were a line of secretaries, human resource officers and marketing and public relations staff who knew nothing about the basics of coding and computer programming.

They were all employees of 4Paradigm, a three-year-old startup founded by Baidu Inc's former principal IT architect and backed by global venture capital powerhouse Sequoia Capital.

The company has been dubbed by investors the Chinese answer to the US big data firm Palantir. It built 4Paradigm·Prophet, an artificial intelligence-powered platform, to help people and organizations comfortably analyze an explosive amount of data which is of increasing importance.

"With machine-assisted analysis, we aim to turn ordinary people into data scientists within a month, so that companies can benefit from artificial intelligence even though they can't afford to build its own AI teams," said Dai Wenyuan, CEO of 4Paradigm.

While working in Baidu from 2009 to 2013, Dai built the company's first commercialized deep-learning search ad platform.

The result of the November statistical modeling competition was impressive with 70 percent of participant groups－with no computer and data background－managing to build models that scored over 0.8 in the AUC test, a well-recognized model evaluation metric. Usually, models with 0.8 are good enough for industrial application.

"But that is not good enough because 30 percent of them failed. There is still a long way to go," Dai added.

Currently, the company has over 100 employees and Yang Qiang, the only Chinese councilor at the Association of the Advancement of Artificial Intelligence, is its principal scientist.

Zhao Ziming, an analyst at Beijing-based internet consultancy Analysys, said the ability to analyze data becomes increasingly important as companies realize that within the voluminous quantity of data lie insights that could give them a competitive edge.

"Without AI-enabled data science, there would be no personalized shopping recommendations on websites like eBay and Amazon and tailored content and advertising on Facebook," he added. "But it is pretty difficult for companies to cash in onto such demand."

In a KPMG survey of senior executives, 99 percent said that analysis of big data was important to their strategy for next year.

Data scientists have become highly sought-after in the marketplace, but there are still relatively few of them. That's exactly where 4Paradigm tries to offer a solution.

In November, it teamed up with China Everbright Bank to set up an AI research lab. It has been offering data analysis to improve precision marketing for the bank's credit card business.

"Compared with rivals, we have the wide-reaching data mining program that enables enterprises to capture what typical users do on the internet." Dai said.

"AI is no longer just capable of profiling groups. We are trying to picture each consumer as precisely as possible."

While other targeted marketing service providers divide 50 million customers into 10 or so categories, the company said it can group them into 500 billion categories based on millions of dimensions.

"Empowered by machine learning, Prophet lowered the demand for computing power and accelerated the analysis process," Dai said, adding that its client's sales performance went up by 60 percent after using the company's service.