Chinese researchers' artificial intelligence outwits humans on verbal IQ test

Microsoft and university researchers in China have proven that we may just talk a good game when it comes to competing verbally with computers.

Computers are known for their mathematical proficiency, but the nuances and whimsy of human verbal expression are usually beyond their ken.

A five-member team developed an artificial intelligence (AI) program with the goal of performing well on verbal sections of IQ tests.

The findings suggest machines could be closer to approaching human intelligence, the researchers wrote in a study, titled Solving Verbal Comprehension Questions in IQ Test by Knowledge-Powered Word Embedding, which they posted to the online database arXivl on June 17.

The researchers gave a set of IQ test questions to their computer program and to a group of 200 people with different levels of education. The test-takers were recruited through Amazon Mechanical Turk, a crowdsourcing platform. The program beat the average score of the test group.

Researchers took an approach known as "deep learning", which involves building up abstract representations of concepts from raw data. The researchers used the method to learn the different representations of words, a technique known as word embedding.

And then they came up with a way to solve the test problems.

The AI's results were surprising, although the machine didn't do as well against people with master's or doctorate degrees.

The report described the approach: "First, we build a classifier to recognize the specific type of verbal questions. According to previous studies, verbal questions usually include sub-types like analogy, classification, synonym and antonym.

"For different types of questions, different kinds of relationships need to be considered and the solvers could have different forms. Therefore,

with an effective question-type classifier, we may solve the questions in a divide-and-conquer manner and achieve high accuracy.

"Second, we obtain distributed representations of words and relations by leveraging a novel word-embedding method that considers the multi-sense nature of words and the relational knowledge among words (or their senses) contained in dictionaries.

"For each polysemous word (those with multiple meanings), we retrieve its number of senses from a dictionary, and conduct clustering on all its context windows in the corpus.

"Third, for each specific type of questions, we propose a simple yet effective solver based on the obtained distributed word representations and relation representations."

The report said the researchers then attached "the example sentences for every sense in the dictionary to the clusters, such that we can tag the polysemous word in each context window with a specific word sense".

It said that "the learning of word-sense representations and relation representations interacts with each other, to effectively incorporate the relational knowledge obtained from dictionaries".

They concluded that "the results are highly encouraging, indicating that with appropriate uses of the deep learning technologies, we could be a further small step closer to human intelligence".

Actually, I kind of had some trouble comprehending the report, which means I would probably lose to the computer.

The researchers were encouraged: "In the future, we plan to leverage more types of knowledge from the knowledge graph … to enhance the power of obtaining word-sense and relation embeddings. Moreover, we will explore new frameworks based on deep learning or other AI techniques to solve other parts of IQ tests beyond verbal comprehension questions."

The research team included Huazheng Wang and Fei Tian, of the Department of Computer Science at the University of Science and Technology of China in Hefei, Anhui province, and researchers Bin Yao, Tie-Yan Liu and Jiang Bian at Microsoft.

Contact the writer at williamhennelly@chinadailyusa.com