DM Washington - If you were already concerned that robots may one day take your job, the latest results from MIT may make difficult reading. An artificial intelligence system called ConceptNet recently took an IQ test designed for children and scored higher than an average four-year-old. After five tests focusing on word reasoning and vocabulary, ConceptNet achieved a total score of 69 while a comparable pre-schooler achieved 50 - and the system is expected to get smarter.

ConceptNet is an open-source project run by the MIT Common Sense Computing Initiative. The team’s ConceptNet 4, used in the study at the University of Chicago, uses ‘concepts’ with what’s known as a ‘relation’. When asked to explain what a fawn is, for example, it understands that a ‘fawn is a deer’ rather than a ‘deer is a fawn’ by understanding the concept of the animal and the relation between the two words and concepts. It can also use a so-called ‘polarity flag’ to indicate a negative correlation, such as penguins can’t fly. 

The WPPSI-III IQ test has 14 subtests and a complete assessment consists of a Performance IQ and a Verbal IQ (VIQ). The Verbal IQ consists of questions designed to test a child’s word reasoning, vocabulary and comprehension. Each IQ score has a mean of 100 and the study concentrated on the Verbal IQ test.

Test questions, such as ‘Why do we shake hands?’, ‘Why do we wear sunscreen in the summer?’ and ‘Why is it bad to put a knife in your mouth?’ were translated into ConceptNet 4 using natural language processing tools. For the study, the researchers led by Stellan Ohlsson from the University of Chicago scored the WPPSI-III subtests once using the top-scored answer to each item. It then scored them a second time using the best answer from among the five top-scoring answers to each item.

They called the former ‘strict scoring’ and the latter ‘relaxed.’  In the Information subtest, ConceptNet scored 20 while scaled scores taken from a four-year-old scored 10. The child scored higher for Word Reasoning and Comprehension, scoring seven in both compared to ConceptNet’s scores of three and two respectively. 

ConceptNet also demonstrated it has a wider vocabulary than the child, with a score of 20 versus 13. ‘The ConceptNet system scored a WPPSI-III VIQ that is average for a four-year-old child, but below average for 5 to 7 year-olds,’ explained the researchers.  ‘Large variations among subtests indicate potential areas of improvement, and Comprehension is the subtest most strongly associated with common sense.

‘The large variations among subtests and ordinary common sense strongly suggest that the WPPSI-III VIQ results do not show that ‘ConceptNet has the verbal abilities a four-year-old.’