This briefing by Nick Bostrom on the dangers of artificial intelligence takes up a serious and legitimate question: Should we be more cautious as we go about trying to improve artificial intelligence? What if an AI became so smart it decided to take over the world?  Silly? I think so. But it’s a question worth exploring, if only to dispel long-standing fear of the mythical “Frankenstein Syndrome.”

Unfortunately, the book is written by a philosopher with an engineering bent, without, apparently, much understanding of human psychology (real intelligence).  Consequently, the book is mostly a sterile exercise and often unintentionally humorous.

Perhaps the most fundamental problem is the failure to define intelligence of the natural kind. Bostrom unthinkingly uses the I.Q. index as a measure of it, but anyone who has studied the matter will agree that IQ equals intelligence only as a matter of convenient social discourse. People who take IQ tests produce scores distributed in a normal distribution and that’s a scientific fact, but there is no theory or explanation of why  or whether answers to the questions on an IQ test have anything to do with intellectual competence or “smarts,” whatever those might be. Here’s an example of the kind of question you might find on an adult IQ test:

Rearrange the following letters to make a word and choose the category in which it fits.

RAPETEK

A. city
B. fruit
C. bird
D. vegetable

Correct answer: bird (parakeet)

If you can answer such a question, what does it mean?  Rapetek is not even a word so you can’t be expected to know it. Perhaps the correct answer shows you have experience with words, letters, and conventional  hierarchical categories of common objects. Does that make you “smart?”  Maybe. Another good answer is “Rape,” and none of the categories presented is appropriate to it.  Is that a less smart answer? (The question did not say I had to use all the letters presented).

The bottom line is that there is no generally accepted explanation for what natural intelligence is. An IQ score is merely a convention for use by educational and legal systems but it explains nothing. If you’re going to write a book about “superintelligence,” I would say you have to do better.

In a related vein, Bostrom seems to have never given two thoughts to the nature of intuition, creativity, agency, subjectivity, empathy, emotion, intrinsic motivation or aesthetics, just to name a few faculties of the intelligent mind that seem important. You would think a philosopher would be at least minimally familiar with current concepts in consciousness studies, such as the debate over qualia. He assumes memory is about storage and retrieval of data, which many people believe, though that is not supported by the scientific research (on humans).

The author proceeds blithely as if there were no question about what intelligence is, so what does he think “super” intelligence is?  It seems to mean symbolic problem solving at a rate much greater than humans can accomplish. Problem-solving slips in there as a new, undocumented re-definition of intelligence. Even if it were, why would “faster” = “smarter?”  What’s the hurry?

Such shortcomings, and many others, including rampant anthropomorphism, leave the discussion ungrounded, a mere exercise for its own sake, leading to nothing. As if that were not bad enough, the writing is execrable. Consider this description of how a “super” AI might solve a problem:

“…the programmer would simply specify a formal criterion of what counts as a success and leave it to the AI to find a solution. To guide its search, the AI would use a set of powerful heuristics and other methods to discover structure in the space of possible solutions. It would keep searching until if found a solution that satisfied the success criterion. (p. 186)”

In other words, the AI would use “methods” to search for a solution. I would do the same myself! Nothing is revealed by the author’s obfuscatory verbiage.

The book is “highly recommended” by Bill Gates, on the front cover. Maybe that should tell you something.  Better choices might be “Artificial Intelligence: A Modern Approach,” by Russell and Norvig, or “The Cambridge Handbook of Artificial Intelligence” by Frankesh and Ramsey.

Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies, Reprint Edition. New York: Oxford University Press, 2014. 415 pp.