Dr Peter Norvig is Director of Research at Google and (co-)author of Artificial Intelligence: A Modern Approach - the most used AI textbook worldwide. In this talk, given at Bay Area AI Meetup at TechShop in Menlo Park, CA in December 2008 he discusses recent directions in AI research at Google and elsewhere. In particular, he discusses the use of Non-Parametric Models in Google's award-sweeping natural language translation systems and recent adoption of non-parametric methods for graphics tasks such as "semantic" editing of snapshot photographs.
Reductionism is an amazingly powerful strategy for leveraging the work of scientists and for disseminating the results in the form of re-usable models of structure and causality. But for some of the "remaining hard problem domains" such as Life (biology, psychology, ecology, etc), the World (world modeling, economies, sociology), Intelligence (understanding the brain, intelligence, and creating Artificial General Intelligences - AGI) and the problem of determining the semantics of language (e.g. text) Reductionism has failed. I claim that reductionist models cannot be created in these domains (which have been named "Bizarre Domains") and that we must use Model Free (Holistic) Methods for these domains. This has important implications for AGI research strategies.
Reductionism - the idea that difficult problems should be attacked by dividing them into simpler problems - is one the most effective strategies in the hard sciences. The justification "The Whole equals the sum of its Parts" has been used for thousands of years. Physics and related sciences, and the support disciplines of mathematics and computer science are all permeated by this Reductionist stance, and for good reason: It has worked really well. We have found compact explanations for all kinds of phenomena and have solved countless problems using these strategies.
But these strategies don't always work in Life Sciences like Biology, Genomics, Psychology, and Ecology. Often "The Whole is larger than the sum of its Parts" and when taking things apart, emergent phenomena like life, quality, intelligence, and meaning simply disappear. All attempts to capture the essence of life using Reductionist models, equations, and theories of living systems have failed. The Life Sciences have for decades managed to get by using other approaches. They use methods that emphasize Whole Systems and where context is to be exploited rather than discarded as a distracting nuisance. These methods adopt a more "Holistic" stance.
What has gone largely unnoticed is that many of these alternative approaches involve using weaker and weaker models, all the way to what we will call "Model Free Methods" (following Lionel S. Penrose, 1935). We have gathered a zoo of such methods and implemented some of them in computers. Amazingly, they allow discovery of solutions to problems "without understanding the problem" in the Reductionist sense. The advent of computers able to manipulate Big Data has suddenly made these Holistic Methods available as concrete and workable tools - not only in the life sciences but in engineering, economy, and other disciplines previously dominated by Reductionist models .
We finish with the claim that Artificial Intelligence failed in the 20th century because "Intelligence is Holistic" but AI was then mostly practiced by programmers - a profession that has, by its nature, always attracted the most hard-lined Reductionists. In the 21st Century, progress in AI will require that we convert AI into a Life Science and start using Model Free Methods the way other Life Sciences do. Researchers at Syntience Inc. have been pursuing this approach to AI since 2001 using an Algorithm named Artificial Intuition.
David Nellessen from the Freiberg Software iGEM 2009 gave me a demo of their hot new google wave robot for synthetic biology, SynBioWave.
To try the robot out, add SynBioWave@appspot.com to a googlewave.
"Synthetic Biology, which aims at constructing whole new genomes, is pushed forward by many users and relies on the assembly of genetic elements to devices and later systems. The construction process needs to be transparent and even at final stages control at the basepair level is required. We are building a software environment enabling multiple distributed users to analyze and construct genetic parts and ultimately genomes with real-time communication. Our current version demonstrates the principle use as well as the power of the underlying Google Wave protocol for collaborative synthetic biology efforts. Many wave-robots with a manageable set of capabilities will divide and conquer the complex task of creating a genome in silico. The initial developments of 'SynBioWave' lay the ground for basic layout, calling and data exchange of wave-robots in a clear and open process, so that future robots can be added and shared easily."
more at: http://2009.igem.org/Team:Freiburg_software