A Crash Course on the Foundations of AI 🤖

Welcome to the first post in our AI class series! Today, we’re diving into the core concepts of Artificial Intelligence.

---

What Exactly Is AI?

The term “Artificial Intelligence” was coined in 1956. But what does it really mean? There isn’t one single definition, but rather four main approaches, categorized by two dimensions:

1. Thinking vs. Acting: Is the goal to replicate human thought processes or just to produce intelligent behavior?
2. Human Performance vs. Rationality: Is success measured by how well it mimics human performance, or by how well it achieves a rational, ideal outcome?

This gives us four categories of AI definitions:

• Acting Humanly: This is the famous Turing Test approach. A computer passes this test if a human can’t tell its responses apart from a person’s. To pass, a computer would need natural language processing, knowledge representation, automated reasoning, and machine learning. A “total Turing Test” would also require computer vision and robotics.
• Thinking Humanly: This approach focuses on cognitive modeling - trying to understand and replicate how the human mind works. We can study this through introspection, psychological experiments, or brain imaging. The interdisciplinary field of cognitive science combines AI computer models with psychology to create testable theories of the human mind.
• Thinking Rationally: This is the “laws of thought” approach, which dates back to Greek philosopher Aristotle. It involves using formal logic to codify “right thinking”. The goal is to build programs that can solve any solvable problem using logical deduction. However, two major hurdles are translating informal knowledge into formal logic and the computational complexity of solving problems with many facts.
• Acting Rationally: This is the rational agent approach. A rational agent is a system that acts to achieve the best possible outcome. This is a more general approach than “laws of thought” because correct inference is just one way to act rationally, and sometimes a reflex action is more successful than a deliberate one.

---

A Look at the Foundations of AI 🧠

AI isn’t a new field; it’s built on centuries of work from various disciplines:

• Philosophy: Philosophers like Aristotle, Descartes, and Thomas Hobbes laid the groundwork by asking fundamental questions about the nature of the mind, where knowledge comes from, and how thought leads to action. This included the idea of reasoning as a form of computation.
• Mathematics: This field provided the formal tools for AI, including logic (thanks to people like Boole and Frege), the theory of computation (from Turing’s work), and probability theory (with key contributions from Cardano, Pascal, and Bayes).
• Economics: Economists formalized the problem of making decisions to maximize “utility” or preferred outcomes. Decision theory combines probability and utility to provide a framework for making decisions under uncertainty.
• Neuroscience: This field studies the brain and nervous system. We know the brain consists of neurons that connect at synapses, and that these connections are thought to be the basis for learning. However, we still have a long way to go in understanding how a collection of simple cells leads to thought and consciousness.
• Psychology: Starting with Wilhelm Wundt’s experimental psychology, and later with the development of cognitive psychology, this field began to view humans and animals as information-processing machines.
• Computer Engineering: The invention and continuous improvement of computers have made AI applications possible. Early breakthroughs include the Z-3 by Konrad Zuse and the ENIAC.
• Control Theory and Cybernetics: This field deals with designing systems that can operate autonomously and regulate their own behavior based on feedback from the environment.
• Linguistics: The work of linguists like Noam Chomsky, who showed that a behaviorist approach couldn’t explain the creativity of language, led to the hybrid field of computational linguistics or natural language processing.

---

The History of AI: Key Periods

• The Gestation Period (1943-1955): Early work by McCulloch and Pitts created a model of artificial neurons. Alan Turing’s influential 1950 paper introduced the Turing Test and other key ideas.
• The Birth of AI (1956): The field was officially born at a workshop at Dartmouth College organized by John McCarthy. Researchers like Newell and Simon presented the Logic Theorist, a program that could prove mathematical theorems.
• Early Enthusiasm (1952-1969): This period saw many early successes, such as the General Problem Solver (GPS) that mimicked human problem-solving, and Arthur Samuel’s checkers program that learned to play better than its creator. John McCarthy developed the Lisp programming language and proposed the “Advice Taker,” an early vision of a complete AI system.
• A Dose of Reality (1966-1973): Early optimism faded as researchers encountered the limitations of their systems. Programs failed when applied to wider problems, a phenomenon known as “combinatorial explosion”.
• Knowledge-Based Systems (1969-1979): The focus shifted from “weak methods” (general search) to using powerful, domain-specific knowledge to solve problems. This led to the development of early expert systems like DENDRAL, which inferred molecular structures, and MYCIN, which diagnosed blood infections.
• AI Becomes an Industry (1980-Present): Expert systems like R1, which configured computer orders for a company, proved to be commercially successful. However, an “AI Winter” followed as many companies failed to deliver on extravagant promises.
• The Return of Neural Networks (1986-Present): The reinvention of the back-propagation algorithm led to a resurgence of interest in neural networks, also known as connectionist models. Today, these approaches are often seen as complementary to symbolic AI.
• AI Adopts the Scientific Method (1987-Present): The field has become more rigorous, with an emphasis on building on existing theories and using hard experimental evidence. This has led to robust applications in speech recognition and machine translation.
• The Emergence of Intelligent Agents (1995-Present): Researchers began to focus on the “whole agent” problem, leading to AI systems that are now common in applications like search engines and recommender systems. This has also brought AI into closer contact with fields like control theory and economics.
• The Availability of Very Large Data Sets (2001-Present): A major shift has occurred with the realization that for many problems, having a massive amount of data can be more important than the specific algorithm used. This has led to breakthroughs in areas like word-sense disambiguation and image manipulation.

That’s a lot of ground to cover! We’ll be diving deeper into these topics throughout the course. Stay tuned for our next post on intelligent agents and environments.

References

1. Artificial Intelligence: A Modern Approach” by Russell and Norvig, Chapter 1