<aside> 💡 **This memo aims to give you a basic understanding of current approaches to building AI systems.

It also defines key terms that you’ll come across when engaging with this topic. We think this background info will be useful for understanding the issues around AI alignment which we’ll explain later on.**

• Explains what people commonly mean when they talk about “artificial intelligence”
• Explains what “machine learning” means and give an intuition for how it works
• Explains the difference between the terms “deep learning”, “supervised learning”, “unsupervised learning”, and “reinforcement learning” </aside>

*This memo is adapted from Richard Ngo’s A short introduction to machine learning* ****

Approaches to artificial intelligence

The field of artificial intelligence attempts to develop computer programs that have the capabilities associated with intelligence in humans: language skills, visual perception, motor control, and so on. The field of developing AI got started around the 1950s. Historically, there have been several different approaches to AI.

In the first few decades of research the dominant paradigm was called symbolic AI. This approach was based around coding in specific knowledge about the world and step-by-step rules for behaviour.

For example, the chess-playing AI system Deep Blue (that famously defeated the world chess champion Garry Kasparov in 1997) was a symbolic system. It essentially executed instructions along the lines of:

• Receive a digital representation of a chessboard, with numbers indicating
  • (a) which chess piece is on each square;
  • (b) which moves would be legal;
  • (c) which board positions would count as checkmate.
• Check how each legal move would modify the board. Then score each possible move by looking through the possible future positions it could lead to and evaluating how "good" each resulting position is, according to rules like: "If the other player's queen has been captured, that's worth 9 points; if Deep Blue's queen has been captured, that's worth -9 points." These rules could be quite complex, but they've all been coded in precisely by humans.
• After evaluating hundreds of millions of possible positions that could result from the different moves it could play, play the move with the highest score.

However, symbolic AI faced a number of difficult issues that were hard to overcome. For example, Deep Blue relied on its ability to evaluate millions of chess positions via brute force; however it wouldn’t have been able to succeed in playing the board game of Go which has far more possible positions than chess. It also relies on humans programming in the rules for how Deep Blue should evaluate different chess positions, programming in human chess expertise and knowledge.

Since the 1990s, the dominant paradigm in AI has instead been machine learning. In machine learning, instead of specifying rules by which AIs find solutions to problems, we "train" an AI to learn its own way to do a task through trial and error.

Today, the most common way of doing this is by programming a computer to create an "artificial neural network" (ANN), which you might think of sort of like a "digital brain" that starts in an empty (or random) state.

For example, AlphaZero - an AI that has been used to master multiple board games including chess and Go - does something more like the following (although it has important elements of "traditional programming" as well, which we’re ignoring for simplicity):

• Plays a chess game against itself (by choosing a legal move, modifying the digital game board accordingly, and then choosing another legal move, etc.) Initially, it's playing by making random moves.
• Every time White wins, it "learns" a small amount, by tweaking the wiring of the ANN ("digital brain") - literally by strengthening or weakening the connections between some "artificial neurons" and others. The tweaks cause the ANN to form a stronger association between game states like what it just saw and "White is going to win." And vice versa when Black wins.
• After a very large number of games, the ANN has become very good at determining - from a digital board game state - which side is likely to win. The ANN can now select moves that make its own side more likely to win.
• The process of "training" the ANN takes a very large amount of trial-and-error: it is initially terrible at chess, and it needs to play a lot of games to "wire its brain correctly" and become good. Once the ANN has been trained once, though, its "digital brain" is now consistently good at the board game it's learned; it can beat its opponents repeatedly.

(The above explanation of Deep Blue and machine learning was adapted from Holden Karnofsky)

Watch this video for an intuitive explanation of what an artificial neural network is:

https://www.youtube.com/watch?v=aircAruvnKk