Neural Networks are a different paradigm for computing:

• von Neumann machines are based on the processing/memory abstraction of human information processing.
• neural networks are based on the parallel architecture of animal brains.

Neural networks are a form of multiprocessor computer system, with

• simple processing elements
• a high degree of interconnection
• simple scalar messages
• adaptive interaction between elements

A biological neuron may have as many as 10,000 different inputs, and may send its output (the presence or absence of a short-duration spike) to many other neurons. Neurons are wired up in a 3-dimensional pattern.

Real brains, however, are orders of magnitude more complex than any artificial neural network so far considered.

Example: A simple single unit adaptive network:

The network has 2 inputs, and one output. All are binary. The output is

1 if W₀ *I₀ + W₁ * I₁ + W_b > 0 

0 if W₀ *I₀ + W₁ * I₁ + W_b <= 0 

We want it to learn simple OR: output a 1 if either I₀ or I₁ is 1.