AI · Neural Network
Not mysterious math, but familiar math stacked deep.
An AI that reads handwritten digits — what mysterious mathematics could be inside?
Distinguish a model output from ground truth.
The XOR that a single-layer network couldn't solve — and pushed AI into a winter.
Open the immersive version →A reading path containing this lesson: Learn by reducing error · 3/3
Why does this question matter?
ChatGPT, face recognition, translation — all neural networks. They look like black boxes, but open one and it's entirely the math you've learned on this site.
One neuron is one function. So stacking layers is composing functions, and because it is a composition, differentiation still works. Since differentiation works, you can nudge the weights in the direction that shrinks the error. That nudging is what we call learning.
Which makes the neural network the journey's terminus and observation deck — where the journeys of functions, change, and uncertainty all meet.
Pause and try it yourself
Make a prediction first. Change one input at a time and compare what happens with what you expected.
Predict first — inside a digit-reading neural network, is there completely new mathematics?
Steer one neuron and build an AND gate
Output = 1 if (w₁x₁ + w₂x₂ + b > 0). Turn the knobs until all four rows show ✓.
The blue region is where the output is 1. Turn the knobs and the line moves. Leave only (1,1) inside the blue and you've done it.
| Input | Output | Target | |
|---|---|---|---|
| (0, 0) | 0 | 0 | ✓ |
| (0, 1) | 1 | 0 | ✗ |
| (1, 0) | 1 | 0 | ✗ |
| (1, 1) | 1 | 1 | ✓ |
Read the idea from another angle
A neuron isn't a brain cell — it's a function.
Take inputs, multiply by weights, add, fire past a threshold — the 'box' from the Function lab, verbatim. Not being fooled by the name is the first step to understanding.
What looks like intelligence is scale plus composition.
One neuron is humble; stack hundreds of millions and tune them with gradient descent, and pattern recognition emerges. Simple rules + vast repetition — how mathematics imitates intelligence.
Now read the definitions with their conditions
What you steered in the experiment, written as formulas, is the whole network.
One neuron
Weighted sum (w·x+b) through an activation σ — the three knobs you steered. A linear function composed with a threshold.
Stacking layers = composing functions
Each layer is a function. The more you compose, the richer the expressible boundaries — why the XOR wall fell.
Learning = gradient descent
Differentiate the loss L by each weight and step against it — the previous lab's update rule verbatim. Passing derivatives backwards through layers is backpropagation.
Make one more distinction
Inside a neural network is mysterious new brain-inspired math.
One neuron is a simple function: multiply by weights, add, fire past a threshold. Nothing mathematically new — the vast composition of simple things creates complex patterns.
Stacking layers just means computing several times.
Composition changes expressive power qualitatively. A single neuron cuts the world with one straight line (no XOR); stacked layers can carve boundaries of any shape — the power of function composition.
Where else can this idea help?
Image recognition
Pixels → edges → shapes → 'cat' — each layer composes progressively more abstract features.
Language models
ChatGPT is essentially one giant function: 'context → probability distribution over the next word'. The Function lab's box, grown to hundreds of billions of parameters.
Recommendation and search
A network outputting 'probability this user clicks' orders your feed.
A tool of science
Protein folding (AlphaFold), weather forecasting, drug discovery — neural networks are already science's microscope.
Explain it in your own words
Try another question
Q1Which logic gate is the neuron y = step(x₁ + x₂ − 1.5)?Show answer ▾
AND. Only (1,1) sums to 2, crossing the 1.5 threshold to output 1. Lower the threshold to 0.5 and it becomes OR — the weights ARE the function.
Q2Explain, as a picture, why a single neuron can't solve XOR.Show answer ▾
One neuron cuts the plane with a single straight line. XOR's positives (0,1),(1,0) and negatives (0,0),(1,1) sit crosswise on the diagonals — no line separates them. Stack layers to carve a curved boundary.
Q3Is any part of a neural network something this site didn't teach?Show answer ▾
No. Neuron=function, layers=composition, learning=derivatives (gradient descent), outputs=probability. Every station of the journey meets here — which is why this lab is the terminus.
Try answering yourself before revealing it — getting it wrong is where learning starts.
What question comes next?
A reading path containing this lesson: Learn by reducing error · 3/3
Previous concept
Gradient Descent
Master the descent (learning) first, and building the mountain (the network) completes it.
← Gradient Descent labLeads to next
Functions
From the journey's end back to its beginning — knowing a neural network is one giant function, the starting point ('the promise of input and output') will look new again.
Go to the Functions lab →Related lab
Functions
The neuron's true identity is the 'box' — the journey's first station becomes the last station's part.
Go to the Functions lab →Related lab
Gradient Descent
How a network 'learns', itself — the station just before this one.
Go to the Gradient Descent lab →