The XOR that a single-layer network couldn't solve — and pushed AI into a winter.
An AI that reads handwritten digits — what mysterious mathematics could be inside?
Not mysterious math, but familiar math stacked deep.
Experiment
Hands-on experiment
Predict first — inside a digit-reading neural network, is there completely new mathematics?
Steer one neuron and build an AND gate
Output = 1 if (w₁x₁ + w₂x₂ + b > 0). Turn the knobs until all four rows show ✓.
The blue region is where the output is 1. Turn the knobs and the line moves. Leave only (1,1) inside the blue and you've done it.
| Input | Output | Target | |
|---|---|---|---|
| (0, 0) | 0 | 0 | ✓ |
| (0, 1) | 1 | 0 | ✗ |
| (1, 0) | 1 | 0 | ✗ |
| (1, 1) | 1 | 1 | ✓ |
Read more — why it exists · insights · common mistakes · formulasExpand ▾
Why
Why does this exist?
ChatGPT, face recognition, translation — all neural networks. They look like black boxes, but open one and it's entirely the math you've learned on this site.
One neuron is one function. So stacking layers is composing functions, and because it is a composition, differentiation still works. Since differentiation works, you can nudge the weights in the direction that shrinks the error. That nudging is what we call learning.
Which makes the neural network the journey's terminus and observation deck — where the journeys of functions, change, and uncertainty all meet.
Insight
Insights from the video
“A neuron isn't a brain cell — it's a function.”
Take inputs, multiply by weights, add, fire past a threshold — the 'box' from the Function lab, verbatim. Not being fooled by the name is the first step to understanding.
“What looks like intelligence is scale plus composition.”
One neuron is humble; stack hundreds of millions and tune them with gradient descent, and pattern recognition emerges. Simple rules + vast repetition — how mathematics imitates intelligence.
Misconception
Common misconceptions
Inside a neural network is mysterious new brain-inspired math.
One neuron is a simple function: multiply by weights, add, fire past a threshold. Nothing mathematically new — the vast composition of simple things creates complex patterns.
Stacking layers just means computing several times.
Composition changes expressive power qualitatively. A single neuron cuts the world with one straight line (no XOR); stacked layers can carve boundaries of any shape — the power of function composition.
Formula
Writing it as math
What you steered in the experiment, written as formulas, is the whole network.
One neuron
Weighted sum (w·x+b) through an activation σ — the three knobs you steered. A linear function composed with a threshold.
Stacking layers = composing functions
Each layer is a function. The more you compose, the richer the expressible boundaries — why the XOR wall fell.
Learning = gradient descent
Differentiate the loss L by each weight and step against it — the previous lab's update rule verbatim. Passing derivatives backwards through layers is backpropagation.
In Real Life
Where you meet it in real life
Image recognition
Pixels → edges → shapes → 'cat' — each layer composes progressively more abstract features.
Language models
ChatGPT is essentially one giant function: 'context → probability distribution over the next word'. The Function lab's box, grown to hundreds of billions of parameters.
Recommendation and search
A network outputting 'probability this user clicks' orders your feed.
A tool of science
Protein folding (AlphaFold), weather forecasting, drug discovery — neural networks are already science's microscope.
Try Yourself
Test yourself
Q1Which logic gate is the neuron y = step(x₁ + x₂ − 1.5)?Show answer ▾
AND. Only (1,1) sums to 2, crossing the 1.5 threshold to output 1. Lower the threshold to 0.5 and it becomes OR — the weights ARE the function.
Q2Explain, as a picture, why a single neuron can't solve XOR.Show answer ▾
One neuron cuts the plane with a single straight line. XOR's positives (0,1),(1,0) and negatives (0,0),(1,1) sit crosswise on the diagonals — no line separates them. Stack layers to carve a curved boundary.
Q3Is any part of a neural network something this site didn't teach?Show answer ▾
No. Neuron=function, layers=composition, learning=derivatives (gradient descent), outputs=probability. Every station of the journey meets here — which is why this lab is the terminus.
Try answering yourself before revealing it — getting it wrong is where learning starts.
Connection
Concepts connect
Previous concept
Gradient Descent
Master the descent (learning) first, and building the mountain (the network) completes it.
← Gradient Descent labLeads to next
Functions
From the journey's end back to its beginning — knowing a neural network is one giant function, the starting point ('the promise of input and output') will look new again.
Go to the Functions lab →Related
Labs worth exploring together
Related lab
Functions
The neuron's true identity is the 'box' — the journey's first station becomes the last station's part.
Go to the Functions lab →Related lab
Gradient Descent
How a network 'learns', itself — the station just before this one.
Go to the Gradient Descent lab →