Brightlab
brightlab.Made for curious mindsbrightlab.Made for curious mindsBrightlab
What does one layer of a neural network do to its input?
Every input to a network is a list of numbers: a vector. Each neuron takes a dot product with its weights. A layer does many at once with a matrix. Then a simple bend, ReLU, folds space so the next layer can separate things with straight lines.
Where this happens: The forward pass: weighted sums in every layer.
Starting knowledge: Coordinates on the plane and arithmetic with decimals. Vectors and matrices are introduced from scratch.
The animations are silent; captions carry the explanation. Pause whenever a caption changes and say what you expect next.
A weight vector and an input vector. The dot product measures how far the input points along the weights; the bias decides which side of a line counts as “on”.
Teal points sit between two groups of orange points, so no straight line separates them. One layer—W, then b, then ReLU—moves every point until a single line can.
Set the four weights and two biases of a layer, switch ReLU on or off, and watch every point move. The straight-line test checks exactly whether any line can separate the colours after the layer.
Two matrix layers in a row do exactly what one matrix does. Put ReLU between them and part of the shape folds flat: something no single matrix can do.
Counterexample challenge. With ReLU on, find a layer that makes things worse, sending a teal point and an orange point to exactly the same place.
A correctly calculated dot product; a W and b that pass the straight-line test with ReLU on; a two-sentence explanation of why the bend is needed.
The animations and lab use two-dimensional data so every step is visible. Real layers work in hundreds or thousands of dimensions and their weights are learned by training, not chosen by hand. The “solving layer” preset was designed for this dataset.