Can a neural network discover a hidden rule that humans can't easily guess?
PatternMachine is a beginner-friendly project that demonstrates how neural networks work as pattern machines — systems that learn mathematical rules purely from examples, without being told what the rule is.
An instructor shows students these two rows and asks: "What is the rule?"
X Y Z
3 4 10
2 3 7
Most students guess Z = 2·X + 1·Y (integer weights, easy to spot).
The instructor then reveals a full dataset generated from:
Nobody guesses 0.8 and 1.2 from a few rows. A neural network, given thousands of examples, discovers those weights on its own — and that is the lesson.
patternmachine/
├── dataset.py # Module 1 — dataset generation
├── train.py # Module 2 — neural network training
├── infer.py # Module 3 — inference
├── app.py # Module 4 — Streamlit interactive UI
├── main.py # CLI entrypoint
├── requirements.txt
├── data/
│ └── dataset.csv # generated training data
├── artifacts/
│ ├── weights.json # learned weights (human-readable)
│ └── loss_history.json
└── venv/ # Python virtual environment
The architecture is intentionally the smallest possible model that can represent the target pattern. This is a teaching project — complexity would obscure the lesson.
Input Layer Hidden Layer(s) Output Layer
────────── ────────────── ────────────
X ──────┐
├──── (w1, w2) ────── Z
Y ──────┘
nn.Linear(in=2, out=1, bias=False)
| Layer | Type | In features | Out features | Bias | Activation |
|---|---|---|---|---|---|
| 1 | nn.Linear |
2 | 1 | No | None |
| Parameter | Symbol | Count |
|---|---|---|
| Weight for X | w1 |
1 |
| Weight for Y | w2 |
1 |
| Bias | — | 0 (disabled) |
| Total | 2 |
The entire knowledge of the trained model is encoded in 2 floating-point numbers.
| Choice | Rationale |
|---|---|
| Single linear layer | The target Z = 0.8·X + 1.2·Y is a pure linear combination. A single linear neuron is the exact minimal representation. |
| No activation function | Activations (ReLU, sigmoid, tanh) introduce non-linearity. Since the pattern is linear, adding them would make weights unreadable and the lesson muddy. |
| No bias term | The true equation has no constant offset (+c). With bias=False, the 2 learned parameters are literally w1 and w2 — the answer you can read from weights.json. |
| No hidden layers | Hidden layers transform the input into new representations. That power is unnecessary and unwanted here — it would prevent direct weight inspection. |
| Hyperparameter | Value | Why this value |
|---|---|---|
| Learning rate | 0.01 |
Small enough to be stable on this convex loss surface; large enough to converge in a few hundred epochs. Too high → weights oscillate. Too low → painfully slow. |
| Optimizer | SGD (Stochastic Gradient Descent) | The most transparent optimiser. Each update is: w ← w − lr × ∇loss. No momentum, no adaptive scaling — the raw gradient story. |
| Loss function | MSE (Mean Squared Error) | Standard for regression. Penalises large errors quadratically; gradient is a clean straight line, ideal for teaching. |
| Batch size | 32 |
Mini-batch gradient descent. Shows learners that the network does not need the full dataset per step — a small random sample is enough. |
| Epochs | 200 (with early stop at loss < 1e-6) |
More than enough for a clean linear problem. Early stopping prevents unnecessary computation once convergence is reached. |
| Train / Val split | 80% / 20% |
Validates the model generalises beyond the training rows. Sanity check against memorisation. |
| Input standardisation | None | Raw X and Y values are used. If we normalised, the learned weights would correspond to the scaled inputs — not 0.8 and 1.2. Keeping raw values preserves the "aha" moment. |
| Weight initialisation | PyTorch default (Kaiming uniform) | Random starting point. The lesson is that the network converges to the correct weights regardless of where it starts. |
| Seed | 42 |
Fixed seed for the dataset RNG and train/val split. Reproducibility for classroom demos. |
| Property | Value |
|---|---|
| Equation | Z = 0.8·X + 1.2·Y + N(0, noise_std) |
| Default samples | 1,000 |
| X range | Uniform(−10, 10) |
| Y range | Uniform(−10, 10) |
| Noise | 0.0 (default) — exact pattern |
| Format | CSV with columns X, Y, Z |
On the default noise-free dataset, convergence is near-instant:
Epoch 1/200 train_loss=3.28e+00 val_loss=2.75e-13 w1=0.8000 w2=1.2000
Early stop at epoch 2: loss=2.49e-13
Learned : w1 = 0.800000, w2 = 1.200000
Target : w1 = 0.8, w2 = 1.2
|Δw1| = 0.000000
|Δw2| = 0.000000
The clean linear dataset is a solved problem for a linear model — the gradient points directly to the answer from step 1.
Tip: Set noise_std > 0 via the Streamlit slider to observe how noise causes the learned weights to fluctuate slightly around the true values — a glimpse of real-world ML.
{
"w1": 0.800000,
"w2": 1.200000,
"epochs_run": 2,
"final_train_loss": 2.49e-13,
"final_val_loss": 2.75e-13,
"target": { "w1": 0.8, "w2": 1.2 }
}A list of per-epoch snapshots used to draw the loss curve in the Streamlit app:
[
{ "epoch": 1, "train_loss": 3.28, "val_loss": 2.75e-13, "w1": 0.8, "w2": 1.2 },
...
]- Windows, Python 3.10+
- The
venv/folder already exists in the project root.
# PowerShell
.\venv\Scripts\Activate.ps1
source .venv/bin/activate
pip install -r requirements.txt# 1. Generate the dataset (1,000 rows by default)
python main.py generate
# 2. Train — watch w1 → 0.8 and w2 → 1.2 in the console
python main.py train
# 3. Predict Z for X=3, Y=4
python main.py infer --x 3 --y 4streamlit run app.pyOpens a browser with three tabs:
- Data — configure and generate the dataset with sliders; view scatter plots
- Train — start training; watch the live loss curve and weight convergence chart
- Infer — enter any X and Y; compare the network's prediction to the exact formula
| Check | Target |
|---|---|
|w1 − 0.8| after training |
< 0.01 |
|w2 − 1.2| after training |
< 0.01 |
Inference error on X=3, Y=4 (exact Z = 7.2) |
< 0.05 |
| Streamlit app: all three tabs work | ✅ |
| Package | Version (installed) | Role |
|---|---|---|
numpy |
2.2.6 | Numerical array ops for dataset generation |
pandas |
2.3.3 | CSV I/O and tabular data display |
torch |
2.12.1 | Neural network, autograd, training loop |
streamlit |
1.58.0 | Interactive browser UI |
matplotlib |
3.10.9 | Loss curve and scatter plots |
| Extension | Concept it teaches |
|---|---|
Set bias=True, observe bias → ~0 |
Bias terms and their meaning |
Increase noise_std to 1–2 |
Model robustness to noisy data |
Swap SGD for Adam (torch.optim.Adam) |
Adaptive learning rates |
| Replace PyTorch with NumPy + manual backprop | Demystifying autograd |
Extend to 3 inputs: Z = 0.5·X + 0.3·Y + 0.2·W |
Scaling to more features |
| Add a hidden layer and a ReLU | Why depth helps on non-linear problems |