Skip to content

About

PatternMachine is a beginner-friendly project that demonstrates how neural networks work as pattern machines — systems that learn mathematical rules purely from examples, without being told what the rule is.

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

PatternMachine 🧠

Can a neural network discover a hidden rule that humans can't easily guess?

PatternMachine is a beginner-friendly project that demonstrates how neural networks work as pattern machines — systems that learn mathematical rules purely from examples, without being told what the rule is.


The Puzzle

An instructor shows students these two rows and asks: "What is the rule?"

X    Y    Z
3    4    10
2    3     7

Most students guess Z = 2·X + 1·Y (integer weights, easy to spot).

The instructor then reveals a full dataset generated from:

$$Z = 0.8 \cdot X + 1.2 \cdot Y$$

Nobody guesses 0.8 and 1.2 from a few rows. A neural network, given thousands of examples, discovers those weights on its own — and that is the lesson.


Project Structure

patternmachine/
├── dataset.py          # Module 1 — dataset generation
├── train.py            # Module 2 — neural network training
├── infer.py            # Module 3 — inference
├── app.py              # Module 4 — Streamlit interactive UI
├── main.py             # CLI entrypoint
├── requirements.txt
├── data/
│   └── dataset.csv     # generated training data
├── artifacts/
│   ├── weights.json    # learned weights (human-readable)
│   └── loss_history.json
└── venv/               # Python virtual environment

Neural Network Architecture

Design Philosophy

The architecture is intentionally the smallest possible model that can represent the target pattern. This is a teaching project — complexity would obscure the lesson.

Input Layer          Hidden Layer(s)       Output Layer
──────────           ──────────────        ────────────
  X ──────┐
           ├──── (w1, w2) ────── Z
  Y ──────┘

        nn.Linear(in=2, out=1, bias=False)

Layer Summary

Layer Type In features Out features Bias Activation
1 nn.Linear 2 1 No None

Parameter Count

Parameter Symbol Count
Weight for X w1 1
Weight for Y w2 1
Bias — 0 (disabled)
Total 2

The entire knowledge of the trained model is encoded in 2 floating-point numbers.

Why this architecture?

Choice Rationale
Single linear layer The target Z = 0.8·X + 1.2·Y is a pure linear combination. A single linear neuron is the exact minimal representation.
No activation function Activations (ReLU, sigmoid, tanh) introduce non-linearity. Since the pattern is linear, adding them would make weights unreadable and the lesson muddy.
No bias term The true equation has no constant offset (+c). With bias=False, the 2 learned parameters are literally w1 and w2 — the answer you can read from weights.json.
No hidden layers Hidden layers transform the input into new representations. That power is unnecessary and unwanted here — it would prevent direct weight inspection.

Hyperparameters

Hyperparameter Value Why this value
Learning rate 0.01 Small enough to be stable on this convex loss surface; large enough to converge in a few hundred epochs. Too high → weights oscillate. Too low → painfully slow.
Optimizer SGD (Stochastic Gradient Descent) The most transparent optimiser. Each update is: w ← w − lr × ∇loss. No momentum, no adaptive scaling — the raw gradient story.
Loss function MSE (Mean Squared Error) Standard for regression. Penalises large errors quadratically; gradient is a clean straight line, ideal for teaching.
Batch size 32 Mini-batch gradient descent. Shows learners that the network does not need the full dataset per step — a small random sample is enough.
Epochs 200 (with early stop at loss < 1e-6) More than enough for a clean linear problem. Early stopping prevents unnecessary computation once convergence is reached.
Train / Val split 80% / 20% Validates the model generalises beyond the training rows. Sanity check against memorisation.
Input standardisation None Raw X and Y values are used. If we normalised, the learned weights would correspond to the scaled inputs — not 0.8 and 1.2. Keeping raw values preserves the "aha" moment.
Weight initialisation PyTorch default (Kaiming uniform) Random starting point. The lesson is that the network converges to the correct weights regardless of where it starts.
Seed 42 Fixed seed for the dataset RNG and train/val split. Reproducibility for classroom demos.

Dataset

Property Value
Equation Z = 0.8·X + 1.2·Y + N(0, noise_std)
Default samples 1,000
X range Uniform(−10, 10)
Y range Uniform(−10, 10)
Noise 0.0 (default) — exact pattern
Format CSV with columns X, Y, Z

Training Behaviour

On the default noise-free dataset, convergence is near-instant:

Epoch   1/200  train_loss=3.28e+00  val_loss=2.75e-13  w1=0.8000  w2=1.2000
Early stop at epoch 2: loss=2.49e-13

Learned : w1 = 0.800000, w2 = 1.200000
Target  : w1 = 0.8,      w2 = 1.2
|Δw1|   = 0.000000
|Δw2|   = 0.000000

The clean linear dataset is a solved problem for a linear model — the gradient points directly to the answer from step 1.

Tip: Set noise_std > 0 via the Streamlit slider to observe how noise causes the learned weights to fluctuate slightly around the true values — a glimpse of real-world ML.


Saved Artefacts

artifacts/weights.json

{
  "w1": 0.800000,
  "w2": 1.200000,
  "epochs_run": 2,
  "final_train_loss": 2.49e-13,
  "final_val_loss": 2.75e-13,
  "target": { "w1": 0.8, "w2": 1.2 }
}

artifacts/loss_history.json

A list of per-epoch snapshots used to draw the loss curve in the Streamlit app:

[
  { "epoch": 1, "train_loss": 3.28, "val_loss": 2.75e-13, "w1": 0.8, "w2": 1.2 },
  ...
]

Setup & Usage

Prerequisites

  • Windows, Python 3.10+
  • The venv/ folder already exists in the project root.

Activate the virtual environment

# PowerShell
.\venv\Scripts\Activate.ps1

source .venv/bin/activate

Install dependencies

pip install -r requirements.txt

Run end-to-end (CLI)

# 1. Generate the dataset (1,000 rows by default)
python main.py generate

# 2. Train — watch w1 → 0.8 and w2 → 1.2 in the console
python main.py train

# 3. Predict Z for X=3, Y=4
python main.py infer --x 3 --y 4

Launch the interactive Streamlit app

streamlit run app.py

Opens a browser with three tabs:

  • Data — configure and generate the dataset with sliders; view scatter plots
  • Train — start training; watch the live loss curve and weight convergence chart
  • Infer — enter any X and Y; compare the network's prediction to the exact formula

Success Criteria

Check Target
|w1 − 0.8| after training < 0.01
|w2 − 1.2| after training < 0.01
Inference error on X=3, Y=4 (exact Z = 7.2) < 0.05
Streamlit app: all three tabs work ✅

Dependencies

Package Version (installed) Role
numpy 2.2.6 Numerical array ops for dataset generation
pandas 2.3.3 CSV I/O and tabular data display
torch 2.12.1 Neural network, autograd, training loop
streamlit 1.58.0 Interactive browser UI
matplotlib 3.10.9 Loss curve and scatter plots

Suggested Extensions

Extension Concept it teaches
Set bias=True, observe bias → ~0 Bias terms and their meaning
Increase noise_std to 1–2 Model robustness to noisy data
Swap SGD for Adam (torch.optim.Adam) Adaptive learning rates
Replace PyTorch with NumPy + manual backprop Demystifying autograd
Extend to 3 inputs: Z = 0.5·X + 0.3·Y + 0.2·W Scaling to more features
Add a hidden layer and a ReLU Why depth helps on non-linear problems

About

PatternMachine is a beginner-friendly project that demonstrates how neural networks work as pattern machines — systems that learn mathematical rules purely from examples, without being told what the rule is.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages