The Lab Handbook

What this lab is

Nine studio tools where algorithms, machine perception, motion, and code are art materials - like paint, paper, and clay. The machine never makes the art alone: it offers a material, and the child makes the visible choices.

Everything runs in the browser on this device. No accounts, no uploads, no cloud. Photos and camera frames never leave the computer, sound is analyzed locally and never recorded, and any code a child writes is tested in a sealed sandbox before it may touch the artwork.

Every lesson ends with the same three questions: What did the algorithm do? What did you choose? What meaning did your choice add?

The nine tools

1. The Machine Brush index.html

An algorithm rebuilds a picture as ink lines, poster colors, halftone dots, mosaic tiles, flow strokes, or a soft field. The child picks the brush and turns four dials.

Brush modes

Edge ink (finds strong brightness changes), Poster colors (fewer colors, like screen print), Halftone (brightness becomes dot size), Mosaic (average tiles), Flow strokes (strokes follow an invisible wind), Soft field (blur as atmosphere).

Dials

Intensity (how strongly the algorithm acts), Scale (mark size), Palette (how many colors survive), Variation (randomness).

Teaches: an algorithm can be a brush; edges, quantization, and sampling in art terms. Try first: Edge ink on the studio sample, then slide Intensity from 0 to 100 and say what appeared and what disappeared.

Teacher's note

Every mode here is a classical image-processing algorithm wearing an art smock. Edge ink computes brightness gradients - how fast neighboring pixels change - which is the very first step of classical computer vision (the same idea inside the Canny edge detector). Poster colors is quantization (fewer allowed values), Halftone is sampling (brightness becomes dot size), Mosaic is block averaging, Flow strokes follow a vector field, Soft field is a blur kernel. These small filters are the direct ancestors of the convolution kernels that convolutional neural networks stack by the million.

Tech deep dive

The photo lives in memory as one flat array: ImageData, four numbers (red, green, blue, alpha) per pixel. At 960x640 that is 2,457,600 numbers (~2.4 MB) - and every mode is a loop that reads them and writes new ones. Edge ink reads each pixel's left/right and up/down neighbors to estimate the gradient; Mosaic averages each tile's pixels into one color. Nothing is "understood" - it is arithmetic over a very long list, which is exactly the honest lesson.

2. Mosaic Studio mosaic.html

A Voronoi rule divides the picture into cells - stained glass, paper quilt, or memory map. Click the artwork to add a cell exactly where you want more detail.

Materials

Stained glass (dark lead lines), Paper quilt (warm stitched pieces), Memory map (muted regions and borders).

Dials

Cells (how many pieces), Gap (space between pieces), Palette (color count), Irregularity (how wild the cell shapes are). Exports PNG and a cuttable SVG.

Teaches: composition under constraints; nearest-neighbor regions; density as emphasis. Try first: click five extra cells on the face of the portrait and watch detail appear only there.

Teacher's note

The cells are a Voronoi partition: every pixel belongs to the nearest seed point, so each seed "owns" a region. Computer vision uses the same grouping instinct: superpixel algorithms (like SLIC) tile an image into small nearest-cluster regions before analyzing it. Clicking extra seeds where you want detail is literally allocating resolution by attention - what matters gets more cells.

Tech deep dive

Memory holds just the seed list - an x, y and a sampled color per seed (a few hundred numbers), not the cells themselves. The cells exist only as a computation: for each pixel, find the nearest seed. Add a seed and every nearby pixel switches owner, which is why one click reshapes the whole neighborhood.

3. Living Pattern Lab pattern.html

Two invisible chemicals feed and fight (reaction-diffusion). Coral, cells, mazes, and bubbles grow on their own; the child plants seeds by dragging and tunes the rule.

Recipes

Coral, Cells, Maze, Bubbles - each is just a different Growth/Decay pair.

Dials

Growth (how fast paint feeds), Decay (how fast it fades), Brush (seed size), Speed (simulation steps per frame). Glazes recolor the world: Coral reef, Ink on paper, Ceramic glaze, Textile dye.

Teaches: art that grows from rules; systems and emergence; freeze-the-moment curation. Try first: pick Maze, drag one line across the canvas, and watch it become a labyrinth.

Teacher's note

This is Gray-Scott reaction-diffusion - two invisible chemicals feed, kill, and spread on a grid, and coral, mazes, and bubbles emerge from purely local rules (Turing's morphogenesis idea). The spreading step is a 3x3 neighborhood convolution (a Laplacian) - the same operation a CNN applies at every layer. A child who tunes feed/kill here has touched the core operation of modern vision networks.

Tech deep dive

Memory: four Float32Array grids of 240x160 (chemical A, chemical B, and a next-step buffer for each) - about 614 KB total. Each simulation step reads every cell's 8 neighbors, applies diffusion + reaction, and writes the next buffer; the Speed dial simply runs more steps per frame. The colors you see are a palette lookup over chemical B's concentration.

4. Layer Collage collage.html

The machine separates the picture into layers; the child decides what material each layer becomes. Two ways of seeing: color regions (no model) or an AI person-cutout that runs entirely on this device.

Backends

Regions (k-means color sorting, instant, honest: "the machine sorts pixels"), AI cutout (a small local segmentation model separates person from background).

Materials per layer

Original (no change), Watercolor, Paper cut, Ink lines, Halftone - plus show/hide per layer.

Teaches: what the machine notices, what it misses, and what the artist adds; layers as collage. Try first: AI cutout on the portrait, keep the person Original, make the world Watercolor - then invert it.

Teacher's note

Two generations of computer vision, side by side. "Regions" is k-means clustering: pixels grouped by color similarity, no learning involved - the classical way. "AI cutout" is a real trained neural network (MediaPipe's selfie segmenter) running entirely on this device, answering one question per pixel: person or not? Comparing where each one succeeds and fails - and what the artist adds on top - is the whole lesson about what machines notice versus what they miss.

Tech deep dive

Memory: a label map - one small number per pixel saying which layer it belongs to (~614 KB), plus the neural network itself: a ~9.6 MB TensorFlow Lite model loaded lazily into WebAssembly the first time you press AI cutout. Its output mask uses 0 for person and 255 for background - getting that mapping backwards was a real bug in this lab once, which is a good story about reading model outputs carefully.

5. Motion Drawing motion.html

Movement is the material. A built-in dancer, your pointer, or local camera motion leaves ribbons, yarn, stars, vines, particles, or pose echoes.

Sources

Orbit dancer (works with no camera), Hand/pointer (drag to draw), Camera motion (opt-in; compares frames on-device, nothing uploaded).

Dials

Trail (how long marks last), Size, Density, Motion (sensitivity/energy).

Teaches: drawing the force, not the object; long-exposure thinking; performance as composition. Try first: Stars brush + slow pointer loops, then the same loops fast - compare the two drawings.

Teacher's note

The camera mode is frame differencing: subtract the previous frame from the current one, pixel by pixel, and whatever is nonzero moved. It is the simplest motion detector and the ancestor of optical flow, which vision systems use to track movement. The pointer modes teach the same idea from the body's side: the drawing records the force of motion over time, like a long-exposure photograph.

Tech deep dive

Memory: the previous camera frame (one ImageData, ~2.4 MB) kept just long enough to compare against the current one - motion is |now - before| per pixel, thresholded. Nothing is recorded or stored beyond that one frame, which is also the privacy story: the camera image never leaves this device and never outlives the comparison.

6. Painterly Gaussian painterly.html

Every dot of paint is a soft gaussian - the photo becomes thousands of pigment drops. Fewer dots = more memory, less photo. Nine pigments, brush-step history, living motion, and music as a force.

Pigments

Watercolor, Oil, Pastel, Charcoal, Ink, Petal, Smoke, Sand, Gold leaf - each a different blob shape, softness, and blending.

Coverage

Whole picture, or Brush areas: the photo melts into paint only where you drag. Every drag is one step with its pigment, dials, and living motion frozen - Undo removes the last step.

Living paint

Still, Breathe, Wind, Alive, Drip, Swirl, Forget, Twinkle + Motion dial + Freeze to capture a moment.

Music as force

Demo beat, microphone (analysis only), or a song file: bass swells the paint, treble sparkles it, beats kick it. Strength dial + a meter showing what the machine hears.

AI material language

By regions or By person: each thing the machine sees gets its own pigment - or stays Original.

Record video

Captures the living painting as a short movie (up to 30 seconds) - great for music-conducted or brush-step animations. Saves MP4 on iPad/Safari, WebM elsewhere.

Teaches: density as style (photographic to abstract), memory and dissolving, sound made visible. Try first: By person + person Original + watercolor world; then Demo beat and watch the world dance while the person holds still.

Teacher's note

The paint atom here is a Gaussian - a soft bell-curve blob - and the painting is thousands of them layered up. That is the 2D cousin of 3D Gaussian Splatting, the technique modern neural rendering uses to rebuild photoreal scenes from soft blobs. Music-as-force uses a real FFT (the browser's audio analyser) to split sound into bass/treble bands that push the paint - signal processing made visible.

Tech deep dive

Memory: a marks array - each mark stores its origin, pigment, layer, rotation, and phase (a handful of numbers each, thousands of marks). Each brush stroke freezes its own pigment AND its own living motion into those marks, which is why one area can drip while another breathes. Stamps (pre-rendered blob textures) are cached by color and shape so drawing stays fast; the living modes never mutate the marks - they re-answer "where is this mark right now" every frame.

7. Draw Alive Studio draw-alive.html

The child draws freely, says what the drawing should become, and the studio wakes it up - animating the child's ORIGINAL strokes, never replacing them with ready-made pictures.

Draw

Eight brush colors plus a full color picker (the rainbow swatch opens every color there is), a size slider, a brush cursor that shows exactly where and how big the next mark will be, a Brush/Eraser toggle (the eraser rubs away whole strokes it touches), Undo, Clear, and Replay (the drawing draws itself back, stroke by stroke, in the order it was made). You can also put a photo on the paper and trace it by hand - the photo stays on the device.

Make it line art

Or let the studio trace the photo for you: "Make it line art" turns the photo into real strokes. A photo OF a drawing (dark pen on light paper) is thinned to single centerlines; any other photo gets its edges traced. Each traced line keeps the photo's own color, so a marker drawing stays a marker drawing. Either way the result is strokes - so a photo of a paper drawing can be woken up, guessed at, and replayed like it was drawn right here.

The studio guesses

Phase 2 is in: after each stroke the studio compares your drawing to little "shape memories" written in its code (a point-cloud matcher - classic pattern-recognition AI, no network) and shows its three best guesses with confidence. Tap a guess to accept it, or correct it with the answer buttons. Press "Wake it up" without choosing and the studio trusts its own top guess.

Wake it up

Pick what it becomes - Fish (swims, tail wiggles, bubbles), Bird (bobs and flaps), Flower (blooms and sways), Rocket (shakes, then lifts off on a flame), Snake (slithers in a wave), Butterfly (wings fold and float), Tree (sways, leaves drift), Sun (pulses with a glow and sparks). Fish, snakes, and side-view birds head the way YOU drew them facing - the studio compares your shape to its left- and right-facing memories and follows the nose. Switch the choice while it is alive and the same drawing becomes something new.

Everything!

Drew more than one thing - or traced a whole photo? The ninth choice treats every separate object on the paper as its own creature: ink that sits close together belongs together, tiny marks (an eye, a cheek) stay with their face, and each object is guessed on its own, faces its own way, and lives on its own clock. A wide flat line is read as the ground - it holds still while everything standing on it comes alive. Trace a photo of a garden and the sun pulses while the flower blooms and the ground stays put.

Keep it

Download the picture, or record up to 30 seconds of the living drawing as a video (MP4 on iPad and Safari, WebM elsewhere).

Teaches: a drawing is data (strokes, points, time); the same marks can be read many ways; motion gives meaning. And the guessing machine teaches the honest AI lesson: it sees patterns, not meaning. When it calls your cat a butterfly, look at your shapes - two big round lobes and a line in the middle IS a butterfly pattern. Try first: draw a circle with eight little lines poking out and watch the guess; then draw a quick blob, press Fish, then Rocket - the same strokes tell two stories.

Teacher's note

This is the lab's densest computer-vision tool, all classical and all on-device. The guesser is a $P point-cloud recognizer: your strokes are resampled into a 48-point cloud and matched against small "shape memories" - nearest-neighbor template matching, the honest ancestor of classification. "Make it line art" is a full classical CV pipeline: Sobel edge detection, Otsu thresholding, Zhang-Suen skeleton thinning, then line tracing. "Everything!" groups strokes into objects with union-find - connected components, the same idea vision uses to find separate things in a scene. Facing detection compares mirrored templates: whole-shape evidence instead of fragile rules.

Tech deep dive

Memory: the drawing is an array of strokes, each { color, size, points: [{x, y, t}] } - position plus time, which is what makes Replay possible (the drawing is data with a clock in it). The recognizer never sees pixels: it works on those points, resampled and normalized, so it is fast and explainable. Photo tracing works at half resolution on a pixel buffer, then scales the traced lines back up.

8. Code Conductor conduct.html

The child writes a small function and it conducts the artwork - every dot, every region, every dancing hand asks the code "what should I be right now?" Full chapter below.

Teacher's note

The deepest idea in the lab: a function is a material. The child's code is called every moment and answers what each dot, region, or hand should be right now - and untrusted code runs in a sealed sandbox first, which is real software safety practice. Per-stage teacher's notes live in the chapter below.

9. Function Brush brush.html

The mirror image of the Code Conductor: there the code conducts the engine's artwork; here the child draws with their own hand, and a small function decides what kind of marks the hand leaves. Twenty-two brushes, each a feeling written in math - and the last one is blank, made to be invented.

Draw

Brush/Eraser toggle, a photo you can paint over or rebuild (with the global Photo background slider), Undo stroke, Clear, Freeze, Record video, Download. Every stroke freezes the code it was drawn with, so one painting can hold many brushes at once.

The brushes

The twelve emotional brushes (Gaussian light ... Firefly), Pixel mosaic, six ports of the conductor's best paint forces (Wave, Spiral, Rain, Galaxy, Confetti, Starry night), Garden ("you are planting seeds"), Dancer (your stroke performs itself), and Your own brush.

Designing a brush with AI

"Your own brush" is the lab's first official AI-collaboration doorway. Its code has no effect at all - but every line of the contract is explained in comments, so a child can copy the whole file to an AI and say what they want in feelings: "make the marks float up slowly like bubbles", "a brush that feels like autumn wind", "rain, but going upward". Three habits to teach: describe the FEELING (not the code), read the AI's answer before running it ("can you find the line that makes the magic?"), and change one number afterwards to make it yours. The sandbox tests every brush before it may touch the artwork, and every value is clamped - nothing can break, so the conversation can be fearless.

Teacher's note

This page is where all the lab's math becomes personal: the same helpers (noise, dist, rand, photo) and the same contract as the conductor, but the child's own hand supplies position, pressure, speed and direction. Per-stroke freezing teaches a real systems idea - work that cannot change is cached (finished strokes bake into one sheet) - and the AI doorway teaches the most important habit of the decade: describing intent precisely, then reading generated code critically.

Tech deep dive

Identical engine to the conductor's brush stage (it graduated from there): a mark is ~21 numbers, the pool holds 15,000, still strokes bake into a cached sheet, and photo-colored stamps live in an LRU cache so photo paintings stay fast. The page shares its math, stamps, sandbox and compiler with the conductor through one module (kidcode.js) - the same code, literally, not a copy.

Chaining artworks (the Studio tray)

Every tool can save its artwork to the tray at the bottom of the page, and most tools can use a saved artwork as their source. One tool's output becomes another's input - effects compose like studio processes.

• Grown stained glass: Pattern Lab (grow a coral) → tray → Mosaic Studio (stained glass) → tray → Machine Brush flow strokes.
• Portrait of paint: Layer Collage (person Original, world Halftone) → tray → Painterly Gaussian, brush-melt just the halftone world.
• Conducted memory: any artwork → tray → Code Conductor paint stage → Galaxy force reorganizes it into a spiral of dots.

The tray keeps the last 8 artworks and survives page changes - it is also the session's history of versions, good for the "which version feels more alive?" conversation.

Code Conductor in depth

One mental model: your function is called every moment and answers what things should be right now. Color, style, material, movement - everything is numbers or functions in the child's hands. The engine only keeps safety rails.

Every run is tested first in a sealed sandbox (a Web Worker with a 1.2-second watchdog). Broken code or endless loops get a friendly message, and a failed run never stops the artwork that is already playing.

Stage 1 - Paint dots: force(mark, t)

~2,600 soft dots are sampled from the photo. Your function runs for every dot, every frame.

Optional second function: function background(t) returning { red, green, blue } - the paper itself is code, and can animate.

The eight paint presets, and the numbers to play with

Wave

Math.sin(t * 2 + mark.x / 60) * 24 - change 2 (wave speed), 60 (wave length), 24 (wave height).

Spiral

Distance + angle around the center; 1.4 - dist / 700 makes the middle spin faster. Flip the sign of 0.5 to reverse the stir.

Rain

(t * 150 * (0.4 + mark.ph)) % 420 - % makes falling repeat; mark.ph gives every drop its own speed; alpha fades with the fall.

Heartbeat

Math.pow(Math.abs(Math.sin(t * 2.2)), 9) - the power 9 sharpens the pulse. Try 2 (soft) vs 20 (spiky).

Galaxy

Teleport showcase: every dot gets an orbit from its ph. Change 0.62 to squash the galaxy, 0.3 for spin speed.

Confetti

Color showcase: three sine waves drive red/green/blue separately. Change the 3 / 2 / 4 speeds and watch colors chase each other.

Starry night

Shape + glow + background: stars twinkle by ph, the paper is a coded night. Change background's blue, or swap "star" for "ring".

Mosaic tiles

Math.round(mark.x / 40) * 40 snaps dots to a grid - change 40 for tile size; soft: 0.1 keeps edges hard.

Teacher's note

The dots are samples of the source image - position plus the color found there - which is precisely how vision systems read pictures: not as objects, as samples. The force never moves the dots permanently; it is asked fresh every frame, like a shader answering "where should this sample appear right now."

Tech deep dive

Memory: one mark per dot - origin x/y, its sampled color, and a personal random number ph (a few numbers times DOTS, so 8,000 dots is still under a megabyte). Per frame, force(mark, t) runs once per dot and the result is drawn from a stamp cache; nothing is mutated, so Freeze is just "stop asking".

Stage 2 - Living pattern: rule(x, y, t)

A reaction-diffusion world (like the Pattern Lab) where the rule can be different at every place and moment. The function is sampled across the picture every frame.

Optional functions: seed(t) returns { x, y, size } (or a list) to drop paint blobs; palette(v, t) maps paint amount v (0-1) to { red, green, blue } - the world's entire palette is a kid function and can change with time.

The eight pattern presets

Stripes

Two living materials in drifting bands: Math.sin(x / 120 + t * 0.3) > 0. Change 120 for band width.

Pulse

The whole world breathes: growth swings ±26 with a sine. Speed lives in 1.6.

Raindrops

seed(t) showcase - a drop falls from the sky along a sine path. Change y: 30 to rain from the side.

Rings

Distance from center → rings of different rules, slowly rippling outward.

Wave pen

paint showcase - paint appears where a traveling wave passes: an invisible pen writing with math.

Checker

Math.floor(x / 120) + Math.floor(y / 120) - integer steps make a checkerboard of two materials.

Lava

palette(v, t) showcase - black rock to glowing lava, and the glow pulses with t.

Neon

A dark-wall palette: almost-black background, electric pink-blue paint.

Teacher's note

Same Gray-Scott chemistry as the Pattern Lab, but now the child's rule(x, y, t) bends the physics per region - and palette(v, t) makes even color a function. The spreading step is a convolution; the child is literally programming the coefficients a neural network would learn.

Tech deep dive

Memory: the same four 240x160 Float32 grids, plus a coarse 48x32 rule map holding the child's growth/decay answers (re-asked about ten times a second, not every step - a deliberate budget so kid code cannot stall the simulation). SPEED sets simulation steps per frame.

Stage 3 - Motion dancer: dancer(t)

Return one point or a list (up to 24). Each point is a dancing hand leaving a ribbon from where it was to where it is. The TRAIL setting (0-100) sets how long ribbons last; Clear trails restarts the canvas.

Optional: function background(t) - the color the trails fade into.

The six dancer presets

Ring

Five hands in a circle - the loop for (let i = 0; i < 5; ...) is the corps de ballet. Change 5 to 12.

Figure eight

One hand, two sines (t * 1.4 and t * 2.8) - a Lissajous dance. Press harder on turns via size.

Fireworks

Bursts with age: sparks fly out and fall under age * age * 30 - kid-written gravity.

Rainbow snake

Ten segments following one path with delays (- i * 0.14) - how followers work.

Comet

Pen + glow + background showcase: a glowing head plus a spray tail crossing a coded night.

Polka dots

The "dots" pen stamps instead of drawing lines - six looping stampers.

Teacher's note

The trails are temporal accumulation - the canvas is faded slightly toward the background each frame, then the new ribbon segment is drawn. That is a long-exposure photograph made of math, and the same accumulation idea graphics uses for motion blur.

Tech deep dive

Memory: almost nothing - just the previous frame's hand positions (up to 24 points). The trail itself is stored in the canvas pixels: TRAIL sets how strongly each frame fades before the new segment lands. There is no history array to grow, which is why dancer never slows down no matter how long it runs.

Stage 4 has graduated

The Function brush grew into its own studio - see Function Brush above (and brush.html in the nav). Everything it learned here - the contract, per-stroke freezing, the photo pipeline - moved with it.

The math toolbox

Each tool of math earns its place by making something beautiful. Meet them in this order:

The lesson ladder

Never start from a blank page. Every lesson climbs the same four rungs:

• 1. Flip one number. Run a preset, change one number, say out loud what changed. ("What did 60 do?")
• 2. Swap one token. Change sin to cos, "star" to "ring", + to -. Predict first, then run.
• 3. Combine two ideas. Rain that falls in a spiral; a heartbeat galaxy; stripes with a lava palette.
• 4. Invent. A blank function and a dare: "make the picture breathe", "make it snow sideways", "write your name in paint".

In the Code Conductor there are no sliders at all: the stage settings are commented constants at the top of every file (DOTS, DOT_SIZE, SPEED, TRAIL) - changing them is the very first rung of the ladder.

Safety, in plain language

• Everything runs on this device. No accounts, no uploads, no cloud services.
• Photos stay in the browser; the AI cutout model is stored inside the lab itself.
• The camera and microphone are opt-in, analyzed live, never recorded, never sent anywhere.
• Written code is tested in a sealed sandbox first; endless loops are stopped after 1.2 seconds with a friendly hint; a broken run never destroys the artwork that is already playing.
• Everything the code returns is clamped to safe ranges - the worst a bug can do is look funny.

Glossary

algorithm

a rule the computer follows exactly - in this lab, a brush you design.

pixel

one tiny square of a picture; the canvas here is 960 wide and 640 tall.

alpha

how see-through something is: 0 invisible, 1 solid.

hue

a position on the color wheel, 0-360 (0 red, 120 green, 240 blue).

function

a small machine made of code: values go in, an answer comes out.

parameter / dial

a number you can turn to change the result without changing the rule.

segmentation

the machine sorting a picture into parts ("person", "background").

reaction-diffusion

two chemicals feeding and fighting - patterns that grow on their own.

growth / decay

the two dials of a living pattern: how fast paint feeds and fades.

palette

the set of colors an artwork is allowed to use.

sandbox

a sealed room where new code is tested before it may touch the art.

watchdog

a timer that stops code that runs forever.

tray

the shelf at the bottom of each tool where artworks wait to become new sources.