Riding Handwired in public

Handwired is a research project by Cheng Lou. The README opens with this line: "Turn your regular JavaScript code into a neural net; no training, exact, yet also fully differentiable!"

I read that line wrong. I thought there was a tool that takes my JS file and gives back a net. There is no such tool. You write the program again, in a special style. This post is me learning that style in public, one wobble at a time.

The widgets below run on a toy I wrote: toy-circuit.ts, about 70 lines. It is a toy of the idea, not Handwired's API. The real runtime is in src/runtime.

What a program is

A Handwired program is a fixed list of FP32 operations. Most are weighted sums and products with a ReLU. ReLU is max(0, x).

You do not write that list by hand. You write builder code. The tutorial calls it "ordinary JavaScript that runs once and appends operations to a Circuit, so nothing in the program branches or loops at runtime."

const circuit = new ToyCircuit()
const a = circuit.input(0, "a")
const above4 = step(circuit, a, 4)
const result = select(circuit, above4, ten, twelve)

const wires = circuit.evaluate(Float32Array.of(4.5))
wires[result] === 11

Wobble 1: the trick

This is the README's first example. The top line is the JS. The bottom two lines are the ops version, which the README marks "For an integer a":

const result = a > 4 ? 10 : 12

const above4 = Math.max(0, a - 4) - Math.max(0, a - 5)
const result = above4 * 10 + (1 - above4) * 12

Here, step builds relu(a-4) - relu(a-5). Move the slider between 4 and 5. At a = 4.5, the JS gives 10 and the ops give 11. The step is not a step there. It is a ramp.

a = 4.5 snap a to the integer grid

a > 4 ? 10 : 12

ops

above4*10 + (1-above4)*12

Different answer. above4 = 0.5, not a clean 0 or 1.

every wire

#	op	label	value
0	input	a	4.5
1	add	sum	0.5
2	relu	relu(x-4)	0.5
3	add	sum	-0.5
4	relu	relu(x-5)	0
5	add	above4	0.5
6	const	10	10
7	const	12	12
8	mul	bit*yes	5
9	add	1-bit	0.5
10	mul	(1-bit)*no	6
11	add	result	11

Turn on the snap. Now a can only be 4 or 5, and the ramp is never hit. This is why Handwired cells declare a range and a grid. The host rounds inputs onto the grid before the program sees them. From the template's toggle:

viewportWidth: input([0, 16_384], {initial: 1280, grid: 1}),
down: {active: event(), x: coordinate(), y: coordinate()},
on: stateBit(),
armed: stateBit(),

Wobble 2: both branches always run

There is no if. Both sides get computed. A 0 or 1 wire picks one with a multiply. Move x and watch the op count. It does not change.

x = 2 (integers 0 to 9)

winner

else

ops run per evaluation

result

102

Both branch wires are yellow. Both get a value on every evaluation. The losing one is multiplied by 0.

#	op	label	value
0	input	x	2
1	add	sum	-2
2	relu	relu(x-4)	0
3	add	sum	-3
4	relu	relu(x-5)	0
5	add	x > 4	0
6	add	if: x*2	4
7	add	else: x+100	102
8	mul	bit*yes	0
9	add	1-bit	1
10	mul	(1-bit)*no	102
11	add	result	102

Wobble 3: wires and the host loop

The program does not own any state. A host loop does four things: write inputs into a Float32Array source table, run one evaluation, write next-state cells back into the table, and write outputs to the DOM.

Click the switch to do all of it at once. Or use the two buttons to do it in slow motion. The real template/toggle.ts does much more: it acts on release, hit-tests the track and springs the thumb. Mine only flips a bit: next = on + click - 2·click·on.

source table: Float32Array [0, 0] (click, on) · evaluations: 0

#	op	label
0	input	click (event)
1	input	on (state)
2	mul	click*on
3	add	next on

Wobble 4: loops become fixed work

A program "runs every operation on every frame and can't stop early, so a loop becomes a count, a tree, a table, or the same work on every item." That line is from docs/patterns.md.

Here is findIndex over 8 numbers. Every item gets compared. Then one pass keeps only the first match. JS stops early. The ops do not. Move the threshold: the op count stays the same wherever the match is. The real patterns doc has smarter versions, such as a count when the test flips once, or a tree.

31728594

find the first item > 6

index

JS findIndex checked

3 items

ops run

every wire

#	op	label	value
0	input	v0	3
1	input	v1	1
2	input	v2	7
3	input	v3	2
4	input	v4	8
5	input	v5	5
6	input	v6	9
7	input	v7	4
8	input	t	6
9	add	v0-t	-3
10	add	sum	-3
11	relu	relu(x-0)	0
12	add	sum	-4
13	relu	relu(x-1)	0
14	add	v0>t	0
15	add	v1-t	-5
16	add	sum	-5
17	relu	relu(x-0)	0
18	add	sum	-6
19	relu	relu(x-1)	0
20	add	v1>t	0
21	add	v2-t	1
22	add	sum	1
23	relu	relu(x-0)	1
24	add	sum	0
25	relu	relu(x-1)	0
26	add	v2>t	1
27	add	v3-t	-4
28	add	sum	-4
29	relu	relu(x-0)	0
30	add	sum	-5
31	relu	relu(x-1)	0
32	add	v3>t	0
33	add	v4-t	2
34	add	sum	2
35	relu	relu(x-0)	2
36	add	sum	1
37	relu	relu(x-1)	1
38	add	v4>t	1
39	add	v5-t	-1
40	add	sum	-1
41	relu	relu(x-0)	0
42	add	sum	-2
43	relu	relu(x-1)	0
44	add	v5>t	0
45	add	v6-t	3
46	add	sum	3
47	relu	relu(x-0)	3
48	add	sum	2
49	relu	relu(x-1)	2
50	add	v6>t	1
51	add	v7-t	-2
52	add	sum	-2
53	relu	relu(x-0)	0
54	add	sum	-3
55	relu	relu(x-1)	0
56	add	v7>t	0
57	const	seen0	0
58	add	sum	0
59	relu	first0	0
60	add	seen1	0
61	add	sum	0
62	relu	first1	0
63	add	seen2	0
64	add	sum	1
65	relu	first2	1
66	add	seen3	1
67	add	sum	-1
68	relu	first3	0
69	add	seen4	1
70	add	sum	0
71	relu	first4	0
72	add	seen5	1
73	add	sum	-1
74	relu	first5	0
75	add	seen6	1
76	add	sum	0
77	relu	first6	0
78	add	seen7	1
79	add	sum	-1
80	relu	first7	0
81	add	seen8	1
82	add	index	2

Wobble 5: it is differentiable

The weights need no training because a person writes them. But the program is still made of sums, products and ReLUs, so gradients can flow through it.

Raise the temperature to soften the step into a smooth curve. Then press train. Gradient descent moves the threshold from 4 toward the gap in the labels, and the loss falls. Press freeze to snap back to a hard step with the learned constant. My curve is a sigmoid, not Handwired's ReLU ramp, to keep the toy short.

temperature = 0.60

threshold

4.00

loss

0.209

correct as hard step

9/12

This is a tiny version of the learned gallery demo. There, a photo layout comes from a small network whose coefficients are source cells, and Adam trains them with gradients from circuit.backpropagateReference.

What it is not

Not a JS-to-net converter. Nothing reads your existing source. You write builder code.
Not unbounded. Cells declare ranges and grids. Off the grid, the tricks give wrong answers, as in wobble 1.
Not early-exit. Every op runs every frame.
Not a GPU runtime by default. src/runtime evaluates on the CPU in JS. The JSON search demo and some lab work use WebGPU.
Not a product. It is one person's research project, with about 1,000 GitHub stars today.

The scale is real, though. The demos page lists the Mini OS program at 9,909 weights in 130 layers, with no training.

What I want to try next

Can learned constants be written back as readable code, like the line freeze prints?
Can a small, exact policy travel as data that an agent carries and runs?
Where does my toy break first when I port one real pattern from the docs?

Start with the repo, the tutorial and the demos.