Training a weight set

Every weight set is the full connectome trained for one task: edge weights, per-neuron state update, adapters and readouts. The wiring and the slot map stay fixed. This page is the outline; the full guide ships with the first public training run.

Two stages

  • Imitation. Recordings of the task with observations and actions at the loop rate. The brain learns the timing and the shape of the behaviour before it learns to optimise it.
  • Reinforcement. A Gym-style environment with a reward. FlyGM showed the connectome converges faster than unconstrained baselines on locomotion; we see the same on cursor and pad tasks.
Python
import flycore

brain = flycore.init("flylabs/flycore", adapters={
    "screen": flycore.io.Screen(), "state": flycore.io.BodyState(), "mouse": flycore.io.Mouse()})

brain.fit(recordings, epochs=20)            # imitation
brain.train(env=FittsEnv(), hours=4)         # reinforcement
brain.save("my-org/cursor-custom")

Compute

Backprop through time over the full network is heavy but fits: with a scalar state per neuron, a batch of 32 sequences of 98 steps at 50 Hz takes about 6 GB and 5 seconds per training step on a 48 GB GPU, roughly 20 minutes per epoch on 13k recorded segments. Larger per-neuron states cost memory and time linearly. Longer horizons use truncated BPTT. Perception weight sets can train on a sub-network (the optic lobes) and still load into the full engine.

Publishing

A weight set is a Hugging Face repo with weights, adapter code, the slot map version and a model card that states what was measured and what is a target. Weight sets on this site follow the same rule.