Game controllerBeta

Pad

Two sticks, twelve buttons, one brain.

Game controller

An opponent that plays with thumbs.

Pad maps game frames to a standard controller: two analogue sticks, triggers and buttons. Trained per game by imitation and reinforcement, on the same brain.

Frames enter through the compound-eye adapter; the sticks leave through the descending neurons as continuous values, the buttons as spikes. The fly's native competence, steering toward a moving target, is exactly what most games ask for.

Reacts in 90 ms.

Frame-to-stick latency of 90 ms end to end at 60 fps, inside the range of human players. Steering tasks converge in under two hours of self-play on one GPU.

target Figures marked "target" are design targets for the first release, not measurements.

90 mstargetReactionframe to stick, 60 fps
< 2 htargetTrainingsteering task, one GPU, self-play
How it is built

Same brain, its own wiring.

Input adapters occupy afferent regions, output adapters read efferent ones. Co-processors feed in through the embedding port.

The 3D view needs WebGPU or WebGL2.
Engine
  • FlyCore v0.1.0
  • weights v0.1.0-beta
Input adapters
  • frames (compound eye)
  • controller state
Output adapters
  • gamepad (sticks, triggers, buttons)
Co-processors
  • none
Python
import flycore

brain = flycore.load("flylabs/pad")      # engine + weights + adapters
agent = brain.attach(frames=env.frames, gamepad=flycore.io.Gamepad())
agent.play(env, hours=2)   # imitation, then self-play
# agent.act(frame) -> GamepadState
Use cases

Two sticks, twelve buttons, one brain.

Sparring partners

Opponents with human reaction times for racing, arena and platform games.

Playtesting at scale

A thousand agents that play like people find the level that is too hard before players do.

Limits
  • Beta: standard XInput layout only.
  • Per-game training required; no zero-shot play.