Object trackingLive demo

Gaze

Follows a grabbed object in video.

The 3D view needs WebGPU or WebGL2.

input neuronsneurons read out

1,581 facets, both compound eyes

Grab an object in a video, and a reticle stays on it.

97%On targetseven objects moving fast
95%In crowded scenesfifteen objects
1,581Facetsboth compound eyes

Gaze sees with the fly’s optic lobes

Gaze tracks with the fly’s optic lobes: switch off their 77,530 neurons, and the reticle stays on the object 27% of the time instead of 98%. The columns that run from the medulla to the lobula, among them T2, T3 and the Tm cells, carry a share of it; in the fly, T2 and T3 respond to small moving objects and feed the lobula’s object detector LC11 (Keleş et al., 2020).

Time on target with one part of the brain switched off
020406080100intactoptic lobescolumnscentral brain98279299on target (%)

Keleş, M. F., Hardcastle, B. J., Städele, C., Xiao, Q. & Frye, M. A. (2020). Inhibitory interactions and columnar inputs to an object motion detector in Drosophila. Cell Reports 30, 2115–2124. doi:10.1016/j.celrep.2020.01.061

In the fly brain

Every video frame reaches the photoreceptors of the fly's compound eyes, and its optic lobes do the seeing.

Senses
    Fly brain138,639neurons15.1 millionconnections
    Where signals enter
    1,299 descending neurons
    Commands