Machines that Learn Like People

Estimated reading time: 5 minutes

Nerve cells, or neurons, are long, thin cells with branching ends. In the cerebral cortex, which is where visual processing happens, each neuron has about 10,000 branches at each end.

Two cortical neurons thus communicate with each other across 10,000 distinct chemical junctions, known as synapses. Each synapse has its own “weight,” a factor by which it multiplies the strength of an incoming signal. The signals crossing all 10,000 synapses are then added together in the body of the neuron. Patterns of stimulation and electrical activity change the weights of synapses over time, which is the mechanism by which habits and memories become ingrained.

A key operation in the branch of mathematics known as linear algebra is the dot-product, which takes two sequences of numbers — or vectors — multiplies their elements together in an orderly way, and adds up the results to yield a single number. In the cortex, the output of a single neural circuit could thus be thought of as the dot-product of two 10,000-variable vectors. That’s a very large calculation that each neuron in the brain can do at a stroke.

Poggio’s group developed an invariant representation of objects that’s based on dot-products. Suppose that you make a little digital movie of an object rotating 360 degrees in a plane — say, 24 frames, each depicting the object as rotated a little bit further than it was in the last one. You store the movie as a sequence of 24 stills.

Suppose next that you’re presented with a digital image of an unfamiliar object. Because the image can be interpreted as a string of numbers describing the color values of pixels — a vector — you can calculate its dot-product with each of the stills from your movie and store that sequence of 24 numbers.

Invariance

Now, if you’re presented with an image of the same object rotated, say, 90 degrees, and you calculate its dot-product with your sequence of stills, you’ll get the same 24 numbers. They won’t be in the same order: What was the dot-product with the first still will now be the dot-product with the sixth. But they’ll be the same numbers.

That list of numbers, then, is a representation of the new object that is invariant to rotation. Similar sequences of stills, which depict an object at various sizes, or at various locations around the frame, will yield sequences of dot-products that are invariant to size and location.

In their new paper, Poggio and his colleagues — first author Fabio Anselmi, a postdoc in Poggio’s group; Joel Leibo, a research affiliate at the McGovern Institute and a research scientist at Google DeepMind; Lorenzo Rosasco, a visiting professor in the Department of Brain and Cognitive Science; and Jim Mutch and Andrea Tacchetti, graduate students in Poggio’s group — demonstrate that, if the goal is to produce an object representation invariant to rotation, size, and location, then the ideal template is a set of images known as Gabor filters. And Gabor filters, it turns out, are known to offer a good description of the image-processing operations performed by the so-called “simple cells” in the visual cortex.

Three dimensions

While this technique works well for visual transformations within a plane, however, it doesn’t work as well for rotation in three dimensions. The dot-product between a new image and that of, say, a car seen straight on would be very different from the dot-product of the same image and that of a car seen from the side.

But Poggio’s group has shown that if the template of still images depicts an object of the same type as the new object, dot-products will still yield adequately invariant descriptions. And this observation accords with recent research by MIT’s Nancy Kanwisher and others, indicating that the visual cortex has regions specialized for recognizing particular classes of objects, such as faces or bodies.

In the work described in PLOS Computational Biology, Poggio and his colleagues — Leibo, Anselmi, and Qianli Liao, a graduate student in electrical engineering and computer science — built a computer system that assembled a set of still images and used the dot-product algorithm to learn to classify thousands of random objects.

For each of the object classes that the system learned, it produced a set of templates that predicted the size and variance of the regions in the human visual cortex devoted to corresponding classes. That suggests, the researchers argue, that the brain and their system may be doing something similar.

The researchers’ invariance hypothesis is “a powerful approach to bridge the large gap between contemporary machine learning, with its emphasis on millions of labeled examples, and the primate visual system that in many instances can learn from a single example,” says Christof Koch, a professor of biology and engineering at Caltech and chief scientific officer of the Allen Institute for Brain Science. “This sort of elegant mathematical framework will be necessary if we are to understand existing natural intelligent systems, on the road to building powerful artificial systems.”

The researchers’ work was sponsored, in part, by MIT’s Center for Brains, Minds, and Machines, which is funded by the National Science Foundation and directed by Poggio.