Hebbian learning

That is the whole idea, and the phrasing is most of why everyone knows it. A synapse between two cells strengthens when those cells are active together. Repeat it enough and the connection has recorded something about the world — no instructor, no answer key, nothing but the coincidence itself.

The line is not Hebb’s. It is a mnemonic generally credited to Carla Shatz some four decades later, and it is looser than what he proposed: Hebb specified repeatedly, and he specified a direction — cell A taking part in firing cell B, rather than the two merely coinciding. Both matter, and the mnemonic drops both. It has survived anyway, because it gets the idea across in seven words and the corrections take a paragraph.

The rule

Strip it to its arithmetic and it is about as simple as a learning rule gets: the change in a connection is proportional to the product of activity on each side of it.

$$\Delta w_{ij} \propto x_i \, y_j$$

Everything on the right-hand side is available at the synapse. Nothing has to be computed elsewhere and delivered — no global error term, no signal that has to travel backwards through the network.

Why an engineer cares about locality

That property is the whole reason this crossed into hardware.

A learning rule requiring a global error signal implies routing: some quantity must be computed centrally and distributed to every adjustable element. On a chip that means wires, memory traffic, and energy — the same costs that make conventional accelerators expensive to run. A rule that needs only the two signals already present at a connection can be implemented in that connection.

This is why spike-timing-dependent plasticity — Hebb’s rule refined so that the relative timing of pre- and post-synaptic spikes sets the sign of the change, not just their coincidence — keeps appearing in neuromorphic designs. Not because it out-performs the alternatives, but because it is the kind of rule you can build out of local circuitry rather than move data around to evaluate.

The instability, which is the interesting part

Pure Hebbian learning does not work. Left alone it is positive feedback: strong weights make the post-synaptic cell more likely to fire, which strengthens those weights further, without bound. Every weight saturates and the representation collapses.

So something has to bound it, and the candidates are all recognisable control problems — normalising the total synaptic weight per cell, making the threshold for strengthening slide with recent activity, or scaling synapses homeostatically over slower timescales. Real neurons appear to use several of these at once, on different timescales.

Which is a nice illustration of a pattern that recurs across this site: the borrowed idea was not sufficient on its own, and what made it usable was stability engineering.

From conjecture to physiology

Worth being clear that Hebb was speculating. He had no evidence for the mechanism in 1949 — it was a postulate offered because it would explain behaviour, not because anyone had seen a synapse change.

Bliss and Lømo’s demonstration of long-term potentiation in 1973 gave it a physiological basis, twenty-four years later. That gap, and the fact that the conjecture survived it, is most of why the idea carries the weight it does.