Sparse codes, part two
The explanation in the previous post was maybe a little confusing. I'll try again with a different analogy.
We're trying to understand how neurons in the brain store memories. In this analogy, each neuron corresponds to one key on a piano. One possibility is that each neuron stores one memory. So playing one note would bring to mind one memory. Obviously, I could only store 88 total memories this way.
Another possibility is that every possible combination of notes brings to mind one memory. This could allow for a huge number of memories-- 2 to the 88th power -- but it would be very tricky to bring to mind any particular memory because I'd have to know whether each of the 88 keys should be up or down.
The third possibility is that each memory is associated with a chord. I have only 10 fingers, so there are about 88 to the 10th power different chords I can play (assuming my fingers can stretch that far!) It's still plenty of possible memories I can play, but I only have to keep track of the position of 10 fingers, rather than all 88 keys. This is called a sparse coding (because the chords form a kind of secret code for the memories) and it's probably the system the brain actually uses.
We're trying to understand how neurons in the brain store memories. In this analogy, each neuron corresponds to one key on a piano. One possibility is that each neuron stores one memory. So playing one note would bring to mind one memory. Obviously, I could only store 88 total memories this way.
Another possibility is that every possible combination of notes brings to mind one memory. This could allow for a huge number of memories-- 2 to the 88th power -- but it would be very tricky to bring to mind any particular memory because I'd have to know whether each of the 88 keys should be up or down.
The third possibility is that each memory is associated with a chord. I have only 10 fingers, so there are about 88 to the 10th power different chords I can play (assuming my fingers can stretch that far!) It's still plenty of possible memories I can play, but I only have to keep track of the position of 10 fingers, rather than all 88 keys. This is called a sparse coding (because the chords form a kind of secret code for the memories) and it's probably the system the brain actually uses.
Comments