Memory as attractor in neural networks

[noha]

Hello,

I am translating a book called Models of Mind by Grace Lindasy and I want to understand what is meant by the following from the extract from Chapter 4:

- will drive the network to fill in the rest. (does it mean that the activation of new neurons form certain memory will drive the network to activate more neurons that form other memories related to this memory )

-  network is in an attractor state (the state of network evolves to attractor state  or the network itself became an attracor?)

- Initial states (of the system of neurons in a network? )

-specific memory attractor ( attractor of memories or the memory itself is the attractor "attractor memory"?)

-memory’s ‘basin of attraction’ (the basin that includes memories related to the attractor memory?)

 

A memory is an attractor because the activation of a few of the neurons that form the memory will drive the network to fill in the rest.
Once a network is in an attractor state, it remains there with the neurons fixed in their ‘on’ or ‘off ’ positions.
Always fond of describing things in terms of energy, physicists consider attractors ‘low energy’ states.
They’re a comfortable position for a system to be in; that is what makes them attractive and stable.
Imagine a trampoline with a person standing on it.
A ball placed anywhere on the trampoline will roll towards the person and stay there.
The ball being in the divot created by the person is thus an attractor state for this system.
If two people of the same size were standing opposite each other on the trampoline, the system would have two attractors.
The ball would roll towards whomever it was initially closest to, but all roads would still lead to an attractor.
Memory systems wouldn’t be of much use if they could only store one memory, so it is important that the Hopfield network can sustain multiple attractors.
The same way the ball is compelled towards the nearest low point on the trampoline, initial neural activity states evolve towards the nearest, most similar memory. The initial states that lead to a specific memory attractor – for example, the picture of your childhood bed that reignites a memory of the whole room or a trip to a beach that ignites the memory of a childhood holiday – are said to be in that memory’s ‘basin of attraction’.

[joigus]

The term 'attractor' or 'strange attractor' comes from chaos theory. Most dynamical systems follow chaotic trajectories in phase space --the space of all dynamical states of a system. Typically they evolve in an unpredictable way, and very close initial conditions differ wildly in very short times. However, they display some patterns, in that sometimes trajectories tend to cluster around certain regions and form certain shapes.

Here's the Wikipedia article.

https://en.wikipedia.org/wiki/Attractor

They have indeed been compared to centres of attraction, although there is no force related to their being formed. It's considered to be an example of emergence, which is another concept you might want to take a look at.

I hope that helped.

[Lorentz Jr]

On 2/8/2023 at 1:01 AM, noha said:

- will drive the network to fill in the rest. (does it mean that the activation of new neurons form certain memory will drive the network to activate more neurons that form other memories related to this memory )

Not new neurons, but neurons that already remember relationships between the network's inputs in existing memories (e.g. visual characteristics of complete images). They will activate other neurons that remember other aspects of the same memories (e.g. various images that share properties represented by that neuron, testing to see what other properties the new perception shares with those memories).

On 2/8/2023 at 1:01 AM, noha said:

-  network is in an attractor state (the state of network evolves to attractor state  or the network itself became an attracor?)

(a) The state of each neuron is represented by a dimension in the net's (very high-dimensional) phase space, (b) each memory corresponds to a point in that space, and (c) the net is designed in such a way that its state will approach (i.e. be "attracted" by) the point that corresponds to a memory if a new perception puts the net in a state that's close enough to that point, i.e. within the point's "basin of attraction".

On 2/8/2023 at 1:01 AM, noha said:

- Initial states (of the system of neurons in a network? )

An initial state of the network is the point in phase space, i.e. the collection of individual neural states, that a new perception (e.g. image) puts the net's neurons into.

On 2/8/2023 at 1:01 AM, noha said:

-specific memory attractor ( attractor of memories or the memory itself is the attractor "attractor memory"?)

This is the flow map of how the net's state can go from various initial states, through other intermediate states, to the state for the specified memory during the net's recognition operation.

On 2/8/2023 at 1:01 AM, noha said:

-memory’s ‘basin of attraction’ (the basin that includes memories related to the attractor memory?)

This is the set of states that will eventually lead to the point for the memory. The endpoint (attractor) is the "lowest-energy" point in the basin, and the boundary of the basin is a "ridge" that separates the basin from other basins.

On 2/8/2023 at 1:01 AM, noha said:

Memory systems wouldn’t be of much use if they could only store one memory, so it is important that the Hopfield network can sustain multiple attractors.

So you can think of the net's phase space as a hilly terrain where each valley corresponds to a memory.

 

Edited February 10, 2023 by Lorentz Jr

[noha]

attractor state

(a) The state of each neuron is represented by a dimension in the net's (very high-dimensional) phase space, (b) each memory corresponds to a point in that space, and (c) the net is designed in such a way that its state will approach (i.e. be "attracted" by) the point that corresponds to a memory if a new perception puts the net in a state that's close enough to that point, i.e. within the point's "basin of attraction".

regarding this point, does state here mean position or condition?

To translate this term, can I say 

the position of attractor

The condition of attractor

 

 

[Lorentz Jr]

On 2/16/2023 at 1:35 AM, noha said:

(a) The state of each neuron is represented by a dimension in the net's (very high-dimensional) phase space, (b) each memory corresponds to a point in that space, and (c) the net is designed in such a way that its state will approach (i.e. be "attracted" by) the point that corresponds to a memory if a new perception puts the net in a state that's close enough to that point, i.e. within the point's "basin of attraction".

does state here mean position or condition?

The state of a neuron is its output value, and the state of the network is the set of all its neurons' states.

Edited February 23, 2023 by Lorentz Jr