Content uploaded by Jerry Waese
Author content
All content in this area was uploaded by Jerry Waese on Nov 28, 2022
Content may be subject to copyright.
Content uploaded by Jerry Waese
Author content
All content in this area was uploaded by Jerry Waese on Nov 27, 2022
Content may be subject to copyright.
Content uploaded by Jerry Waese
Author content
All content in this area was uploaded by Jerry Waese on Nov 27, 2022
Content may be subject to copyright.
EXPERIMENTAL PERCEPTION PROTOTYPE
IMPLEMENTING ASSOCIATIVE MEMORY IN
JAVASCRIPT
Jerry Waese
Prototyping and Development (jerrywaese.ca), Toronto, Ontario, Canada
jerry@jerrywaese.ca
ABSTRACT
This paper explains both the principle of naturally occurring associative memory forming from
synchronous cortical neural activation (eg. gestalt sensation), as well as the principle of a perceptive
reflex that reactivates associative memory from synchronous neural activation cues.
The approach taken here is to graphically simulate an array of 90,000 Cortical Neurons in a 300x300
pixel grid representing a small visual field, and an array of 18,000 highly branched Pyramidal Neurons
inter-dispersed among the Cortical Neurons. Each Pyramidal Neuron in the simulation has an array of
2000 Axon branches which are connected with a random assortment of members of the array of Cortical
Neurons during the simulation start up sequence (loosely analogous to embryonic development). Each
Cortical Neuron therefore has about 400 Pyramidal Neuron axon branches connecting to it.
All of these numbers are very small fractions of total and relative populations of the actual number of
Human Cortical and Pyramidal Neurons in our brains, as well as the number of connections found on
each neuron which can be 50 times more densely interlinked and 800,000 times larger overall in real
human brains,
When a test image is processed, in the simulation, Activated Cortical Neurons induce their neighbouring
Pyramidal Neurons to fire, and when one of the 2000 active Pyramidal Neuron's axon branches connects
with an active Cortical Neuron, simulated ARC protein is deposited at the connection which, enables easy
reactivation later. The simulation lets the experimenter pick any part of a test image to be used as a small
sample to be transferred to the conditioned array of Cortical Neurons, to elicit a perceptive memory reflex
demonstrating that the full memory can be associatively recalled from a limited matching sample.
A working instance of this simulation is provided at https://jerrywaese.github.io/perception/ which is a
single file of source code and eight test image files available as an archive at
https://github.com/jerrywaese/perception . This simple design using HTML 5 Canvas for the graphics
which is standard in all current web browsers and requires no 3rd party libraries at all was selected to
facilitate easy adoption of the core ideas. To create a separate stand alone instance with other images and
code modifications an experimenter can use any editor and web server (eg. xampp is an excellent cost free
cross platform web server environment for stand alone development and experimentation, any web server
can be used including Node.js)
However, the source code can be live inspected and debugged directly from
https://jerrywaese.github.io/perception/ using the browser console and the significant variables are
already exposed in the simple web interface as well.
KEYWORDS
ASSOCIATIVE MEMORY, PERCEPTION, NEURONS, SIMULATION, NEURAL NETWORK.
1. INTRODUCTION
Over the last 4 decades I have been reviewing information about how memory can be “stored”
and “recalled” in the specific sorts of tissues, structures, and cell types that are found in the
brain, Recently the missing links were incidentally revealed in the report of experiments using a
slice of living brain tissue and microscopic electrodes and chemical analysis.
I direct readers to The decade of the dendritic NMDA spike, which was written 12 years ago.
Among other advances, this article clarifies the matter of dendritic back-propagation in activated
Cortical Neurons, It also clearly illustrates that synchronous firing produces localized ARC
protein deposits, and it shows that a plurality of ARC protein deposit sites, when synchronously
activated, will cause a resting Cortical Neuron to fire and to back-propagate, which is, on the
cellular scale, the very essence of perception as a reflex.
The experiments described in the article cited above were done on a small brain slice that did not
include what I regard to be the fundamentally significant thalamic loop portion of natural neural
activation circuitry, but it still managed to articulate the kinds of events that result from a variety
of important in vivo conditions related to memory formation, and perception.
We all know what associative memory is like in its performance in our lives; such as, for
example, when you think of arriving at your home, a mental image of that space may appear,
perhaps it is inside the house or apartment, or perhaps it is at the front door to the building, but
some associated image or feeling comes up, perhaps many; perhaps you were supposed to come
home with flowers? Or maybe you think of a song. Association takes what is active in the mind
and evokes related perceptions that previously had been synchronously combined into associative
memory.
Associative memory formation combines things that are naturally experienced together as a
collection of specific synchronous neural activation's. [Association is equally capable of
combining sequences of frames of things that occur together since the fading edge of one frame
overlaps the cortico-thalamic feedback loops sustaining the next frame.]
Our sensory experiences are electro-physically represented by
1. conducting sensory signals up to specific neurons in the cerebral cortex through the
thalamus,
2. activating those Cortical Neurons such that they fire back to the same thalamic relay
neurons producing a feedback loop, while they also back-propagate electrical fields
through their dendrites in the upper layers of the cortex.
The feedback loop keeps activated Cortical Neurons and their thalamic partner neurons busy for
at least 3 loop cycles in normal states of mind. This can be detected as Alpha and Delta rhythms
at 8-12 hz. Sensations thus involve Bottom-Up looping signals from the Thalamus to the
Cortex.
When activated Cortical neurons back-propagate they trigger nearby Pyramidal Neurons that
each have up to 100,000 axon branches. When any branches of an activated Pyramidal Neuron is
in contact with an active Cortical Neuron, an ARC protein deposit forms that will facilitate later
Cortical Neural reactivation. This is how memory is formed throughout the cerebral cortex.
Figure 1. Neural Circuits which Provide Sensation,
Create Associative Memory,
Activate Perceptive Reflexes,
and Suppress Perception
If any part of the cortex is damaged, the memory remains intact but has been reported to be a bit
less clear, more or less the way a hologram behaves when cut – it retains the whole 3-d image but
that image is less clear than when all of it is intact to generate the 3-d image.
When any similar pattern of neural activation takes place, if more than seven (7) ARC protein
deposits are activated by tiny branches of some activated Pyramidal Neurons on any resting
Cortical Neuron, that Cortical Neuron will become reactivated: it will back-propagate up through
it's dendrites (exciting it's neighbouring Pyramidal Neurons) and reflexively fire down to the
thalamus initiating a Top-Down looping signal with the Thalamus, thus producing a Perception
event.
The Top-Down Perceptive activation condition in a Thalamic Neuron can be discriminated at
the cellular level from a Bottom-Up sensory activation, in that with Top-Down activation, there
is no residue of having been stimulated on the sensory part of the Thalamic Neuron; only the
feedback side in a Top-Down circuit is active, while for a Bottom-Up sensory signal both sides
will be active – at least the sensory signal receiving or input side of the neuron will be in
recovery from having been activated during the previous half of a second. The separation of
Top-Down and Bottom-Up feedback is important in the context of Attention and Awareness
discussed later.
Figure 2. Typical Sequence Showing Sensation T1-T3
Memory Formation T4
Perception T5
and Suppression of Perception T6 enabling full Attention to Sensation.
Associative memory formation therefore is an ongoing process throughout the cortical brain
tissue that creates links in a matter of 1/5 h of a second (e.g. 2 cycles at 10hz) and can elicit
perceptive reflexes in the same time frame.
To explore this phenomenon further, I created the JavaScript Demo program that learns and
recognizes 8 separate images in any order, and which can recognize any of the images when just
a part of the image is shown to it.
To enable better results (in which small sample cues produce the whole image of the originally
learned picture and not parts of other learned pictures) The functional equivalent of sensory
attention (temporarily suppressing perception) was implemented to better attend the sensory
content when the images are first accessed, so that as they are learned, they do not inadvertently
also include contents from other images that might be perceived as related. This is considered as
Hypothalamic suppression of Top-Down looping and it is only done in the simulation when a
new image is first chosen. However it is done many times per day in normal human and animal
behaviour as we attend to arising sensory conditions. This attentive state is like Zen archery, or
like being a child and taking things in fresh, only responding with a perceptive reflex at the end
of a breath or some other review type moment.
In ordinary life, we can usually suppress the bulk of perceptive mental content by shifting our
awareness to our breath, or by sniffing scents which connects the instinctual olfaction reflex we
have inherited that connects to primal behaviours like scenting for food, danger, or sex
pheromones. Presumably the olfactory nerves activate the hypothalamic Top-Down suppression
circuits which promotes attention to new mental content that may be related to a fresh scent.
Another source of non-coding-error that looked like image memory crosstalk or confusion
occurred when I neglected to allow for the short term memory effect.
Short term memory in this scenario is the weighting handicap for perceptive reflexes such that
recent memory patterns are more accessible than fully resting memory (AKA long term
memory). In particular while it takes a minimum of 7 synchronous ARC protein spine activations
to reactivate a resting Cortical Neuron, it only takes 4 synchronous ARC protein spine activations
to reactivate a short term memory Cortical Neuron, i.e. one that had, within the last 5 minutes,
been actively part of active mental contents i.e. some sensation or perception.
The simulation therefore accommodates a tolerable short term memory potentiation that could
appear like a crosstalk effect for the purposes of experiment but it is restricted to 5000
milliseconds. So if the operator changes images faster than 5 seconds some image residue may be
perceived in the tests from the previous active image. The interface allows the operator to change
this duration.
I direct reviewers to point their browsers at https://jerrywaese.github.io/perception/ to see the
JavaScript simulation operate in real time. For Convenience Screen cap Pictures from a typical
session of the simulation are shown below.
Figure 3.
(1) Initial back propagated field effects shown on right side simulating the Activated Cerebral Cortex
after pre-processing (image on left side) an input drawing (the Bottom-Up signals) .
Figure 4.
(2) Clicking on the left side a partial image rectangular sample is fed into the simulated cortex t
activating some of the same neurons in a bottom-up signal feed.
Figure 5.
(3) on right side note reactivation of memory engram of previously learned image after dendritic back
propagation field effects turn on linking Pyramidal Neurons - The blue represents top-down perception of
the associated image while the magenta represents the activation subset or Bottom-Up sensory seed signals.
Figure 6.
(4) The partial sample image (bottom-up input signals in brighter rectangular area) on the right side has a
more dense activation profile than the rest of the reactivated mental contents (top-down perception). Both
however are equally part of active mental contents, and persist as long as feedback with the thalamus is
maintained by these cortical neuron locations (usually 3 feedback cycles).
Figure 7. partial screen capture after Perception:
On right side, clicking on a few reactivation locations we can see the web of Pyramidal axon branches that
were synchronously active and which collaboratively reactivate the resting neurons thus participating in a
perception event filling in the missing parts of the image.
2. PERCEPTION SIMULATION OPERATION GUIDE
After the first image loads (in the far left rectangle) it will be decolourized and rendered as an
outlined figure realized via a user programmable grid called the Processing Matrix mask that
can be user edited and is shown at the bottom of the web page as a 3x3 grid of 1 or 0 or -1.
Figure 8. Full Web Page shown with simulation area at top and
editable parameters below, including the pre-processing mask grid near the bottom
and the Grey Threshold Prep. above etc.
This mask and the code that simulates its effects falls into the category of subliminal post-sensory
image pre-processing, which, in our brain, takes place at high speed and differently in various
visual centres of the brain as well as in auditory, proprioceptive and linguistic centres etc.. The
results of high speed processing in our 6 layer cortex (which I neglected to give attention in this
work) produce separate neural patterns for colour fields, progressions, edges, figure ground, etc.
These processes integrate with the alpha rhythm thalamic loops by suppressing and substituting
which specific cortical neurons can feed back to the thalamus and thus be included in memory
formation and perception.
In this simulation, the mask is effectively used when the image is first loaded, after the turquoise
line is displayed under an image, the session will not use the mask effectively for learning that
image, but it will affect the sample which can produce interesting results related to partial image
fragments that are slightly dissimilar to the original.
I have not experimented very much with this mask part of the simulation as the results obtained
in the default configuration that I implemented are satisfactory to me.
Similarly the Grey Threshold Prep of 50/255 seems to be a reliable post-sensory process setting
for the range of colour and greyscale images being post-sensory processed to useable marks and
shapes in the simulation. The reader will probably not want to change that number, for this set of
images, but could do so in their own instances with their own images etc.
Post-sensory processing is fascinating but it operates outside of my direct interest area of
associative memory formation and perception, even though it enables efficient memory formation
and perception. Because the events of sensory completion or post-sensory processing take place
in the brain it is an area of active brain research, and because it is localized to specific brain
processing areas it is relatively low hanging fruit for researchers and theorists.
It is not to be neglected, as can be seen, a modicum of post-sensory processing makes a wider
range of testing memory possible. It allows the simulation to access images of different kinds
(colour, black and white, greyscale etc.) Not to include some post-sensory processing would have
made the simulation too narrow and of less value to the reader.
The Test Size Pixel Width and Height interface widget provides the reader with a way to use
smaller or larger square sample areas in testing. The default is 150X150 pixels but I have
received encouraging results with test fragment sizes as low as 50 for some images and
reasonable results with 90 pixel squares as well.
2.1. A reliable approach to testing the simulation
The simulation will behave reliably if you wait 5 seconds (or whatever short term memory
potentiation is set to) after the last test or the last new image change. Within the test of any
particular image, you only need wait for the screen to settle before attempting the next test.
To test merely click on the left image – the right side will clear to black and then simulate
introduction of the post-sensory processed image area to the cortex, by activating those neurons
which correspond to the processed image rectangle. the field effects of back propagation, will
appear and the activation of pyramidal neurons. Will take place leading to a sequence of Cortical
neuron reactivation. Dots will appear signalling neurons that are outside of the test area that are
part of the associated group of neurons of which the sample set is part.
You can change the position of where you click on the left side to see how changes in the results
are affected. You can change the size of the sample area and repeat your tests observing how
much of the original image is restored from memory.
To see the Pyramidal Neuron axon branches that are active from any active area of the simulated
cortex you can click on the area of interest in the Associative Cortex Area at the right, and all of
the branches that have connections with ARC protein deposits in this image memory originating
from a clicked pyramidal neuron will be indicated,
To see how active Pyramidal Neurons are linked to any Cortical Neuron, right click on the
neuron and you will see all the branches of all the pyramidal neurons that are involved in the
linkage of that neuron in this image's memory engram.
This can obscure whatever you were studying. But you can restore the screen using the
[RETEST] button which will blank the Cortex Area and re-transfer just the image sample
rectangle and re-display the exact same results without the overlay of the rendered axon branches.
2.2. Changing the test image
Please wait 5 seconds at least before switching to another image by selecting a thumbnail of the
image bar with 8 images. Then check what results you get from different rectangular samples of
that image by clicking in an area on the processed graphic on the left side. When an image has
little or no active neurons in a test area you will not see any perception from that area.
Figure 9. Image Bar with 8 thumbnails
2.3. Images that have already been learned will have a green bar below the image
selection bar.
When an image has been learned you can switch to it for testing purposes and the sample will
appear on the left but nothing will happen on the Associative Cortex area until you pick a sample
area to test with by clicking the mouse on the processed left side.
2.4. Testing on mobile browsers
This simulation works on mobile browsers as well as desktop but you may want to pinch zoom
the screen. An extra button has been provided to switch the click or tap from right click to left
click as mobile has no right click functionality in the browser. The button is located below the
Associative Cortex area and says either [Click above for Pyramidal branches from] or [Click
above for Pyramidal branches to] and that is what will happen when you tap on active points in
the Associative cortex area.
On mobile, while zoomed in, you will miss the kinds of text messages that appear on the right
side of the window on full screen desktop browsers.
CONCLUSIONS
Many direct inferences can be made about Associative Memory and the brain based upon the way
this simulation easily accomplishes the tasks it aims to fulfil, namely:
•The simulation is a reasonable proof of the theory that synchronous neural activation can
and does produce memory engrams which can support image recollection/familiarity
from partial source image cues in a rapid perceptive reflex. This basic familiarity reflex is
probably the basis of all associative memory including body movement, speech
recognition and articulation and thought.
•Wide linkage across the entire cortical field is simulated, such that we can project that
associative reflexes can include not just image completion, but also motor activation,
body movements, speech, or just thoughts and feelings.
•Attention can be achieved by selective suppression of Top-Down perceptions which is
implemented in JavaScript to simulate what is in the figures 1 & 2
•Daydreaming or endless perception sequences can be achieved by Bottom-Up
Suppression in the brain stem which was not implemented or drawn.
•Temporal distortion and memory overlay can be achieved by extending the cortical-
thalamic loop duration – not simulated, but theorized in cases of continuous sensation
and perception. (i.e. in an ongoing stream of consciousness)
•Concepts of Executive controller and reporting consciousness seem obsolete since
associative reflexes and allostatic and homeostatic conditions prevail.
•A clear distinction between associative memory related mental activity at alpha and theta
frequencies should be made with higher speed localized post-sensory pre-processing to
better develop content for associative memory and perceptions.
•The Limbic system which encircles the thalamus is most likely involved in modulating
thalamic modes (loop duration) and the Hippocampus is not the critical location for
memory formation as previously suspected but is certainly involved in Fear responses.
•The traditional views of short term memory, working memory, and memory
consolidation are not supportable but can be efficiently replaced with rapid creation of
long term memory, and the propensity of recently active cortical neurons to be more
easily reactivated in perceptive reflexes than cortical neurons at rest producing the same
effects that are measured as short term memory.
•The traditional view that repetition makes links stronger has to be modified since the
ARC protein links are either there or not there, however, by repetition with slight
alterations in each repetition the memory in question becomes more accessible, or easier
to perceive in a wider range of circumstances. This is achieved by new alternate ARC
protein spines. No memories are stronger than others, but some memories are more
accessible by virtue of being associated through more links and different kinds of links.
•Memories that are out of context are harder to recall. Memories related to the last 5
minutes are the easiest to recall.
•All behaviours are based upon perceptive reflexes.
•The perceptive reflex can be interfered with by shifting attention – e.g. smell or
deliberate slow inhalation.
•Combined with allostatic, homeostatic and instinctive drives to discover through play, to
follow interesting scents while seeking food and sex, and to be defensive when
threatened, this model of associative memory formation + sensation + perceptive reflexes
solves Chalmers Hard problem of explaining consciousness.
ACKNOWLEDGEMENTS
The author would like to thank participants at Research Gate such as Larry Carlson who
introduced many of the problems that Neuroscientists are looking at today (with links and
videos), and who seemed to follow everything I wrote, but may just be that type of
conversationalist; Wieslaw Galus who always disagreed brilliantly; and H.G. Calloway who has a
solid grasp of Functionalism, as well as many others others who have engaged with me in several
discussion forums and have helped me hone these ideas into a functional perspective.
I would also like to thank my brothers, Victor Waese, and Stan Waese, who have read my work
and keep on telling me to make this theory of mind simpler to understand (so that a 6 year old
will be able to make use of it – not there yet), and I have to thank my nephew Dr. Adam Waese
for suggesting that I write code for a simulation to explain the process of memory formation and
perception and who looked at the earliest versions of the simulation and declared it was more
than just very interesting since it was not simply an animation as originally intended but a
working example of the key steps in memory formation and perception.
I would especially also like to thank my wife, Kia, who has supported this work at every turn,
and who has tolerated me speaking technical jargonese every day all day long for decades, and
my daughters Myera and Alice and their children Lev and Zazie, who have been an ongoing
inspiration, and real world examples of the principles of independent exploratory play leading up
to finding one's way in the world between the familiar and the unknown.
REFERENCES
[1] Srdjan D AnticWen & Liang ZhouWen & Liang Zhou & Anna Moore & Katerina D. Oikonomou,
(2010) “The decade of the dendritic NMDA spike”, Journal of Neuroscience Research 88(14):2991-
3001 DOI: 10.1002/jnr.22444
AUTHOR
Jerry Waese Bsc. York University 1974
Declined an offer to engage in experimentation
pursuant to the theory presented in this
paper originally submitted in raw form
as a critique of ongoing research in
Neurophysiology Brain and
Behaviour - a 4th year Science Course
offered at the time. The Critique Paper
was well received and was
accompanied with a serious invitation
to join the Professors who were leaving
for MIT to work on it further.
Jerry, instead, followed a path through
Mechanical Contracting, House
construction and eventually by 1980
had self-educated as a computer
software developer. Specializing in
graphics, video, robotics,
communications protocols, and in the
last decade focussed on Accessibiity
(WCAG 2.0) techniques
implementation and triage for
Accessible Banking applications.
Jerry has had several US Patent applications
filed and registered in robotics and in
video, and pioneered conversationally
interactive multimedia as well as
Mobile Application development at
IBM.
More information regarding technical
qualifications and achievements is
available to interested parties if
required,
In the mean time, since the early 1990's Jerry
has produced a large oeuvre of fine art
works some of which can be perused at
HTTPS://flickr.com/waese