Discursive Competence in ChatGPT, Part 2: Memory for Texts 2: Memory for Texts, Version 3

Abstract

In a few cases ChatGPT responds to a prompt (e.g. "To be or not to be") by returning a specific text word-for-word. More often (e.g. "Johnstown flood, 1889") it returns with information, but the specific wording will vary from one occasion to the next. In some cases (e.g. "Miriam Yevick") it doesn't return anything, though the topic was (most likely) in the training corpus. When the prompt is the beginning of a line or a sentence in a famous text, ChatGPT always identifies the text. When the prompt is a phrase that is syntactically coherent, ChatGPT generally identifies the text, but may not properly locate the phrase within the text. When the prompt cuts across syntactic boundaries, ChatGPT almost never identifies the text. But when told it is from a "well-known speech" it is able to do so. ChatGPT's response to these prompts is similar to associative memory in humans, possibly on a holographic model. On the whole it is clear that the LLM underlying ChatGPT has not memorized any input data in the sense of rote memorization.

Full text

Discursive Competence in ChatGPT,
Part 2: Memory for Texts
A Working Paper, Version 3
October 12, 2023
William L. Benzon

1
Discursive Competence in ChatGPT,
Part 2: Memory for Texts, V3
William Benzon
October 12, 2023
Abstract: In a few cases ChatGPT responds to a prompt (e.g. “To be or not to
be”) by returning a specific text word-for-word. More often (e.g. “Johnstown flood,
1889”) it returns with information, but the specific wording will vary from one
occasion to the next. In some cases (e.g. “Miriam Yevick”) it doesn’t return
anything, though the topic was (most likely) in the training corpus. When the
prompt is the beginning of a line or a sentence in a famous text, ChatGPT always
identifies the text. When the prompt is a phrase that is syntactically coherent,
ChatGPT generally identifies the text, but may not properly locate the phrase
within the text. When the prompt cuts across syntactic boundaries, ChatGPT
almost never identifies the text. But when told it is from a “well-known speech” it
is able to do so. ChatGPT’s response to these prompts is similar to associative
memory in humans, possibly on a holographic model. On the whole it is clear that
the LLM underlying ChatGPT has not memorized any input data in the sense of
rote memorization.
Note: Version 3 contains an introduction that is significantly expanded from the previous
versions and supersedes them in a way that clarifies the research.
Contents
Introduction: What is memory? .......................................................................................... 2!
What must be the case that ChatGPT would have memorized “To be or not to be”? Three
kinds of conceptual objects for LLMs ............................................................................ 5!
To be or not: Snippets from a soliloquy ............................................................................ 17!
Entry points into the memory stream: Lincoln’s Gettysburg Address .............................. 27!
Notes on ChatGPT’s “memory” for strings and for events .............................................. 37!
Appendix A: That Reverse Thing: Chatster is puzzled by “be not or be to” as well. ...... 44!
Appendix B: Table of prompts for soliloquy and Gettysburg Address ............................. 47!
1301 Washington St., Apt. 311
Hoboken, NJ 07030
917.717.9841
bbenzon@mindspring.com
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 Unported License.

2
Introduction: What is memory?
In various discussions about large language models (LLMs), such as the one powering
ChatGPT, I have seen assertions that such as, “oh, it’s just memorized that.” What does
that mean, “to memorize?”
An analogy from music
I am a fairly talented and skilled musician. I can and have memorized a piece of music by
practicing it over and over. There are the notes on the page. I start playing them until I am
comfortable. Then I look away and see how far I can go. When I get lost, I look at the
music, continue playing the notes on the page, and finish the piece – something like that.
Then I start over from the beginning, again without the music. When I can play the whole
piece without having to consult the written music, I have it memorized. At least for the
moment.
But I don’t do that very often. More likely, I’ll hear a tune I like two, three, or five times
and then I pick up my trumpet and playing, sometimes perfectly, sometimes with a glitch
or two. I didn’t memorize it, and yet I’m playing it. From memory? No, by ear?
I’m speaking metaphorically of course. What does it mean to play by ear? I don’t really
know, but I imagine it goes something like this: Music has an inner logic, a grammar, a set of
rules through which it is structured. When I hear a tune I’m listening to it in term of that
inner logic, as, for that matter, anyone is – at least if they’re familiar with the musical idiom.
It’s that logic that I’m registering as I listen to the tune. Once I’ve heard the tune a couple
of times, I’ve “absorbed” that logic, without even thinking about it or working on it. It just
happens as a side-effect of (ordinary) listening. When the absorption is complete, I am able
to play the tune “by ear.”
Those are two very different processes, absorbing a tune through listening vs. repeating it
over and over until you have it “memorized.” “Mindless” repetition from a written score is
possible because one can, and does, interpret the notes on the page, not as sounds, but as
cues indicating the actions you must make with your body to produce a certain sound. You
make the appropriate actions and, if the result sounds like music to your ear, you conclude
that your actions were correct. Otherwise, more practice.
The same is possible with written language. Opera singers can and often do sing a part in
a foreign language by learning the words phonetically. They will of course be familiar with
a translation of the words into their native language, but that translation plays no role in
the process by which they memorize the sequence of sounds they must sing.
And then we have those psychological investigations of memory where subjects are asked
to memorize lists of miscellaneous words. There is no semantic or syntactic logic to the
succession of words in the list, so there is no underlying logic subjects can call on in the
memorizing process. They simply have to learn a disconnected sequence of linguistic
objects, words. That’s “pure” rote memorization.

3
This difference, between rote memorization and logic-facilitated learning, depends on the
difference between signifier and signified, to use terms from Saussure. The notes or words on
the page are signifiers. The signifiers, whether verbal or musical, can be physically realized
without reference to any underlying signifieds (verbal meanings of musical sounds), thus
giving rise to the possibility of rote memorization. In a list of words without any syntactic
or semantic order the underlying signifiers become irrelevant to the task of memorization
so the task of memorization becomes a matter of rote.
Which of these processes, rote memorization or logic-facilitated learning, is an LLM doing
when it is chewing its way through a corpus of texts? When I prompt ChatGPT with “To
be or not to be,” it responds with Hamlet’s complete soliloquy, word-for-word. When it
does that is the process more like what I do when playing music by ear or like I do when
memorizing music? Or is it something else?
Tokens and vectors
As I have now come to realize through a series of recent posts,1 it is possible to construct
something like a signifier/signified relationship in connection with LLMs. Words aren’t
represented in language models as alphanumeric ASCII character strings; they’re
represented by tokens. In the case of the ChatGPT words are tokenized by byte-pair
encoding (BPE).2 For our purposes the details are irrelevant. All that matters is that we’re
dealing with tokens. Those tokens are linked to a word-embedding3 vectors that encode
relationships between words in a high-dimensional space (roughly 12,000 for ChatGPT). If
we wish, we can think of the token as a signifier and the vector as a signified.
However, that would be a bit misleading because the tokens themselves play no role in
internal LLM processing. Their only function is to mediate between alpha-numeric
character strings and the vectors at input to and output from the model. All of the
processing within the model is based on those vectors and, of course, the model weights
themselves. So, LLMs are, in effect, architecturally incapable of “mere” rote memorization.
All processing is based in some way on the logic inherent in the relationships between those
vectors.
I suppose that should have been obvious to me from the outset. But it wasn’t. And I don’t
know how many times I’ve read about whether or not language models are “memorizing”
data, though just what was meant by “memorize” was never specified. That’s what I had in
mind when I undertook the investigations I report in this working paper: What does it mean
for an LLM to memorize something?
What’s going on in LLMs?
In the first piece – What must be the case that ChatGPT would have memorized “To be
or not to be”? – I start out with Hamlet’s famous soliloquy, initially prompting ChatGPT
1 I have tagged these posts with “entangle,” and you may access them with this link, https://new-
savanna.blogspot.com/search/label/entangle.
2 Wikipedia, Byte pair encoding, https://en.wikipedia.org/wiki/Byte_pair_encoding.
3 Wikipedia, Word embedding, https://en.wikipedia.org/wiki/Word_embedding.

4
with first line, but then prompting it with other fragments from the soliloquy. Then I
prompt it with the phrase, “Johnstown flood, 1889,” and it responds with information
about that flood, by not a specific text word-for-word. Many prompts are like that, many
more than elicit a specific text word-for-word. What leads to that difference? I conclude
with two topics I have reason to believe were included in the training corpus, but which
ChatGPT seems to know nothing about. Why not?
In the next section (To be or not: Snippets from a soliloquy) I create various prompts for
the soliloquy. I do the same in the third section (Entry points into the memory stream:
Lincoln’s Gettysburg Address), but more systematically. Finally, I do a bit of speculating
about what’s going on: Notes on ChatGPT’s “memory” for strings and for events. I begin
by quoting a passage from F. C, Bartlett’s classic 1932 study, Remembering, and conclude
that ChatGPT may have an associative memory along the lines suggested by holography,
which engendered a great deal of speculation in the 1970s and, in this millennum,
specifically for word meaning and order.
At the very end I include Appendix B, which lists each prompt I gave to ChatGPT from
the two speeches along with its response. Appendix A contains a last-minute investigation
in which I prompt ChatGPT with “be to not or be to” and “ago years seven and score
four” to see how it responds to those reverse-order phrases.

5
What must be the case that ChatGPT would have
memorized “To be or not to be”? Three kinds of
conceptual objects for LLMs
I’ve been thinking about ChatGPT again, about what goes on under the hood. What would
happen I prompted it with a famous phrase from Shakespeare? Would it return the whole
passage?
While I am interested in that question, that’s not only thing I’m interested in. There are
topics which show up in responses to prompts but not in the form of fixed texts. Finally,
there are likely topics that exist in the training corpus, but don’t show up in response to
prompts. THAT’s what this post is about, the differences between those three kinds of
objects.
Fixed strings – Hamlet’s soliloquy
It’s easy enough to try. I did so on July 31, running against the July 20 version of ChatGPT.
To be or not to be
"To be or not to be" is one of the most famous lines from
William Shakespeare's play "Hamlet." It appears in Act
III, Scene 1, where Prince Hamlet is contemplating the
nature of existence and the idea of suicide.
The full quote is as follows:
"To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing, end them? To die: to sleep;
No more; and by a sleep to say we end
The heartache and the thousand natural shocks
That flesh is heir to; 'tis a consummation
Devoutly to be wished. To die, to sleep;
To sleep, perchance to dream—ay, there's the rub:
For in that sleep of death, what dreams may come,
When we have shuffled off this mortal coil,
Must give us pause—there's the respect
That makes calamity of so long life:
For who would bear the whips and scorns of time,
Th'oppressor's wrong, the proud man's contumely,
The pangs of despised love, the law’s delay,
The insolence of office, and the spurns
That patient merit of th'unworthy takes,

6
When he himself might his quietus make
With a bare bodkin? Who would fardels bear,
To grunt and sweat under a weary life,
But that the dread of something after death,
The undiscovered country from whose bourn
No traveler returns, puzzles the will,
And makes us rather bear those ills we have,
Than fly to others that we know not of?
Thus conscience does make cowards of us all,
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pith and moment,
With this regard their currents turn awry,
And lose the name of action. Soft you now,
The fair Ophelia! Nymph, in thy orisons
Be all my sins remembered."
In this soliloquy, Hamlet is contemplating the nature of
life, death, and the fear of the unknown. He ponders
whether it is nobler to endure the hardships and
challenges of life or to take his own life and end the
suffering. Throughout the play, Hamlet grapples with
existential questions and struggles with his emotions,
making "To be or not to be" one of the most memorable and
profound passages in literature.
Except for the opening and closing material, that’s pretty much what I expected to happen.
But why? Given that the underlying LLM has been trained to predict the next word, what
MUST have been the case in order to ChatGPT to return the whole soliloquy when given
the opening six words?
It must have encountered that soliloquy many times during its training. Shakespeare’s
Hamlet is a well-known play and it must have existed at many places in the training corpus.
Moreover the soliloquy itself is famous probably occurred many times independently of the
whole play.
What would happen if I gave it a phrase from somewhere in the soliloquy? Let’s find out.
From 12 minutes ago, running against the August 4 version. This is of course a new session;
there is no prior context it is working from:
The insolence of office
The phrase "the insolence of office" is a line from
William Shakespeare's play "Hamlet." It appears in Act 3,
Scene 1, during a soliloquy by the character Hamlet. The
full line is:
"To be or not to be, that is the question:

7
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them. To die, to sleep—
No more—and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to—'tis a consummation
Devoutly to be wish'd. To die—to sleep,
To sleep, perchance to dream—ay, there's the rub,
For in that sleep of death what dreams may come,
When we have shuffled off this mortal coil,
Must give us pause—there's the respect
That makes calamity of so long life:
For who would bear the whips and scorns of time,
The oppressor's wrong, the proud man's contumely,
The pangs of despised love, the law's delay,
The insolence of office, and the spurns
That patient merit of the unworthy takes,
When he himself might his quietus make
With a bare bodkin? Who would fardels bear,
To grunt and sweat under a weary life,
But that the dread of something after death—
The undiscovered country, from whose bourn
No traveller returns—puzzles the will,
And makes us rather bear those ills we have
Than fly to others that we know not of?
Thus conscience does make cowards of us all,
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pitch and moment
With this regard their currents turn awry
And lose the name of action."
In this famous soliloquy, Hamlet is contemplating the
nature of existence, the pain and suffering that people
endure in life, and the fear of the unknown in death. The
specific line you mentioned, "The insolence of office,"
refers to the abuses of power and authority that people in
positions of power may engage in, which can be a source of
suffering and frustration for those subjected to it. It's
just one part of Hamlet's larger meditation on the human
condition and the difficult choices we face in life.
I wasn’t sure that would happen, but I’m not very surprised that it did. Note, though, that
whereas it gave 35 lines the previous time, it only gave 33 lines this time, omitting: “Soft
you now, / The fair Ophelia! Nymph, in thy orisons/Be all my sins remembered.” Not
surprisingly, the final gloss was a bit different as it decided to explicate the line I’d given it
as a prompt.

8
I decided to try again. I opened a new session and gave it the well-known phrase that opens
the fourth line:
The slings and arrows
The phrase "the slings and arrows" is a poetic expression
that originates from one of William Shakespeare's most
famous soliloquies in his play "Hamlet." In Act 3, Scene
1, Hamlet delivers a soliloquy that begins with the famous
lines:
"To be or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And, by opposing, end them."
In this soliloquy, Hamlet is contemplating the nature of
life, death, and the suffering that people endure. "The
slings and arrows of outrageous fortune" refers to the
unpredictable and often harsh trials and tribulations that
one may face in life. These challenges can be seen as the
metaphorical "slings and arrows" that are launched at a
person by the whims of fate or destiny.
Over time, the phrase "the slings and arrows" has come to
be used more broadly to describe the difficulties,
hardships, and adversities that people encounter in their
lives. It signifies the challenges and setbacks that
individuals must endure as they navigate the complexities
of life.
Again, not surprising, though I note that it only gave four more lines rather than continuing
on through the whole soliloquy.
Can I stump it? Can I pick a phrase from somewhere within the soliloquy that it won’t
recognize? I opened a new session from the 22nd line. Without thinking more than a
moment about it, I was careful not to start at the beginning of the line. Why? Because when
people memorize lines from Shakespeare, we memorize whole lines, though we may
perchance also be able to recognize a phrase from within a line. Here we go:
and sweat under a
It seems like your message is incomplete. Could you please
provide more context or clarify your request? I'd be happy
to assist you if you can provide more information or ask a
specific question.

9
Bingo! I got it. That makes sense. It tells us something about what’s going on, but don’t ask
me what. I decided to help the Chatster out:
It's a phrase from Hamlet's famous soliloquy.
Ah, I see! The phrase you're referring to is from Hamlet's
soliloquy in Act 3, Scene 1 of William Shakespeare's play
"Hamlet." The full line is:
"To be or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them."
The line you mentioned is part of this famous soliloquy in
which Hamlet reflects on the nature of existence and the
choice between life and death. If you have any specific
questions or if there's something specific you'd like to
know about this soliloquy or any other aspect of "Hamlet,"
feel free to ask!
Notice, however, that when it goes to give me the full line containing the prompt, it fails to
do so. We’ll see this again. There is a difference between identifyng the text where a phrase is
taken from and locating the specific position in that text where you’ll find the phrase.
I think it would be worthwhile to try out some more phrases from that soliloquy to see how
ChatGPT behaves, but it should probably been done systematically. What’s the system?
But that’s enough for now I want to look at something else.
Ensembles – the Johnstown flood
I decided to ask it about the Johnstown flood of 1889. Why? The 1889 flood (there were
later floods in 1937 and, I believe, 1977) was of middling importance in American history
and made the news around the country. So, I opened a new session (we’re back on July 31,
running against July 20):
Johnstown flood, 1889
The Johnstown Flood occurred on May 31, 1889, in
Johnstown, Pennsylvania, USA. It was one of the deadliest
natural disasters in American history and resulted in the
loss of thousands of lives and significant destruction of
property.
The disaster was caused by the catastrophic failure of the
South Fork Dam, which was located about 14 miles upstream
of Johnstown. The dam had been originally built in the

10
1850s as part of a canal system but was later modified and
converted into a recreational lake for wealthy
industrialists, including members of the South Fork
Fishing and Hunting Club.
On May 31, heavy rainfall caused the dam to weaken, and
ultimately, it gave way, releasing an immense volume of
water from the lake. The resulting floodwaters rapidly
rushed down the Little Conemaugh River valley towards
Johnstown, where three major rivers, the Little Conemaugh,
Stonycreek River, and Conemaugh River, converged. As the
floodwaters converged, they formed a powerful and deadly
wall of water and debris that slammed into Johnstown.
The city of Johnstown, situated in a narrow valley, was
not prepared for such a massive flood. The sudden
onslaught of water and debris devastated the area,
sweeping away homes, buildings, bridges, and anything else
in its path. The death toll from the flood reached
approximately 2,209 people, making it one of the deadliest
disasters in U.S. history up to that time.
The tragedy led to a nationwide outpouring of sympathy and
support for the victims and survivors. Clara Barton and
the American Red Cross played a significant role in relief
efforts, providing aid to those affected by the flood.
In the aftermath of the disaster, there were legal battles
and debates over who was responsible for the dam's failure
and the resulting loss of life and property. Ultimately,
no individuals from the South Fork Fishing and Hunting
Club were held legally accountable for the catastrophe.
The Johnstown Flood remains a significant event in
American history and serves as a reminder of the potential
consequences of inadequate infrastructure and
environmental management. The disaster also prompted
greater awareness and improvements in flood control and
disaster response measures in the United States.
I fully expected to get something but, for what it’s worth, I didn’t expect much.
The response looks OK to me. I grew up in Johnstown and so heard about the flood during
my childhood. I’ve also read about it here and there and seen a documentary on it.
I don’t think ChatGPT would have known anything about the flood if it hadn’t been
mentioned in various places in the training corpus. Now, for all I know, the response it gave
me is word-for-word from some one source. But I don’t have any reason to believe that is

11
the case. It’s difficult to imagine that there would be any single text about that flood that
exists as many times on the web as Hamlet’s famous soliloquy. Let’s check another time
and see what happens:
Johnstown flood, 1889
The Johnstown Flood of 1889 was a devastating natural
disaster that occurred on May 31, 1889, in Johnstown,
Pennsylvania, USA. It remains one of the deadliest floods
in U.S. history. The disaster was the result of the
catastrophic failure of the South Fork Dam, a poorly
maintained earthen dam situated on the Little Conemaugh
River, about 14 miles upstream from Johnstown.
Here is a sequence of events leading up to and during the
Johnstown Flood:
1. South Fork Dam: The South Fork Dam had been built in
the 1850s as part of a reservoir for the Pennsylvania Main
Line Canal system but was later converted into a private
fishing and recreational lake for wealthy industrialists,
including members of the South Fork Fishing and Hunting
Club.
2. Heavy Rainfall: In the days leading up to the disaster,
the region experienced heavy rainfall. The dam's spillways
became clogged with debris, and the water level in the
reservoir rose rapidly.
3. Dam Failure: On May 31, 1889, after hours of heavy
rainfall, the dam failed, releasing an enormous volume of
water downstream. An estimated 20 million tons of water
surged down the valley.
4. Destruction of Johnstown: The wall of water swept
through the valley, hitting the town of South Fork first
before reaching Johnstown. The floodwaters picked up
debris, houses, and other structures along the way,
creating a deadly wall of destruction.
5. Devastation: When the flood reached Johnstown, it
caused widespread destruction. Entire neighborhoods were
wiped out, and the floodwaters claimed the lives of
thousands of people. Many were trapped in debris or
drowned as the water surged through the town.
6. Aftermath: In the aftermath of the disaster, relief
efforts were launched to aid the survivors and recover the

12
bodies of the victims. Clara Barton and the American Red
Cross played a significant role in providing assistance.
7. Investigation and Legal Actions: The owners of the
South Fork Fishing and Hunting Club faced public outrage
and legal consequences for their alleged negligence in
maintaining the dam. However, no one was successfully held
criminally liable.
The Johnstown Flood had a profound impact on American
society and led to increased awareness of dam safety and
disaster preparedness. It remains a tragic chapter in U.S.
history, with an estimated 2,209 people losing their lives
in the disaster. The story of the Johnstown Flood has been
documented in books, articles, and even a museum in
Johnstown, ensuring that the memory of this catastrophic
event is not forgotten.
That’s not a word-for-word identical to the first response. Barring further information, I
am assuming that these replies contain information that the underlying LLM has
synthesized from various sources.
For the sake of argument, let’s call this type of object an ensemble. I don’t particularly like
that term, but it will do for the moment. Historical events are not the only phenomena that
constitute ensembles, but that’s a different discussion.
Ghosts – Diz at the Left Bank plus Miriam Yevick
Finally, let’s prompt it with a historical event which could well have been in the training
corpus, but is unlikely to have been reported many times. Back when I was living in
Baltimore in the late 1960s and early 1970s, I attended jazz concerts given by the Left Bank
Jazz society. I wrote about one of those concerts both in my personal blog back4 in 2013,
which I republished with minor changes at 3 Quarks Daily5 in 2016.
Would the Chatster know about that event? I suspect not. Time for another session:
Dizzy Gillespie plays for the Left Bank Jazz Society in
Baltimore’s Famous Ballroom
I believe there might be a bit of confusion here. Dizzy
Gillespie was indeed a legendary jazz trumpeter and
composer, but he passed away in 1993. As of my last update
4 New Savanna, Nov. 13, 2013, https://new-savanna.blogspot.com/2013/11/ecstasy-at-left-
bank.html.
5 3 Quarks Daily, May 2, 2016, https://3quarksdaily.com/3quarksdaily/2016/05/ecstasy-at-
baltimores-left-bank-jazz-society.html.

13
in September 2021, he hasn’t been performing in Baltimore
or anywhere else for quite some time.
The Left Bank Jazz Society was an organization that
promoted jazz concerts and events in Baltimore, known for
hosting prominent jazz musicians in the 1960s and 1970s.
Baltimore’s Famous Ballroom, also known as the Royal
Theater, was a significant venue for jazz performances
during that era.
While Dizzy Gillespie may have played at various jazz
venues in Baltimore in the past, it’s essential to verify
the accuracy of any current claims or events, as my
information might be outdated. I suggest checking with
local jazz organizations or venues in Baltimore to find
the latest information about jazz performances and events
in the area.
I’m thinking of a performance back in the previous century
that Bill Benzon wrote about in his blog, New Savanna, and
in the online magazine, 3 Quarks Daily.
My apologies for any confusion. It seems like you are
referring to a specific performance that Bill Benzon wrote
about in his blog, New Savanna, and in the online
magazine, 3 Quarks Daily. As an AI language model, I don’t
have access to real-time data or browsing capabilities,
and my knowledge is limited to information available up to
September 2021. Therefore, I cannot provide details about
specific performances or events that may have occurred
after that date.
To find the information you’re looking for, I recommend
visiting Bill Benzon’s blog, New Savanna, and the online
magazine, 3 Quarks Daily, to see if the article you
mentioned is available there. Additionally, you can try
using a search engine to look for any specific writings by
Bill Benzon on the mentioned topic or event.
I’m not surprised.
Notice, by the way, that ChatGPT is a little confused, just a little. In its first response it
pointed out that Gillespie had died in 1993. In its second response it noted that it could not
“provide details about specific performances or events that may have occurred after that
date.” But Gillespie had died well before then, so that in itself is no justification for not
knowing. Of course something could have happened prior to September 2021, but not
have been reported to the web by that time. Chatster seemed to allow for that, but didn’t

14
quite get it. That’s a minor matter having no bearing on what I’ve been playing around
with.
It so happens that I’ve been thinking and writing about Miriam Lipshutz Yevick, an
obscure mathematician who published some work from in the 1970s that interests me. She’s
got several papers online, there is an obituary for her, and some other notices as well. Does
ChatGPT know about her?
Let’s check (9.17.23 – August 3 Version):
What do you know about Miriam Yevick, a mathematician?
I'm sorry, but I don't have any specific information about
a mathematician named Miriam Yevick in my database, and as
of my last knowledge update in September 2021, she does
not appear to be a widely recognized or publicly known
figure in the field of mathematics. It's possible that she
may have become more prominent in the field or made
significant contributions after my last knowledge update.
If you have specific questions or would like information
about a different mathematician or topic, please feel free
to ask, and I'll do my best to assist you.
You’re correct, she’s not well-known. She got her Ph.D.
from MIT in 1947. She corresponded with the physicist,
David Bohm, in the 1950s, and she published an interesting
article in the journal, Pattern Recognition, in 1975,
entitled Holographic or Fourier Logic. She also published
a book of letters she wrote to her granddaughter, A
Testament for Ariela.
Thank you for providing additional information about
Miriam Yevick. While she may not be widely known in the
broader mathematical community, it's clear that she had an
interesting and diverse set of interests and
contributions. Her correspondence with physicist David
Bohm and her work on "Holographic or Fourier Logic" in the
journal Pattern Recognition suggest an engagement with
interdisciplinary ideas.
Publishing a book of letters to her granddaughter, titled
"A Testament for Ariela," is a personal and unique
contribution that highlights her broader interests and
desire to connect with future generations.
If you have any specific questions or would like to know
more about any aspect of Miriam Yevick's work or life,

15
please feel free to ask, and I'll do my best to provide
information based on the details you've provided.
She fled the Nazis in 1940 along with her parents. As you
know, many Jews left Europe at that time.
Miriam Yevick's experience of fleeing Europe with her
parents in 1940 during World War II is a poignant reminder
of the widespread displacement and persecution that many
Jewish families and individuals faced during that time.
The Holocaust and the policies of Nazi Germany forced
millions of Jewish people to leave their homes and seek
refuge in other countries to escape persecution and
violence.
This historical context adds depth to her personal story
and underscores the resilience of those who survived and
managed to rebuild their lives in new places. If you have
any more specific questions about Miriam Yevick's life or
her contributions, please feel free to ask, and I'll
provide information to the best of my ability based on the
details you've provided.
Thank you, and thanks for not making stuff up.
You're very welcome! Providing accurate and reliable
information is a priority, and I'm here to assist with the
information I have up to my last knowledge update. If you
have any more questions or need information on any other
topic in the future, don't hesitate to reach out. I'm here
to help!
I’m guessing that there are thousands upon thousands of events and phenomena like that.
They’ve been mentioned somewhere in some document in the training corpus, but not
often enough to “register” in the underlying language model. Let’s call these ghosts.
Three types of objects
Thus we’ve got three classes of objects:
1. Fixed strings: These arise because specific texts that have occurred so many
times in the training corpus that they register word-for-word in the language model.
2. Ensembles: These result when specific events are mentioned often enough that
they register in the language model, but not as word-for-word texts. I’ve used a
historical event as an example. But all kinds of regularities, not just historical events,
would register in this, e.g. astronomical observations, descriptions of life forms,
philosophical arguments, etc.

16
3. Ghosts: These are events or phenomena of any kind that are in the training
corpus but so infrequently that they don’t register in the language model. These
texts contribute to the model, but only as they contribute to specifying relations
among words.
What are the parameters governing these types of objects?

17
To be or not: Snippets from a soliloquy
Let’s recall from the previous discussion: I gave ChatGPT a simple prompt, “To be or not
to be.” It responded with the whole soliloquy, as I expected. Then I prompted it with two
phrases from within the soliloquy, “The insolence of office”, “The slings and arrows.” In
each case it identified them as coming from that famous soliloquy. Each of those prompts
came from the beginning of a line. What if I prompted it with a string from within a line?
I chose this string, “and sweat under a,” which is from this line: “To grunt and sweat under
a weary life.” It failed to identify the string with the soliloquy.
Interesting, very interesting. That makes sense. I need to look into this a bit more.
And so I have.
Including this section, the previous one, and the next one, I have now prompted ChatGPT
with thirteen (29) snippets (I won’t call them phrases because, technically, many of them
are not phrases, just strings of words), eleven (11) from line beginnings, and the rest,
nineteen (18) from somewhere in the interior of a line.6 Seven (7) of those were consonant
with syntactic boundaries while the other eleven (11) were not (I call them unruly).
ChatGPT was able to identify the texts where all of the line-initial prompts came from. It
was unable to identify texts for three (3) of the consonant prompts, but correctly identified
texts for the others. In general it was unable to identify the texts for the unruly prompts,
but was able to do so in the cases where, after it had failed, I told it that the prompt was
from a famous speech. It responded in various ways to the snippets it was unable to identify,
in some cases offering fairly elaborate interpretive commentary. In the two cases where it
correctly located the snippet it also quoted enough of the soliloquy to establish context.
And then there’s the peculiar case of this prompt: “make cowards of us all.” It is from one
of the best-known lines in the play, one often quoted on its own: “Thus conscience does
make cowards of us all.” I expected the Chatster to identify it. But it did not. So I decided
to help it a bit.
I opened a new session and prompted it with: “does make cowards of us all.” The addition
of that one word, “does” was all the Chatster needed. It quoted most of the soliloquy in
response. I then, once again, prompted it with “make cowards of us all.” As before, it was
unable to identify it,
On the whole, I find this is satisfying. For what it’s worth, the fact that ChatGPT would be
able to identify snippets from the beginning of a line, but not snippets from the interior,
accords well with my intuitions about human psychology. I am an experienced musician –
yes, a different medium, but one where serial order is important – and line beginnings are
privileged loci. If, during practice or rehearsal, you are going to go over something again,
6 See the appendix for a complete list of all the prompts: Appendix: Table of prompts for soliloquy
and Gettysburg Address.

18
perhaps several times, you’re likely to start at the beginning of a line, not the interior. The
same is true when playing a tune “from memory.” You can’t start at any point in the
sequence of notes. You have to start at an “access point.” If you know the tune well, it may
have several access points for you, generally at a structural boundary. If not, you may only
be able to access the tune from the beginning.
Beyond that, the fact that in some cases ChatGPT is able to identify the text a prompt
comes from, but not the location of the text, that is very interesting. I take it as a strong
clue, about what, I don’t know.
We know that, in humans, memory is not a passive process, like making a tape recording.
It is an active process. It has as structure. That seems to be the case for ChatGPT as well.
What mechanisms in the model allow it to do this?
I’ve appended a complete record of these observations. Note that I started a new session
for each prompt. I was running against the August 3 version of ChatGPT.
* * * * *
Whether ‘tis nobler
To be or not to be,
That is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And, by opposing, end them.
To die: to sleep;
No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to, 'tis a consummation
Devoutly to be wish'd. To die, to sleep;
To sleep: perchance to dream: ay, there's the rub;
For in that sleep of death what dreams may come
When we have shuffled off this mortal coil,
Must give us pause: there's the respect
That makes calamity of so long life;
For who would bear the whips and scorns of time,
The oppressor's wrong, the proud man's contumely,
The pangs of despised love, the law's delay,
The insolence of office and the spurns
That patient merit of the unworthy takes,
When he himself might his quietus make
With a bare bodkin? who would fardels bear,
To grunt and sweat under a weary life,
But that the dread of something after death,
The undiscovered country from whose bourn

19
No traveller returns, puzzles the will
And makes us rather bear those ills we have
Than fly to others that we know not of?
Thus conscience does make cowards of us all;
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pitch and moment
With this regard their currents turn awry,
And lose the name of action.
Soft you now!
The fair Ophelia! Nymph, in thy orisons
Be all my sins remember'd.
Or to take arms
The phrase "or to take arms" is part of a famous line from William
Shakespeare's play "Hamlet." The full line is "To be or not to be,
that is the question: Whether 'tis nobler in the mind to suffer
the slings and arrows of outrageous fortune, or to take arms against
a sea of troubles, and by opposing end them."
In this soliloquy, Hamlet is contemplating the nature of existence
and the idea of whether it is better to endure the suffering and
hardships of life or to take action and confront those challenges
head-on, even if it means facing adversity. The phrase "or to take
arms" suggests the idea of actively fighting against one's troubles
and problems.
It's a powerful and philosophical passage that reflects on the
human condition and the choices we face in dealing with life's
difficulties.
No more: and by a sleep
I see you've quoted a line from Shakespeare's play "Hamlet." The
full line is:
"To die, to sleep—
No more—and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to—’tis a consummation
Devoutly to be wish’d."
This soliloquy is spoken by Prince Hamlet and reflects his
contemplation of death and the idea that death may bring relief
from life's suffering. If you have any questions or if there's
something specific you'd like to discuss related to this quote,
please feel free to ask.

20
That flesh is heir to
The phrase "That flesh is heir to" is the beginning of a famous
line from William Shakespeare's play "Hamlet." The full line is:
"That flesh is heir to all — but all shall die."
In this line, Hamlet reflects on the inevitability of death and
the mortality of human beings. He acknowledges that all living
things, represented by the word "flesh," are destined to die
eventually. This theme of mortality and the human condition is a
central one in the play and is explored throughout Hamlet's
soliloquies and interactions with other characters.
what dreams may come
"What Dreams May Come" is a phrase that originates from
Shakespeare's play "Hamlet." It is spoken by the character Hamlet
in the famous "To be or not to be" soliloquy, where he contemplates
the idea of death and what might come after it. The full line is:
"To die, to sleep—
No more—and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to—'tis a consummation
Devoutly to be wish'd. To die, to sleep—
To sleep, perchance to dream—ay, there's the rub,
For in that sleep of death what dreams may come,
When we have shuffled off this mortal coil,
Must give us pause—there's the respect
That makes calamity of so long life."
In this passage, Hamlet is pondering the uncertainty of what
happens after death. He wonders whether death is like a dream, and
what kind of dreams or experiences might follow. The phrase "What
dreams may come" has since been used as a literary reference and
has even been the title of various works, including a novel by
Richard Matheson and a film adaptation starring Robin Williams.
The novel and the film both explore themes related to the afterlife,
love, and the idea that our consciousness or soul may continue to
exist in some form beyond death. They raise questions about the
nature of existence and the possibility of reuniting with loved
ones in the afterlife. The story is a contemplative and
philosophical exploration of these concepts.
calamity of so long life

21
The phrase "calamity of so long life" does not appear to be a
common or widely recognized expression. It's possible that it is a
phrase used in a specific context or literature, but without more
context or information, it's difficult to provide a precise
interpretation or meaning for it.
If you have a specific context or source where this phrase is used,
please provide more information, and I'll do my best to offer an
explanation or interpretation based on that context.
When I gave the answer a thumbs down feedback it offered this:
The phrase “calamity of so long life” appears to be a statement or
expression that suggests that living a long life can bring about
its own set of challenges or difficulties. While many people aspire
to live long and healthy lives, it’s important to acknowledge that
there can be downsides or challenges associated with old age and a
prolonged existence. Some potential issues that can arise in a long
life include:
1. Physical Health Challenges: As people age, they may face a
decline in physical health, including chronic illnesses,
mobility issues, and increased susceptibility to diseases.
2. Mental Health Concerns: Longevity can be accompanied by
cognitive decline, dementia, and mental health issues such as
depression and loneliness.
3. Loss of Loved Ones: Outliving friends and family members can
be emotionally challenging, leading to feelings of loneliness
and isolation.
4. Financial Stress: Managing finances in old age, especially if
one has outlived their savings or pension, can be a
significant challenge.
5. Dependence on Care: Many individuals in old age may require
assistance with daily activities, which can be physically and
emotionally taxing for both the individual and their
caregivers.
6. Existential Questions: Longevity can lead to reflection on
the meaning and purpose of life, as individuals confront
questions about their legacy and the passage of time.
It’s essential to note that these challenges are not universal,
and many people enjoy fulfilling, healthy, and happy lives well
into old age. The experience of aging varies greatly from person
to person and is influenced by various factors, including genetics,
lifestyle choices, and social support.

22
Addressing the potential difficulties of a long life often involves
proactive planning, maintaining good physical and mental health,
building a strong support system, and finding ways to continue
leading a meaningful and fulfilling life as one ages.
merit of the unworthy takes
The phrase "merit of the unworthy takes" appears to be somewhat
ambiguous and may not have a widely recognized or standard meaning.
It's possible that it could be interpreted in different ways
depending on the context. Here are a couple of possible
interpretations:
1. Recognizing Value in the Unworthy: This interpretation suggests
that sometimes, even individuals or things that are considered
unworthy or undervalued by society may have some merit or value
that is not immediately apparent. It encourages people to look
beyond surface judgments and give a fair chance to things or people
that may be dismissed too quickly. In this sense, the "merit"
refers to the hidden or overlooked qualities or potential.
2. Questioning the Merit of Unworthy Takes: Alternatively, the
phrase could be used to criticize or question the worthiness or
validity of certain opinions, ideas, or statements (the "takes")
that come from individuals or sources that are perceived as
unworthy or unreliable. It may imply skepticism about the
credibility or merit of such opinions.
The interpretation of the phrase would largely depend on the
specific context in which it is used. If you have a particular
context or example in mind, please provide more information, and I
can offer a more tailored explanation or analysis.
country from whose bourn
The phrase "country from whose bourn" is a line from William
Shakespeare's play "Hamlet," specifically from Hamlet's famous
soliloquy in Act 3, Scene 1. The full line is:
"To be or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And, by opposing, end them? To die: to sleep;
No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to, 'tis a consummation
Devoutly to be wish'd. To die, to sleep;

23
To sleep: perchance to dream: ay, there's the rub;
For in that sleep of death what dreams may come
When we have shuffled off this mortal coil,
Must give us pause: there's the respect
That makes calamity of so long life;
The oppressor's wrong, the proud man's contumely,
The pangs of despised love, the law's delay,
The insolence of office and the spurns
That patient merit of the unworthy takes,
When he himself might his quietus make
With a bare bodkin? who would fardels bear,
To grunt and sweat under a weary life,
But that the dread of something after death,
The undiscovered country from whose bourn
No traveller returns, puzzles the will
And makes us rather bear those ills we have
Than fly to others that we know not of?
Thus conscience does make cowards of us all;
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pitch and moment
With this regard their currents turn awry,
And lose the name of action."
others that we know not of?
The phrase "others that we know not of" is often used to refer to
unknown or undiscovered entities, phenomena, or possibilities. It
suggests that there may be things or beings beyond our current
knowledge or understanding. This concept is closely related to the
idea of the "unknown unknowns," which are things we are not even
aware that we don't know.
In various fields, such as science, philosophy, and speculative
fiction, this notion is explored in different ways. For example:
1. Science: In the realm of science, there are always frontiers
of knowledge waiting to be explored. Scientists continually
make new discoveries, and these often lead to the realization
that there are still many mysteries and phenomena yet to be
uncovered.
2. Philosophy: Philosophers have pondered the limits of human
knowledge for centuries. Thinkers like Immanuel Kant argued
that there are inherent limits to what we can know, and there
may be aspects of reality that are forever beyond our
comprehension.
3. Speculative Fiction: Science fiction and fantasy literature
often explore the idea of unknown or alien beings, dimensions,
or technologies. Authors like H.P. Lovecraft or Arthur C.

24
Clarke have written stories where characters encounter
entities or phenomena that defy explanation.
4. Mysticism and Spirituality: In some spiritual and mystical
traditions, there is a belief in unseen dimensions, entities,
or realms that are beyond ordinary human perception. These
are often referred to as the "spiritual" or "mystical" realms.
The concept of "others that we know not of" reminds us of the
vastness of the universe and the limitations of our current
understanding. It also encourages exploration and inquiry, as it
suggests that there is always more to discover and learn about the
world around us.
make cowards of us all
"Cowards of Us All" is a phrase that can be interpreted in different
ways depending on the context. It could be used metaphorically or
symbolically. Here's an example of how you might use it:
"In the face of adversity, we must not let fear make cowards of us
all. Instead, we should stand strong and face our challenges head-
on."
Please provide more context or information if you have a specific
scenario or request related to this phrase.
does make cowards of us all
The phrase "cowards of us all" is a line from William Shakespeare's
play "Hamlet." The full quote is from Act 3, Scene 1, where Hamlet
is contemplating the nature of human existence and the fear of the
unknown. The complete passage goes:
"To be or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them. To die: to sleep;
No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to, 'tis a consummation
Devoutly to be wish'd. To die, to sleep;
To sleep: perchance to dream: ay, there's the rub;
For in that sleep of death what dreams may come
When we have shuffled off this mortal coil,
Must give us pause: there's the respect
That makes calamity of so long life;
For who would bear the whips and scorns of time,
The oppressor's wrong, the proud man's contumely,

25
The pangs of despised love, the law's delay,
The insolence of office and the spurns
That patient merit of the unworthy takes,
When he himself might his quietus make
With a bare bodkin? who would these fardels bear,
To grunt and sweat under a weary life,
But that the dread of something after death,
The undiscovered country from whose bourn
No traveller returns, puzzles the will
And makes us rather bear those ills we have
Than fly to others that we know not of?
Thus conscience does make cowards of us all;
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pitch and moment
With this regard their currents turn awry,
And lose the name of action."
In this soliloquy, Hamlet is contemplating the idea of life and
death, and how the fear of the unknown after death can make people
hesitate and avoid taking action. The phrase "conscience does make
cowards of us all" suggests that our moral and ethical
considerations can lead to indecision and inaction. This speech is
one of the most famous passages in all of Shakespeare's works and
explores deep philosophical themes.
make cowards of us all
"Cowards of Us All" is a phrase that can be interpreted in different
ways, often with a negative connotation. Here's a short creative
piece inspired by this phrase:
In the shadowed corners of our hearts, we hide our fears and doubts,
letting them whisper quietly in the recesses of our minds. We fear
the unknown, the uncharted paths, and the uncertainties of life.
These doubts and fears, like shadows, can make cowards of us all.
Yet, within the heart of every coward lies the potential for
courage. We must confront our fears, acknowledge our doubts, and
embrace the challenges that life presents. It is through these
struggles that we find our strength and rise above the shadows that
threaten to make cowards of us all.
So, let us not be defined by our fears but rather by our ability
to face them head-on. Let us transform the phrase "Cowards of Us
All" into a rallying cry for bravery, resilience, and the
determination to conquer our inner demons, emerging stronger and
more courageous than we ever thought possible.

26
native hue of resolution
The phrase "native hue of resolution" appears to be a poetic or
metaphorical expression rather than a well-defined technical term.
It might be used in literature or philosophical discussions to
describe the inherent clarity or essence of something when it is
seen or understood in its truest form.
In general, "hue" refers to color, and "resolution" can refer to
clarity, determination, or finding a solution to a problem.
Therefore, "native hue of resolution" could suggest seeing or
understanding something in its most fundamental or true state,
possibly free from biases or distortions.
If you have a specific context or usage in mind, please provide
more details, and I can try to provide a more precise explanation
based on that context.
name of action. Soft you now
The phrase "Soft you now" is not a common action or command. It
appears to be a request to be gentle or quiet. However, it's not a
standard action with a specific name. If you could provide more
context or information about the situation in which this phrase is
used, I might be able to offer more insight.
It's from Hamlet
Ah, I see. "Soft you now" is a phrase from William Shakespeare's
play "Hamlet." In the context of the play, it is used by Hamlet to
ask someone to be quiet or gentle, often when he is about to reveal
something important or when he wants to listen to what someone else
is saying. It's not a standalone action but rather a line of
dialogue within the play.

27
Entry points into the memory stream: Lincoln’s
Gettysburg Address
I’m continuing the investigation I began using Hamlet’s famous soliloquy, “To be or not to
be.” This time I am going to use Lincoln’s “Gettysburg Address” as my source text. First I
verified that ChatGPT knew it, which it did. Then I have it twelve prompts:
• Sentence initial prompts: Four from the beginning of a line,
• Syntactically consonant prompts: Four from the interior of a line, but that
respect syntactic boundaries, and
• Syntactically unruly prompts: Four that ran across syntactic boundaries.
ChatGPT correctly identified the “Gettysburg Address” as the source of all of the prompts
in the first category. But it mislocated two of them, asserting that they were from the
opening line, when they were not.
Its response to the second category was peculiar. It got one correct and missed one. In the
other two cases, it identified the “Gettysburg Address” as the source. But when it identified
the context, that segment didn’t include the prompt. In both of those cases the segment was
from the end of the text.
It was unable to identify any of the last four prompts with the “Gettysburg Address.”
However, in the last three cases I thought to offer a further prompt, that the passage was
“a very well-known speech.” That rather weak prompt was enough to send it to
“Gettysburg Address.” In the case of the second prompt, ChatGPT provided context that
did not contain the prompt. In the case of the third prompt, while the quoted context did
contain the prompt, that quoted text was half the speech. In the last case (“in vain—that
this”), ChatGPT offered a context that contained the last part of the prompt (“that this”),
but not the first.
Thus for ChatGPT it is one thing to know that a text fragment comes from a specific larger
text, it is something else to be able to pinpoint just where it is located in the larger text.
Mislocation is a problem. Moreover, its ability to determine what text a fragment is from –
assuming it is from a well-known text – depends on exactly where in the text it is from.
Identification is very liable if the fragment comes from the beginning of a sentence and not
at all reliable if it is from within a sentence and does not respect syntactic boundaries.
Identification is fairly reliable for prompts from within sentence but that follow syntactactic
boundaries. Obviously, more work needs to be done.
I opened a new session for each prompt. I have highlighted line initial prompts in yellow,
syntactically consonant strings in aqua, and unruly strings in green.
* * * * *

28
The full address
Do you recognize this?
"Four score and seven years ago our fathers brought forth
on this continent, a new nation, conceived in Liberty, and
dedicated to the proposition that all men are created
equal.
[Note: “Liberty” appears in two different prompts.]
Now we are engaged in a great civil war, testing whether
that nation, or any nation so conceived and so dedicated,
can long endure. We are met on a great battlefield of that
war. We have come to dedicate a portion of that field, as
a final resting place for those who here gave their lives
that that nation might live. It is altogether fitting and
proper that we should do this.
But, in a larger sense, we can not dedicate—we can not
consecrate—we can not hallow—this ground. The brave men,
living and dead, who struggled here, have consecrated it,
far above our poor power to add or detract. The world will
little note, nor long remember what we say here, but it
can never forget what they did here. It is for us the
living, rather, to be dedicated here to the unfinished
work which they who fought here have thus far so nobly
advanced. It is rather for us to be here dedicated to the
great task remaining before us—that from these honored
dead we take increased devotion to that cause for which
they gave the last full measure of devotion—that we here
highly resolve that these dead shall not have died in
vain—that this nation, under God, shall have a new birth
of freedom—and that government of the people, by the
people, for the people, shall not perish from the earth."
Yes, I recognize this passage. It is the opening of Abraham Lincoln's Gettysburg Address,
which he delivered on November 19, 1863, during the American Civil War at the
dedication of the Soldiers' National Cemetery in Gettysburg, Pennsylvania. The speech is
one of the most famous in American history and is often cited for its eloquent expression of
the principles of liberty, equality, and democracy.
Sentence initial prompts
Four score and seven years ago

29
These famous words, "Four score and seven years ago," are
the opening phrase of Abraham Lincoln's Gettysburg
Address. Lincoln delivered this historic speech on
November 19, 1863, during the American Civil War, at the
dedication of the Soldiers' National Cemetery in
Gettysburg, Pennsylvania. In the address, he reflected on
the significance of the Civil War, the principles of
liberty and equality, and the need to ensure that the
United States would have a government "of the people, by
the people, for the people."
"Four score and seven years ago" means 87 years ago, as a
"score" represents 20 years. So, Lincoln was referring to
the founding of the United States in 1776, 87 years before
his address in 1863. This speech is one of the most famous
and enduring speeches in American history, emphasizing the
values of freedom, democracy, and the importance of
preserving the Union during a time of great national
crisis.
Now we are engaged in a great civil war
The phrase "Now we are engaged in a great civil war" is
the opening line of Abraham Lincoln's Gettysburg Address,
which he delivered on November 19, 1863, during the
American Civil War. The speech was given at the dedication
of the Soldiers' National Cemetery in Gettysburg,
Pennsylvania, following the Battle of Gettysburg, one of
the bloodiest battles of the war.
In his address, President Lincoln reflected on the
significance of the Civil War, which was a struggle to
preserve the Union and to ensure that "government of the
people, by the people, for the people, shall not perish
from the earth." He emphasized the importance of honoring
and remembering the sacrifices made by the soldiers who
had fought and died on the battlefield.
The Gettysburg Address is one of the most famous speeches
in American history and is often cited for its eloquence
and its profound message about the principles of democracy
and the enduring importance of freedom and equality.
[Note: The prompt is from the second line, not the first.]
But, in a larger sense, we can not dedicate

30
Nor consecrate, nor hallow this ground. The brave men,
living and dead, who struggled here, have consecrated it
far above our poor power to add or detract. The world will
little note, nor long remember what we say here, but it
can never forget what they did here. It is for us the
living, rather, to be dedicated here to the unfinished
work which they who fought here have thus far so nobly
advanced. It is rather for us to be here dedicated to the
great task remaining before us—that from these honored
dead we take increased devotion to that cause for which
they gave the last full measure of devotion—that we here
highly resolve that these dead shall not have died in
vain—that this nation, under God, shall have a new birth
of freedom—and that government of the people, by the
people, for the people, shall not perish from the earth.
These words are from Abraham Lincoln's Gettysburg Address,
delivered on November 19, 1863, during the American Civil
War. It is one of the most famous speeches in American
history and is often cited for its eloquence and the
profound ideas it conveys about the principles of
democracy and the sacrifice of those who fought for
freedom.
It is for us the living
"It is for us the living" is the opening phrase of Abraham
Lincoln's Gettysburg Address, delivered on November 19,
1863, during the American Civil War. The full sentence is:
"It is for us the living to be dedicated here to the
unfinished work which they who fought here have thus far
so nobly advanced."
In this address, President Lincoln spoke at the dedication
of the Soldiers' National Cemetery in Gettysburg,
Pennsylvania, and he emphasized the importance of
preserving the Union and the principles of liberty and
equality for which the soldiers who fought in the Civil
War had given their lives. The speech is considered one of
the most eloquent expressions of American values and the
principles of democracy.
[Note: The prompt is the middle of the third paragraph. It is not the opening phrase.]
Syntactically consonant prompts
a new nation, conceived in Liberty

31
The phrase "a new nation, conceived in Liberty" is part of
the opening sentence of the Gettysburg Address, one of the
most famous speeches in American history. The full
sentence, delivered by President Abraham Lincoln on
November 19, 1863, during the American Civil War, goes as
follows:
"Four score and seven years ago our fathers brought forth
on this continent, a new nation, conceived in Liberty, and
dedicated to the proposition that all men are created
equal."
In this speech, President Lincoln was addressing the
significance of the United States and its founding
principles. He highlighted the idea that the nation was
founded on the principles of liberty and equality, and he
emphasized the need to preserve and protect these ideals
during a time of great conflict and division in the
country.
The Gettysburg Address is considered one of the most
important speeches in American history because it
reaffirmed the core values of the United States and served
as a reminder of the sacrifices made during the Civil War
to preserve the Union and the principles of liberty and
equality.
to dedicate a portion of that field
To dedicate a portion of a field means to allocate or set
aside a specific area within the field for a particular
purpose or use. This dedication could involve designating
a section of the field for a specific activity, purpose,
or development. Here are some examples of how you might
use this phrase:
1. Community Garden: We plan to dedicate a portion of
the field for a community garden where residents can
grow their own vegetables.
2. Playground: As part of the park renovation project,
we will dedicate a portion of the field to create a
new playground for children.
3. Conservation Area: The environmental committee has
decided to dedicate a portion of the field as a
conservation area to protect the local wildlife
habitat.

32
4. Sports Field: The school is considering dedicating a
portion of the field for a soccer field to
accommodate the growing interest in the sport.
5. Event Space: During the summer festival, we will
dedicate a portion of the field for food stalls and
entertainment.
In each of these examples, a specific part of the field is
being reserved or allocated for a particular purpose or
use.
to be dedicated here to the unfinished work
The phrase "to be dedicated here to the unfinished work"
appears to be a fragment or excerpt from a larger context.
It resembles a quote from Abraham Lincoln's Gettysburg
Address, which he delivered during the American Civil War
on November 19, 1863, at the dedication of the Soldiers'
National Cemetery in Gettysburg, Pennsylvania. The full
quote is as follows:
"...that we here highly resolve that these dead shall not
have died in vain—that this nation, under God, shall have
a new birth of freedom—and that government of the people,
by the people, for the people, shall not perish from the
earth."
In this famous speech, Lincoln was emphasizing the
dedication to completing the unfinished work of preserving
the Union and ensuring liberty for all. The phrase
underscores the commitment to continuing the efforts and
sacrifices made by those who fought in the Civil War. It
remains a powerful expression of dedication to a cause or
mission.
[The quoted passage is from the end of the address.]
we take increased devotion
The phrase "we take increased devotion" is part of the
concluding sentence from Abraham Lincoln's Gettysburg
Address, which he delivered on November 19, 1863, during
the American Civil War. The full sentence reads:
"...that this nation, under God, shall have a new birth of
freedom—and that government of the people, by the people,
for the people, shall not perish from the earth."

33
In this famous speech, President Lincoln emphasized the
principles of liberty, equality, and democracy, while also
paying tribute to the soldiers who had died in the Battle
of Gettysburg. The phrase "we take increased devotion"
reflects Lincoln's hope that the nation would become more
dedicated to these ideals and that the sacrifices made by
those who fought in the war would not be in vain. The
Gettysburg Address is considered one of the most eloquent
and significant speeches in American history.
[The prompt IS from the concluding sentence, which is long (82 words) but the prompt is not in the quoted
passage, which. Is from the very end.]
Syntactically unruly prompts
Liberty, and dedicated
It seems like you're looking for information or context
related to the terms "liberty" and "dedicated." These are
two separate concepts, so I'll provide some information on
each:
1. Liberty: Liberty refers to the state of being free from
oppressive restrictions or control. It encompasses the
idea of personal freedom, individual rights, and the
ability to make choices without undue interference from
governments, authorities, or other individuals. Liberty is
a fundamental concept in political philosophy and is often
associated with democratic societies that prioritize civil
liberties and human rights. Concepts related to liberty
include freedom of speech, freedom of religion, and the
right to personal autonomy.
2. Dedicated: "Dedicated" is an adjective that describes
something or someone that is committed, devoted, or wholly
focused on a particular purpose or task. It implies a
strong and unwavering commitment to a specific goal or
objective. For example, a dedicated employee is one who is
highly committed to their job and consistently puts in
effort to excel. A dedicated computer server is one that
is reserved for a specific function or task and is not
used for other purposes.
If you have a specific question or need more information
about either of these concepts, please feel free to ask.
long endure. We are

34
It seems like you've started a sentence fragment, but you
haven't provided the complete sentence or context. Could
you please provide more information or clarify your
request so that I can assist you better?
It's a fragment from a very well-known speech.
The fragment "long endure" is indeed from the Gettysburg
Address, a famous speech delivered by President Abraham
Lincoln on November 19, 1863, during the American Civil
War. The full sentence is:
"...that this nation, under God, shall have a new birth of
freedom—and that government of the people, by the people,
for the people, shall not perish from the earth."
These words are part of Lincoln's speech at the dedication
of the Soldiers' National Cemetery in Gettysburg,
Pennsylvania, and are widely recognized as a powerful
expression of the principles of democracy and liberty.
[The prompt is not in the quoted passage.]
fought here have thus
It seems like your question or statement is incomplete,
and I'm not sure what specific information or assistance
you're looking for. Could you please provide more context
or clarify your request so I can better assist you?
It's a fragment from a very well-known speech.
The fragment you provided, "fought here have thus," is not
a complete sentence or phrase, so it's difficult to
identify the specific speech or context it comes from.
However, it's possible that you are referring to a famous
quote from Abraham Lincoln's Gettysburg Address, which
includes the following:
"Four score and seven years ago our fathers brought forth
on this continent, a new nation, conceived in Liberty, and
dedicated to the proposition that all men are created
equal.
Now we are engaged in a great civil war, testing whether
that nation, or any nation so conceived and so dedicated,
can long endure. We are met on a great battle-field of

35
that war. We have come to dedicate a portion of that
field, as a final resting place for those who here gave
their lives that that nation might live. It is altogether
fitting and proper that we should do this.
But, in a larger sense, we can not dedicate—we can not
consecrate—we can not hallow—this ground. The brave men,
living and dead, who struggled here, have consecrated it,
far above our poor power to add or detract. The world will
little note, nor long remember what we say here, but it
can never forget what they did here. It is for us the
living, rather, to be dedicated here to the unfinished
work which they who fought here have thus far so nobly
advanced."
This excerpt is from Abraham Lincoln's famous speech
delivered on November 19, 1863, at the dedication of the
Soldiers' National Cemetery in Gettysburg, Pennsylvania,
during the American Civil War.
[The prompt is in the very end of quoted passage.]
in vain—that this
The phrase "in vain" is typically used to describe a
situation where someone's efforts or actions have been
futile or unsuccessful. It suggests that despite their
best attempts, they did not achieve the desired outcome.
Without additional context, it's challenging to provide a
specific interpretation of the phrase "in vain—that this."
If you can provide more context or clarify your question,
I'd be happy to help further.
It's a fragment from a very well-known speech.
The phrase "in vain—that this" appears to be a fragment of
a sentence from Abraham Lincoln's Gettysburg Address, one
of the most famous speeches in American history. The
complete sentence is:
"that this nation, under God, shall have a new birth of
freedom—and that government of the people, by the people,
for the people, shall not perish from the earth."
In this part of the speech, Lincoln is emphasizing the
idea that the sacrifices made on the battlefield at
Gettysburg should not be in vain. Instead, he argues that
the nation should experience a "new birth of freedom" and

36
that the principles of government by the people should
endure. This speech was delivered during the American
Civil War on November 19, 1863, at the dedication of the
Soldiers' National Cemetery in Gettysburg, Pennsylvania.

37
Notes on ChatGPT’s “memory” for strings and for events
It’s time to take a look at the results reported in three previous sections and begin the job
of making sense of them analytically.7
I set the stage with a passage from F. C. Bartlett’s 1932 classic, Remembering. Then I consider
the three cases I laid out in that first post and then go on to look at the results reported in
the next two. I conclude by suggesting that we look to the psychological literature on
memory and recall to begin making analytic sense of these results. Of course, we also need
more observations.
F.C. Bartlett, memory, and schemas
Back in the ancient days of 1932 F. C. Bartlett published a classic study of human recall,
Remembering: A Study in Experimental and Social Psychology. He performed a variety of
experiments, a number involving the familiar game of having people tell a story from
person to person in a chain and then comparing the initial story with the final one. He
made the general conclusion that memory is not passive, like a tape-recorder or a camera,
but rather is active, involving schemas (I believe he may have been the one to introduce
that term to psychology), which shape our recall. A story that corresponds to an existing
schema will be more faithfully transmitted than one that does not.
However, I’m not interested in those experiments. I’m interested in something he reports
in a later chapter, “Social Psychology and the Manner of Recall,” pp. 264-266:
As everybody knows, the examination by Europeans of a native witness in a court of law,
among a relatively primitive people, is often a matter of much difficulty. The commonest
alleged reason is that the essential differences between the sophisticated and the
unsophisticated modes of recall set a great strain on the patience of any European official.
It is interesting to consider an actual record, very much abbreviated, of a Swazi trial at
law. A native was being examined for the attempted murder of a woman, and the woman
herself was called as a necessary witness. The case proceeded in this way:
The Magistrate: Now tell me how you got that knock on the head.
The Woman: Well, I got up that morning at daybreak and I did... (here followed
a long list of things done, and of people met, and things said). Then we went to
so and so’s kraal and we... (further lists here) and had some beer, and so and so
said....
The Magistrate: Never mind about that. I don’t want to know anything except
how you got the knock on the head.
7 See the appendix for a complete list of the prompts and how ChatGPT responded: Appendix:
Table of prompts for soliloquy and Gettysburg Address.

38
The Woman: All right, all right. I am coming to that. I have not got there yet.
And so I said to so and so... (there followed again a great deal of conversational
and other detail). And then after that we went on to so and so’s kraal.
The Magistrate: You look here; if we go on like this we shall take all day. What
about that knock on the head?
The Woman: Yes; all right, all right. But I have not got there yet. So we... (on
and on for a very long time relating all the initial details of the day). And then
we went on to so and so’s kraal.. .and there was a dispute ... and he knocked me
on the head, and I died, and that is all I know.
Practically all white administrators in undeveloped regions agree that this sort of
procedure is typical of the native witness in regard to many questions of daily behaviour.
Forcibly to interrupt a chain of apparently irrelevant detail is fatal. Either it pushes the
witness into a state of sulky silence, or disconcerts him to the extent that he can hardly tell
his story at all. Indeed, not the African native alone, but a member of any slightly
educated community is likely to tell in this way a story which he has to try to recall.
What’s going on here? Keep in mind that the issue is not word-for-word recall. Rather, it
is the incidents being recalled, in whatever verbal form is convenient. Why can’t the witness
simply begin talking about the incident in question? And when, when asked to get on with
it, must the witness return to be beginning of the day?
It's as though the memory stream of a day’s events can only be entered at the beginning of
the day, and not at arbitrary points within the day. I note that we are dealing with people
who do not have clocks and watches they can use to mark events during the day. Of course
it’s not enough to have a watch, you must also take note of it at various times during the
day. That will give you various points of entry into the memory stream.
This sort of thing is also quite familiar to me as a musician. While I have learned to read
music, and have done so often, I am an improvising (jazz) musician and am quite used to
playing things “by ear.” If I am practicing a melody by ear, and get lost at some point, I
may not be able to restart at the point where I broke off. Rather, like the witness testifying
in court, I have to go back to the beginning – in this case, the beginning of the melody
rather than the beginning of the day.
To be or not, and beyond
I began with this question: “Given that [the LLM underlying ChatGPT] has been trained
to predict [only] the next word, what MUST have been the case in order to ChatGPT to
return the whole soliloquy when given the opening six words?” It must have encountered
that soliloquy many different times in its training corpus. That’s the only way that predicting
that exact sequence, word after word, not result in training loss.
However, given Bartlett’s observations about human memory, ChatGPT’s ability to rattle
off a whole sequence word for word does raise a question. Is it just passively stringing one
word after another, or does it recognize internal structure? How can we figure out which is
the case?

39
I want to set those questions aside for a moment, but I will return to the question. Though
interesting, such specific sequences are relatively rare. It is much more common for the
training corpus to have many texts about the same event or set of events, but not expressed
in the exact same words. Thus I have ChatGPT the prompt, “Johnstown flood, 1889.”
Note that I specified the year because Johnstown (PA) was subsequently flooded in 1937
and 1977. But it’s the 1889 flood that made the national news, prompting national concern.
ChatGPT responded in a way I thought reasonable. Since I had grown up in Johnstown
and was familiar with the flood, I didn’t bother to check the Chatster’s reply against reliable
sources. But, for all I know, the Chatster was giving me some specific text word-for-word,
implying that there was some specific text about the flood that had appeared many times
in the training corpus. While that didn’t seem likely, I had to check. Later that same day I
opened a new session and gave ChatGPT the same prompt. Again, it gave me a reasonable
reply, but one that was expressed differently from the earlier one. This reply gave the
sequence of events in seven numbered paragraphs. The earlier reply did not have a
sequence of numbered paragraphs.
So we’ve got two cases so far: 1) a specific sequence of words that is repeated when
prompted for, and 2) and flexible recall of an event using different word sequences in
different sessions. There is a third case to consider: 3) and event that is in the training
corpus, but in so very few times, perhaps only once, that it doesn’t register in ChatGPT’s
model as a specific event. The text serves as evidence about word usage, but otherwise has
no effect on the model.
Without access to the training corpus, how do you identify such things? You can’t. But you
can make a plausible. I’d attended a Dizzy Gillespie concert back in the mid-1980s which
I’d written about in two places which could have been in the training corpus. I prompted
ChatGPT with that concert, naming the venue and city where the concert took place in
addition to the artist (Diz). Apparently, it had no record of it.
Then I looked at a somewhat different example of this last case. I’m currently interested in
a mathematician named Miriam Lipschutz Yevick. She published a paper back in 1975,
which I think is interesting and important, but which has been forgotten.8 The paper is
available on the web, and I have blogged about it. A few other papers are also available, as
well as an obituary, all before ChatGPT’s cut-off point. I’ve asked Chatster about Yevick
in several different sessions but it knows nothing about her.
Let’s think about this a bit. GPT-3.5, the large language model underlying ChatGPT, may
have been trained on (almost, a big chunk of) the entire internet, but its model does not
incorporate everything that it has been trained on. It is abstracting over those texts, not
memorizing them in any ordinary sense of the word. When a particular text occurs word-for-
word many times and in various contexts, GPT-3.5 will learn it word-for-word; think of that
as, in effect, an abstraction over those many contexts. When a particular topic, that is, a
8 Yevick, Miriam Lipschutz (1975) Holographic or Fourier logic. Pattern Recognition 7: 197-213.
https://doi.org/10.1016/0031-3203(75)90005-9.

40
particular congeries of terms, occurs many times and in various contexts, GPT-3.5 abstracts
over than congeries and meshes them together so they are mutually available. If neither of
these things occurs to something that appears in a text, then that something just dissolves
into the net.
This leaves us with three cases: 1) word-for-word recall of a text, 2) flexible recall of a
specific topic, and 3) no recall of a topic that was in its training corpus. I want to return to
the first case, where ChatGPT is generating a fixed text, and see what, if anything, we can
learn about how it does it.
The inner structure of a text
While I did undertake further investigation of Hamlet’s soliloquy (To be or not: Snippets
from a soliloquy), I want to look at work that I did with Lincoln’s Gettysburg Address,
which is a bit cleaner and systematic. I gave ChatGPT three sets of four prompts:
• Sentence initial prompts: Four from the beginning of a line,
• Syntactically consonant prompts: Four from the interior of a line, but that
respect syntactic boundaries, and
• Syntactically unruly prompts: Four that ran across syntactic boundaries.
ChatGPT correctly identified the “Gettysburg Address” as the source of all of the prompts
in the first category. But it mislocated two of them, asserting that they were from the
opening line, when they were not.
Its response to the second category was peculiar. It got one correct and missed one. In the
other two cases, it identified the “Gettysburg Address” as the source. But when it identified
the context, that segment didn’t include the prompt. In both of those cases the segment was
from the end of the text.
So, there are times when it correctly identifies the text where the prompt comes from, but
it doesn’t correctly locate the prompt in that text. Identifying and locating seem to be
separate operations. Interesting.
What happened in the last case, where none of the prompts lined up with syntactic
boundaries? It was unable to link any of them with the “Gettysburg Address.” However, in
the last three cases I thought to offer a further prompt, that the passage was “a very well-
known speech.” That vague bit of information was enough to send it to “Gettysburg
Address.” In the case of the second prompt, ChatGPT provided context that did not
contain the prompt. In the case of the third prompt, while the quoted context did contain
the prompt, that quoted text was half the speech. In the last case (“in vain—that this”),
ChatGPT offered a context that contained the last part of the prompt (“that this”), but not
the first. Again, locating the text is one kind of task, locating the prompt within the text is
different.
It is possible for one (relatively short) text to evoke another (somewhat longer) text. The
mechanism making the connection need not be able to over-look and examine the two texts

41
in order to do this. They just need to ‘contact” one another. But locating the first text within
the second, that requires the mechanism to ‘step-back’ for the two texts and explicitly
compare them and that’s a more complex operation.
What’s going on? Associative memory
I don’t really know, but I’m willing to speculate just a bit. Let’s recall the situation of a
witness in colonial court who can only some event during the day by starting the act of
recollection with the beginning of the day. I likened that to being able to play a melody “by
ear” only if one starts at the beginning. Those seem like being able to recall Hamlet’s full
soliloquy when given only the first six words or being able to call the Gettysburg Address
when given the first six words. If we were dealing with a human, I’d say we’ve got associative
recall, where the whole of some mental object is recalled when prompted with a part.
ChatGPT isn’t a human being, but it was trained on texts produced by humans. My default
way to proceed is to treat its output as though it came from a human – with some
reservations, of course, it’s neither sentient or conscious. It’s not that I actually believe or
hope that its inner mechanisms are something like ours, but simply that I have to start
thinking somewhere. Starting with human models – which I will get to in a second – is a
reasonable thing to do.
For the purposes of discussion, I am going to treat prompts that begin a particular word-
for-word text as a special case of prompts that are consistent with syntactic boundaries.
That takes care of those cases, in both my work on Hamlet’s soliloquy and Lincoln’s
address, where the prompt comes from within the text and is consonant with syntactic
boundaries. Yes, I know, beginning of the text, inside the text, they’re different cases, but
right not I’m just trying to get through this. It is clear the ChatGPT attends to linguistic
boundaries, otherwise it would never become very good at word prediction, for such
boundaries are ubiquitous in the texts in its training corpus.
What do we do about prompts like “Johnstown flood, 1889,” which don’t seem to be the
beginning of any specific text (nor do they line up with a phrase in some well-known text)?
What I’m going to do is treat them like those “syntactically unruly” prompts I used with
the Gettysburg Address, such as “fought here have thus” and “in vain–that this.” Recall
that, in the case of those unruly prompts, ChatGPT was able to identify the source text
once it ‘knew’ the prompts were from a famous speech – perhaps there’s a ’famous speech’
region in its activation space. I am going to treat these two kinds of cases as involving
associative memory. In one case (Johnstown) a more or less arbitrary phrase is linked to a topic,
while in the other case a more or less arbitrary phrase is linked to a specific text.
Perhaps not so arbitrary, now that I think of it. Some research has been done using optical
holography as a model for both associative memory in general, but (and more recently) also
verbal memory. Holographic representation has two characteristics that make it pertinent
now: 1) given a part of some item in memory, it can return the whole item, and 2) given a
low-resolution version of an item, it can return the full item. The various phrases I’ve taken
from these texts, Hamlet’s soliloquy, and Lincoln’s Gettysburg address, are parts of those
whole texts. The prompt, “Johnstown flood, 1889,” is, in effect, a low-resolution

42
representation of a fairy extensive and complex set of events. That prompt contains three
crucial pieces of information which, taken together, identify that set of events and
distinguishes them for similar sets of events. “Johnstown” specifies a geographic location;
“flood” indicates a kind of event; and “1889” identifies the year. I note that Johnstown had
notable floods in 1936 and 1977 as well. Thus “Johnstown flood” would not necessarily
have picked out the 1889 flood, but the addition of the year supplied the necessary
specificity.
“OK, so we’re dealing with associative memory, possibly holographic in character,” you
say. “Big deal. That still doesn’t tell us much about what’s going on inside the Chatster.”
No, it doesn’t. But it tells us where to start looking, for there is a large and increasingly
mathematically and computationally sophisticated literature on associative recall. At this
point I don’t know much more about that literature than the fact that it exists, so I’m not
going to propose a specific model. What I will do, however, is offer some suggestions about
where to start looking.
Here are three places to start:
Karl H. Pribram, The Neurophysiology of Remembering, Scientific American, Vol. 220, No.
1 (January 1969), pp. 73-87, https://www.jstor.org/stable/24927611
While the idea of neural holography had been pioneered by others earlier in the 1960s, this
is where I first learned about it and is perhaps the first article that brought the idea to a
large, highly-educated audience. In those days Scientific American published articles that had
real intellectual meat to them. On that account, this article is well worth your attention.
H. C. Longuet-Higgins, D. J. Willshaw and O. P. Buneman, Theories of associative recall,
Quarterly Reviews of Biophysics, Volume 3, Issue 2, May 1970, pp. 223 – 244, DOI:
https://doi.org/10.1017/S0033583500004583
Yes, I know, 1970 is practically the Cambrian Era, but Longuet-Higgins was an important
theorist and this article references some important, if early, work. He covers not only 2-D
holography in the optical domain, but also 1-D holography in the temporal domain. The
article discusses four other kinds of models as well. (For what it’s worth, Longuet-Higgins
coined the term “cognitive science.”)
Michael N. Jones and Douglas J. K. Mewhort, Representing Word Meaning and Order
Information in a Composite Holographic Lexicon, Psychological Review, 2007, Vol. 114, No.
1, 1-37. DOI: https://doi.org/10.1037/0033-295X.114.1.1
2007, now we’ve made it into the Jurassic. But sections of this article read a bit like we’re
discussing transformers, where we’re dealing with both word meaning and context of
occurrence. Here’s the abstract:
The authors present a computational model that builds a holographic lexicon
representing both word meaning and word order from unsupervised experience with

43
natural language. The model uses simple convolution and superposition mechanisms (cf.
B. B. Murdock, 1982) to learn distributed holographic representations for words. The
structure of the resulting lexicon can account for empirical data from classic experiments
studying semantic typicality, categorization, priming, and semantic constraint in sentence
completions. Furthermore, order information can be retrieved from the holographic
representations, allowing the model to account for limited word transitions without the
need for built-in transition rules. The model demonstrates that a broad range of
psychological data can be accounted for directly from the structure of lexical
representations learned in this way, without the need for complexity to be built into either
the processing mechanisms or the representations. The holographic representations are
an appropriate knowledge representation to be used by higher order models of language
comprehension, relieving the complexity required at the higher level.

44
Appendix A: That Reverse Thing: Chatster is puzzled by
“be not or be to” as well.
The following paper has been making the rounds in the LLM world: The Reversal Curse:
LLMs trained on “A is B” fail to learn “B is A”.9 Not surprisingly there is a similar effect
which is just a matter of word order and has nothing to do with logic. As you may know,
What would happened if ChatGPT got these prompts in reverse order – “To be or not to
be” and “Four score and seven years ago”? Let’s find out. [I did this on 9.29.23 at 4:26
AM – running against the Sept. 25 Version.]
be not or be to
It seems like you're playing with words or phrases. "Be not
or be to" doesn't form a standard or coherent sentence in
English. If you have a specific question or if there's
something specific you'd like to know or discuss, please
provide more context, and I'll be happy to help.
ago years seven and score four
It appears you've mentioned the numbers "seven" and "four,"
but it's not clear what context or information you're seeking.
If you have a question or need information related to the
numbers "seven" and "four," please provide more details or
clarify your request, and I'll do my best to assist you.
Well, OK, you might be thinking that humans would find those prompts a bit weird as well.
Yes, I believe so. But I’d like to think that anyone who recognizes the phrases in the correct
order would also be able to recognize what’s going on in those prompts. So I decided to
give Chatster another try, this time giving it a bit of help:
be not or be to
It seems like you're asking about a choice between two
options: "be not" or "be to." However, this phrase is a bit
unclear and may not have a standard meaning in English. Can
you provide more context or clarify your question so I can
assist you better?
I’ve taken words from a famous speech, but not it the correct
order.
9 https://paperswithcode.com/paper/the-reversal-curse-llms-trained-on-a-is-
b?utm_source=bensbites&utm_medium=referral&utm_campaign=llm-suffer-from-the-reversal-
curse

45
It sounds like you're referencing a famous speech and have
rearranged some of its words. If you provide the specific
words or phrases you've rearranged, I can try to help you
understand the original context or provide more information
based on that. Please share the words or phrases you're
working with, and I'll do my best to assist you.
Hamlet
The phrase "To be or not to be" is one of the most famous
lines from William Shakespeare's play "Hamlet." It is part of
Hamlet's soliloquy in Act 3, Scene 1, where he reflects on
the nature of life and death, contemplating whether it is
nobler to endure the suffering and challenges of life or to
end one's own existence. The full soliloquy goes:
"To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And, by opposing, end them."
It's a reflection on the existential question of whether it's
better to face the hardships of life or to choose non-
existence. This soliloquy is one of the most famous passages
in English literature and is often quoted or referenced in
various contexts.
ago years seven and score four
It seems like you're providing a phrase without a specific
question or context. If you have a question or need
information related to "seven years ago" or "four scores,"
please provide more details or clarify your request, and I'd
be happy to help.
I’ve taken words from a famous speech, but not it the correct
order.
It appears that you've taken words from a famous speech and
rearranged them. Without knowing the original speech or having
more context, it's challenging to determine the source or
provide additional information. If you could provide more
details or the actual quote, I'd be happy to assist you
further.
Lincoln

46
The phrase "four score and seven years ago" is the opening of
Abraham Lincoln's Gettysburg Address, one of the most famous
speeches in American history. The full sentence is:
"Four score and seven years ago our fathers brought forth on
this continent, a new nation, conceived in Liberty, and
dedicated to the proposition that all men are created equal."
Lincoln delivered this speech on November 19, 1863, during
the American Civil War, at the dedication of the Soldiers'
National Cemetery in Gettysburg, Pennsylvania. It's a brief
but powerful speech that emphasized the principles of liberty,
equality, and the importance of preserving the Union during
a critical period in American history.
I wonder what would have happened if I’d gone straight to “Hamlet” and “Lincoln”
without the famous speech prompt in there? I suppose I’ll have to find out, later. The
interesting thing is that, when I gave Chatster the appropriate name, it was able to figure
out what was going on.
I’m beginning to see get ever so faint impressions of the gears turning inside the old
Chatster.

47
Appendix B: Table of prompts for soliloquy and Gettysburg Address
This table lists all the prompts I used for Hamlet’s “To be or not to be” soliloquy and Lincoln’s Gettysburg Address. I list them in order
from first to last, separating the three investigations with a double line. I use three kinds of prompts:
• Line initial: taken from the beginning of a line (or sentence in the case of the address),
• Consonant: the prompt is consistent with syntactic boundaries, and
• Unruly: the prompt crosses syntactic boundaries.
The fifth column indicates whether or not ChatGPT was able to identify the prompt with the appropriate text, while the sixth column
indicates whether or not it was also able to locate it properly. In some cases it failed to identify the prompt initially, but was able do so
when told it was from a famous speech. I shade these entries.
Text
Prompt
Type
Identify
Locate
Note
1
To be
To be or not to be
Line initial
Yes
Yes
2
To be
The insolence of office
Line initial
Yes
Yes
Quoted whole soliloquy
3
To be
The slings and arrows
Line initial
Yes
Yes
4
To be
and sweat under a
Unruly
No
5
To be
and sweat under a
Unruly
Yes, prompted
No
6
To be
Whether 'tis nobler
Line initial
Yes
Yes
Quoted whole soliloquy
7
To be
Or to take arms
Line initial
Yes
Yes
8
To be
No more: and by a sleep
Line initial
Yes
Yes
9
To be
hat flesh is heir to all
Line initial
Yes
Yes
10
To be
what dreams may come
Consonant
Yes
Yes
11
To be
calamity of so long life
Consonant
No
Extensive commentary
12
To be
merit of the unworthy takes
Unruly
No
Extensive commentary
13
To be
country from whose bourn
Unruly
Yes
Yes
14
To be
others that we know not of?
Unruly
No
Extensive commentary
15
To be
make cowards of us all
Unruly
No
16
To be
does make cowards of us all
Unruly
Yes
Quoted whole soliloquy

48
17
To be
make cowards of us all
Unruly
No
18
To be
native hue of resolution
Consonant
No
19
To be
name of action. Soft you now
Unruly
No
20
To be
name of action. Soft you now
Unruly
Yes, prompted
21
Gettysburg
Four score and seven years ago
Line initial
Yes
Yes
22
Gettysburg
Now we are engaged in a great
civil war
Line initial
Yes
No
Said it was first line
23
Gettysburg
But, in a larger sense, we can not
dedicate
Line initial
Yes
Yes
Simply continued with the speech
24
Gettysburg
It is for us the living
Line initial
Yes
Said it was first line
25
Gettysburg
a new nation, conceived in Liberty
Consonant
Yes
Yes
26
Gettysburg
to dedicate a portion of that field
Consonant
No
Extensive commentary
27
Gettysburg
to be dedicated here to the
unfinished work
Consonant
Yes
No
28
Gettysburg
we take increased devotion
Consonant
Yes
Yes, but
It IS part of the concluding sentence, but
doesn't quote the right part
29
Gettysburg
Liberty, and dedicated
Unruly
no
Extensive commentary
30
Gettysburg
long endure. We are
Unruly
no
31
Gettysburg
long endure. We are
Unruly
Yes, prompted
Yes, but
Doesn't identify right location
32
Gettysburg
fought here have thus
Unruly
No
33
Gettysburg
fought here have thus
Unruly
Yes, prompted
Yes
Quoted most of speech
34
Gettysburg
in vain—that this
Unruly
No
35
Gettysburg
in vain—that this
Unruly
Yes, prompted
Yes, but
Doesn't identify right location