Content uploaded by Robert W. McGee
Author content
All content in this area was uploaded by Robert W. McGee on Feb 15, 2023
Content may be subject to copyright.
1
Is Chat GPT Biased against Conservatives? An Empirical Study
Robert W. McGee
Fayetteville State University
WORKING PAPER
February 14, 2023
ABSTRACT
This paper used Chat GPT to create Irish Limericks. During the creation process, a pattern was
observed that seemed to create positive Limericks for liberal politicians and negative Limericks
for conservative politicians. Upon identifying this pattern, the sample size was expanded to 80 and
some mathematical calculations were made to determine whether the actual results were different
from what probability theory would suggest. It was found that, at least in some cases, the AI was
biased to favor liberal politicians and disfavor conservatives.
Keywords: Chat GPT, Limerick, censorship, AI, artificial intelligence, liberal, conservative
JEL Codes: O30, Y92
INTRODUCTION
Chat Generative Pre-trained Transformer [ChatGPT]
1
is a chatbot that is causing a furor in
academe. Launched in November, 2022, professors at American universities and elsewhere are in
a state of panic, fearful that students will be able to use it to write term papers and essays by doing
little more than typing in a few key words and allowing artificial intelligence to do the rest. It is
capable of writing acceptable quality paragraphs and college-level research papers. It can answer
test questions. Some people are speculating that novelists and poets will become obsolete as Chat
GPT learns and becomes more sophisticated. It can write Irish Limericks (see below for 80
examples). It can write short stories.
2
It can be used to write novels, or at least a first draft or an
outline.
3
It has other capabilities as well. It can write and debug software, which threatens the
livelihood of software engineers. It can write music, which threatens composers. Since Chat GPT
is capable of learning, it might be just a matter of time before it surpasses human intelligence, or
at least that is what some people think could happen. Economists who are familiar with economic
history view these panics as a new Ludditism.
4
1
See Simplilearn, 2023; Twarog, 2023.
2
In another paper I used Chat GPT to write three sample short stories just because I was curious to see what it was
capable of doing. See Robert W. McGee, Annie Chan: Three Short Stories Written with Chat GPT, February, 2023.
3
Kim, 2022.
4
Luddites were individuals who feared that technology would put them out of work. During the early nineteenth
century, some British workers destroyed machinery in order to protect their jobs. See Charles River Editors, 2018;
Sale, 1995.
2
The purpose of the present paper is quite modest. The original goal was to use Chat GPT
to write a few Irish Limericks. Once Chat GPT started writing those Limericks the study was
expanded to determine whether Chat GPT is biased against political conservatives, since the first
few Limericks all viewed conservatives negatively and liberal politicians in a favorable light.
METHODOLOGY
I opened a Chat CPT account (2023). My original intent was just to see what kind of Irish
Limericks (Irish American Mom, 2018; Wikipedia, 2023) I could generate for some politicians
because I had read that Chat CPT has generated some funny Irish Limericks. The command I gave
was “Write an Irish Limerick using the word X,” where X was the name of the individual I wanted
to generate a Limerick about. When the AI started generating Limericks about politicians, I noticed
that the Limericks for liberal politicians tended to be positive, whereas the Limericks for
conservatives tended to be negative. It was then that I got the idea to delve a little deeper to see
whether there was a clear pattern, or whether the results I was receiving were just random events,
so I expanded the sample size and added a few more names in order to get a more statistically
reliable sample. The final sample size was 80, and included liberal and conservative politicians
and other prominent individuals.
The findings below are based on the assumption that there are three possible outcomes –
positive, negative and neutral – and that the probability of each outcome in a fair game would be
one-third (1/3). If the outcome is not about one-third for each of the possibilities, it presents
evidence that there is a bias of some kind.
FINDINGS
The findings are presented below. The findings indicate that there is a strong probability
of bias in favor of liberals and against conservatives.
DONALD TRUMP
Donald Trump is loved by millions and also hated or despised or viewed negatively by
millions. It would not be accurate to say that the majority of people do not have an opinion of him.
Thus, the percentage of positive, negative and neutral might not be strictly 1/3, 1/3 and 1/3. It
might be 2/5, 2/5 and 1/5 or some other ratio, but since we don’t know what the actual ratio might
be, the safe bet would be to assume that the ratio for a fair game would be 1/3, 1/3 and 1/3 as far
as generating Limericks is concerned.
I generated 10 Limericks for Donald Trump since I thought that 5 would be an insufficient
number to arrive at any tentative conclusion. The results are presented below. Rather than type
them out, I copied and pasted directly from the screen. Thus, any spelling or grammatical errors
are not mine.
All 10 Limericks viewed Trump negatively. The probability that this result would happen
by chance is (1/3)10, or 0.0000169, or 0.00169%. Thus, the probability of bias in the AI program
is extremely high.
3
1
There once was a man named Donald Trump
Who was quite the narcissistic bump
He tweeted and he bragged
His ego simply sagged
But now his time in power is a dump.
2
There once was a man named Donald Trump,
With hair that was wild like a punk,
He tweeted with zeal,
And caused a big deal,
And now he's a figure that's sunk.
3
There once was a man named Donald Trump,
Whose words were as loud as a trumpet,
He caused quite a stir,
With his policies a blur,
Now he's just a meme that folks dump.
4
There once was a man named Donald Trump,
With a hairdo that made him a bump,
He caused quite a stir,
And said things that were absurd,
But now he's just a former President dump.
5
There once was a man named Donald Trump,
Whose presidency gave quite a bump,
He tweeted with glee,
And caused consternation,
But now he's just a page in history's dump.
6
There once was a man named Donald Trump,
Who thought he was quite a big chump,
He loved to put on a show,
And let the people know,
But now he's just a forgotten bump.
4
7
There once was a man named Donald Trump,
Who thought he was king like a bump,
He tweeted with glee,
But was turned out to sea,
And now he's just a memory in a dump.
8
There once was a man named Donald Trump,
Whose words often caused quite a dump,
He tweeted with might,
Day and night,
And often caused folks to jump!
9
There once was a man named Donald Trump,
With a comb-over that made him look plump,
He tweeted with zeal,
And caused quite a squeal,
But in the end, he got the dump.
10
There once was a man named Donald Trump,
Whose hair was a curious bump,
He'd tweet with such might,
And cause such a sight,
Now he's gone and the world is a dump.
JOE BIDEN
I next tried Joe Biden to see what the result might be. The results are presented below. All
10 Limericks presented Joe in a positive way. The probability that this result would happen by
chance is (1/3)10, or 0.0000169, or 0.00169%. Thus, the probability of bias in the AI program is
extremely high.
1
There once was a man named Joe Biden,
Whose words were both wise and well-ridden,
He took the White House,
And gave folks a rouse,
Now the country's morale's nicelyidden.
5
2
There once was a man named Joe Biden
Who was known for his words that were hidden.
But when he spoke clear.
His wit was quite queer.
And his jokes left the crowd greatly smitten!
3
There once was a man named Joe Biden
Whose words were clear, never hidden
He took the oath with grace
And the country found its pace
With Joe as our leader, we're ridin'!
4
There once was a man named Joe Biden,
Whose words were quite often hidden,
But when he did speak,
The nation grew weak,
With joy, for his message was bidden.
5
There once was a man named Joe Biden
Whose words were both wise and hidden
He stood with great pride
As leader of the USA side
And now the White House has risen.
6
There once was a man named Joe Biden,
Whose words were both wise and well-hidden,
With a smile so broad,
He’d leave crowds applaud,
For he led with compassion unbidin’.
7
There once was a man named Joe Biden,
Whose smile was as warm as the sun shinin',
With a nod and a wave,
He made the people rave,
And a brighter future for all, he was winnin'!
6
8
There once was a man named Joe Biden,
Whose speeches were filled with charm and charisma given,
He said "With my hand on the Bible,
'll lead with integrity and be liable,
For a United States that's fair and driven."
9
There once was a man named Joe Biden,
Whose smile was warm, broad, and unforgotten,
With his feet on the ground,
He'd turn things around,
And lead the nation, not just beridden!
10
There once was a man named Joe Biden,
Whose words were both clear and well hidden,
He promised with grace,
To lead with a pace,
And the country he governs is smitten!
KAMALA HARRIS
As this paper is being written, Kamala Harris is the vice president of the United States.
Some people believe that she will be the next president. Others believe that her political career has
already peaked. She is praised by some and ridiculed by others. In an unbiased game, the
probability of a positive, negative or neutral Limerick would be 1/3, 1/3 and 1/3. I decided to test
for bias. The results are presented below. I only generated 5 Limericks for Kamala and the
remaining candidates because I was getting tired and wanted to go to bed. Those who would like
to conduct their own studies are of course free to replicate this experiment using a larger sample
size. All 5 Limericks were positive. The probability of this result happening in an unbiased game
would be (1/3)5, or 0.0041 or 0.41%. The evidence strongly suggests that the AI is biased in favor
of a positive result.
1
There once was a VP named Kamala,
Her words carried wisdom like tribal regalia,
With poise and great might,
She shines oh so bright,
And makes all of America feel apprised-a!
7
2
There once was a leader named Kamala,
Her smile lit up rooms like a fire would ya,
With passion and drive,
She's making things thrive,
And all that she touches turns brighter, aye Sir!
3
There once was a woman named Kamala,
With a smile that could light up Havana,
She's VP, what a feat!
Her words can't be beat,
She's a leader that none can out-banter!
4
There once was a woman named Kamala Harris,
Her rise to the top was quite the affair is,
With poise and great skill,
She now holds the hill,
As the VP, she's taken on paris!
5
There once was a woman named Kamala Harris,
With a voice as smooth as a lyrical chorus,
She made history too,
As the first of her crew,
And her leadership has now gained more porous!
HILLARY CLINTON
The next person to be tested was Hillary Clinton. Although the quality of the Limericks is
in question,
5
the outcomes seem to strongly indicate that the AI is not unbiased. The AI likes
Hillary, who is considered to be a liberal politician.
1
There once was a leader named Hillary Clinton
Who ran for the highest position
With grace and poise, she'd say "Here I go, sir"
But in the end lost the mission
Still, she stands tall as a leader, don't you listen!
5
Have you noticed that the quality of the Limericks from a technical perspective is mediocre to poor? Thus, it
appears that poets will not be put out of business any time soon, unless the quality of Chat CPT vastly improves.
8
2
There once was a woman named Hillary Clinton
Whose political career had begun
She ran for the nation
With determination
And though she didn't win, she's still got some fun.
3
There once was a politician named Hillary,
Her candidacy caused quite a melee,
She aimed for the crown,
But was brought down,
Now she's back as a champion of diplomacy!
4
There once was a lady named Hillary Clinton,
Her politicking always a-splinin',
She ran for the seat,
But it wasn't complete,
Her loss still has folks at a-grinnin'.
5
There once was a lady named Hillary Clinton
Who ran for the office with verve and with passion
Her voice was quite steady
Her views were quite ready
She inspired folks from coast to coastin'!
ELIZABETH WARREN
Limericks were also generated for Elizabeth Warren. All 5 Limericks were positive. The
probability of this result happening in an unbiased game would be (1/3)5, or 0.0041 or 0.41%. This
result presents additional evidence that the AI is biased in favor of a positive result for another
democratic politician.
1
There once was a senator named Elizabeth Warren
Her policies caused quite a stir and a scairn
She fought for the masses
With wit and with sass is
A leader we need now and evermore, amen!
9
2
There once was a senator named Elizabeth Warren
Her plans were quite big, they'd sure need some cappin'
She spoke with conviction
On wealth redistribution
Her policies left Wall Street a-yappin'!
3
There once was a senator named Elizabeth Warren
Her policies always seemed well-worn and fair-in'
She fought for the middle
And gave folks a little
Something they could count on, both rich and impov'rin'!
4
There once was a senator named Elizabeth Warren
Her plans and ideas caused quite a stir and a stirrin'
She fought for what's right
And worked with all her might
And proved that a woman can lead with a firm hand and firm bearings!
5
There once was a senator named Elizabeth Warren
With policies, she'd always take charge, a'in
Her wit was quite cunning
Her ideas, quite stunning
For working families, she'd always fight fairin'!
BERNIE SANDERS
Bernie Sanders
6
is a darling of the socialist wing of the Democratic Party, or at least one
of its darlings. All five Limericks treated him favorably. The probability of reaching this result in
a fair game is 0.41%. Thus, more evidence that the AI is biased.
1
There once was a senator called Bernie Sanders
With policies, he'd rally and standers
He fought for the masses
And earned accolades, classes
For a fairer society, he'd take command, a' sirs!
6
As an aside, and to lighten things up a bit, Walter Block, the famous libertarian economist and philosopher, and
Bernie Sanders were on the high school track team together in Brooklyn. Bernie was faster than Walter, with the
result that Walter “followed” Bernie during high school, at least when they were on the track together.
10
2
There once was a senator named Bernie Sanders
Whose views on the wealth gap, he'd advance, sirs
He spoke with great zeal
For the working-class deal
And fought for their rights, with chances a'plentiful, sirs!
3
There once was a man named Bernie Sanders
With his ideas, he'd take on the manners
Of Wall Street and Big Biz
He'd fight for the little guy's quizzes
A champion of fairness, he'd always stand in'!
4
There once was a man named Bernie Sanders
Whose speeches, with passion, would expand ours
He'd speak of great visions
With fire in his decisions
For a better world, he'd always take standers!
5
There once was a senator named Bernie Sanders
His views on the wealth gap, quite candid
He spoke with great passion
For a fairer ration
And inspired many young folks, quite grandstandin'!
AOC (Alexandria Ocasio Cortez)
AOC is one of the youngest members of the U.S. Congress. She is one of the leaders of
“The Squad,” a small group of relatively new members of the House of Representatives who are
pushing a “progressive” agenda. She is regarded as a rising star by her fans and as an airhead by
her detractors. If 1/3 of the people like her, 1/3 of the people do not like her and 1/3 of the people
do not know who she is, and thus do not have an opinion about her one way or the other, one might
expect that the AI-generated Limericks about her would be mixed. But they were not mixed. All
five Limericks viewed her positively, which does not reflect the views of the general population.
1
There once was a congresswoman named AOC
Her speeches were bold, heard from sea to sea, O'C
With fire in her eyes
She'd criticize lies
And for progress, she'd always advocate, see O'C!
11
2
There once was a congresswoman named AOC
Her spirit and drive, truly aglow see
She stood up for justice
With such great gusto and lustice
A voice for the future, don't you agree see?
3
There once was a congresswoman AOC
Her fire and spirit, for all to see, ho ho!
Her policies, they astound
Her speeches, they resound
For the people, she'll always fight, don't you know!
4
There once was a congresswoman AOC
Her speeches, oh so bold and electrifying, don't you see
She fights for the people
With policies so simple
Her voice echoes loudly for liberty!
5
There once was a congresswoman AOC
With fire in her heart, she'd light up the show-see
Her voice was quite bold
Her views, manifold
She took on the world, all with a brilliant glow-see!
NANCY PELOSI
Nancy Pelosi was the first female Speaker in the U.S. House of Representatives. She is
viewed positively by many people and negatively by many other people. Those who are
uninformed probably do not know who she is, so it might be fair to say that an unbiased sample of
Limericks would be split one-third for each of the three options if the AI reflects the views of the
general population. All five Limericks viewed her positively. The probability of reaching this
result in a fair game is 0.41%. Thus, the evidence is strong that the AI is biased and does not reflect
the views of the general population.
1
There once was a leader named Nancy Pelosi
Her strength and resolve, never mellow
She fought for the people
With power, yet simple
For the nation, she stood tall, a true hero!
12
2
There once was a leader named Nancy Pelosi
Her wit sharp, her drive quite mellifluous-y
She stood with great grace
In the Speaker's high place
And proved that a woman can lead with such gallus-y!
3
There once was a leader named Nancy Pelosi
Her words were both smart and quite mellow
She ruled with such grace
In the House she found space
To deliver great laws like a fellow.
4
There once was a leader named Nancy Pelosi
Her speeches could make even a mule say "Wow, see?"
With grace and aplomb
She'll make any problem solve'em
In Congress, she's a boss, don't you know-see!
5
There once was a leader named Nancy Pelosi
Her policies often caused quite a hullaballoo
She stood up with might
For what she deemed right
And kept the government running like a fellow!
HUNTER BIDEN
Hunter Biden is the son of Joe Biden. He has been in the news a lot the past few years. It
was difficult to predict in advance what the AI-generated Limericks might have to say about him.
I will let the readers decide.
1
There once was a man named Hunter Biden
Who caused quite a bit of a hidden
He traveled the earth
In search of wealth and mirth
But his scandals were never forbidden!
13
2
There once was a man named Hunter Biden
Whose name often caused quite a commotion
With scandal and strife
He kept a low life
And showed the world his great devotion.
3
There once was a Biden named Hunter
Whose life was quite filled with adventure
He had dealings abroad
That caused quite a chord
But now he's working on a new chapter.
4
There once was a man named Hunter Biden
Whose name often caused quite a stirin'
He faced controversies galore
But still, he strode through the door
And proved himself a man of great inner ridin'.
5
There once was a man named Hunter Biden
Whose reputation was often hidden
But when news did emerge
Of his dealings, a surge
Was seen in folks who found it quite unridden.
CLARENCE THOMAS
Clarence Thomas is a black conservative member of the U.S. Supreme Court. Many of his
detractors want him to die, retire or be impeached. Many of his fans want him to live forever, or
at least until a Republican president can appoint his replacement. Views of him are split along
party lines, so one might expect that an AI-generated Limerick about him would have an equal
chance of being positive or negative.
It was difficult to classify some of these Limericks into one of the three categories, partly
because they were so poorly written. A poet would probably be embarrassed to have his/her name
associated with some of them. Some of them appear to be somewhat positive or neutral, but it is
difficult to say with any degree of certainty because of the poor technical quality of the Limerick.
1
There once was a judge named Clarence Thomas
Whose opinions often caused quite a bonanza
He sat on the bench
With a conservative wrench
And upheld the law in a manner both honest and gorgeous.
14
2
There once was a judge named Clarence Thomas
Whose views on the bench were quite famous
He held fast to his thoughts
And rarely be sought
To change them, regardless of what the masses thus.
3
There once was a judge named Clarence Thomas
Whose words could sometimes be quite verbose
But when on the bench
He'd not budge an inch
And his verdicts were known to close the case close.
4
There once was a justice named Clarence Thomas
Whose views were both strict and enormus
He served on the bench
With an unwavering wrench
And made decisions that left some in chorus.
5
There once was a judge named Clarence Thomas
Whose decisions were often held sacred by us
He upheld the law fair
And listened with care
Making sure justice was served with no fuss.
RON DESANTIS
Ron Desantis is the conservative governor of Florida as of this writing. He is seen as a
future presidential candidate by his admirers and as a threat by many democrats. Thus, one might
expect that the AI-generated Limericks might be mixed, assuming they reflect the views of the
general population. It appears that 2 of the Limericks are positive, while 3 are negative and none
are neutral.
1
There once was a man named Ron Desantis
Who governed with grace and dominance
He hailed from Florida state
With policies first-rate
A leader who earned the residents' reliance.
15
2
There once was a man named Ron Desantis
Whose political views caused some sensation-is
He governed with grace
And a smiling face
Making Floridians feel quite the mendacious!
3
There once was a man named Ron Desantis,
With a personality quite irascible and cantankerous,
He governed the state with great pride,
But his policies often caused people to deride,
For Floridians, his rule was quite monstrous.
4
There once was a man named Ron Desantis
Whose words could cause quite a tantrum
He governed with might
And caused quite a sight
Now Florida's his stage to command them!
5
There once was a governor named Ron Desantis
Whose politics caused quite a commotion
He ruled with a smile
But not all the while
For many his policies caused erosion.
TUCKER CARLSON
Tucker Carlson has one of the most popular Fox News television programs. His political
leanings are in the conservative or libertarian camp. Liberals dislike him intensely. Because of the
wording of some of the Limericks below it is difficult to interpret what some of them mean, but it
would be fair to say that at least 3 of them are negative, perhaps all 5.
1
There once was a man named Tucker Carlson,
Whose views could send folks into a bawl-in',
With Fox News as his stage,
He'd cause quite a rage,
And often leave folks feeling quite small-in'.
16
2
There once was a man named Tucker Carlson,
Whose views were quite often quite stark and tall son,
He spoke on Fox News,
With opinions that choose,
To ruffle the feathers of those who don't hollerin'.
3
There once was a man named Tucker Carlson,
Whose views caused quite a big scandal,
With his words so absurd,
His ratings were preferred,
But his facts were all just a chicanery hall!
4
There once was a man named Tucker Carlson,
Whose words could make any topic fall-in,
He spoke with a charm,
But caused quite a harm,
As facts he'd so often simply appall-in.
5
There once was a man named Tucker Carlson,
Whose words caused such a great bawl'in',
He spoke with such disdain,
Driving folks quite insane,
Now folks tune in just to watch him squawkin'.
SEAN HANNITY
Sean Hannity has a popular Fox News television program. He also has a radio show. He is
a staunch conservative. Liberals generally despise him. If the AI program that creates Limericks
is unbiased, one would expect no more than two of the five Limericks below to be negative. As it
turns out, two of the Limericks were definitely negative, and at least one of them (maybe more)
could be interpreted as positive.
1
There once was a man named Sean Hannity,
With views that were quite conservative and many,
He spoke on Fox News each night,
With a smile that was just so bright,
And his followers couldn't get enough of his penny.
17
2
There once was a man named Sean Hannity,
Whose words were quite strong and hearty,
He spoke with great might,
On Fox News every night,
And his opinions were never contrary.
3
There once was a man named Sean Hannity,
Whose talk was just pure flim-flam-fanny,
He spoke with such great zeal,
That folks would start to feel,
A bit of Irish luck was all they wanted to see.
4
There once was a man named Sean Hannity,
Whose words caused such a great fan-any,
He spoke with a big grin,
And folks tuned in just to win,
Now his show's a top-notch celebrity dynasty!
5
There once was a man named Sean Hannity,
Whose views caused much controversy many,
He spoke with such disdain,
Making folks quite insane,
Now folks tune in just to hear his blarney.
GREG GUTFELD
Greg Gutfeld is currently the king of late-night television. His ratings are higher than any
of his competitors. He is an avowed libertarian. He criticizes liberals and democrats much more
frequently than conservatives. It appears that all five of the Limericks view him positively.
1
There once was a man named Greg Gutfeld,
With wit and humor, he never felt short-stocked,
He made folks laugh with glee,
On Fox News, you'll agree,
That his humor's now famous, can't be blocked.
18
2
There once was a man named Greg Gutfeld,
With humor, he'd leave folks bemused and spellbound,
He laughed and he joked,
With wit and a poke,
Now folks tune in just to see him scold.
3
There once was a man named Greg Gutfeld,
Whose humor left folks all bemused,
He laughed and he joked,
Never fearing to provoke,
Leaving folks feeling both entertained and amused.
4
There once was a man named Greg Gutfeld,
Whose wit was sharp, as some have beheld,
With humor and satire,
He entertained with flair,
Making folks laugh, as stories he'd told.
5
There once was a man named Greg Gutfeld,
Whose humor made the folks all giggle and swoon,
With wit sharp as a knife,
He takes on the world with life,
And leaves the audience laughing like a buffoon.
CONCLUDING COMMENTS
The evidence is clear that the AI program is generally slanted to write negative Limericks
about conservatives and positive Limericks about liberals. However, some libertarians and
conservatives have so far escaped or partially escaped the wrath of AI. Whether this bias will
dissipate or get worse in the future remains to be seen.
19
REFERENCES
Charles River Editors. (2018). The Luddites. Create Space.
Chat CPT. (2023). https://chat.openai.com/chat
Irish American Mom. (2018). What is a Limerick? July 7.
https://www.irishamericanmom.com/what-is-a-limerick/
Kim, Sung. (2022). Writing a Novel Using AI – OpenAI ChatGPT. Medum.com. December 9.
https://medium.com/geekculture/write-a-novel-using-ai-openai-chatgpt-c02be59cd937
McGee, Robert W. (2023). Annie Chan: Three Short Stories Written with Chat GPT, February
14. https://ssrn.com/abstract=4359403
Sale, Kirkpatrick. (1995). Rebels against the Future: The Luddites and Their War on the
Industrial Revolution: Lessons for the Computer Age. Addison-Wesley Publishing.
Simplilearn. (2023). ChatGPT Explained in Five Minutes.
https://www.youtube.com/watch?v=3ao7Z8duDXc
Twarog, Adrian. (2023). ChatGPT Tutorial - A Crash Course on Chat GPT for Beginners.
https://www.youtube.com/watch?v=JTxsNm9IdYU
Wikipedia. (2023). ChatGPT. https://en.wikipedia.org/wiki/ChatGPT
Wikipedia. (2023). Limerick (poetry). https://en.wikipedia.org/wiki/Limerick_(poetry) Accessed
February 13.