Content uploaded by Shalin Hai-Jew
Author content
All content in this area was uploaded by Shalin Hai-Jew on Oct 22, 2017
Content may be subject to copyright.
Exploring the
Google Books Ngram Viewer
for “Big Data” Text Corpus
Visualizations
SHALIN HAI-JEW
KANSAS STATE UNIVERSITY
SIDLIT 2014 (OF C2C)
JULY 31 –AUG. 1, 2014
Presentation Overview
As part of the Google Books digitization project, the Google Books Ngram Viewer
(https://books.google.com/ngrams) was released in late 2010 to enable public querying of a “shadow dataset”
created from the tens of millions of digitized books. The texts are from a 500-year span (1500-2000+), with new texts
added fairly continuously, and there are a range of datasets of different text corpuses (and in different languages,
like Italian, French, German, Spanish, Russian, Hebrew, and simplified Chinese). The name of the tool comes from a
computer science term referring to strings of alphanumeric terms in particular order: a unigram (or one-gram)
consists of one entity, a bigram (or two-gram) consists of two entities, onwards. (Its precursor was a prototype
named “Bookworm.”) Users may acquire the (de-contextualized) word or phrase or symbol frequency counts of
terms in books—which provide a lagging indicator of trends (over time), public opinion, and other phenomena.
The Ngram Viewer has been used to provide insights on diverse topics such as the phenomena of fame (and the
fields which promote fame), collective forgetting, language usage, cultural phenomena, technological
innovations, and other insights. The data queries that may be made with this tool are virtually unanswerable
otherwise. The enablements of the Google Books Ngram Viewer provide complementary information sourcing for
designed research questions as well as free-form discovery. This tool is also used for witty data visualizations (such
as simultaneous queries of “chicken” and “egg” to see which came first) based on the resulting plotted line chart.
The tool also enables the download of raw dataset information of the respective ngrams, and the findings are
released under a generous intellectual property policy. This presentation will introduce this semi-controversial tool
and some of its creative applications in research and learning.
2
3
Welcome!
Hello! Who are you?
Any direct experiences with n-grams? Any research angles that may be
informed by n-grams?
What are your interests in terms of the Google Books Ngram Viewer? What
is your level of experience with this Viewer?
4
What is the
Google Books
Ngram Viewer?
A SIMPLE OVERVIEW
5
History
Google Books project (conceptualized from 1996, official secret launch in
2002, announcement of “Google Print” project in 2005, new user
interface in 2007, known also as Google Book Search)
Over 100 million digitized books from 1500s to the present
Book Search interface available in 35 languages
Over 10,000 publishers and authors from 100+ countries in the Book Search
Partner Program
Integrated with Google Web Search
Public domain works in full view, copyrighted works with snippets (“Google
Books”)
6
History (cont.)
Derived shadow dataset: Bookworm Ngrams -> Ngram Viewer
Based on a “bag of words” approach
Launched in late 2010
Google Books Ngram Viewer prototype (then known as “Bookworm”)
created by Jean-Baptiste Michel, Erez Aiden, and Yuan Shen…and then
engineered further by The Google Ngram Viewer Team (of Google
Research)
7
History (cont.)
Includes a number of corpuses across
many languages (finer details of each
corpus)
Current corpuses
American English 2012, American
English 2009
British English 2012, British English 2009
Chinese 2012, Chinese 2009
English 2012, English 2009
English Fiction 2012, English Fiction 2009
English One Million
French 2012, French 2009
German 2012, German 2009
Hebrew 2012, Hebrew 2009
Spanish 2012, Spanish 2009
Russian 2012, Russian 2009
Italian 2012
8
Some Terminology
Digital humanities: Research from computation and disciplines in the
humanities
Big data: Datasets with an “n of all,” large datasets with a large number of
records (such as in the millions)
N-gram: A contiguous sequence of n items from text or speech (unigram,
bigram / digram, trigram, four-gram, five-gram, and so on), often
representing a concept
Text corpuses: Collections of text such as manuscripts or microblogging
streams or other texts
Shadow dataset: Masked or de-identified extrapolated data
9
Some Terminology (cont.)
Frequency count: The number of times a particular n-gram appears in a
text corpus
Smoothing: The softening of spikes by averaging data from preceding and
following years to indicate a “moving average” (a smoothing of 1 means
a datapoint on the linegraph is the average of the count for 1 year to
each side; a smoothing of 2 means the average of the count for 2 years to
each side, etc.)
Data visualization: The image-based depiction of data for the
identification of patterns
10
N-grams
Number of Ngrams
Examples (comma
-separated)
Unigram
or one-gram
time, $, Julia, pi, 3.14159265359
Bigram or digram or two
-gram
borrow money, return home, déjà vu,
golden mean
Trigram
her own purse, the
trip abroad
Four
-gram
the time it took, the Merchant of
Venice
Five
-gram
when he left the store, after she fed
the dog
Six
-gram
I plan to travel very soon
Seven
-gram
The president
will visit the state
tomorrow
11
<3.14159265359>
12
A “Shadow Dataset” of Google Books
Collections
Shadowing the dataset to protect against privacy infringement or reverse
engineering of manuscripts
The machine extraction of n-grams from the de-contextualized texts
Pure frequency counts of the n-grams in various sequences (but only if they
reach a threshold of ngrams that appear in 40 or more books to keep the
processing manageable)
Tagging of parts of speech (POS) that structure language but do not hold
semantic value
The elimination of unique phrase sequences to avoid potential hacking and
reverse-engineering of particular texts
13
A “Shadow Dataset” of Google Books
Collections (cont.)
Depiction of frequency counts over time (with defined and editable start-and-
end years) for broad-scale trending
Ability to compare multiple words and phrases
Value Added Capabilities
Downloadable n-gram datasets (for further analysis)
Interactive visualizations from mouseovers
Machine-highlighted years of interest
Linkage to original texts (on Google)
Choices from dozens of different and multilingual corpuses (French, simplified
Chinese, Italian, Russian, Spanish, Hebrew, German, and others)
14
Anomalous Years of Interest; Links
15
<foot and mouth disease, FMD, hoof
and mouth disease>
16
Downloadable Experimental Datasets
17
Some Controversies with the Ngram
Viewer
Decontextualized machine “analysis” vs. contextualized reading and
human expertise
Machine “(non)reading” (in frequency counts) vs. human reading
(symbolic decoding), a quantitative vs. a qualitative focus, an
overbalance into computational understandings (a quantity of words
separated from conscious expressed meaning and author hand)
Example from Karen Reimer’s “Legendary, Lexical, Loquacious Love” (1996), a
deconstructed book which consisted of lists of alphabetized contents (per
Uncharted…)
Inability to verify results outside of Google Books Ngram Viewer
Some degree of elusive “black box” lack of knowledge about functionality
18
19
Research Potential?
On first blush, what do you think can be learned from such data
extractions? Why?
What can be asserted from the linegraphs?
Would publishers ever accept a deconstructed book for publication (or do
you think these are mainly one-offs??
20
A Cursory Overview of Research
Findings So Far from Ngram Viewer
Fame: Who gets famous, and how? What sorts of professions lead to
fame?
Collective memory: In terms of human memories of events, how long do
people tend to remember? What is the trajectory of collective
consciousness from knowing to not knowing?
Adoption of innovations: What is the typical time length for people to
accept technological and other innovations? How do cultural
phenomena affect human populations over time?
Language evolution: How does language evolve over time? How do rules
of language become normalized?
21
A Cursory Overview of Research
Findings So Far from Ngram Viewer
(cont.)
First-use of terms: When was the time when a term was first used? (such as
terminology linked to technological innovations) (a form of fact-checking)
Popularity: Between various artists / scientists / politicians, who was more
popular in his / her day?
Government Censorship: What was the role of Nazi censorship of certain
Jewish artists in terms of their reputations and mentions / non-mentions in
the literature?
22
Some Examples USING GENERAL DATA
EXTRACTIONS
23
Single Extraction
<dimsum>
24
Two Element Comparison
<dimsum,tapas>
25
Two Element Comparison (cont.)
<future,past>
26
Multi-Element Comparisons and Contrasts
<height,weight> <diet,exercise>
27
Year(s)
<1984, 1999, 2010>
28
Symbols
<&, ampersand>
29
Phrases: Multiword Strings
<crossing the Rubicon>
30
(Somewhat
More)
Advanced
Searches
•WILDCARD EXTRACTIONS
•INFLECTION SEARCHES
•CASE SENSITIVITY / CASE
INSENSITIVITY
•ACCESSING TAGGING (AS FOR
PARTS OF SPEECH)
•RICHER COMBINATIONS
•SOME BOOLEAN-BASED
QUERIES
•STARTS AND ENDS OF
SENTENCES
•DEPENDENCY RELATIONS
•ROOTS OF PARSE TREES
31
Wildcard Search *
The use of * to stand in for a word so the Ngram Viewer will display the top
ten substitutes (of most common stand-in words) for the asterisk
32
<the * of his life>
33
<the * of her life>
34
<a betrayal of the *>
35
Inflection Search with _INF
Differentiation of various word forms
(infinite verb form)
-ed
-ing
-s
(irregular spellings)
36
<tell_INF>
37
Case Sensitive / Insensitive Searches
Case Sensitive
Capitalizations and lower cases
matter
Case Insensitive
Capitalizations and lower cases do not
matter
Case insensitivity will result in a variety
of capitalization / lower case mixes
and variations for a particular search
term
38
Case Sensitive
<RICO>
39
Case Insensitive
<RICO>
40
Part-of-Speech Tags
Disambiguation of term to defining its usage as a part-of-speech to
capture the conceptual usage
May be used as stand-alones (_VERB_) or appended to a verb (play_VERB)
41
42
TAGS
APPLICATION
_NOUN_
Noun
_VERB_
Verb
_ADJ_
Adjective
_ADV_
Adverb
_PRON_
Pronoun
_DET_
Determiner or article
_ADP_
Adposition (preposition or
postposition)
_NUM_
Numeral
_CONJ_
Conjunction
_PRT_
Particle
_ROOT_
Root of the parse tree
_START_
Start of a sentence (sentence
boundary)
_END_
End of a sentence (sentence
boundary)
<play_NOUN, play_VERB>
43
Some Combinations
Inflection keyword with part-of-speech text
buy_INF, buy_VERB_INF (buy, buying, bought, buys)
Dependencies with wildcards
ride=>*_NOUN (ride car; ride bike; ride bus)
44
<buy_INF, buy_VERB_INF>
45
<ride=>*_NOUN>
46
Ngrams at the Starts and Ends of
Sentences
Sentence Boundary Indicators
_START_
_END_
47
<_START_ The USSR,_START_ The US>
48
Dependency Relations with =>
Operator
main noun => descriptor (the main noun dependent on the descriptor)
49
<home=>sweet>
50
<apocalypse=>zombie>
51
<_ROOT_=>win,_ROOT_=>lose>
53
Ngram
Compositions
USING OPERATORS
( )
54
Operators ( )
Operators
Functions
+
sums expressions on the left and the
right to combine
multiple ngram time series into one line in the linegraph
(can combine multiple added sequences)
-
subtracts expression on right from the left (need spaces on
either side of the minus sign)
/
divides string on left by expression on right
*
multiplies expression on left by number on right to compare
ngrams of different frequencies with entire ngram in ( ) so
the asterisk is seen as a multiplication sign and function
:
applies ngram on left to the text corpus on the right to
enable comparisons against different corpuses
55
+ operator
<Russian Federation, Russia, USSR,
RussianFederation+Russia+USSR>
56
- operator ( + ) –( + )
<breakfast,lunch,dinner,breakfast+lunch+dinn
er,(breakfast+lunch+dinner)-(snacks)>
57
/ operator
<traumatic brain injury, TBI, mTBI,(traumatic
brain injury / (traumatic brain injury+TBI+mTBI)>
58
* multiplication operator
<HUMINT,GEOINT,MASINT,OSINT,SIGINT,TECHINT,(CYBINT*
1000),DNINT,(FINNT*1000)>
To explore visualizations between texts with widely varying frequencies
The Data Extractions
HUMINT,GEOINT,MASINT,OSINT,SIGINT,TECHINT,CYBINT,DNINT,FINNT
HUMINT,GEOINT,MASINT,OSINT,SIGINT,TECHINT,(CYBINT*1000),DNINT,(FINNT*
1000) (to no avail since “CYBINT” and “FINNT” do not have sufficient
occurrences to count in the Ngram Viewer)
59
60
: operator
<Beijing:eng_gb_2012,Beijing:chi_sim_2012,
Beijing:eng_us_2012>
Beijing:eng_gb_2012,Beijing:chi_sim_2012,Beijing:eng_us_2012
Some definitions of the pointed-to datasets
eng_gb_2012: “Books predominantly in the English language that were
published in Great Britain”
chi_sim_2012: “Books predominantly in simplified Chinese script”
eng_us_2012: “Books predominantly in the English language that were
published in the United States”
61
62
Two Steps to Privacy: 隐私
63
Popularization of Term with Broader
Interactions Globally
<隐私>
64
Any Ideas for Using the Ngram Viewer
for your Fun, Work, and Research?
Fun
•Stories to tell friends
Work
•Insights that may affect
knowledge and decision-
making
Research
•Citable information from the
world’s collective knowledge
conveyed through books…
65
Ngram Viewer
Applications for
Visual Wordplay
and Wit
A ONE-SCREEN TEXT-
BASED VISUAL TO
ACCENTUATE A
WEBSITE,
PRESENTATION, OR
PUBLICATION?
66
<d’oh>
67
Chicken or the Egg?
<chicken,egg>
68
Five Elements
<metal,wood,water,fire,earth>
69
Good and Evil
<good,evil>
70
<Buddhism, Christianity, Hinduism,
Islam,Judaism, Mormonism, Sikhism>
71
<Oriental, Asian>
72
Ivy League Institutions
<Brown University, Columbia University, Cornell University,
Dartmouth College, Harvard University, Princeton
University, the University of Pennsylvania, Yale University>
73
Caveat
The ease of accessing and understanding the visualization may mean a
potential misunderstanding of the underlying information… This is partly a
product of the cognitive bias known as the “availability heuristic,” with
more easeful and faster ideas coming to mind accepted as truth.
Visualizations like this are highly overly simplified as compared to the
underlying realities.
Researchers need to make sure to head off potential misunderstandings
with such Ngram Viewer linegraph visualizations when using these as
“accents” or as “invitations” to people to learn more.
74
Ngram Viewer
Uses in Research EARLY THOUGHTS
75
Some Possible Research Applications
of the Ngram Viewer
Variations of the following:
Competition between languages and phrases (their origins and trajectories /
trends over time, word and phrase gists over time, multilingual queries, and
others)
Cultural understandings and cross-cultural insights; popular sentiment and
understandings
Analysis of research capabilities and understandings (historically and through
the present)
Population readiness for accepting particular ideas through big data text
corpus analysis
Literary terms of art and their uses over time
76
Some Possible Research Applications
of the Ngram Viewer (cont.)
Effects of historical events (governance, social phenomena, wars, health issues,
and others) on language
Biographical insights on historical figures (particularly comparative insights)
Research lead creation; research source identification
and many others
77
78
Qualifiers and Clarifications
Words in books as a lagging (vs. leading) indicator
Changing authorship and access to literacy and publication over time
(with the changing roles of books from years of formalism to much less
formalism in the present day)
Word frequency counts as one information stream among many
Still a critical role for close readings of select publications
Ngram Viewer counts are much more effective and informative when
used with complementary streams of information and in-depth analysis
79
Particular Researcher Requirements
General
Understandings
Language literacy, optimal
multilingual literacy
Digital (and computational)
literacy
Understandings of history
Understandings of the changing
roles of authors and books
Understandings of big data and
big data analyses
Domain +
Computational
Understandings
Deep knowledge of particular
domain fields and related fields
Understandings of language uses
and publications in-the-field
Computational set thinking
Uses of the Ngram
Viewer Tool
Openness to discovery learning
Knowing what to query and how
(particularly with creative query
setups, year adjustment
parameters, links to documents)
Knowing what may be asserted
and what may not be asserted
(ability to qualify assertions)
Knowing when to conduct
complementary and follow-on
research (including close
readings)
80
Initial Ngram Viewer Tips
Start simple. Once the basic extractions are acquired, try the more
complex ones using tagging and combinatorial approaches. Broaden out
to foreign languages.
Reload the Ngram Viewer if a “flatline” is attained because “under heavy
load, the Ngram Viewer will sometimes return a flatline”.
Text corpuses accessed by the Ngram Viewer are always changing, and
more data is added all the time. It may help to capture a sequence of
data extractions to what changes there may be.
81
Tips on Research Approaches
Shape a data query both for need-to-know and to the limits of the massive
dataset.
Err on the side of making a number of various runs for a data query. Keep
good records of the data extractions.
Take time to actually analyze the results. Sometimes, because the
extractions occur in milliseconds, just making a cursory look at the
linegraph seems sufficient…but much more can be learned by interacting
with the visual. (One can explore years, resources, and other aspects, for
example.)
Keep a research journal of observations and findings. Note your learning
about the tool as well.
82
Tips on Research Approaches (cont.)
Spend some time to discover the tool by making various runs purely for
discovery learning. Structure some of these explorations with thought
experiments.
Branch out beyond the Ngram Viewer by analyzing the extracted datasets
in other tools. One freeware and open-source one is the Ngram Statistics
Package.
Also, use other datasets (such as from social media platform-extracted big
data corpuses) for analysis. One such publicly available set is the Rovereto
Twitter N-Gram Corpus.
83
Tips on Research Approaches (cont.)
There is a broad and wide literature on the machine analysis of human
language: natural language processing, stylometry, computational
linguistics, sentiment analysis, personality analysis, speech recognition, and
others. There are automated text summaries (with efforts towards
accuracy and “grammaticality”). There are language models used for
speech recognition and machine translation between languages. A core
unit underlying these approaches are n-grams. It may help to delve more
deeply for certain types of research to more fully contextualize research-
based approaches.
84
References
Aiden, E. & Michel, J.-B. (2013). Uncharted: Big Data as a Lens on Human
Culture. New York: Riverhead Books.
Mayer-Schönberger, V. & Cukier, K. (2013). Big Data: A Revolution that will
Transform How We Live, Work, and Think. New York: Houghton Mifflin
Harcourt Publishing Company.
86
