Conference PaperPDF Available

Experience is the Best Teacher: Personalized Vocabulary Building Within the Context of Instagram Posts and Sentences from GPT-3

Authors:

Abstract and Figures

Although language learners have different contexts and motivations, sensing personal backgrounds to optimize learning materials has still been challenging. By focusing on the huge movement of Social Networking Services (SNS) such as Instagram, we came up with the idea of utilizing social posts, in particular images, as learning materials. This paper presents our working prototype of the proposed system that extracts keywords from these images and leverages GPT-3 to generate sentences for acquiring new vocabulary around the keywords. By conducting a pilot study involving three users, we found that 2.2 words appeared as unknown words for the user in one generated sentence on average, and there is room for improvement in the proposed system. These findings can be utilized in a large-scale evaluation designed in the future.
Content may be subject to copyright.
Experience is the Best Teacher: Personalized Vocabulary Building
Within the Context of Instagram Posts and Sentences from GPT-3
KANTA YAMAOKA, Osaka Prefecture University, Japan
KO WATANABE, University of Kaiserslautern & DFKI GmbH, Germany
KOICHI KISE, Osaka Metropolitan University, Japan
ANDREAS DENGEL, University of Kaiserslautern & DFKI GmbH, Germany
SHOYA ISHIMARU, University of Kaiserslautern & DFKI GmbH, Germany
An Instagram post Generated sentence from the Instagram post using GPT-3 Users discover new words and learn them by heart
Fig. 1. The proposed method to find new words within the context of Instagram posts and sentences generated from GPT-3
Although language learners have dierent contexts and motivations, sensing personal backgrounds to optimize learning materials
has still been challenging. By focusing on the huge movement of Social Networking Services (SNS) such as Instagram, we came up
with the idea of utilizing social posts, in particular images, as learning materials. This paper presents our working prototype of the
proposed system that extracts keywords from these images and leverages GPT-3 to generate sentences for acquiring new vocabulary
around the keywords. By conducting a pilot study involving three users, we found that 2.2 words appeared as unknown words for the
user in one generated sentence on average, and there is room for improvement in the proposed system. These ndings can be utilized
in a large-scale evaluation designed in the future.
CCS Concepts: Applied computing E-learning;Human-centered computing Social media.
Additional Key Words and Phrases: Context-Aware Language Learning; Personalized Learning; Intelligence Augmentation; GPT-3
ACM Reference Format:
Kanta Yamaoka, Ko Watanabe, Koichi Kise, Andreas Dengel, and Shoya Ishimaru. 2022. Experience is the Best Teacher: Personalized
Vocabulary Building Within the Context of Instagram Posts and Sentences from GPT-3. In Proceedings of the 2022 ACM International
Joint Conference on Pervasive and Ubiquitous Computing (UbiComp/ISWC ’22 Adjunct), September 11–15, 2022, Cambridge, United
Kingdom. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3544793.3560382
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not
made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page. Copyrights for components
of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on
servers or to redistribute to lists, requires prior specic permission and/or a fee. Request permissions from permissions@acm.org.
©2022 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Manuscript submitted to ACM
1
UbiComp/ISWC ’22 Adjunct, September 11–15, 2022, Cambridge, United Kingdom Yamaoka, et al.
1 INTRODUCTION
Our experiences are considered to be stored in the brain as episodic memory. According to research in psychology
and neuroscience elds, applying new information to such existing memory is more eective than doing so with
non-familiar topics [
1
]. Associating new knowledge with existing memory with emotions, interests, or past experiences
is one of the promising ways to learn things eectively while having fun.
In the past, we had few clues to determine such personal backgrounds other than asking questions in person, which is
almost impossible considering the eort it requires. Today, however, social media for sharing photos like Instagram are
widespread among the young generation. People describe their experiences, thoughts, or emotions on such a platform,
along with pictures. These posts are informative in that people post them, especially when they are impressed, touched,
or excited about something in their personal lives. Therefore, such photos can be good sources for sensing or estimating
learners’ interest, his/her targets of emotions, or past experiencesto personalize content to learn and contexts where
the new content can be associated to make the new content easier to remember.
This paper proposes a working-in-progress learning system to generate sentences from learners’ Instagram posts
(see Figure 1). A sentence generated from the system is shown on the right side of the screen. On the left side, a user’s
Instagram picture is shown. Our system is not intended to provide image-to-text captioning. Instead, we want to give an
example sentence likely to relate to the user’s experiences or interests and discover unknown but valuable vocabulary
for their daily situations. By such design, users are supposed to click words in a sentence if they do not know the
meanings, allowing them to show its lemma and lemma’s translation in the native language of the user.
We hypothesized that associating a new vocabulary from one’s personal experience visible on his/her Instagram
posts is one of the promising ways to optimize learning materials. With this idea, we aim to enhance learning
performance by nding and learning new vocabulary within the context as a sentence and an image that is
personalized to their lives. In order to evaluate our hypothesis, we conducted a pilot study including recall tests and
compared the retention ratio compared with two baselines.
2 BACKGROUND AND RELATED WORK
2.1 Context-Aware Language Learning Systems
As many researchers have reported, the context of a learner contains the potential to personalize learning materials.
For instance, Jacquet et al. proposed Vocabulometer, which recommends text based on the user’s vocabulary estimated
from reading activity sensed by eye-tracking devices [
4
]. To extend their study, Yamaguchi et al. proposed Mobile
Vocabulometer, where users choose topics of interest within the application, and it recommends English articles based
on the selections to discover new vocabulary within the context [
6
]. The following topics were provided within
the application, “entertainment, economy, environment, lifestyle, politics, sport, and science. While they provide
preliminary work to adapt interests by user topic selection, our interest is more specic in reality. Take the topic sport,
for example; some people are into baseball while others might like football. Our interests and experiences in daily lives
are diverse; we need to sense such backgrounds, which is another motivation for our study.
Hautasaari et al. proposed an application called VocaBura which allows language learners to acquire vocabulary using
their idle hours [
3
]. According to the study, this audio-based application allows users to nd new vocabulary based on
the user’s location history. While they focus on personalization using location history, we focus on personalization
using social image data.
2
Experience is the Best Teacher: Personalized Vocabulary Building
Within the Context of Instagram Posts and Sentences from GPT-3 UbiComp/ISWC ’22 Adjunct, September 11–15, 2022, Cambridge, United Kingdom
Instagram posts
of a human learner
Multi-label detection model API
Text-to-text model API
Nalural Language Processing API
Content Pool
Syntax info.
e.g., lemma, part-of-speech
Unknown word extraction within generated sentences
along with images by the human learner
Translation API
Images
Labels
Sentences
STARTING FROM HERE
Images
Batch Preprocessing-side
(Content-generation)
Client-side
(Unknown word extraction)
ENDING HERE
Unknown words
List of personalized vocabulary
Web browser interface hosted on the cloud
Sentences
Personalized
dictionary
Fig. 2. System overview of out proposed system.
(i) Our prompt to generate a sentence from keywords extracted from an image
Contents within curly braces were replaced.
Parameter
Value
model
text-davinci-002
temperature
max_tokens
100
top_p
1
frequency_penalty
0
presence_penalty
0
(ii) Some of our parameters for GPT-3
Fig. 3. Summary of GPT-3 configurations, prompt (le) and main parameters (right).
2.2 Text-Generation Using GPT-3
GPT-3 is a text-to-text model where both input and output are text, allowing tasks and few-shot demonstrations by text
interaction according to the original paper where GPT-3 is introduced [
2
]. GPT-3 is learned with 175 billion parameters,
and it often provides natural responses. For example, if a text input often called a prompt "Can I visit Japan from
Germany by foot?" is given to the GPT-3, the model would respond like this: "No, you cannot visit Japan from Germany
by foot. You would need to take a plane, bus, or train." While GPT-3 was announced in 2020, its API is open to everyone
at the end of 2021
1
. Due to its novelty, its potential use cases or usability in an educational context still has room for
investigation. Therefore, our study is exploratory.
3 PROPOSED SYSTEM
As shown in Figure 2, our system comprises multiple steps. Harnessed by cloud computing APIs, our system generates
labels from exported Instagram images, labels to sentences, analyzes generated sentences, and prepares translations.
3.1 Detecting Keyword-Objects from Instagram Image
Given images from learners’ exported Instagram data, we feed them into a multi-label detection API Google Cloud
Vision API, providing up to ten labels per image. Take an example image from Figure 1; the following labels were
detected from the image: Cloud, Skyscraper, Building, Sky, Daytime, Window, etc. We feed these labels into GPT-3 so that
people can discover new words (a) co-occurrences suggested by GPT-3 and (b) detected labels related to the context
strongly tied with the learner’s experiences.
1https://openai.com/blog/api-no- waitlist/
3
UbiComp/ISWC ’22 Adjunct, September 11–15, 2022, Cambridge, United Kingdom Yamaoka, et al.
Find and learn unknown words Distraction Task (Addition Task, 1 min.) Recall Task (Given a translation, spell an unknown word)
Proposed Method C1, Two Baselines: C2, and C3 Evaluation
Fig. 4. Overview of evaluation procedure of our system.
3.2 Generating Sentences by Keywords and GPT-3
We integrated GPT-3’s completion API, which provides sentences to our system with our prompt and keyword labels.
Currently, GPT-3 [
2
] is publicly accessible as a beta version. By nature of the text-to-text model, instructions to the
model are given by sentences. Our prompt is described in Figure 3. Lines 1-3 clarify what the model is expected to do.
Our system inserts keyword labels to the line starting from Keywords. GPT-3 appends a sentence after One-Sentence.
Some work, e.g., [
5
], tries to nd a better prompt. However, we keep our prompt simple as it is not our scope of work.
3.3 Providing Translations for Each Word
After all the sentences are generated, we analyze syntax by using Google Cloud Natural Language API. Through this
process, the API oers morphological analysis capabilities, providing us with the following helpful information for our
system: part-of-speech and lemma. We need lemmas to prevent the system from extracting multiple but essentially the
same words over and over, such as car and cars. Users can click each sentence word within our system to learn them
with standard forms and the corresponding translation of the standard forms.
4 EVALUATION
We conducted a pilot study shown in Figure 4 to evaluate our hypothesis: “Displaying a picture taken by the user
and related to a sentence is useful in exploring unknown words for language learning. Since this hypothesis can be
decomposed into two factors (1: my image or someone’s image, 2: displaying or not displaying an image), the following
three conditions were mixed on the system, and recall ratios of unknown words of each condition were compared.
C1: Displaying an image provided by the user and a sentence generated from the image (Proposed)
C2: Displaying an image provided by someone and a sentence generated from the image (Baseline 1)
C3: Displaying only a sentence generated by an image provided by someone (Baseline 2)
4.1 Experimental Design
We recruited three Japanese undergraduate students who had posted at least 30 images on Instagram. This requirement of
many photos was to collect data on a sucient number of unknown words. As a pre-task of the experiment, participants
submitted photos excluding sensitive data, screenshots, potential copyright issues, or texts as the main content. Pictures
by others for C2 and C3 were provided by the authors. Participants who completed tasks received 1,000 JPY as a reward.
This study has been approved by the ethics committee of Osaka Metropolitan University.
4
Experience is the Best Teacher: Personalized Vocabulary Building
Within the Context of Instagram Posts and Sentences from GPT-3 UbiComp/ISWC ’22 Adjunct, September 11–15, 2022, Cambridge, United Kingdom
P1 examples selected out of 8
P2 examples selected out of 29
P3 examples selected out of 16
1
I am a masterful journalist and can depict daily
life with my advanced vocabulary, whether it's
describing someone's personal style or a new
product on the market.
The wooden structure is an impressive
landmark with its intricate architecture and
symmetrical lines.
The magenta-colored dish was sweet
and delicious, and it was made with
love.
2
The art of cheesemaking is a complex and
delicate process, requiring the utmost care and
precision at every stage.
The brown wood beam is a material
property that has a ceiling of symmetry and
tints and shades of pattern.
The dish was garnished with a leaf
vegetable and served with a seafood
recipe.
3
The food of Germany is as varied as its people,
with each region boasting its own specialties.
The wood cabinetry was a beautiful fawn
color with hardwood floors that matched the
chair perfectly.
The jaw-dropping recipe for the dish
was simple, yet the cuisine was
exquisite.
4
The plant is responsible for the production of
automotive lighting and rolling stock for the
locomotive and freight car industry.
The table was a perfect addition to the
room and really brought the whole space
together. [no unknown words]
The clouds in the sky are like a
painting, with their different shapes
and colors.
5
The sky is dark and the buildings are illuminated
by the car headlights and the tire lights.
The sky is a beautiful blue and there are
some fluffy white clouds.
The temple was crowded with people
enjoying the public space.
[no unknown words]
P1 P2 P3
0
2
4
6
8
Score (maximum 10)
C1
C2
C3
Fig. 5. Results of the pilot study. Le: Random samples of generated sentences where extracted words are highlighted with red. Right:
Recall task scores of all participants.
In the learning phase, participants clicked unknown words to check their meanings. After selecting all unknown
words in one sentence, participants clicked the button to display the next sentence. This process continued until the
number of unknown words reached 10 in three conditions (30 words in total) or all images prepared for C1 were shown.
As in the middle of Figure 4, the system displayed a distraction task after users extracted unknown words from the
three conditions. The simple addition task of three one-digit numbers was carried out for over 1 minute. The next recall
task prompted the user to spell out English words within the user’s unknown word list in a shued order, providing
scores for each condition.
4.2 Results
Figure 5 (left) shows examples of the sentences shown to participants during the experiment. To collect 30 unknown
words (10 for each condition), 8, 29, 16 sentences were required to be displayed, respectively. To put in another form, 2.2
words appeared as unknown words in one sentence on average (
(
30
/
8
+
30
/
29
+
30
/
16
)/
3
=
2
.
2). We found that the ratio
of words extracted per sentence varied among the users depending on their extracted words and their language skills.
We compared submitted answers and expected answers after converting them into lowercase. Scores of the recall
test for each condition and participant are shown in Figure 5 (right). In P1, C1’s score outweighs other conditions. In P2,
scores are the same with C1 and C3. In P3, C3 outweighs other conditions.
4.3 Discussion
Even if we had a limited number of participants, we observed ndings and the following challenges that can be addressed
before conducting a large-scale experiment.
Word diculty and language skill The number of words extracted per sentence varied depending on users. One
of the possible explanations is that we focused on evaluating the proof of concept, and the system does not consider
learners’ English levels. In future research, we need to make our system aware of learners’ English levels [6].
Optimization of GPT-3 parameters In the rst sentence in the case of P1 from Figure 5, we can see that our
prompt, as in Figure 3, is similar to the generated sentence. This type of sentence, i.e., overtting to line 1 of our prompt,
was found only once among all the sentences, including sentences we could not show in Figure 5. Considering the
amount of generated sentences shown to participants until they nished word extraction were 8, 29, and 16 (respectively
for P1-P3), this unwanted behavior is considered negligible. In the future, it is better if we can prepare automatic
guidelines to select proper sentences from the pedagogical point of view because GPT-3 is a novel technology.
5
UbiComp/ISWC ’22 Adjunct, September 11–15, 2022, Cambridge, United Kingdom Yamaoka, et al.
The number of image resources In formal English education, the levels of words are often aligned when learners
before they take exams; therefore, evaluating their performance can be done by simply comparing scores within the
same level, e.g., 3rd grade in high school. In our system, however, extracting words is a dynamic process. Therefore,
comparing answer correctness within similar vocabulary diculties is crucial to evaluate our software. To do so,
increasing the number of trials within an individual is essential. This, however, brings another challenge: Instagram
Posts Are Finite. As a future direction, the learner’s smartphone’s camera rolls instead of Instagram posts. We used
Instagram posts mainly because they reect their experiences, emotion, or interests. In this sense, a tradeo between
sample size and the strength of impressions toward images is inevitable.
5 CONCLUSION
We proposed a working-in-progress learning system to generate sentences from learners’ Instagram posts. We hypoth-
esized that associating a new vocabulary with one’s personal experience is one of the promising ways to optimize
learning materials. We conducted a pilot study with three Japanese students. Our experiment claried that the number of
sentences they needed to extract a certain amount of unknown words varied depending on the participants. To address
the issue, we need to make our system aware of learners’ English levels in the future. Also, our system allows learners’
to extract words, unlike formal education where word levels are aligned. Therefore, our future investigations should
compare scores within similar word groups of diculties. To do so, we need to increase posts from learners. However,
since Instagram posts are nite, it is not an easy task. Therefore, using learners’ camera rolls on their smartphones
instead of their Instagram posts is one of our possible future research directions. While the pilot study did not provide
an answer to our hypothesis, we will further extend our research from the lessons the study supplied.
ACKNOWLEDGMENTS
This work was supported by DFG International Call on Articial Intelligence Learning Cyclotron, JSPS Fostering Joint
International Research (B) (Grant Number: 20KK0235), and JASSO Student Exchange Support Program.
REFERENCES
[1]
Garvin Brod, Markus Werkle-Bergner, and Yee Lee Shing. 2013. The inuence of prior knowledge on memory: a developmental cognitive neuroscience
perspective. Frontiers in behavioral neuroscience 7 (2013), 139.
[2]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry,
Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jerey
Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam
McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models Are Few-Shot Learners. In Proceedings of the 34th International
Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS’20). Curran Associates Inc., Red Hook, NY, USA, Article 159,
25 pages.
[3]
Ari Hautasaari, Takeo Hamada, Kuntaro Ishiyama, and Shogo Fukushima. 2019. VocaBura: A Method for Supporting Second Language Vocabulary
Learning While Walking. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 4, Article 135 (dec 2019), 23 pages.
[4] Clément Jacquet, Olivier Augereau, Nicholas Journet, and Koichi Kise. 2018. Vocabulometer, a Web Platform for Ubiquitous Language Learning. In
Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable
Computers (Singapore, Singapore) (UbiComp ’18). Association for Computing Machinery, New York, NY, USA, 361–364.
[5]
Laria Reynolds and Kyle McDonell. 2021. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. In Extended Abstracts of
the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York,
NY, USA, Article 314, 7 pages.
[6]
Kohei Yamaguchi, Motoi Iwata, Andrew Vargo, and Koichi Kise. 2020. Mobile Vocabulometer: A Context-Based Learning Mobile Application to
Enhance English Vocabulary Acquisition. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing
and Proceedings of the 2020 ACM International Symposium on Wearable Computers (Virtual Event, Mexico) (UbiComp ’20). Association for Computing
Machinery, New York, N Y, USA, 156–159.
6
... Furthermore, when used for text summarization (Phillips et al, 2022), they help synthesize content and improve the student's abstraction capabilities. Their use as learning software in virtual assistants is highly relevant to flexible learning (Wang et al, 2022;Yamaoka et al, 2022). Furthermore, their language intelligence capabilities make them an appropriate tool for code correction (MacNeil et al, 2022b). ...
... The evaluation of the solution relied on a discussion of features like readability and orthographic correctness of the generated text. Unlike the works above, which focused on textual input data, Yamaoka et al (2022) used the gpt-3 model to exploit social media data, particularly from Instagram, for learning purposes. The proposed pipeline comprises (i ) detecting the relevant objects in the images, (ii ) extracting keywords to generate sentences related to those keywords, and (iii ) providing linguist information about the words that composed the sentence. ...
Preprint
Full-text available
Transformer architectures contribute to managing long-term dependencies for Natural Language Processing, representing one of the most recent changes in the field. These architectures are the basis of the innovative, cutting-edge Large Language Models (LLMs) that have produced a huge buzz in several fields and industrial sectors, among the ones education stands out. Accordingly, these generative Artificial Intelligence-based solutions have directed the change in techniques and the evolution in educational methods and contents, along with network infrastructure, towards high-quality learning. Given the popularity of LLMs, this review seeks to provide a comprehensive overview of those solutions designed specifically to generate and evaluate educational materials and which involve students and teachers in their design or experimental plan. To the best of our knowledge, this is the first review of educational applications (e.g., student assessment) of LLMs. As expected, the most common role of these systems is as virtual tutors for automatic question generation. Moreover, the most popular models are GTP-3 and BERT. However, due to the continuous launch of new generative models, new works are expected to be published shortly.
... Furthermore, when used for text summarization (Phillips et al., 2022), they help synthesize content and improve the student's abstraction capabilities. Their use as learning software in virtual assistants is highly relevant to flexible learning (Wang et al., 2022;Yamaoka et al., 2022). Furthermore, their language intelligence capabilities make them an appropriate tool for code correction (MacNeil et al., 2022b). ...
... The evaluation of the solution relied on a discussion of features like readability and orthographic correctness of the generated text. Unlike the works above, which focused on textual input data, Yamaoka et al. (2022) used the GPT-3 model to exploit social media data, particularly from Instagram, for learning purposes. The proposed pipeline comprises (i) detecting the relevant objects in the images, (ii) extracting keywords to generate sentences related to those keywords, and (iii) providing linguist information about the words that composed the sentence. ...
Article
Full-text available
Transformer architectures contribute to managing long-term dependencies for natural language processing, representing one of the most recent changes in the field. These architectures are the basis of the innovative, cutting-edge large language models (LLMs) that have produced a huge buzz in several fields and industrial sectors, among the ones education stands out. Accordingly, these generative artificial intelligence-based solutions have directed the change in techniques and the evolution in educational methods and contents, along with network infrastructure, towards high-quality learning. Given the popularity of LLMs, this review seeks to provide a comprehensive overview of those solutions designed specifically to generate and evaluate educational materials and which involve students and teachers in their design or experimental plan. To the best of our knowledge, this is the first review of educational applications (e.g., student assessment) of LLMs. As expected, the most common role of these systems is as virtual tutors for automatic question generation. Moreover, the most popular models are GPT-3 and BERT. However, due to the continuous launch of new generative models, new works are expected to be published shortly.
... In this paper, we mainly explain E2, the large-scale study, because E2 has more participants and improvements in the design, which we explain in the next section. It is noted that the initial report of E1 and results were elaborated on our previous work [29]. This paper is an extension of the previous work. ...
Article
Full-text available
Psychological studies highlights the importance of combining new knowledge with one's prior experience. Hence personalization for a learner plays a key role for vocabulary acquisition. However, this faces two challenges: probing a learner's experiences in their lives and crafting tailored material for every different individual. With the prevalence of visual social media, such as Instagram, people share their photos from favorite moments, providing rich contexts, and emerging generative AI would create learning material in an effortless fashion. We prototyped an online vocabulary exploration system, which displays a learner's selected photos from their Instagram along with a generated sentence using image recognition and a language model, GPT-3. The system lets a learner to find new words that are strongly tied to their daily life with the approximated context. We carried out our within-subject design evaluation of the system with 23 participants with three conditions: contexts grounded with learner's Instagram photos, contexts grounded from general images, and text-only modality. From learners' recall tasks accuracy, we found that having a context grounded with a learner's social image allowed them to find difficult words to quickly learn than having context generated by someone's image, or text only modality-although this finding is statistically insignificant. The Zipf frequency comparisons revealed that generally having image-based context allowed learners to extract difficult vocabulary than having text-only context. We also discuss quantitative and qualitative results regarding participants' acceptance of the personalization system using their personal photos from social media. Generally, they reported positive impressions for our system such as high engagement. While our system prioritizes user privacy with opt-in data control and secure design, we explore additional ethical considerations. This paves the way for a future where personalized language learning, grounded in real-world experiences and generative AI, benefits learners. INDEX TERMS Context-aware language learning, HCI, large language models, generative AI.
... Moreover, smartphone applications can adapt their services to individual learners, making language learning more efficient [8][9][10]. Personalizing learning materials and the notification timing is gradually becoming common [11][12][13]. ...
Article
Full-text available
Vocabulary learning is essential for language study, and it is crucial to identify unknown words for users. Conventional English vocabulary learning methods specify words to be learned based on the frequency of vocabulary use and cannot suggest unknown words tailored to individual learners. In this study, we propose a novel smartphone-based unknown English word estimating system called VocabMe. We leverage learner-independent textual features and learner-dependent reading activity behavior data from smartphone sensors. We combined reading time, smartphone acceleration, scroll speed, and rereading patterns to reflect the learners’ reading comprehension abilities. The results showed that the proposed method, including smartphone sensors, outperformed the accuracy of unknown English word estimation in 75 out of 79 participants with different cultural backgrounds, compared to the conventional method of estimating unknown words from texts. This study shows that mobile technology for personalized language learning opens new possibilities for learners to engage more effectively in English vocabulary acquisition.
... Since cloud computing provides web-based solutions and thereby does not require the purchase and maintenance of costly hardware, it can be a cost-effective choice for a GPT model. By utilizing cloud computing, the GPT model can only pay for the computing resources it uses [56]. The other added advantage of cloud computing in GPT is, it gives GPT models the freedom to access computing resources whenever it wants, from any location in the world. ...
Article
Full-text available
The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward the development of machines that can understand and communicate using language in a manner that closely resembles that of humans. GPT is based on the transformer architecture, a deep neural network designed for natural language processing tasks. Due to their impressive performance on natural language processing tasks and ability to effectively converse, GPT have gained significant popularity among researchers and industrial communities, making them one of the most widely used and effective models in natural language processing and related fields, which motivated to conduct this review. This review provides a detailed overview of the GPT, including its architecture, working process, training procedures, enabling technologies, and its impact on various applications. In this review, we also explored the potential challenges and limitations of a GPT. Furthermore, we discuss potential solutions and future directions. Overall, this paper aims to provide a comprehensive understanding of GPT, its enabling technologies, their impact on various applications, emerging challenges, and potential solutions.
... These aspects, coupled with the recent publication dates of most papers, suggest a rapidly growing interest in Generative AI applications in education, setting a promising trajectory for future research endeavors. [30]. Such applications underscore the transformative role of Large Language Models in catering to the unique educational needs and preferences of each learner. ...
Preprint
Full-text available
This paper critically examines the transformative role of Generative Artificial Intelligence (AI), specifically Large Language Models like ChatGPT and Google Bard, in the realm of education. It delves into the evolution of Generative AI, its current applications in learning environments, and its potential future impact. The study highlights how Large Language Models are reshaping educational methodologies, offering new perspectives on teaching and learning, while also addressing the critical challenges they pose. Key issues such as AI bias, ethical concerns, and the risk of undermining traditional educational values are explored. Furthermore, the paper investigates the balance between technological advancement and the maintenance of personalized, inclusive learning experiences. Through an analytical approach, it aims to provide a comprehensive understanding of the benefits and pitfalls of integrating Generative AI in educational settings, offering insights and recommendations for future research and practical application in this rapidly evolving field.
... Chat GPT will also assist them with vocabulary. ChatGPT has the remarkable ability to aid students in their quest for lexical enrichment [34] through various engaging activities. By offering comprehensive explanations, alternate expressions, and vivid illustrations of unfamiliar terms, this exceptional language companion enables students to broaden their lexicon enjoyably and interactively. ...
Article
In the ever-evolving landscape of education, integrating innovative technologies can enhance the learning experience for students. ChatGPT, a cutting-edge language processing tool developed by OpenAI, offers exciting possibilities for teaching writing. This advanced AI model can be a powerful asset in the classroom, providing students with valuable resources and support as they develop their writing skills. Seventy-three college students participated in the quasi-experiment. The findings demonstrate that AI chatbot-based instruction reduces students' anxiety about learning English writing. AI chatbots offer instant feedback, allowing students to correct errors immediately. This quick feedback loop can prevent students from ruminating over their mistakes, thus reducing anxiety. With AI chatbot, students can learn at their own pace. They can take time to understand concepts, practice writing, and receive feedback without feeling rushed. This flexibility can alleviate the pressure of strict deadlines in traditional classroom settings. The findings imply teachers to implement chatbot-based learning in the classroom.
... Personalization is one of the key concepts in the field of knowledge transfer. Hence, understanding what participants know or experience is crucial to discover patterns of decoding [37]. The level of understanding may vary depending on the person's domain knowledge. ...
Conference Paper
Full-text available
Knowledge Transfer is one of the essential principles in education. The teacher’s knowledge is encoded through speech and writing and transmitted to the student. The student then decodes the transmitted information according to individual capabilities and absorbs it as knowledge. This paper presents an approach to accelerate knowledge transfer using sensor technology and social-cognitive states. So far, we have worked on quantifying meeting discussions, analyzing lecture studies, and estimating domain knowledge from web browsing. The contribution of this study is to estimate the degree of achievement of knowledge transfer and to accelerate it based on the estimated results
Article
Full-text available
The integration of Artificial Intelligence (AI) into applied (linguistics) has sparked significant interest due to its transformative potential in language teaching, learning, and research. With the advent of AI language models such as ChatGPT and GPT-4, a comprehensive understanding of their role and impact in the field is imperative. This study provides an extensive content analysis of existing literature on the application of AI, especially focusing on GPT models, to identify how these tools are influencing linguistic scholarship and practice. A systematic review of 73 scholarly articles was conducted, using a content analysis design to categorise literature based on their perspective towards AI in linguistics—whether supportive, opposing, or mixed. Statistical analyses, including a non-parametric one-way ANOVA and Chi-square tests, were utilized to discern patterns and correlations within the data. The review revealed a diverse range of applications and viewpoints. While some studies reported the efficacy of AI in language pedagogy and research, others pointed out challenges such as ethical concerns and the quality of AI-generated content. The results were synthesised into a comprehensive table, detailing each study’s aim, findings, position, and the specific area of linguistic application. The findings suggest that while AI models like ChatGPT possess considerable promise for enhancing applied and linguistic tasks, their deployment must be tempered with ethical considerations and a commitment to maintaining content quality and authenticity. This study underscores the need for guidelines to navigate the ethical use of AI in (applied) linguistics and highlights the importance of digital competencies for educators and researchers in the field.
Article
Full-text available
Across ontogenetic development, individuals gather manifold experiences during which they detect regularities in their environment and thereby accumulate knowledge. This knowledge is used to guide behavior, make predictions, and acquire further new knowledge. In this review, we discuss the influence of prior knowledge on memory from both the psychology and the emerging cognitive neuroscience literature and provide a developmental perspective on this topic. Recent neuroscience findings point to a prominent role of the medial prefrontal cortex (mPFC) and of the hippocampus (HC) in the emergence of prior knowledge and in its application during the processes of successful memory encoding, consolidation, and retrieval. We take the lateral PFC into consideration as well and discuss changes in both medial and lateral PFC and HC across development and postulate how these may be related to the development of the use of prior knowledge for remembering. For future direction, we argue that, to measure age differential effects of prior knowledge on memory, it is necessary to distinguish the availability of prior knowledge from its accessibility and use.
Article
Learning a new language is difficult and time-consuming. Apart from dedicated classroom study, second language (L2) learners often lack opportunities to switch their attention to vocabulary learning over other daily routines. In this work, we propose a method that enables L2 learners to study new vocabulary items during their dead time, such as when commuting to school or work. We developed a smartphone application, VocaBura, which combines audio learning with location-relevant L1-L2 word pairs to allow users to discover new vocabulary items while walking past buildings, shops and other locations. Our evaluation results indicated that Japanese beginner level English learners were able to retain more vocabulary items with the proposed method compared to traditional audio-based study despite being less aware of L2 vocabulary acquisition having occurred. In our second study, we report on the level of English vocabulary coverage for L2 learning achievable with our proposed method. We discuss several design implications for educational technologies supporting second language learning.
Conference Paper
The Vocabulometer is a reading assistant system designed to record the reading activity of a user and to extract mutual information about the users and the read documents. The Vocabulometer stands as a web platform and offers services for analyzing which word is read at which time, estimating the reading skill of the user, the level of difficulty of a document, and recommending reading materials according to the user's vocabulary. For now the system is based on eye tracking but in the future we plan to make a smartphone application which do not need additional hardware.
Ilya Sutskever, and Dario Amodei. 2020. Language Models Are Few-Shot Learners
  • Tom B Brown
  • Benjamin Mann
  • Nick Ryder
  • Melanie Subbiah
  • Jared Kaplan
  • Prafulla Dhariwal
  • Arvind Neelakantan
  • Pranav Shyam
  • Girish Sastry
  • Amanda Askell
  • Sandhini Agarwal
  • Ariel Herbert-Voss
  • Gretchen Krueger
  • Tom Henighan
  • Rewon Child
  • Aditya Ramesh
  • Daniel M Ziegler
  • Jeffrey Wu
  • Clemens Winter
  • Christopher Hesse
  • Mark Chen
  • Eric Sigler
  • Mateusz Litwin
  • Scott Gray
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models Are Few-Shot Learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS'20). Curran Associates Inc., Red Hook, NY, USA, Article 159, 25 pages.