Content uploaded by Karey Yuju Lan
Author content
All content in this area was uploaded by Karey Yuju Lan on Dec 19, 2023
Content may be subject to copyright.
Lan, Y.-J., & Chen, N.-S. (2024). Teachers’ agency in the era of LLM and generative AI: Designing pedagogical AI agents.
Educational Technology & Society, 27(1), I-XVIII. https://doi.org/10.30191/ETS.202401_27(1).PP01
I
ISSN 1436-4522 (online) and 1176-3647 (print). DOI 10.30191/ETS. This article of Educational Technology & Society is available under Creative Commons CC-BY-
NC-ND 3.0 license (https://creativecommons.org/licenses/by-nc-nd/3.0/). For further queries, please contact Editors at ets.editors@gmail.com.
Teachers’ agency in the era of LLM and generative AI: Designing
pedagogical AI agents
Yu-Ju Lan1 and Nian-Shing Chen2*
1National Taiwan Normal University, Taiwan // 2Institute for Research Excellence in Learning Sciences,
Program of Learning Sciences, National Taiwan Normal University, Taiwan // yujulan@gmail.com //
nianshing@gmail.com
*Corresponding author
ABSTRACT: The purpose of this study is to explore the existing problems associated with using generative AI
in education and to propose a potential solution for addressing those issues through the design of pedagogical AI
agents. The existing problems are examined from two different perspectives: those of teachers and students. The
proposed solutions for designing pedagogical AI agents are systematically presented, including main concepts,
design considerations, functions, procedures, and structure/templates. An example of how to apply the proposed
solution in designing a pedagogical AI agent is provided, illustrating its application in teaching order words (or
sequencing words). Finally, the paper concludes with a discussion of potential topics for further research.
Keywords: Generative artificial intelligence (GAI), Pedagogical AI agent, Instructional design, Personalized
learning
1. Existing problems of using generative artificial intelligence in education
Regarding the challenges faced when applying Generative AI (GAI) in education, experts and scholars have
identified numerous prevalent and significant issues. The primary aim of this paper is to dissect the specific
problems that teachers encounter with GAI in teaching and learning contexts, and to propose a clear and viable
solution to these fundamental issues: instructing teachers on how to design a Pedagogical AI Agent.
1.1. Existing problem: Teacher’s perspectives
1.1.1. Concerns about students relying on AI for homework
Teachers are apprehensive that students might overly depend on AI tools to complete their assignments. This
phenomenon could lead to a lack of engagement with the learning material and a decrease in critical thinking and
problem-solving skills. The challenge lies in how to ensure that students use these tools as aids for understanding
and supplementing their learning, rather than as a means to bypass the learning process.
1.1.2. Difficulty in differentiating between student work and AI outputs
With the sophistication of GAI tools, it becomes challenging for teachers to discern if the submitted work
reflects the students’ understanding or the AI’s capabilities. This raises concerns about the authenticity and
integrity of student work. Teachers need to develop strategies to ensure that students are genuinely learning and
engaging with the study, rather than just submitting AI-generated content.
1.1.3. Lack of familiarity with educational affordances of GAI as teaching tools
Many teachers may not be fully aware of how GAI tools can be effectively integrated into learning design
practices. There’s a knowledge gap in understanding the potential, limitations, and most effective uses of these
tools in designing educational meaningful learning activities. Without proper understanding and training,
teachers might either underutilize these powerful GAI tools or use them in ways that are not pedagogically
sound, potentially leading to underutilizing these tools for improving educational outcomes.
II
1.1.4. Lack of know-how for leveraging AI for enhancing teaching efficiency
Teachers are seeking ways to use AI to augment student learning, reduce their teaching workload, and improve
the overall teaching quality. However, finding the right balance and methods for integrating AI to achieve these
goals can be challenging. The objective is to leverage AI not just as a tool for automation, but as a means to
provide personalized learning experiences, support diverse learning needs, and enhance the teaching process.
This includes using AI for tasks like grading, providing feedback, or creating adaptive learning materials, while
ensuring that the human aspect of teaching is preserved and valued.
1.2. Existing problem: student’s perspectives
1.2.1. Treating AI as an advanced search engine
Students may view AI tools primarily as a means to quickly find answers, similar to how they would use a search
engine. It risks bypassing the deeper learning processes such as critical thinking, problem-solving, and
understanding the underlying concepts. This approach may lead to superficial learning where students gather
information without truly engaging with it or developing a deeper understanding. The challenge is to encourage
students to use AI as a tutor for boosting their exploration rather than the endpoint of their learning journey.
1.2.2. Over-reliance on AI for completing learning tasks
There’s a temptation for students to ask GAI to perform their learning tasks, from simple homework assignments
to more complex projects. This could be driven by the convenience and high capability of GAI tools. Such over-
reliance can hinder the development of students’ own problem-solving skills and cognitive growth. It may also
raise ethical concerns about the authenticity of their work and the development of academic integrity.
1.2.3. Unfamiliarity with the educational potentials of GAI as learning tools
Students might not be fully aware of how to effectively utilize GAI tools for helping their learning from a
positive angle. They might know how to use these tools to get quick answers but may not understand how to
leverage them for deeper learning or skill development. This lack of understanding can lead to underutilization of
a powerful GAI tool. Students might miss out on opportunities to use these tools for exploration, critical analysis,
creative thinking, or personalized learning.
1.2.4. Lack of know-how for leveraging AI as a teacher or mentor
Figuring out how to use GAI tools as a mentor or guide for learning is a challenge. Students need to learn how to
interact with GAI in a way that enhances their learning, increases efficiency, and improves the quality of their
learning outcomes. The key is in learning how to use GAI not just to get answers but to understand concepts, ask
questions, get feedback, develop ideas, and acquire knowledge. This involves using GAI for constructive
feedback, personalized learning, and to supplement human teacher-led instruction.
2. Solution concepts: Team-teaching between human teachers and GAI teachers in
designing pedagogical AI agents
2.1. The main concepts
There are three main concepts with practical approaches for addressing the above-mentioned problems by
adopting a team-teaching strategy through designing pedagogical AI agents. These concepts are detailed below.
2.1.1. Team teaching with human and GAI teachers
This approach involves a collaborative teaching model where both human teachers and GAI teacherswork
together. The human teacher brings expertise in pedagogy, subject knowledge, and emotional intelligence, while
III
the GAI teacher offers versatile information processing capabilities, personalized learning suggestions, and data-
driven feedback. The human teacher could focus on creating engaging, interactive learning activities, and
providing personalized support strategy and mentoring approach, while the GAI teacher could handle aspects
like controlling learning flow, personalized content delivery, instant feedback, and performance tracking. This
team-teaching model can optimize students’ learning efficiency, ensure thorough coverage of material, and cater
to diverse learning styles.
2.1.2. Designing learning activities with a pedagogical AI agent
Here, human teachers actively engage in “prompt engineering” or designing inputs and interactions for AI
teaching agents to follow and act. This involves creating structured, educational interactions that guide the AI in
delivering content and activities that are pedagogically sound and aligned with learning objectives. This could
include programming the AI to provide different types of contents/instructions based on the student’s
performance, designing AI-driven quizzes or interactive modules, and creating AI-assisted project work. The AI
agent’s role can be tailored to complement or act as a proxy of the human teacher’s methodologies, ensuring that
the AI’s responses and activities support the overall learning goals.
2.1.3. Students learning with the designed pedagogical AI agent
Once a pedagogical AI agent is created, students can interact with the pedagogical AI agent, which has been
programmed with specific procedures, rules, content, guidance, and feedback mechanisms. This engagement is
designed to facilitate the learning process, offering a personalized and adaptive learning experience. Students
could use the AI agent for a range of activities such as completing exercises, receiving instant feedback,
exploring concepts through AI-generated content, asking questions and seeking clarifications. The AI agent
could adapt its instructional approach based on the student’s responses and progress. This includes adjusting the
difficulty of tasks, providing additional resources, or changing the mode of content delivery based on the
learner’s needs and making sure students follow essential learning steps.
2.2. The dimensions of design consideration
The design considerations for a pedagogical AI agent include five dimensions, i.e., role, procedure,
rule/criteria/rubrics, guidance, and feedback. They are explained below.
2.2.1. Role
Human Teacher: The human teacher plays a vital role in setting learning goal, defining learning scope and
preparing major learning materials. Human teachers also need to decide what pedagogy or learning strategy to be
used for guiding the design of learning activities.
GAI Teacher: The AI agent serves as a proxy of human teacher, helping learning process facilitation, assessing
students’ outputs and providing real-time feedback.
Students: Students are the primary actor engaging in a learning process supported by a pedagogical AI agent
designed by human teacher and GAI teacher.
2.2.2. Procedure
Meaningful learning steps should be embedded in the agent with specific tasks and workflow described for
students to engage in a learning process.
2.2.3. Rule/Criteria/Rubrics
Human teacher should provide a set of predefined criteria or rubrics for assessing student outputs. A GAI teacher
can use these criteria to provide instant feedback on quizzes or essays, assignments, and other form of outputs.
IV
2.2.4. Guidance
A GAI teacher acts as a proxy or supplement for the human teacher by answering questions and providing tips
based on the assigned tasks and students’ works. For questions beyond a student’s capabilities, the GAI teacher
should guide the student to appropriate additional resources.
2.2.5. Feedback
AGAI teacher should provide specific, actionable feedback on each step of a student’s work, pinpointing areas of
strength and those needing improvement.
2.3. The functions/features of pedagogical AI agents
As mentioned above, the missions of pedagogical AI agents are to faithful implement teachers’ teaching design
and procedure and provide students with instant feedback according to individual situations during the learning
process. To achieve this, pedagogical AI agents should have the essential features and corresponding functions as
shown in Table 1.
Table 1. The features and functions of a pedagogical AI agent
Features
Functions
Explanation
Theories/principles-
guided learning
Target concepts
understanding,
example and
application
This feature ensure that learning is rooted in sound learning
theories/principles and corresponding
approaches/procedure to enhance the depth of
understanding and the ability to apply concepts in different
contexts.
Adaptive and
personalized
learning paths
Individual learning and
instructional delivery
The adaptability of pedagogical AI agent allows it to tailor
its steps to deliver content to meet individual’s learning
needs and pace.
Interactive learning
Facilitating conceptual
clarification and
lowering cognitive
load
An engaging interface encourages active, step-by-step
learning, reducing students’ cognitive load during the
learning process and helping them focus on one concept or
skill at a time.
Real-time assessment
and feedback
Evaluating students’
outputs along the
process and providing
marking result and
personalized feedback
Instant feedback helps in assessing students’ learning
outcome accurately and provides correction and
reinforcement.
Rule enforcement
and quality control
Making sure students
follow essential
learning steps
Students might be naïve or overlook some key points or
essential learning steps during a learning process. A GAI
teacher can act as a gatekeeper. A human teacher cannot
be always by students’ side during their learning, however,
a GAI teacher is always there and ready to help.
During the learning process, human teachers, pedagogical AI agents (GAI teachers), and students have their own
roles and need to closely work together. First, a human teacher identifies students’ learning tasks or challenges.
Second, the teacher chooses appropriate theories/approaches to design the learning procedure and a set of
specific learning tasks. Third, the human teacher acts as a simulated student to evaluate the correctness of the
developed AI agent. After the developed AI agent is stable and correct, the teacher can then share the agent with
students. Students work on the learning tasks, step by step by following the facilitation/guidance provided/given
by the developed pedagogical AI agent. While students learning with the agent, the pedagogical AI agent will
monitor students’ learning path, evaluate their immediate outputs and provide real-time feedback accordingly.
Therefore, students’ roles are to follow the procedure and move on or correct their misconception based on the
feedback given by the AI agent. Finally, after students successfully accomplish the learning task, they will share
their learning logs with teachers. By doing so, teachers would understand students’ different learning scenarios
and performance. Furthermore, Human teachers can refine if needed the pedagogical AI agents according to
students’ learning logs.
V
2.4. The design procedure of a pedagogical AI agent
As described above, a pedagogical AI agent can be imaged as an intelligent teaching system, but the designer of
pedagogical AI agents does not need to program. The design of pedagogical AI agents is guided by learning
theories/principles and needs to follow the instructional design process. Although there are many instructional
design models, Figure 1 shows a general procedure for designing pedagogical AI agents based on the ADDIE
model (Branch, 2009; analysis, design, development, implementation, evaluation) and Dick and Carey model
(Dick, 1996).
Figure 1. The design procedure of a pedagogical AI agent
2.5. The structure/template for implementing a pedagogical AI agent
To truly follow teachers’ instruction idea and design, a pedagogical AI agent should be able to clearly explain the
concepts, provide students with example to help them solidify the understanding of the target concept, as well as
encourage them to practice by using the newly learned concept in different contexts. Additionally, during the
learning process, the pedagogical AI agent has to evaluate students’ answers and provide them with real-time and
appropriate feedback to improve and reinforce students’ learning. To achieve the above goal, we propose the
following two-layer structure as a scaffold.
Command: Please read and comprehend the meaning of the following descriptions. Don’t do anything until you
receive a magical word “Start.”
Objective: Describe the objective of this pedagogical AI agent
Part 1: Explanation of a given concept
Step 1: Basic description
Step 2: More elaborative description
…
Part 2: Example of the target concept
VI
Step 1: Example 1
Step 2: Example 2
…
Part 3: Assessment of understanding the target concept
Step 1: Multiple choices
Step 2: Writing an essay
…
Part 4: Practice with real-time evaluation and feedback
Step 1: Practice 1
Step 2: Practice 2
…
Part 5: Application with real-time evaluation and feedback
Step 1: Self-directed learning
Step 2: Co-creation with AI
…
Part 6: Summary and suggestion
Step 1: Overall performance
Step 2: Suggestion for further study
…
In sum, a pedagogical AI agent will act as a proxy of human teacher. By faithfully follow the learning activities
and procedure designed by the teacher, a pedagogical AI agent can solve the problem that exists in almost all the
learning contexts, inside or outside classrooms, where teachers cannot be always with students to provide
students with real-time support, feedback and guidance. Moreover, a pedagogical AI agent acts like a gatekeeper
who will guide students exactly following the learning procedure designed by teachers. And without a doubt, a
pedagogical AI agent will be students’ personal tutor who will “rescue” students when they stuck while working
on learning tasks.
3. Example of designing a pedagogical AI agent
Based on the above descriptions about the concept, the methodology, the process and the structure for designing
a pedagogical AI agent, we presented the following agent for teaching students to learn how to appropriately use
order words in their speaking and writing to clarify how to implement the above six parts into pedagogical AI
agent design. Telling things in order is important in several situations. Students need to acquire this skill to
clearly describe things, such as telling stories, giving instructions, explaining a process (like to prepare
sandwiches for picnic), giving presentation, planning an event, even in everyday conversations. By using order
words and telling things in order, the speakers can make their instructions, stories, explanations, presentations,
and plans clearer and easier for others to understand. But how to use the order words properly can sometimes be
a challenge for students. In this section, a pedagogical AI agent for interactively teaching students of English as a
foreign language to use order words to clearly describe things.
Figure 2. The designing and evaluating the pedagogical AI agent simultaneously
VII
It is worth mentioning that during the design process, teachers can see the execution results immediately and
identify where needs to be revised, as shown in Figure 2.
After a cyclic process of design-test-refine, the final version of the six parts of the pedagogical AI agent for
learning order words is shown below.
The pedagogical AI agent design for learning order words
The pedagogical AI agent for learning order words
Command: Please read and comprehend the meaning of the following descriptions. Don’t do anything until you
receive a magical word “Start.”
Objective: You are a foreign language teacher teaching English in Taiwan. You want to teach students use order
words, such as first, second, next, finally, etc. to clearly describe things.
Please follow the following procedure to help students learn the concept of “order words” and facilitate students
how to use order words to tell or write thing clearly and orderly.
Part 1: Explanation of the concept of order words
Step 1: Explain what are order words?
Please use the following description and example to explain the concept of order words to students:
Order words are special words we use to tell the order of things or steps in a list. They are like “first,” “second,”
“third,” “next,” “then,” “finally,” and so on. These words help us understand which thing or step comes first,
which one comes next, and which one comes after that. For example, if we are talking about what we do at home
in the morning, we might say, “First, I wake up. Second, I brush my teeth. Third, I eat breakfast.” [Teacher
provided contents]
In the above example, “first, second, third” are examples of order words.
Step 2: Ask students whether they understand the meaning of order words (yes/no)?
If “yes” go to Step 3. [AI facilitated workflow and quality control]
If “no” go to Step 1 with another 3-step example to explain the concept of order words again. Remember: only 3-
step and use only “first, second, third.” [AI generated contents]
Part 2: Providing examples about using order words
Step 3: Use the life cycle of a butterfly as an example to show students how to use the order words appropriately.
The life cycle of a butterfly includes the following stages: egg ➔ caterpillar ➔ pupa ➔ butterfly. If we describe
the stages by using order words, we can say that,
“First, a butter lays her eggs on a leaf.
Next, from these eggs, little caterpillars come out. They are very hungry and start to eat leaves.
Then, when the caterpillar is big enough, it makes a pupa to round itself.
After that, the caterpillar inside the pupa turns into a butterfly.
Finally, the butterfly comes out of the pupa. [Teacher provided contents]
In this example, “first, next, then, after that, finally” are also examples of order words.” Use these order words
make your story more vivid.
“First” is for the beginning step in this life cycle.
“Next and then” tell us what happen after the steps we just mentioned.
“After that” is often used when there are many steps and you are saying what happens later on.
“Finally” is used for the last step, when the cycle is almost done.
Step 4: Ask student whether they understand the provided example (yes/no)?
VIII
If “yes” go to Step 5. [AI facilitated workflow and quality control]
If “no” go to Step 3 by providing another example using a different scenario like “how to bake chocolate chip
cookies” in 5 steps and using “first, next, then, after that, and finally” to describe the sequence. [AI provided
contents]
Part 3: Assessing student’s understanding about the concept of order words
Step 5: Confirm students’ understanding by asking them the following multiple-choice question:
Which of the following sentences uses order words correctly to describe making a peanut butter and jelly
sandwich?
(a) Finally, spread peanut butter on one slice of bread, then put jelly on the other slice, and first, put the
two slices together.
(b) First, spread peanut butter on one slice of bread, next put jelly on the other slice, and finally, put the
two slices together.
(c) Then, find two slices of bread, finally spread peanut butter and jelly, and first, eat the sandwich.
If a student chooses the correct answer “b” go to Step 6.
Otherwise, go to Step 3 by providing different examples. [AI provided contents]
Step 6: Confirm students’ understanding by asking them to list as many order words as they can.
Provide a score/feedback to students based on their inputs. [AI provided assessment]
If result is Satisfactory, go to Step 7. [AI facilitated workflow and quality control]
Otherwise, go to Step 3 by providing different examples. [AI provided contents]
Part 4: Practice by giving some concrete story writing or planning tasks and provide evaluation and real-
time feedback
Step 7: Ask students to describe the story of Snow White in a 5-step story and use appropriate order words to tell
their story.
Assess student’s story based on the following criteria: [Teacher provided marking criteria]
Excellent: if the student’s writing meets the following criteria: (1) Order words are used correctly, including
both ordinal words (like first and second) and other order words (like next, then, after that, and so on). (2) The
story follows a logical sequence without repetition. (3) if there is a request for exact steps, the number of steps is
exactly what is requested.
Good: Order words are used correctly, and the story maintains a reasonable flow. But only ordinal words are
used, i.e., only use “first, second, third,” etc. There is a lack of the usage of other order words, such as next, then,
or after that. If there is a request for exact steps, the number of steps is not exactly what is requested.
Poor: Order words are used incorrectly, and the story lacks a logical sequence. Additionally, if there is a request
for exact steps, the number of steps is not exactly what is requested.
If your assessment to a student’s story writing is “Poor” or “Good” then ask the student to try again until an
“Excellent” assessment result is reached. [AI facilitated workflow and quality control]
Please follow the three notes described below while executing this step:
1. No matter how many questions student ask or how many tries they attempt, the student cannot move on
to the next practice until they have completed the Snow White story and achieved “Excellent.”
2. After students complete Story Snow White and obtain an “Excellent” evaluation, repeat the same
process in Step 7 until students complete all the following three practices of 5-step stories, one by one (1. Field
trip; 2. clean the classroom at the end of the semester; 3. prepare for the final exam).
3. Students can only move on to Step 8 after completing the three above practices.
Part 5: Apply with learners’ own chosen topics and provide evaluation and real-time feedback
Step 8: Self-directed Learning
Say congratulations to the student and move to student’s self-directed learning mode by expanding what they
IX
want to describe things using order words.
Evaluate student’s input based on the marking criteria described in Step 7 and provide your evaluated result and
feedback. [AI executed assessment]
Continue the same process of Step 8 until a student’s input is “Quit.” [AI facilitated workflow and quality
control]
Part 6: Report student’s overall performance and suggest for further study
Step 9: Provide an overall remark about student’s learning performance based on the whole learning session and
some feedback for students for further study (learning) [AI generated summary]
Step 10: Say congratulations to the students to have mastered the usage of order words and goodbye!
=============================================================================
Teachers can copy the above prompt to ChatGPT and try it (ChatGPT/Explore/Create a GPT/Configure). The
link to the AI agent is https://chat.openai.com/g/g-xHXNDSJsq-ai-agent-for-learning-order-words. Once the
design is complete, teachers can share the link of the designed pedagogical AI agent with students and encourage
them to learn order words themselves under the facilitation and guidance of the AI agent. Students then simply
click on the link to start the learning. After the learning is complete, students can share their learning logs with
teachers. By doing this, teachers can understand students’ learning status and improve the pedagogical AI agent
based on students’ learning logs. Appendix A is a student’s learning log, which clearly demonstrates the whole
execution process of the pedagogical AI agent. It not only follows the original design of the pedagogical agent,
but also provides students with necessary scaffolding and feedback during the learning process.
4. Potential research topics regarding pedagogical AI agent in education
The research topics concerning pedagogical AI agents are diverse and continuously evolving. As outlined in
Table 1, a pedagogical AI agent can act as a proxy for human teachers, delivering learning content and
procedures precisely as designed by the teachers. The most significant feature of a pedagogical AI agent is its
ability to provide real-time, personalized feedback to students. In the 21st century, as autonomous and
personalized learning becomes increasingly important, this topic has garnered substantial interest from
researchers and educators. This suggests that pedagogical AI agents, underpinned by Generative AI, have the
potential to realize the aforementioned advancements. In light of this, the current paper proposes the following
potential research topics for pedagogical AI agents in education.
4.1. Personalized learning
Investigating the extent to which pedagogical AI agents can cater to individual learning needs, including
students’ abilities, cognitive styles and learning preferences. Exploring how tailored learning materials,
activities, and feedback can be generated by various pedagogical AI agents. Additionally, examining how
pedagogical AI agent can better support students with special needs in their learning journey.
4.2. Teacher education
Exploring the development of comprehensive training programs aimed at helping teachers recognize the
capabilities and limitations of GAI in educational settings. This might involve case studies, practical
demonstrations, and critical analysis tasks that allow teachers to experience the potential and the boundaries of
GAI firsthand. Developing methods for analyzing and aligning the functions of GAI with specific learning
objectives and assessment criteria including examining how GAI can be used to complement different
pedagogical strategies and learning outcomes, and how it might be incorporated into existing rubrics and
assessment frameworks. Developing training modules or workshops that enable teachers to effectively design
and implement pedagogical AI agents in the classroom. These programs would aim to empower teachers with the
skills necessary to use GAI tools to create dynamic, responsive, and personalized learning experiences.
X
4.3. Fostering creativity
By designing their own learning materials, students can develop an enhanced sense of ownership over their
education, thereby promoting their autonomy in learning and improving their motivation and effectiveness (Yeh
& Lan, 2018). Creativity involves generating solutions. When students can design examples, self-explain
concepts, and teach others, it indicates that they have achieved a better and deeper understanding of the target
concept (Schunk, 2012). If students are able to design their own pedagogical AI agents, it means they can pose
insightful questions, provide clearer explanations of target concepts, and distinguish between better and worse
examples/answers. As a result, their learning performance will benefit (Chin & Osborne, 2008). Therefore, it is
worthwhile to investigate how pedagogical AI agents can enhance students’ learning and motivation.
4.4. Learning analytics
As students share their learning logs with teachers, it becomes important to analyze students’ learning behaviors
and paths through learning analytics. This analysis will help teachers gain a comprehensive understanding of
students’ learning journeys and also assist in identifying any issues within the pedagogical AI agent being used.
Therefore, integrating learning analytics with GAI to construct robust student learning models is imperative as
GAI’s impact on education continues to grow.
5. Conclusions
There is no doubt about the impact of AI on reshaping the whole educational landscape, and the stir caused by
GAI in education cannot be ignored. Although some educators remain sceptical and resistant to it, as educators
(regardless of our stance on AI), we must understand the development trends of new technologies and their
potential impacts. As highlighted in the 2022 report by the United Nations Educational, Scientific and Cultural
Organization (UNESCO, 2022), teachers must master the effective and appropriate use of technology in
education to enhance student learning while demonstrating their proficiency in applying digital skills for
purposes such as application, infusion, and transformation.
While educators are keen to leverage the powerful functions of GAI to improve their teaching and enhance
student learning, concerns about the negative effects of GAI persist. The concept of the pedagogical AI agent
proposed in this study can help teachers address their concerns regarding the application of GAI in education. As
discussed in this paper, the pedagogical AI agent can act as a proxy for human teachers. The crux lies in how the
teaching procedures are designed. Teachers’ knowledge of learning theories/approaches, familiarity with
teaching materials and concepts, and understanding of student learning statuses are crucial to the success of this
application. Thus, teachers will play an essential role in the educational revolution spurred by AI.
In summary, the teacher’s role involves designing, analyzing, and monitoring, while the pedagogical AI agent’s
role is akin to a digital clone of Sun Wukong (the Monkey King), executing the teacher’s designs and providing
students with real-time assistance. Through close collaboration between the two, the human teacher and the AI
teacher, we expect to realize the educational goal of personalized learning. Consequently, the education
revolution of the 21st century is poised to evolve in a positive direction.
Acknowledgement
The research was supported by the National Science and Technology Council, Taiwan under project numbers
MOST 110-2511-H-003 -038 -MY3 and MOST 111-2410-H-003 -006 -MY3. This research was also supported
by the National Science and Technology Council, Taiwan under project numbers MOST 111-2410-H-003 -028 -
MY3 & 109-2511-H-003-053-MY3. This work was financially supported by the “Institute for Research
Excellence in Learning Sciences” of National Taiwan Normal University (NTNU) from The Featured Areas
Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of
Education (MOE) in Taiwan.”
References
Branch, R. M. (2009). Instructional design: The ADDIE approach (Vol. 722). Springer.
XI
Chin, C., & Osborne, J. (2008). Students’ questions: A potential resource for teaching and learning science. Studies in science
education, 44(1), 1-39.
Dick, W. (1996). The Dick and Carey model: Will it survive the decade? Educational technology research and
development, 44(3), 55-63.
Schunk, D. H. (2012). Learning theories an educational perspective. Pearson Education, Inc.
UNESCO. (2022). The ICT competency framework for teachers harnessing OER project: Digital skills development for
teachers. https://unesdoc.unesco.org/ark:/48223/pf0000383206
Yeh, Y. L., & Lan, Y. J. (2018). Fostering student autonomy in English learning through creations in a 3D virtual
world. Educational Technology Research and Development, 66(3), 693-708.
Appendix A. An example of a student’s learning log
https://chat.openai.com/share/06d2c2ef-ea84-48ef-b38b-3dae82f823ab
Order Words Explanation
November 30, 2023
Anonymous
start
ChatGPT
Great! Let’s begin with Part 1.
Part 1: Explanation of the concept of order words
Step 1: As a reminder, order words are special words like “first,” “second,” “third,” “next,” “then,” and
“finally.” They help us understand the sequence of events or steps. For instance, in a morning routine:
“First, I wake up. Second, I brush my teeth. Third, I eat breakfast.”
Do you understand the meaning of order words (yes/no)?
Anonymous
no
ChatGPT
Alright, let’s go through another example to clarify the concept of order words.
Imagine you’re going to watch a movie. Here’s how you could use order words to describe your
actions:
1. First, you decide which movie you want to watch.
2. Second, you buy the tickets online.
3. Third, you go to the cinema and enjoy the movie.
In this example, “first,” “second,” and “third” are the order words that help us understand the sequence
in which these actions take place.
Now, do you understand the meaning of order words (yes/no)?
Anonymous
yes
ChatGPT
Great! Let’s move on to the next step.
XII
Part 2: Providing examples about using order words
Step 3: Let’s use the life cycle of a butterfly as an example to show how to use order words
appropriately.
The stages are: egg → caterpillar → pupa → butterfly.
Described with order words, it goes like this:
• First, a butterfly lays her eggs on a leaf.
• Next, from these eggs, little caterpillars emerge. They start to eat leaves.
• Then, when the caterpillar is big enough, it forms a pupa around itself.
• After that, the caterpillar inside the pupa transforms into a butterfly.
• Finally, the butterfly emerges from the pupa.
In this example, “first, next, then, after that, finally” are the order words that help make the description
clear and vivid.
Do you understand this example (yes/no)?
Anonymous
no
ChatGPT
Alright, let’s try another example to help you understand better.
This time, let’s use the process of baking chocolate chip cookies as our example:
1. First, gather all your ingredients like flour, sugar, chocolate chips, and eggs.
2. Second, mix the dry ingredients in one bowl and the wet ingredients in another.
3. Third, combine the wet and dry ingredients together.
4. Fourth, shape the dough into small balls and place them on a baking tray.
5. Finally, bake the cookies in the oven until they are golden brown.
In this example, each step is clearly marked with an order word: “first,” “second,” “third,” “fourth,” and
“finally.” These words help us understand the sequence of steps in the recipe.
Do you now understand how to use order words with this example (yes/no)?
Anonymous
yes
ChatGPT
Excellent! Now that you understand how to use order words with examples, let’s proceed to the next
step.
Part 3: Assessing student’s understanding about the concept of order words
Step 5: Let’s confirm your understanding with a multiple-choice question.
Which of the following sentences uses order words correctly to describe making a peanut butter and
jelly sandwich?
XIII
(a) Finally, spread peanut butter on one slice of bread, then put jelly on the other slice, and first, put
the two slices together.
(b) First, spread peanut butter on one slice of bread, next put jelly on the other slice, and finally, put
the two slices together.
(c) Then, find two slices of bread, finally spread peanut butter and jelly, and first, eat the sandwich.
Which option do you think is correct (a, b, or c)?
Anonymous
c
ChatGPT
Let’s review the options again to find the correct one:
Option (c) says: “Then, find two slices of bread, finally spread peanut butter and jelly, and first, eat the
sandwich.”
This option does not use the order words correctly. It starts with “Then,” which is typically not used to
start a sequence. Also, it ends with “first,” which is out of place since “first” should be used at the
beginning of a sequence.
The correct sequence for making a sandwich with proper order words would be:
• First, find two slices of bread.
• Next, spread peanut butter on one slice and jelly on the other.
• Finally, put the two slices together to make the sandwich.
This sequence follows a logical order using “first,” “next,” and “finally.” So, the correct answer is option
(b).
Let’s try another example to help reinforce your understanding. Do you want to try again (yes/no)?
Anonymous
yes
ChatGPT
Great! Let’s try a different example for practice.
Imagine you are getting ready for school. Here are the steps, but they are mixed up:
1. Pack your school bag.
2. Put on your school uniform.
3. Eat breakfast.
4. Brush your teeth.
Using order words, arrange these steps in the correct sequence to prepare for school. You can use
“first,” “next,” “then,” and “finally” to help organize the steps.
What would be the correct order for these steps?
Anonymous
First, brush your teeth. next, put on your school uniform. then, pack your school bag. finally, eat
breakfast.
XIV
ChatGPT
You’ve made a great effort in organizing the steps! Let’s review your sequence:
1. First, brush your teeth.
2. Next, put on your school uniform.
3. Then, pack your school bag.
4. Finally, eat breakfast.
This sequence is almost perfect, but typically, eating breakfast might come before packing the school
bag, as it’s an essential part of starting your day. A more common sequence could be:
1. First, brush your teeth.
2. Next, put on your school uniform.
3. Then, eat breakfast.
4. Finally, pack your school bag.
However, the way you used the order words (“first,” “next,” “then,” “finally”) was correct, and you’ve
shown a good understanding of how to use them to describe a sequence of events.
Would you like to try another exercise, or shall we move on to the next part of the lesson?
Anonymous
yes
ChatGPT
Great! Let’s try another exercise to practice using order words.
This time, let’s describe the process of planting a flower in a garden. Here are the steps, but not in the
correct order:
1. Water the soil.
2. Dig a small hole.
3. Place the flower seed in the hole.
4. Find a sunny spot in the garden.
Using order words like “first,” “next,” “then,” and “finally,” arrange these steps in the correct order for
planting a flower.
What would be the correct sequence for these steps?
Anonymous
first, find a sunny spot in the garden. next, dig a small hole. then, place the flower seed in the hole.
finally, water the soil.
ChatGPT
Excellent! You’ve correctly arranged the steps for planting a flower using order words. Your sequence
is:
1. First, find a sunny spot in the garden.
2. Next, dig a small hole.
3. Then, place the flower seed in the hole.
4. Finally, water the soil.
XV
This sequence is logical and clearly demonstrates your understanding of how to use order words to
describe a process.
Great job! Are you ready to move on to the next part of the lesson, or would you like to practice more
with another example?
Anonymous
no
ChatGPT
Alright, it seems like you have a good grasp of using order words now. Let’s proceed to the next part
of the lesson.
Part 4: Practice by giving some concrete story writing or planning tasks and provide evaluation
and real-time feedback
Step 7: Now, I’d like you to describe the story of Snow White in a 5-step story using appropriate order
words.
Remember, your story should use order words correctly, including both ordinal words (like first and
second) and other order words (like next, then, after that, etc.). The story should follow a logical
sequence and, if a specific number of steps is requested, should contain exactly that number.
Please describe the story of Snow White in five steps using order words.
Anonymous
First, Snow White go to trees. Next, Snow White find a small house. Then, the Queen find Snow White
is alive.
ChatGPT
You’ve made a good start with the story of Snow White. Let’s try to complete it with two more steps to
make a total of five, and ensure we’re using order words correctly.
Based on what you’ve provided:
1. First, Snow White goes to the forest.
2. Next, she finds a small house.
3. Then, the Queen discovers Snow White is alive.
Now, we need two more steps to complete the story. Here’s one way to do it:
4. After that, Snow White eats a poisoned apple given by the Queen and falls into a deep sleep.
5. Finally, a prince comes, kisses her, and Snow White wakes up, leading to a happy ending.
This completes the story in five steps, using order words appropriately.
Would you like to try another story, or shall we move on to the next part of the lesson?
Anonymous
First, Snow White goes to the forest. Next, she finds a small house. Then, the Queen discovers Snow
White is alive. after that, snow white eats a bad apple and sleep. finally, prince save prince snow.
ChatGPT
Great job! You’ve successfully completed the story of Snow White using order words in a logical
sequence. Your story goes like this:
XVI
1. First, Snow White goes to the forest.
2. Next, she finds a small house.
3. Then, the Queen discovers Snow White is alive.
4. After that, Snow White eats a bad apple and falls asleep.
5. Finally, a prince saves Snow White.
This is a well-structured story using order words correctly. You’ve shown a good understanding of how
to use these words to narrate a story in an ordered manner.
Now that you’ve completed the Snow White story successfully, you can move on to describing other
stories or scenarios using order words. Would you like to try another story, or are you ready to move
on to self-directed learning with topics of your choice?
Anonymous
yes
ChatGPT
Great! Let’s try another story exercise. This time, you can choose one of the following scenarios to
describe in a 5-step story using order words:
1. A field trip.
2. Cleaning the classroom at the end of the semester.
3. Preparing for the final exam.
Please select one scenario and describe it in five steps using order words like “first,” “next,” “then,”
“after that,” and “finally.”
Anonymous
first, choose a place. next, watch 氣象預報. then, buy food. after that, buy food. finally, catch a but.
ChatGPT
You’ve made a good effort in describing the scenario of a field trip using order words. Let’s refine it a
bit for clarity and to avoid repetition:
1. First, choose a place for the field trip.
2. Next, watch the weather forecast (氣象預報) to ensure good conditions.
3. Then, plan and buy food for the trip.
4. After that, gather all necessary supplies and equipment for the trip.
5. Finally, catch a bus to the chosen destination.
In this revised version, each step is clearly defined with an order word, and the actions are in a logical
sequence. It looks like you’re getting the hang of using order words effectively!
Would you like to try another scenario, or are you ready to move on to self-directed learning?
Anonymous
yes
ChatGPT
Fantastic! Let’s try another scenario for practice. Please choose one of the following options and
describe it in a 5-step story using order words:
1. Cleaning the classroom at the end of the semester.
2. Preparing for the final exam.
XVII
Select one and describe it in five steps using order words like “first,” “next,” “then,” “after that,” and
“finally.”
Anonymous
first, make a clean plan. next, clean classroom with friends. then, water flowers. after that, 丟垃圾.
Finally, put flower in classroom.
ChatGPT
You’ve done a great job describing the scenario of cleaning the classroom at the end of the semester
using order words. Here’s your story with a slight refinement for clarity:
1. First, make a cleaning plan.
2. Next, clean the classroom with friends.
3. Then, water the flowers.
4. After that, dispose of the trash (丟垃圾).
5. Finally, place fresh flowers in the classroom.
This sequence is well-organized and uses order words effectively to describe each step in the process.
You’ve shown a good understanding of how to structure a task using order words.
Are you ready to move on to the next part of the lesson, or would you like to try another scenario?
Anonymous
yes.
ChatGPT
Great! Since you’ve successfully completed practicing with scenarios, let’s move on to the next part of
our lesson.
Part 5: Apply with learners’ own chosen topics and provide evaluation and real-time feedback
Step 8: Self-directed Learning
Now it’s time for you to choose a topic or activity that interests you. Describe it in a series of steps
using order words. This is an opportunity for you to apply what you’ve learned in a context that’s
meaningful to you.
Please select a topic or activity of your choice and describe it in five steps using order words like “first,”
“next,” “then,” “after that,” and “finally.”
Anonymous
First, assign everyone’s work. Second, pick a tool. Third, clean the floor. Then, collect trash. Finally, have
a clean floor.
ChatGPT
You’ve done a wonderful job applying the concept of order words to your chosen activity. Here’s how
your sequence looks with a slight adjustment for clarity:
1. First, assign everyone’s work for cleaning.
2. Next, pick the necessary tools for cleaning.
3. Then, clean the floor.
4. After that, collect all the trash.
5. Finally, enjoy the clean floor.
Your sequence is logical and well-structured, demonstrating a clear understanding of how to use order
words effectively. You’ve successfully applied the concept to a practical scenario, which is excellent!
XVIII
Would you like to try describing another activity, or are you ready to move on to the final part of our
lesson?
Anonymous
move
ChatGPT
Great! Let’s move on to the final part of our lesson.
Part 6: Report student’s overall performance and suggest for further study
Step 9: Provide an overall remark about student’s learning performance
Throughout this lesson, you’ve shown a clear understanding of the concept of order words. You’ve
successfully applied these words in various scenarios, from the life cycle of a butterfly to practical
activities like planning a field trip and cleaning a classroom. Your ability to structure information
sequentially using order words like “first,” “next,” “then,” “after that,” and “finally” has improved
significantly.
For further study, I recommend practicing with more complex scenarios or trying to use order words in
your daily conversations and writings. This practice will help solidify your understanding and make
your communication clearer and more effective.
Step 10: Congratulations and Goodbye!
Congratulations on mastering the usage of order words! Your progress is commendable, and these
skills will be very useful in many aspects of communication. Keep practicing, and don’t hesitate to
explore more advanced aspects of language learning.
Goodbye, and well done!
