ArticlePDF Available

Impact of Chatbots on Student Learning and Satisfaction in the Entrepreneurship Education Programme in Higher Education Context

Authors:

Abstract and Figures

There are many ways to learn how to be entrepreneurs and one of the powerful ways is to learn from successful entrepreneurs. However, it is difficult to reach and interview those entrepreneurs about their best practices in doing business in real lives. Chatbot technology can come into play in mimicking conversation of successful entrepreneurs and providing pre-programmed responses of their best practices drawn from interviews published in newspapers, books and articles. Therefore, this research aimed to examine the impact of chatbots in the form of successful entrepreneurs with 24 first-year graduate students, who enrolled in a master's degree of entrepreneurship education at Kasetsart university. Data analysis involved mean, standard deviation, frequency, percentage, and content analysis. The research findings showed that the developed chatbots were appropriate at a very high level (Mean= 4.75, S.D. = 0.22). The impact of chatbots was positive. Students perceived that their learning was better and their satisfaction was at a very high level (Mean = 4.65, S.D. = 0.44) with thoughts that chatbots were an interesting, innovative, and fun teaching way. This study indicated that chatbot technology positively impacted student learning and satisfaction. It can be implemented as a powerful tool to teach entrepreneurship in entrepreneurship education programmes in higher education context.
Content may be subject to copyright.
International Education Studies; Vol. 15, No. 6; 2022
ISSN 1913-9020 E-ISSN 1913-9039
Published by Canadian Center of Science and Education
15
Impact of Chatbots on Student Learning and Satisfaction in the
Entrepreneurship Education Programme in Higher Education Context
Patchara Vanichvasin1
1 Faculty of Education, Kasetsart University, Bangkok, Thailand
Correspondence: Patchara Vanichvasin, Faculty of Education, Kasetsart University, Bangkok, Thailand.
Received: May 1, 2022 Accepted: August 9, 2022 Online Published: November 21, 2022
doi:10.5539/ies.v15n6p15 URL: https://doi.org/10.5539/ies.v15n6p15
Abstract
There are many ways to learn how to be entrepreneurs and one of the powerful ways is to learn from successful
entrepreneurs. However, it is difficult to reach and interview those entrepreneurs about their best practices in doing
business in real lives. Chatbot technology can come into play in mimicking conversation of successful
entrepreneurs and providing pre-programmed responses of their best practices drawn from interviews published in
newspapers, books and articles. Therefore, this research aimed to examine the impact of chatbots in the form of
successful entrepreneurs with 24 first-year graduate students, who enrolled in a master's degree of
entrepreneurship education at Kasetsart university. Data analysis involved mean, standard deviation, frequency,
percentage, and content analysis. The research findings showed that the developed chatbots were appropriate at a
very high level (Mean= 4.75, S.D. = 0.22). The impact of chatbots was positive. Students perceived that their
learning was better and their satisfaction was at a very high level (Mean = 4.65, S.D. = 0.44) with thoughts that
chatbots were an interesting, innovative, and fun teaching way. This study indicated that chatbot technology
positively impacted student learning and satisfaction. It can be implemented as a powerful tool to teach
entrepreneurship in entrepreneurship education programmes in higher education context.
Keywords: impact, chatbots, student learning, satisfaction, entrepreneurship education
1. Introduction
Entrepreneurs play a key role in promoting economic development. They and their businesses can generate jobs
and wealth, which help reduce social and regional inequality (Westhead & Wright, 2013). Entrepreneurial
development is expected to boost the economy by creating more jobs and generating new incomes (Song et al.,
2021). Teaching entrepreneurship is therefore fundamental for building stronger economy, better income and more
flexible societies. As such, it is important to include entrepreneurship teaching in the educational system (Klapper,
2004) due to the fact that entrepreneurship education contributes to the development of business for entrepreneurs
(Souitaris et al., 2007).
There are many ways in teaching entrepreneurship and one of the powerful ways is to learn from successful
entrepreneurs. Interviewing with entrepreneurs is the learning way used in teaching entrepreneurship (Solomon et
al., 2002) and considered as a good starting point to find out what it takes to be successful in businesses. However,
it is quite difficult to reach and interview successful entrepreneurs about their best practices in doing business in
real lives due to their busy schedules and lack of business network. Chatbot technology can come into play in
mimicking human conversation of successful entrepreneurs and providing pre-programmed responses of best
practices in doing business drawn from interviews published in newspapers, books and articles, which can be used
as learning tasks to teach entrepreneurship and equip students with right knowledge about how to be entrepreneurs
from experiences and perspectives of successful entrepreneurs in the form of chatbots like real humans. With
chatbot technology, learning entrepreneurship from the best is made possible by turning successful entrepreneurs
into chatbots in order that students can learn from the best.
A chatbot (also known as bot) is defined as a software program, which acts like a human when interacted with via
message or voice over the internet to simulate a conversation in a scripted way, understand human languages (Xu
et al., 2017) and reply to a conversation automatically (Benotti et al., 2014). The requirements of chatbot design
include accurate knowledge representation, strategy for answering, and predefined responses to reply when user
input is not understood (Augello, Gentile, & Dignum, 2018) or not in a system. Although developing chatbot
requires technical expertise (e.g., machine learning and conversation design) that is different from developing
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
16
traditional software systems, the use of chatbot framework and platform provided by Microsoft, Facebook, Google
and LINE application nowadays makes the chatbot development become much easier (Abdellatif et al., 2020).
Two types of chatbots were mostly used. Firstly, a chatbot with pre-programmed answers is called a rule-based
chatbot as it is responsible for matching pre-programmed answers with text messages or speeches inputted. It
chooses the system response from pre-programmed answers, based on the input text without creating new texts.
The responses from the chatbot were coded, organized and presented in the format of conversational patterns.
Secondly, a chatbot generating suitable responses is called a machine learning-based chatbot as it is responsible for
generating suitable responses drawn from user input via natural language processing and deep machine learning.
Therefore, it can generate answers better than the first type, based on user messages because it acts like human by
using algorithms of machine learning and techniques of deep learning (Hien et al., 2018). Concerning chatbot
design, scalability and usability can be major issues since they have a direct impact on the user experience of a
chatbot. Scalability refers to the way a chatbot design handles the increase of users, interactions, and content
contained in the chatbot, and usability refers to the actual usability of a chatbot design and if users are able to
perform the desired tasks (de Andrade et al., 2022).
Chatbots are useful in applications (Shawar & Atwell, 2007), which can be used as a tool to learn or study a new
language, access an information system, visualize the contents of a corpus; and give answers to questions (Bayan,
2005). Benefits involve saving costs, time, and effort (Abdellatif et al., 2020), increasing interaction, active
learning, and creative learning (Bii, 2013), promoting retention and engagement (Benotti et al., 2014), motivating
students to learn (Benotti et al., 2014), and enhancing user satisfaction (Vanichvasin, 2021).
As a result, chatbot technology has very high potential to use in various contexts. However, there is still limited
research on chatbots in education, with most of the research focusing on health and language learning (Nilros &
Lidén, 2020). Although chatbots have been tested in education to some extent with language learning being one of
the few fields where chatbots have been adopted and widely used (Winkler & Söllner, 2018; Fryer, Nakao, &
Thompson, 2019), there are not many chatbots used in higher education context to support students in their
learning process. In Thailand, researches were conducted in different contexts such as customer service, medical
service, hotel service, tourism and entertainment but there is still limited research in education, especially in
teaching and learning. Offering student services by chatbots were most popular topics of research studies.
Therefore, this research aims to focus on higher education context with educational intentionality because chatbots
can create a learning environment where students use them to increase the quality of learning and provide
individual support when the classrooms get bigger and individual support becomes more challenging (Winkler &
Söllner, 2018). This study is also built on the researcher’s previous research (Vanichvasin, 2021) but with different
chatbot framework and platform.
Due to very high potential of chatbot technology and only few studies on chatbots in education in higher education
context in Thailand, this research study tends to use chatbot technology to examine the impact of student learning
and satisfaction. It is anticipated that the developed chatbots can be used to teach entrepreneurship in
entrepreneurship education programme with positive impact of student learning and satisfaction.
2. Literature Review
A chatbot (also known as bot) is defined as a software program, which acts like a human when interacted with via
message or voice over the internet to simulate a conversation in a scripted way, understand human languages (Xu
et al., 2017) and reply to a conversation automatically (Benotti et al., 2014). The requirements of chatbot design
include accurate knowledge representation, strategy for answering, and predefined responses to reply when user
input is not understood (Augello, Gentile, & Dignum, 2018) or not in a knowledge base. Although developing
chatbot requires technical expertise (e.g., machine learning and conversation design) that is different from
developing traditional software systems, the use of chatbot framework and platform provided by Microsoft,
Facebook, Google and LINE application nowadays makes the chatbot development become much easier
(Abdellatif et al., 2020). Two types of chatbots were mostly used. Firstly, a chatbot with pre-programmed answers
is called a rule-based as it is responsible for matching pre-programmed answers with text messages or speeches
inputted. It chooses the system response from pre-programmed answers, based on the input text without creating
new texts. The responses from the chatbot were coded, organized and presented in the format of conversational
patterns. Secondly, a chatbot generating suitable responses is called a machine-learning based chatbot as it is
responsible for generating suitable responses drawn from user input via natural language processing and deep
machine learning. Therefore, it can generate answers better than the first type, based on user messages because it
acts like human by using algorithms of machine learning and techniques of deep learning (Hien et al., 2018). Some
chatbot framework and platform operates both types of chatbots.
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
17
Concerning chatbot design, scalability and usability can be major issues since they have a direct impact on the user
experience of a chatbot. Scalability refers to the way a chatbot design handles the increase of users, interactions,
and content contained in the chatbot, and usability refers to the actual usability of a chatbot design and if users are
able to perform the desired tasks. For rule-based chatbots, the developer needs to list all possibilities in advance
and the chatbot does not work if the input message differs from the predefined patterns, because the generalization
is very limited. The conversation context is only understood if the developer programs it directly and objectively.
For machine-learning chatbots, they are trained on the datasets called knowledge bases (Agarwal & Wadhwa,
2020). The user does not need to send messages exactly as they are in the knowledge base, because, after training
the chatbot, it can generalize new messages. The conversation context is comprehended the same way if the
knowledge base contains conversation flows. The course of action is determined based on the probabilities of an
incoming message being an intention registered on the knowledge base. The action chosen is the one with the
highest probability or none if the probabilities are not high enough (Andrade et al., 2022).
In educational settings, chatbots can be categorized into those with educational intentionality and those without.
Chatbots without educational intentionality are used in administrative tasks such as student guidance and
assistance (Fernoagă, 2018). Chatbots with educational intentionality are used in fostering teaching and learning.
Within this category, there are chatbots, which provide the framework of the learning process, that is, select and
arrange contents to fit the students’ needs and speed, and help in reflection and learning motivation. These bots act
as a learning companion which provides dialogue, collaboration and reflection (Molnár & Szüts, 2018).
Radziwill & Benton (2017) extracted quality attributes from each of the 32 papers and 10 articles, and grouped
them based on similarity. After two or three iterations, they noticed that in general, they were aligned with the ISO
9241 concept of usability, that is, 1) effectiveness, 2) efficiency and 3) satisfaction with which specified users
achieve specified goals in particular environments (Abran et al., 2003). In particular, effectiveness refers to the
accuracy and completeness with which specified users achieve their goals such as performance while efficiency
refers to how well resources are applied to achieve those goals such as functionality and humanity (Radziwill &
Benton, 2017). Satisfaction refers to affect, ethics and behavior, and accessibility. As a result, effectiveness,
efficiency, and satisfaction can be used as a checklist to design, develop, and evaluate chatbots.
3. Theoretical Background
The theoretical background was based from chatbot technology and its quality attributes covering effectiveness,
efficiency, and satisfaction. This study was conducted by using chatbot technology and the interaction between
chatbots and students in one of the entrepreneurship education programmes in higher education context. The
adoption of chatbot technology promoted interaction in 2 ways, that is, through pre-programmed answers
(Rule-based chatbot) and suitable answers (Machine-learning based chatbot). Concerning chatbot design,
scalability and usability have a direct impact on the user experience of a chatbot. Scalability is how chatbots handle
the increase of users, interactions and content contained in chatbots. Usability is how chatbots perform tasks that
users desire. In this study, chatbots were designed with scalability to handle users, interaction, and content of best
practices from successful entrepreneurs and usability to perform assigned learning tasks. In educational settings,
chatbots can be categorized into those with educational intentionality and those without. There were many quality
attributes to evaluate chatbots, that is, effectiveness, efficiency, and satisfaction. Prior research findings reported
the high impact of learning and satisfaction when using chatbot technology. In this study, Dialogflow was
employed as chatbot framework as it operated both types in order to support learning with education intentionality.
Student learning and satisfaction were used as learning outcomes. Therefore, student learning and satisfaction
were measured to see how chatbot technology supported learning in the selected entrepreneurship education
programme in higher education context.
4. Objectives
This study sought to answer two research questions: 1) What do chatbots look like? and 2) How students benefit
from the quality attributes of chatbot technology? through the following objectives:
1) To design and develop the chatbots of successful entrepreneurs
2) To examine the impact of the chatbots from student perceptions of learning and satisfaction
This will add to the literature on the impact of using chatbot technology to support teaching and learning in
education settings.
5. Method
The researcher used the quasi-experimental research design with mixed methods. The quantitative research was
concerned with the applicability of chatbots, and students’ perceptions of their satisfaction towards the chatbots.
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
18
The qualitative research was concerned with experts’ recommendations towards the chatbot improvement,
students’ perceptions of their learning, and suggestions towards the chatbot technology. The research was
conducted in 3 stages: 1) the chatbot and research instrument design, 2) the chatbot and research instrument
development, and 3) the chatbot implementation to examine the impact of the chatbots with a reflective journal of
student learning and a satisfaction questionnaire.
Stage 1 was concerned with reviewing literature of chatbot technology and studying best practices of successful
entrepreneurs from interviews published in newspapers, books and articles in order to use their best practices in
designing chatbots and research instruments.
Stage 2 was concerned with the chatbot and research instrument development. Dialogflow was used as chatbot
framework with Thai as a main language. LINE application was used as a platform to communicate with students.
Interactions with students and content contained in chatbots were designed in the form of questions and answers
like interviewing successful entrepreneurs about their best practices in doing business. Then, chatbots were created
with educational intentionality to support teaching and learning. Experts evaluated the applicability of chatbots
from key elements of chatbot technology. After making changes according to experts’ comments, a learning task
was assigned to students to ask and interact with chatbots through texts. Default answers were given to continue
interviewing when students sent a sticker, an emoji or unmatched texts. The researcher built a knowledge base to
contain content of best practices in doing business from successful entrepreneurs. Three different answers were
stored in the knowledge base to make chatbots interact like real humans, not repeated answers. Once the
knowledge base was completed, the researcher integrated Dialogflow with LINE application and named two
chatbots as 1) Masked Entrepreneur and 2) Unmasked entrepreneur. Then, quality attributes were examined.
Student learning and satisfaction were selected as quality attributes to measure the impact of chatbots in
educational settings. Relevant instruments to measure student learning and satisfaction were developed. Students
were encouraged to add two chatbots as friends to Line application and started to interview them about how
successful entrepreneurs succeeded in doing their business. After using chatbots, students were distributed a
reflective journal and a satisfaction questionnaire in order to measure how chatbots impacted their learning and
satisfaction.
Stage 3 was concerned with the chatbot evaluation and verification of relevant research instruments. 3 experts
were used in this stage. The pilot study of 20 non-targeted students was also included. Feedbacks from experts and
results from the pilot study were collected in order to analyze and use data to improve the chatbots. Next, the
chatbots were implemented with 24 first-year graduate students, who enrolled in a master’s degree of
entrepreneurship education programme at Kasetsart university, to examine the impact of the chatbots on student
learning and satisfaction.
5.1 Population and Sampling Procedures
The population was 30 graduate students under business education and entrepreneurship education curriculum at
the Faculty of Education, Kasetsart university. Only 24 first-year graduate students, who enrolled in a master’s
degree of entrepreneurship education programme, were chosen by using a purposive sampling technique as they
had to study entrepreneurship under entrepreneurship education curriculum.
5.2 Instruments
Two chatbots were designed and developed to examine the impact of the chatbots on student learning and
satisfaction with combination of 3 research instruments, that is, an evaluation questionnaire of the chatbot
applicability, a reflective journal of students learning, and a student satisfaction questionnaire. Details were given
below.
The researcher designed and developed two chatbots from reviewing the literature of chatbot technology and best
practices of successful entrepreneurs drawn from interviews published in newspapers, books and articles. 3 experts
used an evaluation questionnaire to examine the chatbot applicability. The pilot study of 20 non-targeted students,
was conducted.
The first instrument was an evaluation questionnaire related to the chatbot applicability from the key elements of
chatbot technology. It consisted of 3 aspects: content knowledge with 3 statements, content presentation with 4
statements, and benefits of chatbots with 5 statements. Therefore, 12 items of statements with a 5-point Likert
scale (from lowest to highest) with 1 open-ended question were generated. The evaluation criteria for average
scores (Mean) were set. 4.51-5.00 means highest level. 3.51-4.50 means higher level. 2.51-3.50 means average
level. 1.51-2.50 means lower level, and 1.00-1.50 means lowest level. The acceptable level of the chatbot
applicability was set at 4.51-5.00. The research instrument was then distributed to 3 experts to seek their opinions
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
19
of the chatbot applicability before testing with targeted students. The researcher then reviewed scores results and
utilized recommendations to improve the chatbots.
The second instrument was a reflective journal of student learning according to their perceptions in 3 areas, that is,
key learning points for success as entrepreneurs, application of key learning points, and reflection on learning. All
areas consisted of open-ended questions to identify student learning. The instrument was then distributed to
students to check their perceptions of learning from two chatbots. The item-objective congruence (IOC) index
ranged between 0.67-1.00 according to experts’ opinions.
The third instrument was a satisfaction questionnaire based on using chatbot technology to teach entrepreneurship.
It covered 4 areas: content of best practices from successful entrepreneurs with 5 statements, communication about
best practices from successful entrepreneurs with 3 statements, application of chatbots as learning tasks to teach
entrepreneurship with 4 statements, and benefits of chatbots in teaching entrepreneurship with 5 statements. 17
items of statements with a 5-point Likert scale (from lowest to highest) and 1 open-ended question for students to
provide thoughts on the use of chatbots were generated. The evaluation criteria for average scores (Mean) were set.
4.51-5.00 means highest level. 3.51-4.50 means higher level. 2.51-3.50 means average level. 1.51-2.50 means
lower level, and 1.00-1.50 means lowest level. The research instrument was then distributed to students to check
their perceptions of satisfaction, feedback and comments toward the use of two chatbots. The item-objective
congruence (IOC) index ranged between 0.67-1.00 from experts’ opinions. The Cronbach Alpha coefficient value
was 0.92 from the pilot study of 20 non-targeted students.
5.3 Data Collection
The researcher, as an instructor, asked 3 experts to examine the chatbot applicability and assigned learning tasks to
students to interview two successful entrepreneurs in the form of chatbots to find out who they were and what took
them to be successful entrepreneurs. Students were asked after completing the learning tasks to reflect on their
learning through a reflective journal and rate their satisfaction towards the use of chatbots including feedback and
recommendation on their uses. Finally, all instruments were checked for completeness and accuracy before
analyzing data.
5.4 Data Analysis
Data were obtained and analyzed from all research instruments, that is, an evaluation questionnaire of the chatbot
applicability, a reflective journal of student learning, and a student satisfaction questionnaire. Data of the chatbot
applicability, and students’ perceptions of their satisfaction were analyzed. A 5-point Likert scale (lowest to
highest) was applied to examine the chatbot applicability from experts’ opinions and rate the perceptions of
satisfaction from students’ opinions while data from all open-ended questions were collected to analyze experts’
recommendations, students’ perceptions of their learning, and suggestions for the use of chatbot technology. All
data were analyzed by mean, standard deviation, frequency, percentage, and content analysis.
6. Results and Discussion
The key findings of this research were given below:
6.1 The Chatbot Development
The researcher designed and developed two chatbots from reviewing the literature of chatbot technology and best
practices in doing businesses from successful entrepreneurs drawn from interviews published in newspapers,
books and articles. Two chatbots were both rule-based and machine learning-based types that generated as suitable
responses as possible from the pre-programmed answers with the ability to operate 24 hours while handling all
students at the same time. Students could use their own language to communicate with two chatbots and interview
as many times as they preferred by typing questions, keywords or texts that they wanted to know via smart phones,
tablets or computers.
The chatbots were developed by Dialogflow and implemented on LINE application that students were accustomed
to without need to install any extra application. Two chatbots were built, that is, the unmasked entrepreneur and the
masked entrepreneur, to act as two successful entrepreneurs and converse with students through texts, pictures and
stickers like real humans. Students were assigned a learning task to interview two chatbots of successful
entrepreneurs about their best practices in doing businesses; find out who they were, and what it took for them to
be successful. Pre-programmed and suitable responses when typing questions, keywords or texts were provided
without waiting time. Examples of two chatbots were shown in Figure. 1.
ies.ccsenet.
3 experts
w
applicabil
i
Tab le 1.
M
Ch
Co
n
1.
C
2.
C
3.
C
Co
n
4.
P
5.
P
6.
P
7.
P
Be
n
8.
C
9.
C
10.
11.
12.
To
t
Table 1 ill
u
mean and
high level
s
experts re
c
necessary
students.
T
more attra
chatbots b
org
w
ith extensiv
e
i
ty and recom
m
M
ean & standa
r
atbot Applicabilit
y
tent Knowledge
C
ontent is releva
n
C
ontent is in the s
C
ontent of entrep
r
n
tent Presentatio
n
P
resentation in th
e
P
resentation in an
P
resentation in a
v
P
resentation 24 h
o
n
efits of Chatbots
C
hatbots are easy
C
hatbots can be u
s
Chatbots are a g
o
Chatbots provid
e
Chatbots inspire
t
al
u
strated that t
h
standard devi
s
(Mean = 4.5
commended
a
changes to th
e
T
he results w
e
ctive. Picture
s
efore using w
i
Figure 1. Ex
a
e
knowledge a
n
m
ended the ch
a
r
d deviation o
f
y
n
t to best practice
s
ame language stu
d
r
eneurship is suit
a
n
e
human convers
a
interesting form
a
v
ariety of devices
o
urs
to use at anytime
s
ed many times a
s
o
od learning reso
u
e
responses witho
u
students to be en
t
h
e overall ave
r
ation of cont
e
6, S.D. = 0.3
8
a
dding more
c
e
chatbots fro
m
e
re positive wi
t
s
and stickers
w
i
th the targete
d
Internatio
n
a
mples of two
n
d experience
a
tbot improve
m
f
the chatbot a
p
s
from successful
e
dents use, which
i
a
ble
a
tion format
at
and anywhere
s
preferred
u
rce of best practi
c
u
t waiting time
t
repreneurs
r
age was at a
v
e
nt knowledge
,
8
, Mean = 4.8
3
c
onversational
m
experts’ opin
t
h recommen
d
w
ere added ac
c
d
students.
n
al Education St
u
20
chatbots of su
c
in the field o
f
m
ent. The app
l
p
plicability
e
ntrepreneurs
i
s informal and e
a
c
es from success
fu
v
ery high level
, content pres
e
3
, S.D. = 0.17
words to act
ions. The chat
b
d
ation to add
m
c
ording to the
u
dies
c
cessful entre
p
f
education an
d
l
icability of c
h
a
sy to understand
fu
l entrepreneurs
(Mean = 4.75
e
ntation, and
b
, Mean = 4.7
8
more like re
a
b
ots were the
n
m
ore pictures
a
non-targeted
s
p
reneurs
d
technology
v
h
atbots were il
l
𝑋
S.D.
5.00 0.00
4.33 0.58
4.33 0.58
4.67 0.58
4.67 0.58
5.00 0.00
5.00 0.00
5.00 0.00
4.67 0.58
4.67 0.58
5.00 0.00
4.67 0.58
4.75 0.22
, S.D. = 0.22).
b
enefits of ch
a
8
, S.D. = 0.38
,
a
l humans. T
h
n
pilot tested
w
a
nd stickers t
o
s
tudents’ opin
i
Vol. 15, No. 6;
v
erified the c
h
l
ustrated in Ta
b
Interpretation
Very high
High
High
Very high
Very high
Very high
Very high
Very high
Very high
Ve r y H i g h
Very high
Ve r y H i g h
Very high
In each aspe
c
a
tbots were at
,
respectively)
h
e researcher
m
w
ith 20 non-tar
g
o
make the ch
a
i
ons to impro
v
2022
atbot
b
le 1.
t, the
very
.
The
m
ade
g
eted
a
tbots
v
e the
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
21
6.2 Students’ Perceptions of Learning
The results of the students’ perceptions of learning were demonstrated in Tables 2, 3 and 4. Similar comments by
students were summarized. Tables 2, 3 and 4 showed the students perceptions of learning on key learning points
for success as entrepreneurs, application of key learning, and reflection on learning, respectively.
Table 2. Percentage & frequency of key learning points for success as entrepreneurs
Key Learning Points for Success as Entrepreneurs Frequency Percentage
1. How to solve problems and make decisions 17 70.83
2. How to create changes with creativity and innovation 13 54.16
3. How to build connection in and outside organization 10 41.66
4. How to improve oneself all the times 6 25.00
5. How to start business with passion 5 20.83
6. How to learn business by doing business 3 12.50
Table 2 showed the key learning points for success as entrepreneurs. 70.83% or 17 students learnt that solving
problems and making decisions made entrepreneurs successful. 54.16% or 13 students learnt that creating changes
with creativity and innovation made entrepreneurs successful. 41.66% or 10 students learnt that building
connection in and outside organizations made entrepreneurs successful. 25.00% or 6 students learnt that improving
oneself all the times made entrepreneurs successful. 20.83% or 5 students learnt that starting business with passion
made entrepreneurs successful. 12.50% or 3 students learnt that learning by doing made entrepreneurs successful.
Table 3. Percentage & frequency of application of key learning points
Application of Key Learning Points Frequency Percentage
1. Taking more reasonable risks to create changes with creativity and
innovation 15 62.50
2. Learning from mistakes, problems and decisions 14 58.33
3. Finding what one loves to do 12 50.00
4. Planning business with tools and technology 7 29.16
5. Taking business ideas to actions 5 20.83
6. Looking for networking opportunities 4 16.66
Table 3 showed the application of key learning points. 62.50% or 15 students learnt that they should take more
reasonable risks to create changes with creativity and innovation. 58.33% or 14 students learnt that they should
learn from mistakes, problems and decisions. 50.00% or 12 students learnt that they should find what they love to
do. 29.16% or 7 students learnt that they should plan business with tools and technology. 20.83% or 5 students
learnt that they should take business ideas to actions. 16.66% or 4 students learnt that they should look for
networking opportunities.
Table 4. Reflection on student learning
Students Reflection on Learning
Student 1 “I liked it. I felt that I really talked to real entrepreneurs to find out who they were so I had to concentrate and thought
carefully what to ask”
Student 2 “I liked it. I needed to plan questions and think carefully on answers received so that I could ask proper questions to find
out who were the successful entrepreneurs that I was talking to.
Student 3 “I liked the way chatbots led me to think and analyse identities of entrepreneurs from answers”
Student 4 “I liked learning with fun”
Student 5 “I liked learning that could happen at anytime and anywhere”
Student 6 “I liked learning that was interesting and colourful”
Student 7 “I liked learning through questioning and analysing knowledge gained from answers”
Student 8 “I liked this new way of learning entrepreneurship through chatbots”
Student 9 “I liked it because I had opportunities to practice questioning, thinking, reading, analysing and synthesizing”
Student 10 “I didn’t like this way of learning because some answers didn’t match my questions”
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
22
Student 11 “I liked it because I gained knowledge from questions I asked”
Student 12 “I liked active learning and I thought that I learnt by doing the learning tasks”
Student 13 “I liked it because I learnt instantly and enjoyed my own learning.
Student 14 “I liked it although I found it hard to concentrate on my learning tasks.
Student 15 “I liked learning with no boundaries and freedom to learn the way I questioned”
Student 16
Student 17
Student 18
Student 19
Student 20
“I liked fast learning that could happen 24 hours”
“I liked learning at my convenient time, space, and pace. I also enjoyed answers from questions that I inputted as
responses from chatbots were not the same everytime”
“I liked this new learning channel”
“I didn’t like it because chatbots were not real humans that I could meet in person. I knew that I was communicating
with a computer programme, not real successful entrepreneurs”
“I liked it because it was easy to ask and get answers as many times as possible until I learnt”
Student 21 “I liked it because I had freedom to set my own questions and get only answers that I really wanted to know”
Student 22 “I liked it. Learning was made easy”
Student 23 “I liked it. Learning was made easy”
Student 24 “I liked it. I learnt different perspectives from two successful entrepreneurs in forms of chatbots”
Table 4 showed that a majority of (91.66%) students’ perceptions of learning was positive because they liked
teaching entrepreneurship with chatbots and thought that chatbot technology highly supported their learning while
a minority of (8.34%) students’ perceptions of learning did not like teaching entrepreneurship with chatbots
because they knew that chatbots they communicated with were not real humans and sometimes gave answers not
matched with questions asked.
Based on the results, the impact of chatbots on student learning was very positive. Most students perceived that
they learnt better. They felt that they were equipped with right entrepreneurship knowledge to succeed as an
entrepreneur.
6.3 Students’ Perceptions of Satisfaction
Students were given a satisfaction questionnaire after the completion of the learning tasks to measure satisfaction
towards the use of chatbots. Table 5 illustrated students’ perceptions of satisfaction.
Table 5. Mean & standard deviation of perceived satisfaction of students
Perceived Satisfaction of Students
𝑋
S.D. Interpretation
Content of Best Practices from Successful Entrepreneurs
1. Content is easy to understand and informal 4.33 0.92 High
2. Content is related to best practices from successful entrepreneurs 4.63 0.58 Very high
3. Content is attractive 4.67 0.56 Very high
4. Content is various 4.63 0.58 Very high
5. Content suits student needs 4.50 0.72 High
Communication of Best Practices from Successful Entrepreneurs
6. Communication channel to access best practices of successful
entrepreneurs 4.67 0.64 Very high
7. Communication of best practices from successful entrepreneurs in
the format of conversation 4.54 0.72 Very high
8. Communication can be done as many times as students prefer 4.75 0.44 Very high
Application of chatbots as learning tasks to teach entrepreneurship
9. Chatbots are easy to use with no extra technical skills 4.75 0.44 Very high
10. Chatbots can be used at anytime and anywhere convenient to
students 4.92 0.28 Very high
11. Chatbots can be accessed via any device 4.83 0.38 Very high
12. Chatbots cost no extra expenses 4.96 0.20 Very high
Benefits of chatbots in teaching entrepreneurship
13. Learning from the best 4.54 0.66 Very high
14. Learning resource of best practices from successful
entrepreneurs 4.71 0.55 Very high
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
23
15. Motivation to learning 4.58 0.65 Very high
16. Inspiration to be entrepreneurs 4.58 0.65 Very high
17. Practical way to teach entrepreneurship 4.54 0.59 Very high
Total 4.65 0.44 Very high
Table 5 illustrated that the perceived satisfaction towards the chatbot use was at a very high level (Mean = 4.65,
S.D. = 0.44). In each area, students perceived satisfaction in content of best practices from successful
entrepreneurs, communication of best practices from successful entrepreneurs, application of chatbots as learning
tasks to teach entrepreneurship, and benefits of chatbots in teaching entrepreneurship at very high levels (Mean =
4.55, S.D. = 0.56, Mean = 4.65, S.D. = 0.52, Mean = 4.86, S.D. = 0.24, Mean = 4.59, S.D. = 0.55, respectively).
Students thought that the chatbots were an interesting, innovative, and fun teaching way with game-like learning
tasks to do at anytime, anywhere and many times as they preferred.
The main findings indicated that chatbot technology positively impacted student learning and satisfaction. Results
showed that most of students learnt about how to solve problems and make decisions while few students learnt
about how to learn business by doing business. Furthermore, most of students wanted to apply their learning by
taking more reasonable risks to create changes with creativity and innovation while few students wanted to apply
their learning by looking for networking opportunities. Also, most of students felt that chatbot technology
supported teaching and learning while few students felt that chatbot technology obstructed teaching and learning
because chatbots were not real humans and not able to provide matching answers, which was in accordance with
Sandu (2020) who found that biggest concerns for students involved getting incorrect answers by using chatbots.
In addition, students were highly satisfied with chatbot technology, which was in accordance with Tavichaiyuth
and Rattagan (2021), who reported that some of the most popular and appropriate ways to assess a chatbots
performance are user satisfaction wh ile Chaipram et al. (2020) conducted the satisfaction survey and found out that
the chatbot helped students well and students were satisfied with the ease of use the most. Although many useful
quality attributes were suggested, there was still no guidance to determine what to apply (Radziwill & Benton,
2017). The lack of reliable and efficient chatbots’ evaluation method limits the correct testing and comparison of
different models (Caldarini, Jaf, & McGarry, 2022). Apart from that, using Dialogflow as chatbot framework was
also one of the reasons leading to learning and satisfaction as previous researches suggested that Dialogflow was
outperforming other NLUs in intents classification and entity extraction (Abdellatif et al., 2021). All findings
suggested that chatbot technology seemed to have more advantages than disadvantages in supporting teaching and
learning if it was properly designed and developed with educational intentionality according to the scalability and
usability of chatbots. Therefore, it can be implemented as a powerful tool to teach entrepreneurship in
entrepreneurship education programmes in higher education context. The findings were consistent with previous
researches. They were in line with Vanichvasin (2021) who found that the application of chatbot technology in
education context to promote student knowledge of research yielded positive views of the chatbot as it created
positive learning outcomes and provided better individualized learning experience while Winkler and Söllner
(2018) reported that chatbots in education promised to have a significant good impact on learning success and
satisfaction of students, which was in accordance with Alepsis and Virvou (2011) who reported that their study
showed success in the use of conversational systems with high acceptance from both students and teachers that
perceived the system as a very useful tool for health education and Preecha (2020) who reported that learning with
an artificial intelligence chatbot was efficient as students can quickly find answers anytime and anywhere. Positive
impact may stem from the researcher’s experience in conducting this kind of research before but with different
type of chatbot frameworks and platforms. Also, the ability of chatbots to mimick conversation of successful
entrepreneurs like real humans is powerful because chatbots can do a task that would otherwise not be possible or
take a longer time (Fryer, Nakao & Thompson, 2019) like interviewing real entrepreneurs, who are difficult to
access and interview about their best practices in doing business in real lives. Apart from that, students felt
comfortable to interview as many times as possible without worries about using proper or unproper
communication to find out what it took to be successful entrepreneurs, which was in accordance with Hill et al.
(2015), who analyzed 100 messaging conversations and found that humans carried on significantly longer
messaging conversations with the chatbot than with other humans. While each message sent to the chatbot was
shorter and the vocabulary not as rich, this finding demonstrated the ease with which humans can communicate
with chatbots and the quantity of conversational engagement that was possible. The findings helped instructors
gain a better understanding of chatbot technology and provided strong evidence about how to integrate chatbot
technology as learning tasks to achieve better student learning and satisfaction.
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
24
7. Conclusion and Recommendation
This study was limited using chatbots to perform a learning task and interact with algorithm only in the form of
interviewing successful entrepreneurs while there were many types of learning tasks and algorithm that can be
used to support teaching and learning. The researcher acknowledged this limitation but strongly believed that the
findings would add to the literature on the impact of chatbot technology in 2 areas, that is, how to design and
develop a chatbot with educational intentionality according to its scalability and usability to support teaching and
learning in educational settings and how students benefited from quality attributes of a chatbot. As a result, the
findings reflected that chatbot technology positively impacted student learning and satisfaction. With its high
potential, it can be implemented as a powerful tool to teach entrepreneurship in entrepreneurship education
programmes in higher education context. Teachers can formulate strategies for using chatbot technology in
designing and developing chatbots (rule-based or machine learning based type) to match their learning tasks
(interviewing or other possible learning tasks), platform to use (LINE application or other possible platforms) and
quality attributes to measure (learning, satisfaction or other possible quality attributes) as the results indicated that
students learnt better and satisfied more with this technology. It is hoped that chatbot technology will be
extensively used in education settings with educational intentionality due to its’ positive impact on student learning
and satisfaction. However, the current study used a small sample size and only focused in the entrepreneurship
education programme. Therefore, it is anticipated that the next study may use a large sample size and apply
chatbots in other education programmes that there is entrepreneurship teaching in the programmes to further
investigate learning outcomes with a broader extent in chatbot technology and different programmes in higher
education context for more insights in order to establish best practices of teaching entrepreneurship. In addition,
the negative impact may be considered in future studies to find out problems in using chatbot technology for better
design and development although the research objectives were achieved, there were few students reflecting that
chatbots could not perform their functions well enough. As such, algorithms for interactions may be revised for
better results.
Acknowledgements
Many thanks to experts for their kind assistance and invaluable advice during the chatbot development and the
verification of the chatbot applicability including great recommendation on its improvement. Also, very special
thanks to students in a master’s degree of entrepreneurship education programme at Kasetsart University for their
time and participation. This research study would not have been possible without their kind support.
References
Abdellatif, A., & Costa, D., & Badran, K., & Abdalkareem, R., & Shihab, E. (2020). Challenges in chatbot
development: A study of stack overflow posts. In Proceedings of the 17th International Conference on
Mining Software Repositories (pp. 174-185). https://doi.org/10.1145/3379597.3387472
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003, April). Consolidating the ISO usability models. In
Proceedings of 11th international software quality management conference (pp. 23-25).
https://doi.org/10.1023/A:1025869312943
Agarwal, R., & Wadhwa, M. (2020). Review of state-of-the-art design techniques for chatbots. SN Computer
Science, 1(5), 1-12. https://doi.org/10.1007/s42979-020-00255-3
Alepsis, E., & Virvou, M. (2011). Automatic generation of emotions in tutoring agents for affective e-learning in
medical education. Expert Systems with Applications, 38(8), 9840-9847.
https://doi.org/10.1016/j.eswa.2011.02.021
Andrade, G. G., Francisco, C. M. D., Santas, G. A., Da Costa, J. P., & De Sousa, R.T. (2022). A
conversation-driven approach for chatbot management. IEEE Access (pp. 1-1).
https://doi.org/10.1109/ACCESS.2022.3143323
Augello, A., Gentile, M., & Dignum, F. (2018). An overview of open-source chatbots social skills. In S. Diplaris,
A. Satsiou, A. Følstad, M. Vafopoulos, & T. Vilarinho (Eds.), INSCI 2017. LNCS (Vol. 10750, pp. 236-248).
Springer, Cham. https://doi.org/10.1007/978-3-319-77547-0_18
Bayan, A. S. (2005). A corpus based approach to generalising a chatbot system. Retrieved from
https://etheses.whiterose.ac.uk/1323/1/abushawar.pdf
Benotti, L., Martínez, M. C., & Schapachnik, F. (2014). Engaging high school students using chatbots. In
Proceedings of the 2014 Innovation and Technology in Computer Science Education Conference (pp.
63-68). ACM, New York, USA. https://doi.org/10.1145/2591708.2591728
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
25
Bii, P. (2013). Chatbot technology: A possible means of unlocking student potential to learn how to learn.
Educational Research, 4(2), 218-221.
Caldarini, G., Jaf, S., & McGarry, K. A. (2022). Literature Survey of Recent Advances in Chatbots. Information,
13(41), 1-22. https://doi.org/10.3390/info13010041
Chaipram, N., Tumnut, N., & Rattanapoka, C. (2020). Educational response and notification system using LINE
Bot. Journal of Information Science and Technology, 10(2), 59-70.
Fernoagă, V., Stelea, G. A., Gavrilă, C., & Sandu, F. (2018). Intelligent Education Assistant Powered by Chatbots.
eLearning & Software for Education, 2.
Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences,
interest and competence. Computers in Human Behaviour, 93, 279-289.
https://doi.org/10.1016/j.chb.2018.12.023
Hien, H. T., Cuong, P.-N., Nam, L. N. H., Nhung, H. L. T. K., & Thang, L. D. (2018). Intelligent assistants in
higher-education environments: the FIT-EBot, A chatbot for administrative and learning Support. In
Proceedings of the 9th International Symposium on Information and Communication Technology (pp.
69-76). ACM, New York, USA. https://doi.org/10.1145/3287921.3287937
Hill, J., Ford, W. R., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison
between human-human online conversations and human-chatbot conversations. Computers in Human
Behaviour, 49, 245-250. https://doi.org/10.1016/j.chb.2015.02.026
Klapper, R. (2004). Government goals and entrepreneurship education–An investigation at a Grand Ecole in
France. Education & Training, 46(3), 27-37. https://doi.org/10.1108/00400910410531787
Molnár, G., & Szüts, Z. (2018). The role of chatbots in formal education. In 2018 IEEE 16th International
Symposium on Intelligent Systems and Informatics (SISY). https://doi.org/10.1109/SISY.2018.8524609
Nilros, K., & Lidén, A. (2020). Perceived benefits and limitations of chatbots in higher education. Retrieved
from http://www.diva-portal.org/smash/get/diva2:1442044/FULLTEXT01.pdf
Preecha, T. (2020). Application of Artificial Intelligence Chatbot for Learning. Royal Thai Air Force Medical
Gazette, 66(2), 64-73.
Radziwill, N., & Benton, M. (2017). Evaluating quality of chatbots and intelligent conversational agents.
arXiv:1704.04579. https://doi.org/10.48550/arXiv.1704.04579
Sandu, R. (2020). Adoption of AI-Chatbots to Enhance Student Learning Experience in Higher Education in
India. https://doi.org/10.1109/ITHET46829.2019.8937382
Shawar, B. A., & Atwell, E. (2007). Chatbots: Are they really useful? Journal for Language Technology and
Computational Linguistics, 22(1), 29-49.
Solomon, G. T., Duffy, S., & Tarabishy, A. (2002). The State of Entrepreneurship Education in the United States:
A Nationwide Survey and Analysis. International Journal of Entrepreneurship Education, 1(2), 65-86.
Song, S. M., Thominathan, S., & Khalid, N. A. (2021). Entrepreneurial Intention of UiTM Students and the
Mediating Role of Entrepreneurship Education. Asian Journal of University Education (AJUE), 17(2),
236-251. https://doi.org/10.24191/ajue.v17i2.13405
Souitaris, V., Zerbinati, S., Al-Laham, A. (2007). Do Entrepreneurship programmes raise Entrepreneurial
intention of science and Engineering students? The Effect of Learning, Inspiration and Resources. Journal
of Business Venturing, 22(4), 566-591. https://doi.org/10.1016/j.jbusvent.2006.05.002
Tavichaiyuth, N., & Rattagan, E. (2021). Developing chatbots in higher education: A case study of academic
program chatbot in Thailand. Retrieved from
http://https://as.nida.ac.th/wp-content/uploads/2021/11/6210422022_Nicha_Paper_PDF.pdf
Vanichvasin, P. (2021). Chatbot development as a digital learning tool to increase students’ research knowledge.
International Education Studies, 14(2), 44-53. https://doi.org/10.5539/ies.v14n2p44
Westhead, P., & Wright, M. (2013). Entrepreneurship: A very short introduction. Oxford University Press.
https://doi.org/10.1093/actrade/9780199670543.001.0001
Winkler, R., & Söllner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis.
https://doi.org/10.5465/AMBPP.2018.15903abstract
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
26
Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017). A new chatbot for customer service on social media.
In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 3506-3510).
ACM, New York. https://doi.org/10.1145/3025453.3025496
Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution
license (http://creativecommons.org/licenses/by/4.0/).
... These studies found that Generative AI has significant benefits in higher education. For instance, studies by P. Vanichvasin [53] and L. Zhou et. Al [60] acquired the highest OR and 95-CI. ...
... No Framework Result [53] 30 students ...
... No Framework Result [53] 30 students Qasi-experimental research Chat bot has a significant impact on student learning and satisfaction. [9] 844/students Survey quantitative analyzed ...
Article
The rapid advancement of genitive AI, like Chat GPT, has initiated a profound transformation in higher education. It offers customized learning experiences, automates administrative tasks, and provides personalized support to students and educators. Following PRISMA guidelines, this paper presents a systematic review that delves into the implications of genitive AI, a cutting-edge language model, in higher education. We adopted ChatGPT as an example of this study. It thoroughly examines the potential advantages and constraints of integrating ChatGPT into educational environments, assessing the quality of 35 selected articles and conducting a comprehensive meta-analysis of their findings. This study yields fresh insights into the multifaceted consequences of employing ChatGPT in higher education and underscores the intricate landscape associated with AI integration in academic settings. It emphasizes the imperativeness of addressing ethical, legal, and pragmatic challenges while capitalizing on the potential benefits of AI technology in education. Our systematic review reveals a consistent reservation trend regarding generative AI integration within educational contexts. These concerns encompass many issues, emphasizing the necessity for judicious implementation and ro bust safeguards to mitigate potential challenges.
... An AI chatbot is a software program that acts like a human when communicated with message or voice over the internet to create a scripted conversation, understand human languages, and automatically respond to a conversation (Vanichvasin, 2022). When a client asks a question in an AI chatbot system, an intelligent robot investigates the reply system for relevant material that matches the topic of the customer's inquiry and then responds instantly. ...
... These interactive consumer experiences are the result of the rise of a computer-mediated marketing environment as technology advances and the Internet grows at a rapid pace. (Zheng et al;2022). The key to successful customer engagement is to create meaningful and valued connections that foster strong relationships between the company and its clients and earn their trust. ...
... There are two types of chatbots: The first type: act according to the pre-defined, it is described as rule based. It selects the system response from a list of pre-programmed answers depending on the input text rather than developing fresh text (Vanichvasin, 2022). They are less useful in consumer computing since they demand a certain input and are programmed to produce a certain outcome, leaving little room for varied conversation styles (Markoski et al., 2018). ...
... While some education has experimented with chatbots, language acquisition is one of the few domains in which they have been extensively embraced and enacted (Fryer et al., 2019;Winkler & Sollner, 2018). The use of chatbots to assist students in the learning process in higher education is uncommon (Vanichvasin, 2022). Utilizing the emergence of generic artificial intelligence (AI), this study designs and prototypes a program that can remotely assist users' independent education and self-evaluation activities beyond educational settings. ...
Article
Full-text available
The present study aimed to develop and implement an interactive chatbot in a language-learning setting. For this purpose, fifty Omani EFL students were divided into two equal groups: control and experimental. All the pretests were validated and piloted before the treatment to ensure practicality. The chatbot was designed using the WhatsApp platform, and instructions were given to the students. Some researcher-made tests, such as writing tasks, grammar, and reading comprehension activities, were assigned to both groups. In addition, an adapted questionnaire initially developed by Williamson (2007) was used as the pretest and posttest. The control group received all the materials using the traditional face-to-face teaching method, while the experimental group received all the instructions, explanations, and tests through the chatbot. The findings of the study revealed that the control group did not show any interest in completing the activities; on the contrary, the majority of participants in the experimental group engaged themselves in task achievements. Moreover, the self-directed learning questionnaire results showed that the experimental group progressed significantly from the pretest to the posttest in self-directed abilities, including awareness, learning strategies, activities, evaluation and interpersonal skills, with a statistically significant difference compared to the control group. The results of this study could be helpful for educators, students, and self-access centre staff.
... Summary Of Research on The Benefits and Limitations of AI Applications in Learning and Studying English LanguagueBenefitsLimitations Students enjoy learning and applying AI. AI supports students in writing (Enik Rukiati, Julien Arief Wicaksono, Gullit Tornado Taufan, Degita Danur Suharsono, 2023),(Patchara Vanichvasin, 2022) Technology problems, limited AI capabilities, and lack of language standardization (EnikRukiati, Julien Arief Wicaksono, Gullit Tornado Taufan, Degita Danur Suharsono, 2023) Learners perceive positively about using AI applications, learners enjoy using AI in doing homework(Dian Toar Y. G. Sumakul, Fuad Abdul Hamied & Didi Sukyadi, 2021) Increase motivation to study and actively study (Ching-Yi Chang, Gwo-Jen Hwang, Meei-Ling Gau, 2022) Develop language skills (Esteban Vázquez-Cano, Santiago Mengual-Andrés, and Eloy López-Meneses, 2021) ...
Article
Full-text available
Artificial Intelligence tools such as Grammarly and ChatGPT are used in the learning process, including learning foreign languages for high school students. This paper aims to synthesize the benefits and limitations of using AI in learning foreign languages, and the current situation of using AI tools in studying English for high school students in Vietnam. The authors have systematized theories about the benefits and limitations of using AI tools in learning English in about 30 papers in peer-reviewed journals from 2018 to 2024. The authors developed a survey to collect data from 300 students in high school at Hanoi in Vietnam to see and know the current status of using AI tools in learning English. The results of data analysis from 297 responses calculated the AI usage situation as follows: Using AI tools in studying English students is very common with a high frequency of use including four skills listening, speaking, reading, and writing. Most of the students are also well aware of the benefits and limitations of AI tools. However, students' use of AI tools is still spontaneous and not supported by teachers and schools. To better use AI for students' English learning, the article makes recommendations to schools and teachers.
... Sin embargo, al tratarse de un campo tan reciente, los estudios que abordan este tema específico están en pleno desarrollo. En el ámbito académico y de la educación superior la mayoría de los trabajos se enfocan en describir las potencialidades de este tipo de IA, los retos y limitaciones, o analizar aplicaciones en algún aprendizaje más específico como el de lenguas (Wu y Yu, 2024) o emprendimiento (Vanichvasin, 2022). ...
Article
Full-text available
Introducción: Tras la irrupción de la inteligencia artificial (IA) en el ámbito educativo, la enseñanza se encuentra en un proceso de cambio y adaptación. Este estudio surge de la necesidad de comprender cómo el estudiantado universitario integra la IA en sus procesos de aprendizaje y qué percepciones tiene sobre su uso y regulación. Metodología: Utilizando una encuesta ad-hoc con preguntas cerradas, de escala Likert y abiertas, se recopiló información de una muestra casual de estudiantes. Los datos se han analizado con técnicas estadísticas de carácter exploratorio. Resultados: Los hallazgos muestran que más del 94% de los encuestados utiliza la IA, principalmente en generación y traducción de textos y elaboración de presentaciones. Los estudiantes valoran el aprendizaje sobre IA para mejorar su competitividad laboral, aunque las opiniones sobre su regulación varían: algunos prefieren no regular, mientras otros sugieren limitar su uso o promover habilidades no automatizables, como la creatividad y el análisis crítico. Conclusiones: Se destaca la necesidad de formación del profesorado, así como el rediseño de las actividades educativas para que integren la IA. Deberá realizarse desde una perspectiva ética y complementaria, aprovechando, por un lado, el potencial de la IA y evitando, por otro, reemplazar el aprendizaje del alumnado.
... This integration has facilitated the development of diverse chatbot applications (Sreelakshmi & Prathap, 2020). Moreover, the infusion of AI into chatbot systems has extended its utility across educational spheres, comprising educators, students, and administrative functions (Vanichvasin, 2022). The rise of AI and chatbots leads to transformation in education and offers the potential to reshape learning paradigms and knowledge exploration (Kooli, 2023). ...
Article
Full-text available
Aim/Purpose: The main goal of this study is to investigate the factors affecting students’ satisfaction and continuous usage of ChatGPT in an educational context, using the Expectation-Confirmation Model (ECM) as the theoretical framework. Specifically, this investigation focuses on identifying how user expectations, perceived usefulness, and satisfaction influence the continuous usage of ChatGPT in education. Background: ChatGPT is an AI-based chatbot that can generate natural language and support multi-turn dialogue. ChatGPT can be used in education for various purposes, such as academic writing, language translating, coding assistance, and information searching. Research indicates that ChatGPT is highly beneficial for students. Specifically, it aids in time-saving by synthesizing information across diverse fields and offers feedback tailored to individual learning needs and progress. Moreover, it enhances learning experiences through a personalized learning process, as well as aids in the translation of learning materials into different languages for improved accessibility. Despite the potential of ChatGPT in education, there remains a gap in understanding the determinants of user satisfaction and continuous usage. Therefore, this study was conducted to bridge this gap by assessing the impact of expectation confirmation, perceived usefulness, and satisfaction on continuous usage of ChatGPT. Methodology: This research employed a quantitative approach, which utilized a five-point Likert scale questionnaire to measure four variables with a total of 14 items. Data were collected from 435 students from eight Vietnamese universities through non-probability convenience sampling. The internal consistency of the questionnaire was examined using Cronbach’s alpha. Four-factors Confirmatory Factor Analysis (CFA) and Structural Equation Modeling (SEM) were then employed through IBM SPSS 26 and AMOS 24 to assess the proposed model and hypotheses, allowing for a comprehensive examination of the relationships between 4 variables: perceived usefulness, expectation confirmation, satisfaction, and continuous usage. Contribution: This paper contributes to the literature on ChatGPT in education by identifying how expectation confirmation and perceived usefulness affect students’ satisfaction and their intention to continue using ChatGPT in their learning. This study also has practical implications for educators and students for applying ChatGPT in their learning and teaching. It provides recommendations for effective use of ChatGPT in education. Findings: The study’s results show that students with greater expectations, either met or surpassed by ChatGPT, perceived the tool as more beneficial and satisfying for their needs. Additionally, students who perceived ChatGPT as useful and valuable for their learning goals were more inclined to continue to use it. Furthermore, the findings indicate that higher satisfaction with the tool was associated with a greater likelihood of continuous usage. Nevertheless, the results did not reveal a significant relationship between perceived usefulness and satisfaction regarding ChatGPT. This implies that perceiving ChatGPT as useful does not guarantee satisfaction. This finding may be attributed to various factors influencing student satisfaction, such as price value, using habit, facilitating conditions, trust, reliability, and quality. Recommendations for Practitioners: Educators and students should consider the findings of this study to make informed decisions about the use of ChatGPT in educational settings, emphasizing the importance of managing users’ expectation confirmation and ensuring perceived usefulness and satisfaction on continuous usage. Recommendation for Researchers: Researchers can use the findings of this study as a foundation for exploring a deeper understanding of user expectation confirmation and the factors influencing perceived usefulness and satisfaction with ChatGPT in diverse educational settings. Impact on Society: This study contributes to the broader field of the application of AI technologies in education, potentially reshaping the dynamics of interaction between students and educators with AI technologies for educational purposes. Its contributions have the potential to elevate the overall quality of education by promoting the appropriate and effective use of AI technologies. Future Research: Future research should investigate the roles of ChatGPT in education more deeply and explore its long-term effects on both learners and educators. This includes conducting comparative studies to evaluate the efficacy and advantages of ChatGPT-assisted learning in comparison to traditional teaching methods. Additionally, there is a crucial need to assess the broader impact of ChatGPT on students’ academic performance, cognitive development, and critical thinking skills.
... Summary of research on the benefits and limitations of AI applications in learning EnglishBenefitsLimitations Students enjoy learning and applying AI. AI supports students in writing (Enik Rukiati, Julien Arief Wicaksono, Gullit Tornado Taufan, Degita Danur Suharsono, 2023),(Patchara Vanichvasin, 2022) Technology problems, limited AI capabilities, and lack of language standardization (EnikRukiati, Julien Arief Wicaksono, Gullit Tornado Taufan, Degita Danur Suharsono, 2023) Learners perceive positively about using AI applications, learners enjoy using AI in doing homework (Dian Toar Y. G.Sumakul, Fuad Abdul Hamied & Didi Sukyadi, 2021) Increase motivation to study and actively study (Ching-Yi Chang, Gwo-Jen Hwang, Meei-Ling Gau, 2022) Develop language skills (Esteban Vázquez-Cano, Santiago Mengual-Andrés, and Eloy López-Meneses, 2021) The answer is misleading (Patchara Vanichvasin, 2022) Improved learning performance, self-assessment, and selfregulation of learning (Víctor González-Calatayud, Paz Prendes-Espinosa, and Rosabel Roig-Vila, 2021) ...
Article
Full-text available
AI tools are used in many fields, including learning English for high school pupils. This study aims to synthesize the benefits and limitations of using AI in learning English and the current situation of using AI in learning English for high school pupils in Vietnam. The authors have systematized theories about the benefits and limitations of using AI tools in learning English in about 32 articles in peer-reviewed journals from 2020-2024. To find out the status of using AI tools in learning English, the authors developed a survey to collect data from 297 high school pupils in Vietnam. Data processing using Microsoft Excel has shown the following results: Using AI in learning English is very popular for the four skills of listening, speaking, reading, and writing; with high frequency of use; and with personal spontaneity. Most of the people are also well aware of the benefits and limitations of AI tools. To effectively use AI in learning English, the article has proposed several discussions to address the limitations of AI tools for pupils and schools in Vietnam.
Article
Full-text available
Design thinking is a systematic process of creating innovations that centers around users, emphasizing diverse ideas along with the possibilities of modern digital technologies under the digital environment nowadays to address user-related problems that affect users, which results in building prototypes to be rapidly tried, tested, and refined until a solution that addresses the user-related problems is achieved. The process of creating innovations according to design thinking can be applied to innovate within educational contexts, allowing teachers to use innovations derived from this process to cope with student-related problems of learning management appropriately and adapt to the challenging digital environment that has significantly transformed the way students learn nowadays. As a result, teachers play a crucial role in creating educational innovations to keep up with modern digital technologies, making them suitable in accordance with digital trends and beneficial for students under today’s digital environment. The design thinking process consists of five key steps that focus on students as the central point, that is, empathizing, defining, ideating, prototyping, and testing. When it is applied along with educational application guidelines, these five steps of design thinking can effectively and efficiently address learning management problems in the digital age, ultimately leading to improved learning quality. Therefore, this article aims to present a design thinking process to help teachers understand and apply it to innovate in education, thereby addressing learning challenges in the digital age that have resulted in ineffective and inefficient student learning. This will ultimately lead to better quality learning experiences for students in the future.
Article
Full-text available
Artificial Intelligence (AI) chatbot usage is becoming increasingly common in schools, as students today are digital natives accustomed to instant access to information and personalized experiences of modern technology. This descriptive correlational design determined the level of benefits of using AI chatbots and their significant relationship to student engagement. The adapted questionnaires were distributed to 206 respondents from Divine Word College of San Jose senior high school department, representing diverse academic tracks. This study revealed that the students have a moderate agreement regarding the benefits of using AI chatbots; moreover, a high degree of student engagement was revealed in student engagement. A significant relationship exists between the perceived benefits of AI chatbots and students’ engagement levels. As students perceive greater benefits from using AI chatbots, their engagement level also increases, highlighting the potential of these tools to foster active participation and meaningful interactions in the learning process. This study provides insight into the benefits of using AI chatbots and student engagement of senior high school students. This study recommends that policymakers and administrators address potential challenges, such as data privacy and digital equity, to create a supportive environment for the widespread adoption of AI chatbots in education. Future researchers may also explore and study the specific school activities in which students are getting more engaged in using AI chatbots in an educational context. Keywords: benefits of AI chatbots, student engagement, artificial intelligence, modern technology, learning process
Article
Full-text available
The objective of this study was to systematically assess and rank the features and characteristics of online consultation pages and virtual prescription applications, with a particular focus on their offerings in nutrition and exercise. The study aimed to identify which attributes of these digital health solutions most significantly influence user engagement and satisfaction, thereby guiding the optimization of such platforms for improved health management. Utilizing a qualitative research design, this study combined semi-structured interviews with 40 participants and a thorough review of 102 Instagram pages and applications related to nutrition and exercise. Participants comprised a diverse group including page users and academic experts. The data collection focused on aspects such as the presence and quality of workout and nutrition plans, availability of physical consultation options, and the involvement of recognized experts in the field. Data analysis involved coding and thematic analysis to identify and rank the key features and characteristics deemed most valuable by users. The analysis revealed a preference for online platforms that facilitate effective and direct communication with users, offering diversified content across different sports disciplines and providing services tailored to the specific needs of men and women. The inclusion of specialized teamwork, featuring collaboration among healthcare professionals, also emerged as a critical aspect. These preferences underscore the importance of personalization, interactivity, and credibility in digital health solutions catering to nutrition and exercise. This study highlights the essential features that digital health platforms should embody to enhance the management of non-communicable diseases and promote overall wellness through nutrition and exercise. The findings emphasize the necessity for digital solutions to prioritize user-centric communication, offer varied and personalized content, and engage multidisciplinary experts to meet user needs effectively.
Article
Full-text available
Managing and evolving a chatbot’s content is a laborious process and there is still a lack of standardization. In this context of standardization, the absence of a management process can lead to bad user experiences with a chatbot. This work proposes the Chatbot Management Process, a methodology for content management on chatbot systems. The proposed methodology is based on the experiences acquired with the development of Evatalk, the chatbot for the Brazilian Virtual School of Government. The focus of this methodology is to evolve the chatbot content through the analysis of user interactions, allowing a cyclic and human-supervised process. We divided the proposed methodology into three distinct phases, namely, manage, build, and analyze. Moreover, the proposed methodology presents a clear definition of the roles of the chatbot team. We validate the proposed methodology along with the creation of the Evatalk chatbot, whose amount of interactions was of 22,771 for the 1,698,957 enrolled attendees in the Brazillian Virtual School of Government in 2020. The application of the methodology on Evatalk’s chatbot brought positive results: we reduced the chatbot’s human hand-off rate from 44.43% to 30.16%, the chatbot’s knowledge base examples increased by 160% whilst maintaining a high percentage of confidence in its responses and keeping the user satisfaction collected in conversations stable.
Article
Full-text available
Chatbots are intelligent conversational computer systems designed to mimic human conversation to enable automated online guidance and support. The increased benefits of chatbots led to their wide adoption by many industries in order to provide virtual assistance to customers. Chatbots utilize methods and algorithms from two Artificial Intelligence domains: Natural Language Processing and Machine Learning. However, there are many challenges and limitations in their application. In this survey we review recent advances on chatbots, where Artificial Intelligence and Natural Language processing are used. We highlight the main challenges and limitations of current work and make recommendations for future research investigation.
Article
Full-text available
The emphasis on entrepreneurship by the Malaysian government clearly displays their effort in strengthening the entrepreneurial development in the country, which is expected to boost the economy by creating more jobs and generating new incomes. The idea is extended to local higher education institutions so as to nurture and to sustain entrepreneurial engagement among the students through formal entrepreneurial programs. The initiatives are expected to help in reducing graduate unemployment level in the country. Despite all the effort taken by the government, the uptake of entrepreneurship among young minds is still low, and there is growing unemployment in the country. Thus, this study aims to determine the factors that affect entrepreneurial intentions among UiTM graduating students and the mediating effect of entrepreneurship education towards entrepreneurial intention through attitude, perceived behaviour control and self-efficacy. The sample consists of 429 UiTM students who have taken the entrepreneurship course. Factor analysis and Partial Least Squares methods were used to analyse the data. The findings reveal a significant relationship of attitude, perceived behaviour control and entrepreneurship education towards entrepreneurial intention, while self-efficacy was insignificantly related. However, the study found that entrepreneurship education partially mediates the relationship between attitude and perceived behaviour control towards entrepreneurship intention and a full mediation on self-efficacy. The findings will help the policymakers or the higher education institutes in developing a suitable policy and program in promoting entrepreneurship to the university graduates at the national level. Keywords: Entrepreneurial intention, IBM, Partial Least Squares, TPB.
Article
Full-text available
The research aimed to: 1) develop the chatbot; 2) evaluate its effectiveness; and 3) investigate its effects on students’ research knowledge. The sample consisted of 36 Thai university students. The research instruments consisted of: 1) the chatbot; 2) an evaluation form; 3) an effectiveness questionnaire; and 4) research tests. Data analysis used was mean, standard deviation, content analysis and a t-test. The findings indicated that: 1) the chatbot was evaluated by experts with the applicability at a very high level ( = 4.67, S.D. = 0.08) with recommendation to add more research content and interactive learning. The pilot test was done with 14 non-target group of students. Students perceived the chatbot’s effectiveness at a high level ( = 4.43, S.D. = 0.35) with comments to add more examples and graphics to make the chatbot more interesting; 2) the 36 target group of Thai university students perceived the chatbot as an effective technology to use as a digital learning tool at a high level ( = 4.37, S.D. = 0.48). They thought that chatbot technology was easy to use, easy to understand, innovative and fun for learning. They could get answers instantly and be able to seek specific information without waiting for responses. However, in response to questions not matched keywords specified, further details of finding proper answers such as links should be provided instead of leaving those questions unanswered. Also, the chatbot only provided responses when typing correctly so there should be an option to choose from a list of questions or keywords; and 3) the post-test scores were significantly higher than the pre-test scores at the 0.05 level of significance. In conclusion, using chatbot technology in education settings to increase students’ research knowledge gave positive results as it led to positive learning outcomes and helped provide better personalized learning experience for students.
Article
Full-text available
Amazon’s Alexa, Apple’s Siri, Google Assistant and Microsoft’s Cortana, clearly illustrate the impressive research work and potentials to be explored in the field of conversational agents. Conversational agent, chatter-bot or chatbot is a program expected to converse with near-human intelligence. Chatbots are designed to be used either as task-oriented ones or simply open-ended dialogue generator. Many approaches have been proposed in this field which ranges from earlier versions of hard-coded response generator to the advanced development techniques in Artificial Intelligence. In a broader sense, these can be categorized as rule-based and neural network based. While rule-based relies on predefined templates and responses, a neural network based relies on deep learning models. Rule-based are preferable for simpler task-oriented conversations. Open-domain conversational modeling is a more challenging area and uses mostly neural network-based approaches. This paper begins with an introduction of chatbots, followed by in-depth discussion on various classical or rule-based and neural-network-based approaches. The evaluation metrics employed for chatbots are mentioned. The paper concludes with a table consisting of recent research done in the field. It covers all the latest and significant publications in the field, the evaluation metrics employed, the corpus which is used as well as the possible areas of enhancement that exist in the proposed techniques.
Conference Paper
Full-text available
Chatbots are becoming increasingly popular due to their benefits in saving costs, time, and effort. This is due to the fact that they allow users to communicate and control different services easily through natural language. Chatbot development requires special expertise (e.g., machine learning and conversation design) that differ from the development of traditional software systems. At the same time, the challenges that chatbot developers face remain mostly unknown since most of the existing studies focus on proposing chatbots to perform particular tasks rather than their development. Therefore, in this paper, we examine the Q&A website, Stack Overflow, to provide insights on the topics that chatbot developers are interested and the challenges they face. In particular, we leverage topic modeling to understand the topics that are being discussed by chatbot developers on Stack Overflow. Then, we examine the popularity and difficulty of those topics. Our results show that most of the chatbot developers are using Stack Overflow to ask about implementation guidelines. We determine 12 topics that developers discuss (e.g., Model Training) that fall into five main categories. Most of the posts belong to chatbot development, integration , and the natural language understanding (NLU) model categories. On the other hand, we find that developers consider the posts of building and integrating chatbots topics more helpful compared to other topics. Specifically, developers face challenges in the training of the chatbot's model. We believe that our study guides future research to propose techniques and tools to help the community at its early stages to overcome the most popular and difficult topics that practitioners face when developing chatbots.
Article
Full-text available
Conversation practice, while paramount for all language learners, can be difficult to get enough of and very expensive. In this mobile age, chatbots are an obvious means of filling this gap, but have yet to realize their potential as practice partners. The current study was undertaken to examine why chatbots are not yet a substantial instrument for language learning engagement/practice, and to provide direction for future practice and chatbot development. To this end, building on a recent experimental study examining chatbot novelty effects, students undertook a pair of conversation activities: human-human and human-chatbot (via speech-to-text software). Immediately following the practice conversations, students' interest in the two partners was surveyed and open- ended textual feedback was collected. With these data sources and prior standardised test results, regression and content analysis of the data was undertaken. Findings indicated: 1) prior interest in human conversation partners was the best single predictor of future interest in chatbot conversations; 2) prior language competency was more strongly linked to interest in chatbot than human conversations; 3) that the qualitative experience of having "learned more" with the chatbot was strongly connected to task interest, even when reporting communication difficulties. Implications for practicing languages with currently available chatbots, for chatbots and related educational technology as sources of student interest and directions for chatbots future development are discussed. -----------> FREE COPY UNTIL FEB 2019 -----------------HERE----------------------->https://authors.elsevier.com/a/1YHMZ2f~UW4mGl
Conference Paper
Full-text available
The purpose of this paper is to discuss about smart learning environments and present the FIT-EBot, a chatbot, which automatically gives a reply to a question of students about the services provided by the education system on behalf of the academic staff. The chatbot can play the role of an intelligent assistant, which provides solutions for higher-education institutions to improve their current services, to reduce labor costs, and to create new innovative services. Various artificial intelligence techniques such as text classification, named entity recognition are used in this work to enhance the system performance.
Book
Entrepreneurship: A Very Short Introduction navigates the debates about entrepreneurship, providing a guide to the entrepreneurial process. There has been an explosion of interest in entrepreneurs in the popular media, as well as in business, policy, and education. The actions of entrepreneurs have always been shaped by the external environment and the availability of resources. They can be found in all sorts of organizations and they come from a diversity of backgrounds. Entrepreneurs have a significant impact on modern market economies and there is a certain future for entrepreneurship in our increasingly globalized world. But what do entrepreneurs do? What is entrepreneurship and why is it important? What is distinctive about entrepreneurs? And where do they come from?