Available via license: CC BY 4.0
Content may be subject to copyright.
International Education Studies; Vol. 15, No. 6; 2022
ISSN 1913-9020 E-ISSN 1913-9039
Published by Canadian Center of Science and Education
15
Impact of Chatbots on Student Learning and Satisfaction in the
Entrepreneurship Education Programme in Higher Education Context
Patchara Vanichvasin1
1 Faculty of Education, Kasetsart University, Bangkok, Thailand
Correspondence: Patchara Vanichvasin, Faculty of Education, Kasetsart University, Bangkok, Thailand.
Received: May 1, 2022 Accepted: August 9, 2022 Online Published: November 21, 2022
doi:10.5539/ies.v15n6p15 URL: https://doi.org/10.5539/ies.v15n6p15
Abstract
There are many ways to learn how to be entrepreneurs and one of the powerful ways is to learn from successful
entrepreneurs. However, it is difficult to reach and interview those entrepreneurs about their best practices in doing
business in real lives. Chatbot technology can come into play in mimicking conversation of successful
entrepreneurs and providing pre-programmed responses of their best practices drawn from interviews published in
newspapers, books and articles. Therefore, this research aimed to examine the impact of chatbots in the form of
successful entrepreneurs with 24 first-year graduate students, who enrolled in a master's degree of
entrepreneurship education at Kasetsart university. Data analysis involved mean, standard deviation, frequency,
percentage, and content analysis. The research findings showed that the developed chatbots were appropriate at a
very high level (Mean= 4.75, S.D. = 0.22). The impact of chatbots was positive. Students perceived that their
learning was better and their satisfaction was at a very high level (Mean = 4.65, S.D. = 0.44) with thoughts that
chatbots were an interesting, innovative, and fun teaching way. This study indicated that chatbot technology
positively impacted student learning and satisfaction. It can be implemented as a powerful tool to teach
entrepreneurship in entrepreneurship education programmes in higher education context.
Keywords: impact, chatbots, student learning, satisfaction, entrepreneurship education
1. Introduction
Entrepreneurs play a key role in promoting economic development. They and their businesses can generate jobs
and wealth, which help reduce social and regional inequality (Westhead & Wright, 2013). Entrepreneurial
development is expected to boost the economy by creating more jobs and generating new incomes (Song et al.,
2021). Teaching entrepreneurship is therefore fundamental for building stronger economy, better income and more
flexible societies. As such, it is important to include entrepreneurship teaching in the educational system (Klapper,
2004) due to the fact that entrepreneurship education contributes to the development of business for entrepreneurs
(Souitaris et al., 2007).
There are many ways in teaching entrepreneurship and one of the powerful ways is to learn from successful
entrepreneurs. Interviewing with entrepreneurs is the learning way used in teaching entrepreneurship (Solomon et
al., 2002) and considered as a good starting point to find out what it takes to be successful in businesses. However,
it is quite difficult to reach and interview successful entrepreneurs about their best practices in doing business in
real lives due to their busy schedules and lack of business network. Chatbot technology can come into play in
mimicking human conversation of successful entrepreneurs and providing pre-programmed responses of best
practices in doing business drawn from interviews published in newspapers, books and articles, which can be used
as learning tasks to teach entrepreneurship and equip students with right knowledge about how to be entrepreneurs
from experiences and perspectives of successful entrepreneurs in the form of chatbots like real humans. With
chatbot technology, learning entrepreneurship from the best is made possible by turning successful entrepreneurs
into chatbots in order that students can learn from the best.
A chatbot (also known as bot) is defined as a software program, which acts like a human when interacted with via
message or voice over the internet to simulate a conversation in a scripted way, understand human languages (Xu
et al., 2017) and reply to a conversation automatically (Benotti et al., 2014). The requirements of chatbot design
include accurate knowledge representation, strategy for answering, and predefined responses to reply when user
input is not understood (Augello, Gentile, & Dignum, 2018) or not in a system. Although developing chatbot
requires technical expertise (e.g., machine learning and conversation design) that is different from developing
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
16
traditional software systems, the use of chatbot framework and platform provided by Microsoft, Facebook, Google
and LINE application nowadays makes the chatbot development become much easier (Abdellatif et al., 2020).
Two types of chatbots were mostly used. Firstly, a chatbot with pre-programmed answers is called a rule-based
chatbot as it is responsible for matching pre-programmed answers with text messages or speeches inputted. It
chooses the system response from pre-programmed answers, based on the input text without creating new texts.
The responses from the chatbot were coded, organized and presented in the format of conversational patterns.
Secondly, a chatbot generating suitable responses is called a machine learning-based chatbot as it is responsible for
generating suitable responses drawn from user input via natural language processing and deep machine learning.
Therefore, it can generate answers better than the first type, based on user messages because it acts like human by
using algorithms of machine learning and techniques of deep learning (Hien et al., 2018). Concerning chatbot
design, scalability and usability can be major issues since they have a direct impact on the user experience of a
chatbot. Scalability refers to the way a chatbot design handles the increase of users, interactions, and content
contained in the chatbot, and usability refers to the actual usability of a chatbot design and if users are able to
perform the desired tasks (de Andrade et al., 2022).
Chatbots are useful in applications (Shawar & Atwell, 2007), which can be used as a tool to learn or study a new
language, access an information system, visualize the contents of a corpus; and give answers to questions (Bayan,
2005). Benefits involve saving costs, time, and effort (Abdellatif et al., 2020), increasing interaction, active
learning, and creative learning (Bii, 2013), promoting retention and engagement (Benotti et al., 2014), motivating
students to learn (Benotti et al., 2014), and enhancing user satisfaction (Vanichvasin, 2021).
As a result, chatbot technology has very high potential to use in various contexts. However, there is still limited
research on chatbots in education, with most of the research focusing on health and language learning (Nilros &
Lidén, 2020). Although chatbots have been tested in education to some extent with language learning being one of
the few fields where chatbots have been adopted and widely used (Winkler & Söllner, 2018; Fryer, Nakao, &
Thompson, 2019), there are not many chatbots used in higher education context to support students in their
learning process. In Thailand, researches were conducted in different contexts such as customer service, medical
service, hotel service, tourism and entertainment but there is still limited research in education, especially in
teaching and learning. Offering student services by chatbots were most popular topics of research studies.
Therefore, this research aims to focus on higher education context with educational intentionality because chatbots
can create a learning environment where students use them to increase the quality of learning and provide
individual support when the classrooms get bigger and individual support becomes more challenging (Winkler &
Söllner, 2018). This study is also built on the researcher’s previous research (Vanichvasin, 2021) but with different
chatbot framework and platform.
Due to very high potential of chatbot technology and only few studies on chatbots in education in higher education
context in Thailand, this research study tends to use chatbot technology to examine the impact of student learning
and satisfaction. It is anticipated that the developed chatbots can be used to teach entrepreneurship in
entrepreneurship education programme with positive impact of student learning and satisfaction.
2. Literature Review
A chatbot (also known as bot) is defined as a software program, which acts like a human when interacted with via
message or voice over the internet to simulate a conversation in a scripted way, understand human languages (Xu
et al., 2017) and reply to a conversation automatically (Benotti et al., 2014). The requirements of chatbot design
include accurate knowledge representation, strategy for answering, and predefined responses to reply when user
input is not understood (Augello, Gentile, & Dignum, 2018) or not in a knowledge base. Although developing
chatbot requires technical expertise (e.g., machine learning and conversation design) that is different from
developing traditional software systems, the use of chatbot framework and platform provided by Microsoft,
Facebook, Google and LINE application nowadays makes the chatbot development become much easier
(Abdellatif et al., 2020). Two types of chatbots were mostly used. Firstly, a chatbot with pre-programmed answers
is called a rule-based as it is responsible for matching pre-programmed answers with text messages or speeches
inputted. It chooses the system response from pre-programmed answers, based on the input text without creating
new texts. The responses from the chatbot were coded, organized and presented in the format of conversational
patterns. Secondly, a chatbot generating suitable responses is called a machine-learning based chatbot as it is
responsible for generating suitable responses drawn from user input via natural language processing and deep
machine learning. Therefore, it can generate answers better than the first type, based on user messages because it
acts like human by using algorithms of machine learning and techniques of deep learning (Hien et al., 2018). Some
chatbot framework and platform operates both types of chatbots.
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
17
Concerning chatbot design, scalability and usability can be major issues since they have a direct impact on the user
experience of a chatbot. Scalability refers to the way a chatbot design handles the increase of users, interactions,
and content contained in the chatbot, and usability refers to the actual usability of a chatbot design and if users are
able to perform the desired tasks. For rule-based chatbots, the developer needs to list all possibilities in advance
and the chatbot does not work if the input message differs from the predefined patterns, because the generalization
is very limited. The conversation context is only understood if the developer programs it directly and objectively.
For machine-learning chatbots, they are trained on the datasets called knowledge bases (Agarwal & Wadhwa,
2020). The user does not need to send messages exactly as they are in the knowledge base, because, after training
the chatbot, it can generalize new messages. The conversation context is comprehended the same way if the
knowledge base contains conversation flows. The course of action is determined based on the probabilities of an
incoming message being an intention registered on the knowledge base. The action chosen is the one with the
highest probability or none if the probabilities are not high enough (Andrade et al., 2022).
In educational settings, chatbots can be categorized into those with educational intentionality and those without.
Chatbots without educational intentionality are used in administrative tasks such as student guidance and
assistance (Fernoagă, 2018). Chatbots with educational intentionality are used in fostering teaching and learning.
Within this category, there are chatbots, which provide the framework of the learning process, that is, select and
arrange contents to fit the students’ needs and speed, and help in reflection and learning motivation. These bots act
as a learning companion which provides dialogue, collaboration and reflection (Molnár & Szüts, 2018).
Radziwill & Benton (2017) extracted quality attributes from each of the 32 papers and 10 articles, and grouped
them based on similarity. After two or three iterations, they noticed that in general, they were aligned with the ISO
9241 concept of usability, that is, 1) effectiveness, 2) efficiency and 3) satisfaction with which specified users
achieve specified goals in particular environments (Abran et al., 2003). In particular, effectiveness refers to the
accuracy and completeness with which specified users achieve their goals such as performance while efficiency
refers to how well resources are applied to achieve those goals such as functionality and humanity (Radziwill &
Benton, 2017). Satisfaction refers to affect, ethics and behavior, and accessibility. As a result, effectiveness,
efficiency, and satisfaction can be used as a checklist to design, develop, and evaluate chatbots.
3. Theoretical Background
The theoretical background was based from chatbot technology and its quality attributes covering effectiveness,
efficiency, and satisfaction. This study was conducted by using chatbot technology and the interaction between
chatbots and students in one of the entrepreneurship education programmes in higher education context. The
adoption of chatbot technology promoted interaction in 2 ways, that is, through pre-programmed answers
(Rule-based chatbot) and suitable answers (Machine-learning based chatbot). Concerning chatbot design,
scalability and usability have a direct impact on the user experience of a chatbot. Scalability is how chatbots handle
the increase of users, interactions and content contained in chatbots. Usability is how chatbots perform tasks that
users desire. In this study, chatbots were designed with scalability to handle users, interaction, and content of best
practices from successful entrepreneurs and usability to perform assigned learning tasks. In educational settings,
chatbots can be categorized into those with educational intentionality and those without. There were many quality
attributes to evaluate chatbots, that is, effectiveness, efficiency, and satisfaction. Prior research findings reported
the high impact of learning and satisfaction when using chatbot technology. In this study, Dialogflow was
employed as chatbot framework as it operated both types in order to support learning with education intentionality.
Student learning and satisfaction were used as learning outcomes. Therefore, student learning and satisfaction
were measured to see how chatbot technology supported learning in the selected entrepreneurship education
programme in higher education context.
4. Objectives
This study sought to answer two research questions: 1) What do chatbots look like? and 2) How students benefit
from the quality attributes of chatbot technology? through the following objectives:
1) To design and develop the chatbots of successful entrepreneurs
2) To examine the impact of the chatbots from student perceptions of learning and satisfaction
This will add to the literature on the impact of using chatbot technology to support teaching and learning in
education settings.
5. Method
The researcher used the quasi-experimental research design with mixed methods. The quantitative research was
concerned with the applicability of chatbots, and students’ perceptions of their satisfaction towards the chatbots.
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
18
The qualitative research was concerned with experts’ recommendations towards the chatbot improvement,
students’ perceptions of their learning, and suggestions towards the chatbot technology. The research was
conducted in 3 stages: 1) the chatbot and research instrument design, 2) the chatbot and research instrument
development, and 3) the chatbot implementation to examine the impact of the chatbots with a reflective journal of
student learning and a satisfaction questionnaire.
Stage 1 was concerned with reviewing literature of chatbot technology and studying best practices of successful
entrepreneurs from interviews published in newspapers, books and articles in order to use their best practices in
designing chatbots and research instruments.
Stage 2 was concerned with the chatbot and research instrument development. Dialogflow was used as chatbot
framework with Thai as a main language. LINE application was used as a platform to communicate with students.
Interactions with students and content contained in chatbots were designed in the form of questions and answers
like interviewing successful entrepreneurs about their best practices in doing business. Then, chatbots were created
with educational intentionality to support teaching and learning. Experts evaluated the applicability of chatbots
from key elements of chatbot technology. After making changes according to experts’ comments, a learning task
was assigned to students to ask and interact with chatbots through texts. Default answers were given to continue
interviewing when students sent a sticker, an emoji or unmatched texts. The researcher built a knowledge base to
contain content of best practices in doing business from successful entrepreneurs. Three different answers were
stored in the knowledge base to make chatbots interact like real humans, not repeated answers. Once the
knowledge base was completed, the researcher integrated Dialogflow with LINE application and named two
chatbots as 1) Masked Entrepreneur and 2) Unmasked entrepreneur. Then, quality attributes were examined.
Student learning and satisfaction were selected as quality attributes to measure the impact of chatbots in
educational settings. Relevant instruments to measure student learning and satisfaction were developed. Students
were encouraged to add two chatbots as friends to Line application and started to interview them about how
successful entrepreneurs succeeded in doing their business. After using chatbots, students were distributed a
reflective journal and a satisfaction questionnaire in order to measure how chatbots impacted their learning and
satisfaction.
Stage 3 was concerned with the chatbot evaluation and verification of relevant research instruments. 3 experts
were used in this stage. The pilot study of 20 non-targeted students was also included. Feedbacks from experts and
results from the pilot study were collected in order to analyze and use data to improve the chatbots. Next, the
chatbots were implemented with 24 first-year graduate students, who enrolled in a master’s degree of
entrepreneurship education programme at Kasetsart university, to examine the impact of the chatbots on student
learning and satisfaction.
5.1 Population and Sampling Procedures
The population was 30 graduate students under business education and entrepreneurship education curriculum at
the Faculty of Education, Kasetsart university. Only 24 first-year graduate students, who enrolled in a master’s
degree of entrepreneurship education programme, were chosen by using a purposive sampling technique as they
had to study entrepreneurship under entrepreneurship education curriculum.
5.2 Instruments
Two chatbots were designed and developed to examine the impact of the chatbots on student learning and
satisfaction with combination of 3 research instruments, that is, an evaluation questionnaire of the chatbot
applicability, a reflective journal of students learning, and a student satisfaction questionnaire. Details were given
below.
The researcher designed and developed two chatbots from reviewing the literature of chatbot technology and best
practices of successful entrepreneurs drawn from interviews published in newspapers, books and articles. 3 experts
used an evaluation questionnaire to examine the chatbot applicability. The pilot study of 20 non-targeted students,
was conducted.
The first instrument was an evaluation questionnaire related to the chatbot applicability from the key elements of
chatbot technology. It consisted of 3 aspects: content knowledge with 3 statements, content presentation with 4
statements, and benefits of chatbots with 5 statements. Therefore, 12 items of statements with a 5-point Likert
scale (from lowest to highest) with 1 open-ended question were generated. The evaluation criteria for average
scores (Mean) were set. 4.51-5.00 means highest level. 3.51-4.50 means higher level. 2.51-3.50 means average
level. 1.51-2.50 means lower level, and 1.00-1.50 means lowest level. The acceptable level of the chatbot
applicability was set at 4.51-5.00. The research instrument was then distributed to 3 experts to seek their opinions
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
19
of the chatbot applicability before testing with targeted students. The researcher then reviewed scores results and
utilized recommendations to improve the chatbots.
The second instrument was a reflective journal of student learning according to their perceptions in 3 areas, that is,
key learning points for success as entrepreneurs, application of key learning points, and reflection on learning. All
areas consisted of open-ended questions to identify student learning. The instrument was then distributed to
students to check their perceptions of learning from two chatbots. The item-objective congruence (IOC) index
ranged between 0.67-1.00 according to experts’ opinions.
The third instrument was a satisfaction questionnaire based on using chatbot technology to teach entrepreneurship.
It covered 4 areas: content of best practices from successful entrepreneurs with 5 statements, communication about
best practices from successful entrepreneurs with 3 statements, application of chatbots as learning tasks to teach
entrepreneurship with 4 statements, and benefits of chatbots in teaching entrepreneurship with 5 statements. 17
items of statements with a 5-point Likert scale (from lowest to highest) and 1 open-ended question for students to
provide thoughts on the use of chatbots were generated. The evaluation criteria for average scores (Mean) were set.
4.51-5.00 means highest level. 3.51-4.50 means higher level. 2.51-3.50 means average level. 1.51-2.50 means
lower level, and 1.00-1.50 means lowest level. The research instrument was then distributed to students to check
their perceptions of satisfaction, feedback and comments toward the use of two chatbots. The item-objective
congruence (IOC) index ranged between 0.67-1.00 from experts’ opinions. The Cronbach Alpha coefficient value
was 0.92 from the pilot study of 20 non-targeted students.
5.3 Data Collection
The researcher, as an instructor, asked 3 experts to examine the chatbot applicability and assigned learning tasks to
students to interview two successful entrepreneurs in the form of chatbots to find out who they were and what took
them to be successful entrepreneurs. Students were asked after completing the learning tasks to reflect on their
learning through a reflective journal and rate their satisfaction towards the use of chatbots including feedback and
recommendation on their uses. Finally, all instruments were checked for completeness and accuracy before
analyzing data.
5.4 Data Analysis
Data were obtained and analyzed from all research instruments, that is, an evaluation questionnaire of the chatbot
applicability, a reflective journal of student learning, and a student satisfaction questionnaire. Data of the chatbot
applicability, and students’ perceptions of their satisfaction were analyzed. A 5-point Likert scale (lowest to
highest) was applied to examine the chatbot applicability from experts’ opinions and rate the perceptions of
satisfaction from students’ opinions while data from all open-ended questions were collected to analyze experts’
recommendations, students’ perceptions of their learning, and suggestions for the use of chatbot technology. All
data were analyzed by mean, standard deviation, frequency, percentage, and content analysis.
6. Results and Discussion
The key findings of this research were given below:
6.1 The Chatbot Development
The researcher designed and developed two chatbots from reviewing the literature of chatbot technology and best
practices in doing businesses from successful entrepreneurs drawn from interviews published in newspapers,
books and articles. Two chatbots were both rule-based and machine learning-based types that generated as suitable
responses as possible from the pre-programmed answers with the ability to operate 24 hours while handling all
students at the same time. Students could use their own language to communicate with two chatbots and interview
as many times as they preferred by typing questions, keywords or texts that they wanted to know via smart phones,
tablets or computers.
The chatbots were developed by Dialogflow and implemented on LINE application that students were accustomed
to without need to install any extra application. Two chatbots were built, that is, the unmasked entrepreneur and the
masked entrepreneur, to act as two successful entrepreneurs and converse with students through texts, pictures and
stickers like real humans. Students were assigned a learning task to interview two chatbots of successful
entrepreneurs about their best practices in doing businesses; find out who they were, and what it took for them to
be successful. Pre-programmed and suitable responses when typing questions, keywords or texts were provided
without waiting time. Examples of two chatbots were shown in Figure. 1.
ies.ccsenet.
3 experts
w
applicabil
i
Tab le 1.
M
Ch
Co
n
1.
C
2.
C
3.
C
Co
n
4.
P
5.
P
6.
P
7.
P
Be
n
8.
C
9.
C
10.
11.
12.
To
t
Table 1 ill
u
mean and
high level
s
experts re
c
necessary
students.
T
more attra
chatbots b
org
w
ith extensiv
e
i
ty and recom
m
M
ean & standa
r
atbot Applicabilit
y
n
tent Knowledge
C
ontent is releva
n
C
ontent is in the s
C
ontent of entrep
r
n
tent Presentatio
n
P
resentation in th
e
P
resentation in an
P
resentation in a
v
P
resentation 24 h
o
n
efits of Chatbots
C
hatbots are easy
C
hatbots can be u
s
Chatbots are a g
o
Chatbots provid
e
Chatbots inspire
t
al
u
strated that t
h
standard devi
s
(Mean = 4.5
commended
a
changes to th
e
T
he results w
e
ctive. Picture
s
efore using w
i
Figure 1. Ex
a
e
knowledge a
n
m
ended the ch
a
r
d deviation o
f
y
n
t to best practice
s
ame language stu
d
r
eneurship is suit
a
n
e
human convers
a
interesting form
a
v
ariety of devices
o
urs
to use at anytime
s
ed many times a
s
o
od learning reso
u
e
responses witho
u
students to be en
t
h
e overall ave
r
ation of cont
e
6, S.D. = 0.3
8
a
dding more
c
e
chatbots fro
m
e
re positive wi
t
s
and stickers
w
i
th the targete
d
Internatio
n
a
mples of two
n
d experience
a
tbot improve
m
f
the chatbot a
p
s
from successful
e
dents use, which
i
a
ble
a
tion format
at
and anywhere
s
preferred
u
rce of best practi
c
u
t waiting time
t
repreneurs
r
age was at a
v
e
nt knowledge
,
8
, Mean = 4.8
3
c
onversational
m
experts’ opin
t
h recommen
d
w
ere added ac
c
d
students.
n
al Education St
u
20
chatbots of su
c
in the field o
f
m
ent. The app
l
p
plicability
e
ntrepreneurs
i
s informal and e
a
c
es from success
fu
v
ery high level
, content pres
e
3
, S.D. = 0.17
words to act
ions. The chat
b
d
ation to add
m
c
ording to the
u
dies
c
cessful entre
p
f
education an
d
l
icability of c
h
a
sy to understand
fu
l entrepreneurs
(Mean = 4.75
e
ntation, and
b
, Mean = 4.7
8
more like re
a
b
ots were the
n
m
ore pictures
a
non-targeted
s
p
reneurs
d
technology
v
h
atbots were il
l
𝑋
S.D.
5.00 0.00
4.33 0.58
4.33 0.58
4.67 0.58
4.67 0.58
5.00 0.00
5.00 0.00
5.00 0.00
4.67 0.58
4.67 0.58
5.00 0.00
4.67 0.58
4.75 0.22
, S.D. = 0.22).
b
enefits of ch
a
8
, S.D. = 0.38
,
a
l humans. T
h
n
pilot tested
w
a
nd stickers t
o
s
tudents’ opin
i
Vol. 15, No. 6;
v
erified the c
h
l
ustrated in Ta
b
Interpretation
Very high
High
High
Very high
Very high
Very high
Very high
Very high
Very high
Ve r y H i g h
Very high
Ve r y H i g h
Very high
In each aspe
c
a
tbots were at
,
respectively)
h
e researcher
m
w
ith 20 non-tar
g
o
make the ch
a
i
ons to impro
v
2022
atbot
b
le 1.
t, the
very
.
The
m
ade
g
eted
a
tbots
v
e the
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
21
6.2 Students’ Perceptions of Learning
The results of the students’ perceptions of learning were demonstrated in Tables 2, 3 and 4. Similar comments by
students were summarized. Tables 2, 3 and 4 showed the students’ perceptions of learning on key learning points
for success as entrepreneurs, application of key learning, and reflection on learning, respectively.
Table 2. Percentage & frequency of key learning points for success as entrepreneurs
Key Learning Points for Success as Entrepreneurs Frequency Percentage
1. How to solve problems and make decisions 17 70.83
2. How to create changes with creativity and innovation 13 54.16
3. How to build connection in and outside organization 10 41.66
4. How to improve oneself all the times 6 25.00
5. How to start business with passion 5 20.83
6. How to learn business by doing business 3 12.50
Table 2 showed the key learning points for success as entrepreneurs. 70.83% or 17 students learnt that solving
problems and making decisions made entrepreneurs successful. 54.16% or 13 students learnt that creating changes
with creativity and innovation made entrepreneurs successful. 41.66% or 10 students learnt that building
connection in and outside organizations made entrepreneurs successful. 25.00% or 6 students learnt that improving
oneself all the times made entrepreneurs successful. 20.83% or 5 students learnt that starting business with passion
made entrepreneurs successful. 12.50% or 3 students learnt that learning by doing made entrepreneurs successful.
Table 3. Percentage & frequency of application of key learning points
Application of Key Learning Points Frequency Percentage
1. Taking more reasonable risks to create changes with creativity and
innovation 15 62.50
2. Learning from mistakes, problems and decisions 14 58.33
3. Finding what one loves to do 12 50.00
4. Planning business with tools and technology 7 29.16
5. Taking business ideas to actions 5 20.83
6. Looking for networking opportunities 4 16.66
Table 3 showed the application of key learning points. 62.50% or 15 students learnt that they should take more
reasonable risks to create changes with creativity and innovation. 58.33% or 14 students learnt that they should
learn from mistakes, problems and decisions. 50.00% or 12 students learnt that they should find what they love to
do. 29.16% or 7 students learnt that they should plan business with tools and technology. 20.83% or 5 students
learnt that they should take business ideas to actions. 16.66% or 4 students learnt that they should look for
networking opportunities.
Table 4. Reflection on student learning
Students Reflection on Learning
Student 1 “I liked it. I felt that I really talked to real entrepreneurs to find out who they were so I had to concentrate and thought
carefully what to ask”
Student 2 “I liked it. I needed to plan questions and think carefully on answers received so that I could ask proper questions to find
out who were the successful entrepreneurs that I was talking to.
Student 3 “I liked the way chatbots led me to think and analyse identities of entrepreneurs from answers”
Student 4 “I liked learning with fun”
Student 5 “I liked learning that could happen at anytime and anywhere”
Student 6 “I liked learning that was interesting and colourful”
Student 7 “I liked learning through questioning and analysing knowledge gained from answers”
Student 8 “I liked this new way of learning entrepreneurship through chatbots”
Student 9 “I liked it because I had opportunities to practice questioning, thinking, reading, analysing and synthesizing”
Student 10 “I didn’t like this way of learning because some answers didn’t match my questions”
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
22
Student 11 “I liked it because I gained knowledge from questions I asked”
Student 12 “I liked active learning and I thought that I learnt by doing the learning tasks”
Student 13 “I liked it because I learnt instantly and enjoyed my own learning.
Student 14 “I liked it although I found it hard to concentrate on my learning tasks.
Student 15 “I liked learning with no boundaries and freedom to learn the way I questioned”
Student 16
Student 17
Student 18
Student 19
Student 20
“I liked fast learning that could happen 24 hours”
“I liked learning at my convenient time, space, and pace. I also enjoyed answers from questions that I inputted as
responses from chatbots were not the same everytime”
“I liked this new learning channel”
“I didn’t like it because chatbots were not real humans that I could meet in person. I knew that I was communicating
with a computer programme, not real successful entrepreneurs”
“I liked it because it was easy to ask and get answers as many times as possible until I learnt”
Student 21 “I liked it because I had freedom to set my own questions and get only answers that I really wanted to know”
Student 22 “I liked it. Learning was made easy”
Student 23 “I liked it. Learning was made easy”
Student 24 “I liked it. I learnt different perspectives from two successful entrepreneurs in forms of chatbots”
Table 4 showed that a majority of (91.66%) students’ perceptions of learning was positive because they liked
teaching entrepreneurship with chatbots and thought that chatbot technology highly supported their learning while
a minority of (8.34%) students’ perceptions of learning did not like teaching entrepreneurship with chatbots
because they knew that chatbots they communicated with were not real humans and sometimes gave answers not
matched with questions asked.
Based on the results, the impact of chatbots on student learning was very positive. Most students perceived that
they learnt better. They felt that they were equipped with right entrepreneurship knowledge to succeed as an
entrepreneur.
6.3 Students’ Perceptions of Satisfaction
Students were given a satisfaction questionnaire after the completion of the learning tasks to measure satisfaction
towards the use of chatbots. Table 5 illustrated students’ perceptions of satisfaction.
Table 5. Mean & standard deviation of perceived satisfaction of students
Perceived Satisfaction of Students
𝑋
S.D. Interpretation
Content of Best Practices from Successful Entrepreneurs
1. Content is easy to understand and informal 4.33 0.92 High
2. Content is related to best practices from successful entrepreneurs 4.63 0.58 Very high
3. Content is attractive 4.67 0.56 Very high
4. Content is various 4.63 0.58 Very high
5. Content suits student needs 4.50 0.72 High
Communication of Best Practices from Successful Entrepreneurs
6. Communication channel to access best practices of successful
entrepreneurs 4.67 0.64 Very high
7. Communication of best practices from successful entrepreneurs in
the format of conversation 4.54 0.72 Very high
8. Communication can be done as many times as students prefer 4.75 0.44 Very high
Application of chatbots as learning tasks to teach entrepreneurship
9. Chatbots are easy to use with no extra technical skills 4.75 0.44 Very high
10. Chatbots can be used at anytime and anywhere convenient to
students 4.92 0.28 Very high
11. Chatbots can be accessed via any device 4.83 0.38 Very high
12. Chatbots cost no extra expenses 4.96 0.20 Very high
Benefits of chatbots in teaching entrepreneurship
13. Learning from the best 4.54 0.66 Very high
14. Learning resource of best practices from successful
entrepreneurs 4.71 0.55 Very high
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
23
15. Motivation to learning 4.58 0.65 Very high
16. Inspiration to be entrepreneurs 4.58 0.65 Very high
17. Practical way to teach entrepreneurship 4.54 0.59 Very high
Total 4.65 0.44 Very high
Table 5 illustrated that the perceived satisfaction towards the chatbot use was at a very high level (Mean = 4.65,
S.D. = 0.44). In each area, students perceived satisfaction in content of best practices from successful
entrepreneurs, communication of best practices from successful entrepreneurs, application of chatbots as learning
tasks to teach entrepreneurship, and benefits of chatbots in teaching entrepreneurship at very high levels (Mean =
4.55, S.D. = 0.56, Mean = 4.65, S.D. = 0.52, Mean = 4.86, S.D. = 0.24, Mean = 4.59, S.D. = 0.55, respectively).
Students thought that the chatbots were an interesting, innovative, and fun teaching way with game-like learning
tasks to do at anytime, anywhere and many times as they preferred.
The main findings indicated that chatbot technology positively impacted student learning and satisfaction. Results
showed that most of students learnt about how to solve problems and make decisions while few students learnt
about how to learn business by doing business. Furthermore, most of students wanted to apply their learning by
taking more reasonable risks to create changes with creativity and innovation while few students wanted to apply
their learning by looking for networking opportunities. Also, most of students felt that chatbot technology
supported teaching and learning while few students felt that chatbot technology obstructed teaching and learning
because chatbots were not real humans and not able to provide matching answers, which was in accordance with
Sandu (2020) who found that biggest concerns for students involved getting incorrect answers by using chatbots.
In addition, students were highly satisfied with chatbot technology, which was in accordance with Tavichaiyuth
and Rattagan (2021), who reported that some of the most popular and appropriate ways to assess a chatbot’s
performance are user satisfaction wh ile Chaipram et al. (2020) conducted the satisfaction survey and found out that
the chatbot helped students well and students were satisfied with the ease of use the most. Although many useful
quality attributes were suggested, there was still no guidance to determine what to apply (Radziwill & Benton,
2017). The lack of reliable and efficient chatbots’ evaluation method limits the correct testing and comparison of
different models (Caldarini, Jaf, & McGarry, 2022). Apart from that, using Dialogflow as chatbot framework was
also one of the reasons leading to learning and satisfaction as previous researches suggested that Dialogflow was
outperforming other NLUs in intents classification and entity extraction (Abdellatif et al., 2021). All findings
suggested that chatbot technology seemed to have more advantages than disadvantages in supporting teaching and
learning if it was properly designed and developed with educational intentionality according to the scalability and
usability of chatbots. Therefore, it can be implemented as a powerful tool to teach entrepreneurship in
entrepreneurship education programmes in higher education context. The findings were consistent with previous
researches. They were in line with Vanichvasin (2021) who found that the application of chatbot technology in
education context to promote student knowledge of research yielded positive views of the chatbot as it created
positive learning outcomes and provided better individualized learning experience while Winkler and Söllner
(2018) reported that chatbots in education promised to have a significant good impact on learning success and
satisfaction of students, which was in accordance with Alepsis and Virvou (2011) who reported that their study
showed success in the use of conversational systems with high acceptance from both students and teachers that
perceived the system as a very useful tool for health education and Preecha (2020) who reported that learning with
an artificial intelligence chatbot was efficient as students can quickly find answers anytime and anywhere. Positive
impact may stem from the researcher’s experience in conducting this kind of research before but with different
type of chatbot frameworks and platforms. Also, the ability of chatbots to mimick conversation of successful
entrepreneurs like real humans is powerful because chatbots can do a task that would otherwise not be possible or
take a longer time (Fryer, Nakao & Thompson, 2019) like interviewing real entrepreneurs, who are difficult to
access and interview about their best practices in doing business in real lives. Apart from that, students felt
comfortable to interview as many times as possible without worries about using proper or unproper
communication to find out what it took to be successful entrepreneurs, which was in accordance with Hill et al.
(2015), who analyzed 100 messaging conversations and found that humans carried on significantly longer
messaging conversations with the chatbot than with other humans. While each message sent to the chatbot was
shorter and the vocabulary not as rich, this finding demonstrated the ease with which humans can communicate
with chatbots and the quantity of conversational engagement that was possible. The findings helped instructors
gain a better understanding of chatbot technology and provided strong evidence about how to integrate chatbot
technology as learning tasks to achieve better student learning and satisfaction.
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
24
7. Conclusion and Recommendation
This study was limited using chatbots to perform a learning task and interact with algorithm only in the form of
interviewing successful entrepreneurs while there were many types of learning tasks and algorithm that can be
used to support teaching and learning. The researcher acknowledged this limitation but strongly believed that the
findings would add to the literature on the impact of chatbot technology in 2 areas, that is, how to design and
develop a chatbot with educational intentionality according to its scalability and usability to support teaching and
learning in educational settings and how students benefited from quality attributes of a chatbot. As a result, the
findings reflected that chatbot technology positively impacted student learning and satisfaction. With its high
potential, it can be implemented as a powerful tool to teach entrepreneurship in entrepreneurship education
programmes in higher education context. Teachers can formulate strategies for using chatbot technology in
designing and developing chatbots (rule-based or machine learning based type) to match their learning tasks
(interviewing or other possible learning tasks), platform to use (LINE application or other possible platforms) and
quality attributes to measure (learning, satisfaction or other possible quality attributes) as the results indicated that
students learnt better and satisfied more with this technology. It is hoped that chatbot technology will be
extensively used in education settings with educational intentionality due to its’ positive impact on student learning
and satisfaction. However, the current study used a small sample size and only focused in the entrepreneurship
education programme. Therefore, it is anticipated that the next study may use a large sample size and apply
chatbots in other education programmes that there is entrepreneurship teaching in the programmes to further
investigate learning outcomes with a broader extent in chatbot technology and different programmes in higher
education context for more insights in order to establish best practices of teaching entrepreneurship. In addition,
the negative impact may be considered in future studies to find out problems in using chatbot technology for better
design and development although the research objectives were achieved, there were few students reflecting that
chatbots could not perform their functions well enough. As such, algorithms for interactions may be revised for
better results.
Acknowledgements
Many thanks to experts for their kind assistance and invaluable advice during the chatbot development and the
verification of the chatbot applicability including great recommendation on its improvement. Also, very special
thanks to students in a master’s degree of entrepreneurship education programme at Kasetsart University for their
time and participation. This research study would not have been possible without their kind support.
References
Abdellatif, A., & Costa, D., & Badran, K., & Abdalkareem, R., & Shihab, E. (2020). Challenges in chatbot
development: A study of stack overflow posts. In Proceedings of the 17th International Conference on
Mining Software Repositories (pp. 174-185). https://doi.org/10.1145/3379597.3387472
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003, April). Consolidating the ISO usability models. In
Proceedings of 11th international software quality management conference (pp. 23-25).
https://doi.org/10.1023/A:1025869312943
Agarwal, R., & Wadhwa, M. (2020). Review of state-of-the-art design techniques for chatbots. SN Computer
Science, 1(5), 1-12. https://doi.org/10.1007/s42979-020-00255-3
Alepsis, E., & Virvou, M. (2011). Automatic generation of emotions in tutoring agents for affective e-learning in
medical education. Expert Systems with Applications, 38(8), 9840-9847.
https://doi.org/10.1016/j.eswa.2011.02.021
Andrade, G. G., Francisco, C. M. D., Santas, G. A., Da Costa, J. P., & De Sousa, R.T. (2022). A
conversation-driven approach for chatbot management. IEEE Access (pp. 1-1).
https://doi.org/10.1109/ACCESS.2022.3143323
Augello, A., Gentile, M., & Dignum, F. (2018). An overview of open-source chatbots social skills. In S. Diplaris,
A. Satsiou, A. Følstad, M. Vafopoulos, & T. Vilarinho (Eds.), INSCI 2017. LNCS (Vol. 10750, pp. 236-248).
Springer, Cham. https://doi.org/10.1007/978-3-319-77547-0_18
Bayan, A. S. (2005). A corpus based approach to generalising a chatbot system. Retrieved from
https://etheses.whiterose.ac.uk/1323/1/abushawar.pdf
Benotti, L., Martínez, M. C., & Schapachnik, F. (2014). Engaging high school students using chatbots. In
Proceedings of the 2014 Innovation and Technology in Computer Science Education Conference (pp.
63-68). ACM, New York, USA. https://doi.org/10.1145/2591708.2591728
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
25
Bii, P. (2013). Chatbot technology: A possible means of unlocking student potential to learn how to learn.
Educational Research, 4(2), 218-221.
Caldarini, G., Jaf, S., & McGarry, K. A. (2022). Literature Survey of Recent Advances in Chatbots. Information,
13(41), 1-22. https://doi.org/10.3390/info13010041
Chaipram, N., Tumnut, N., & Rattanapoka, C. (2020). Educational response and notification system using LINE
Bot. Journal of Information Science and Technology, 10(2), 59-70.
Fernoagă, V., Stelea, G. A., Gavrilă, C., & Sandu, F. (2018). Intelligent Education Assistant Powered by Chatbots.
eLearning & Software for Education, 2.
Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences,
interest and competence. Computers in Human Behaviour, 93, 279-289.
https://doi.org/10.1016/j.chb.2018.12.023
Hien, H. T., Cuong, P.-N., Nam, L. N. H., Nhung, H. L. T. K., & Thang, L. D. (2018). Intelligent assistants in
higher-education environments: the FIT-EBot, A chatbot for administrative and learning Support. In
Proceedings of the 9th International Symposium on Information and Communication Technology (pp.
69-76). ACM, New York, USA. https://doi.org/10.1145/3287921.3287937
Hill, J., Ford, W. R., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison
between human-human online conversations and human-chatbot conversations. Computers in Human
Behaviour, 49, 245-250. https://doi.org/10.1016/j.chb.2015.02.026
Klapper, R. (2004). Government goals and entrepreneurship education–An investigation at a Grand Ecole in
France. Education & Training, 46(3), 27-37. https://doi.org/10.1108/00400910410531787
Molnár, G., & Szüts, Z. (2018). The role of chatbots in formal education. In 2018 IEEE 16th International
Symposium on Intelligent Systems and Informatics (SISY). https://doi.org/10.1109/SISY.2018.8524609
Nilros, K., & Lidén, A. (2020). Perceived benefits and limitations of chatbots in higher education. Retrieved
from http://www.diva-portal.org/smash/get/diva2:1442044/FULLTEXT01.pdf
Preecha, T. (2020). Application of Artificial Intelligence Chatbot for Learning. Royal Thai Air Force Medical
Gazette, 66(2), 64-73.
Radziwill, N., & Benton, M. (2017). Evaluating quality of chatbots and intelligent conversational agents.
arXiv:1704.04579. https://doi.org/10.48550/arXiv.1704.04579
Sandu, R. (2020). Adoption of AI-Chatbots to Enhance Student Learning Experience in Higher Education in
India. https://doi.org/10.1109/ITHET46829.2019.8937382
Shawar, B. A., & Atwell, E. (2007). Chatbots: Are they really useful? Journal for Language Technology and
Computational Linguistics, 22(1), 29-49.
Solomon, G. T., Duffy, S., & Tarabishy, A. (2002). The State of Entrepreneurship Education in the United States:
A Nationwide Survey and Analysis. International Journal of Entrepreneurship Education, 1(2), 65-86.
Song, S. M., Thominathan, S., & Khalid, N. A. (2021). Entrepreneurial Intention of UiTM Students and the
Mediating Role of Entrepreneurship Education. Asian Journal of University Education (AJUE), 17(2),
236-251. https://doi.org/10.24191/ajue.v17i2.13405
Souitaris, V., Zerbinati, S., Al-Laham, A. (2007). Do Entrepreneurship programmes raise Entrepreneurial
intention of science and Engineering students? The Effect of Learning, Inspiration and Resources. Journal
of Business Venturing, 22(4), 566-591. https://doi.org/10.1016/j.jbusvent.2006.05.002
Tavichaiyuth, N., & Rattagan, E. (2021). Developing chatbots in higher education: A case study of academic
program chatbot in Thailand. Retrieved from
http://https://as.nida.ac.th/wp-content/uploads/2021/11/6210422022_Nicha_Paper_PDF.pdf
Vanichvasin, P. (2021). Chatbot development as a digital learning tool to increase students’ research knowledge.
International Education Studies, 14(2), 44-53. https://doi.org/10.5539/ies.v14n2p44
Westhead, P., & Wright, M. (2013). Entrepreneurship: A very short introduction. Oxford University Press.
https://doi.org/10.1093/actrade/9780199670543.001.0001
Winkler, R., & Söllner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis.
https://doi.org/10.5465/AMBPP.2018.15903abstract
ies.ccsenet.org International Education Studies Vol. 15, No. 6; 2022
26
Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017). A new chatbot for customer service on social media.
In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 3506-3510).
ACM, New York. https://doi.org/10.1145/3025453.3025496
Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution
license (http://creativecommons.org/licenses/by/4.0/).