ArticlePDF Available

Abstract and Figures

Current lifelong learning platforms offer users a query option to select a wide variety of courses. However, finding a suitable course among the seemingly endless catalogs of options presented by the platforms is not straightforward. We argue that digital counseling can enhance this process. In this paper, we present a set of three formative studies where we explored the main aspects that can provide the counseling needed. The methods comprise an analysis of user profile characteristics and learning analytics indicators (e.g., learning progress/self-regulation) by means of an expert workshop, evaluating the feasibility of current technologies (e.g., natural language processing) for automatically assessing users' competencies, and a survey on the use of Chatbots as the interaction interface between the users and the lifelong learning portals. The analysis resulted in the extraction of basic requirements for digital counseling. We conclude the paper by presenting a system design derived from these studies.
Content may be subject to copyright.
Caught in the Lifelong Learning Maze: Helping People with
Learning Analytics and Chatbots to Find Personal Career Paths
Atezaz Ahmad*, Natalie Kiesler, Daniel Schiffner, Jan Schneider, and Sebastian Wollny
AbstractCurrent lifelong learning platforms offer users a
query option to select a wide variety of courses. However,
finding a suitable course among the seemingly endless catalogs
of options presented by the platforms is not
straightforward. We argue that digital counseling can enhance
this process. In this paper, we present a set of three formative
studies where we explored the main aspects that can provide the
counseling needed. The methods comprise an analysis of user
profile characteristics and learning analytics indicators (e.g.,
learning progress/self-regulation) by means of an expert
workshop, evaluating the feasibility of current technologies (e.g.,
natural language processing) for automatically assessing users'
competencies, and a survey on the use of Chatbots as the
interaction interface between the users and the lifelong learning
portals. The analysis resulted in the extraction of basic
requirements for digital counseling. We conclude the paper by
presenting a system design derived from these studies.
Index TermsLearning analytics, indicators, dashboard,
Chatbot, lifelong learning, natural language processing
I. INTRODUCTION
Lifelong learning is the continuous development of
knowledge and skills, which happens throughout a lifetime [1,
2]. It plays an essential role in improving societal and
individual well-being [3] and increases the employability,
adaptability, and mobility of educated professionals [4].
Moreover, it helps professionals, such as IT experts, in
keeping up with important technological advances like new
programming languages, which tend to appear faster in
recent years [5, 6].
In contrast to primary, secondary, and higher education,
lifelong learning is currently a primarily decentralized
process. To provide learners with an overview of learning
opportunities, portals for lifelong learning courses [79]
were established in Germany. However, finding suitable
courses is not straightforward: Learners need to be aware of
their competence level [10], translate their learning desires
into keywords, and perform a respective selection of suitable
courses. This process is usually accompanied and guided by
counseling via trained personnel and hence is limited by the
availability of these.
We argue that the enhancement of lifelong learning portals
with digital counseling can help to address the previously
mentioned challenges. We propose three driving
technologies for the implementation of this digital counseling
that helps people to navigate toward their desired career path.
First, we consider it important to have a user profile that
Manuscript received September 25, 2022; revised October 23, 2022;
accepted November 23, 2022.
The authors are with the DIPF | Leibniz Institute for Research and
Information in Education, Frankfurt, Germany.
*Correspondence: a.ahmad@dipf.de (A.A.)
stores and visualizes the competencies of the users,
where competency is the set of knowledge, cognitive abilities,
skills, and dispositions of an individual in the context of a
task [11, 12]. With the help of Learning Analytics (LA)
techniques, this profile can show/visualize the users’ current
and projected learning progress. LA refers to the collection,
analysis, and reporting of learners’ data for understanding
and improving learning [13]. It has been acknowledged that
LA Dashboards (LADs) play a role in learners’ academic
performance, understanding level, self-regulated learning,
and academic motivation [1416].
Second, for the competencies that are not explicitly stated
or whose level is not well defined in the user profile, we
contemplate the option of having a light assessment that can
help users to find available courses suitable for their
competency level and in turn update their profile. Finally, to
provide digital counseling, we opt for the interaction between
user and portal to be mediated by a Chatbot, as it has been
shown that Chatbots have the potential to function as mentors
in educational settings [17].
This paper presents several preliminary evaluations where
we explored the suitability, feasibility, and design concepts
for the use of the three aforementioned technologies in the
development of a digital counseling solution for a lifelong
learning portal. We guided our research through the
following research questions:
The main research question is: how can we support
learners in finding suitable courses through digital
counseling?
To answer this overarching research question, we derived
the three following ones that are linked to our proposed
driving technologies to support digital counseling.
Concerning the creation of user profiles for a lifelong
learning platform, first, we want to know the important
characteristics of the user profiles that lead to our first
research question.
RQ1: What user-profile characteristics based on LA
instruments can be used to help learners find more suitable
courses?
It is not feasible to manually create light assessments for
all competencies and available courses. Hence we want to
explore whether technological advancements can help
automate this process leading to our second research
question.
RQ2: To what extent can we use Natural Language
Processing (NLP) techniques to automatically extract
questions for the light assessment?
To build a Chatbot that supports digital counseling for
lifelong learning, it is important to address some of its
constraints and main features, thus leading to our third
research question.
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
423
doi: 10.18178/ijiet.2023.13.3.1822
RQ3: What characteristics of Chatbots are relevant to
providing learners with counseling regarding the course
selection process?
II. CURRENT SOLUTIONS
There are several non-commercial, state-supported
information portals where course providers can promote their
training and users can search for lifelong learning courses.
The main goal of lifelong learning portals is to provide easy
access to information and knowledge to help users to find
further educational courses. In this context, traditional
German open lifelong learning portals (see Fig. 1) usually
collect information about available learning materials
(courses, seminars, etc.) from different providers and store
them in a central database. The main feature of respective
platforms is a meta-search engine, which offers courses for
further education of any kind, whereas course providers may
be commercial and non-commercial. Users can then query
the database to search for courses based on filters such as
topics, location, schedule, etc. In some cases, such as the
state-supported IWWB portal (German ―InfoWeb
Weiterbildung‖, in English ―InfoWeb Further Education‖),
users have to select a suitable course out of over three million
course offerings [7]. Other examples of further education
portals are the portal provided by the state of Hesse, Germany,
with more than 15 thousand course offerings [8], and the
further education portal by the federal government of
Germany, with more than 28 thousand course offerings [9].
All three examples are publicly available. We are currently
not aware of comparable portals that are commonly used in
other countries and regions.
In contrast to these portals, there are several other
commercial or semi-commercial, non-open platforms related
to lifelong learning who offer the search for their own
lifelong learning courses, for example:
Khan Academy [18]: a non-commercial online learning
platform based in the United States that relies on the use
of a search bar for users to find courses free of charge.
Coursera [19]: a commercial platform started in the
United States, which offers university-level courses and
certification programs that use categorized search and
registration is required to find courses.
Iversity [20]: a commercial platform based in Berlin,
Germany, that provides online courses and higher
education lectures that rely on a search bar for users and
categories to find learning materials.
Futurelearn [21]: another commercial online learning
platform based in the United Kingdom that offers course
categories and a search bar to find relevant courses.
Linkedin Learning [22]: formerly known as Lynda.com,
is a commercial educational platform based in the United
States. They use profile logs and Artificial Intelligence
(AI) to recommend courses to get a dream job. Due to
their commercial nature and the lack of respective
research, the approach can neither be evaluated nor
replicated.
In-depth analyses and comparisons of MOOC platforms
for lifelong learning are presented in [2325]. The main
difference is that the German lifelong learning portals are
open (i.e., search without registration), supported by the
government, and they combine different course providers
and their offers.
With such a huge amount of offerings, finding a suitable
course that will support the further professional development
of users is not straightforward. With the exception of a few
cases that also provide basic course recommendations, most
current platforms expect the user to hopefully find a suitable
course in the ocean of available courses with the use of basic
search functionalities. Therefore, we propose to enhance
lifelong learning portals via digital counseling.
Fig. 1. Lifelong learning platform system architecture. Users query the
underlying database by formulating queries or following prestructured
pages.
III. METHOD
To realize digital counseling, which can enhance a lifelong
learning portal, the plan is to follow a design-based research
methodology. This is an iterative process whose objective is
to support the development of solutions and interventions to
complex problems [26]. In this paper, we present the
preliminary findings that will lead to the design and
development of the prototypes that will be developed for the
first iteration.
For the investigation of basic requirements for the content
and visualizations of the user profile mentioned in RQ1, we
conducted a profile workshop with 15 experts in the field of
technology-enhanced learning. Experts were divided into
three groups of five persons each. They had 45 minutes to use
their expertise in LA to design a dashboard that will allow the
users to visualize their competencies so that it can support
users in the search for suitable lifelong learning courses. To
design such a dashboard, we asked the participants to use
both Google Drawing [27] and Open Learning Analytics
Indicator Repository (OpenLAIR) [28] for selecting relevant
LA indicators. OpenLAIR is a LA tool that helps course
designers, teachers, students, and educational researchers to
make informed decisions about the selection of learning
activities, LA indicators, and metrics or measurements for
their course design or LA dashboard [28].
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
424
We then wanted to explore whether lifelong learning
portals need to create an individualized user profile or if they
could import generic user profiles. Therefore, in a second
step, we analyzed the fields and metadata available at
Europass [29], which is an online platform designed to create
and share Curricula Vitae. We investigated to what extent an
integration with such a platform can be used to create a user
profile that helps with lifelong learning digital counseling
considering the basic requirements of the user profiles
extracted from the profile workshop.
To examine the use of NLP techniques for the automatic
extraction of light assessment, we evaluated the metadata of
the course description of the available ―JavaScript‖ courses
in IWWB [7] and in the following books [3032] following
the methods and scripts defined in [33], that leverage
artificial jabbering for automatic text comprehension
question generation. The restriction to programming
languages has been made in advance to reduce the overall set
of possible hits.
To explore the relevant characteristics of Chatbots to
provide lifelong learners with counseling regarding the
course selection process, we also conducted an exploratory
survey with 89 university students where we asked the
following questions:
1) Have you ever searched for further training online
(seminar, course, etc.)?
2) Do you already have some experience with Chatbots?
Chatbots are applications that allow you to chat with a
technical system (e.g., Siri, Amazon customer service).
3) Assuming you could use a Chatbot to find a suitable
course, which features would be important to you?
4) What data are you willing to share with a Chatbot?
5) Which courses should the Chatbot display to you?
Besides the survey, we held a Chatbot workshop with the
experts in technology-enhanced learning. As part of the
workshop, we asked them to design the communication flow
chart of users interacting with a Chatbot in three groups of
five, whereby the aim of the Chatbot was to help users find
suitable courses. Experts had 45 minutes to complete this
task.
IV. RESULTS
RQ1: The designs of the three groups within the profile
workshop pointed out the relevance of knowing the metrics
or measurements that determine the different competency
levels of a user. For example, the number of courses taken in
a programming language (metrics) will help to determine the
programming competency of the user (indicator) [34]. All
three groups considered presenting these competencies as a
spider chart. They argued that this indicator could help users
to achieve the desired career path (see Fig. 2.). Two groups
mentioned the relevance of showing in the dashboard
visualization how taking the available courses would impact
their current competency level.
The mockup of the first group further proposed a weight
balance scale visualization. This feature allows users to select
courses from a list of available options that are aligned with
their desired competencies. Once a course is selected, the
scale then visualizes the impact that the selected course has in
helping the user to achieve their desired competency level.
This group also proposed to use a word cloud (also known as
a tag cloud) visualization of the users’ skills, experience, and
keywords that may help understand the level of a user.
Furthermore, they proposed the use of a visualization, where
it is possible to compare the current competencies of the user
against the competencies from the desired career path of the
user that could be extracted from job offerings.
Fig. 2. Mock-up design of the first expert group.
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
425
We examined the metadata included in Europass to see to
what extent such an open profile can help with digital
counseling for lifelong learning. In the case of language
learning, the Europass profile has excellent metadata because
it includes the already established Common European
Framework of Reference (A1 to C2) for each of the
languages included in the users’ profiles. For all other
competencies, most data fields are free text where users type
the title of their job or the studies. Important information to
determine the competencies of the user, such as the contents
of the courses taken or job tasks, is missing. Therefore, in its
current state, the data from Europass is not automatically
processable and hence not very helpful in providing digital
counseling in this regard.
RQ2: To answer our RQ2 concerning the use of NLP, we
automatically extracted questions for a potential light
assessment, i.e., a rough estimation of the competency level.
As a first step, we looked at the available ―JavaScript‖
courses in several databases and the respective information
provided in the course description. This way, we identified
1316 different offerings. We then analyzed the description of
20 randomly selected course offers. On average, the course
description was 279.45 words long with a standard deviation
of 309.5. The course with the longest description had 1450
words, and the one with the shortest description had 38. The
examination of the descriptions revealed that 12 of them
included logistics about the course offering e.g., number of
sessions, schedule of the sessions, etc. Eight of them included
information about the relevance of the course and the
discussed technologies. Six of them provided theories about
the technologies taught during the course. Five of them
described the course via the forms of conducted exercises.
Four of them comprised only keywords of the topics covered
in the course. These fundamental results show how difficult it
is to automatically extract questions that may help assess the
level of users’ competencies. Without a supervised model,
for example, it would be easy to get a vast number of
questions asking for the contact email of the teacher
providing the course or on which days of the week someone
can learn about JavaScript. These questions, however, are
unsuitable for the assessment of a user’s competencies.
To look for further possibilities to automatically extract
questions that can help to assess the competency level of
users, we used the script and methods defined in [33] in the
first chapter of the following books [3032]. We extracted 71
questions along with their corresponding answers (both of
them in German, translated by the authors). Two examples of
a question and answer pair are illustrated below:
Example 1:
Question: What are mnemonics?
Answer: To simplify the effort of writing directly in
machine language, so-called assembler languages appeared
from the 1940s onward. Assembler languages represent
machine code in a readable form. For the (manageably)
numerous machine commands, short English-like commands
(so-called mnemonics) were created, which can be translated
into machine code with one-to-one correspondence.
Example 2:
Question: What are tags in HTML?
Answer: Tags are symbols that mark the beginning and the
end of an HTML element.
Despite minor language errors, the question and answer
pairs represent the contained concepts reasonably well.
Therefore, this first exploration seems to be a promising
approach that is worth further investigation and assessment in
a greater scope.
RQ3: To address the third research question concerning
the use of Chatbots for lifelong learning counseling, we first
developed an exploratory online survey comprising five
questions. Most of the 89 respondents were university
students. Not all participants replied to all questions. In the
following, we will highlight the main results.
Fig. 3. Assuming you could use a Chatbot to find a suitable course, which
features would be important to you? (multiple choice question, n=89).
Fig. 3 shows the results regarding the features of the
Chatbot that are important to potential users. 76 of the 89
participants (85%) opted for a goal-oriented conversation
that should quickly yield a suitable course. In addition, 58 out
of 89 participants (65%) selected that a Chatbot should be
accessible at any time so that the users are not limited in terms
of time. Another important criterion worth mentioning is that
a Chatbot should be able to answer general questions about
the recommended courses, such as the place, price, duration,
and time of the course. This option was selected by 54 out of
89 participants (60%). What is particularly striking is that
few participants chose the appealing User Interface (15 of 89
participants, 16%) or having an engaging/stirring
conversation (12 of 89 participants, 13%) as criteria.
Fig. 4. Which courses should the Chatbot display to you? (Single choice
question, n=89).
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
426
The responses to the question ―Which courses should the
Chatbot display to you?‖ are summarized in Fig. 4. Most
participants (47 out of 89 participants, 53%) prefer to see a
small selection of potentially relevant courses and then
decide on a course. The answers also indicate that fewer
participants are interested in courses that are rated high by
others (22 out of 89 participants, 25%).
Another aspect worth mentioning is that only 8 of the 89
participants (9%) prefer to select courses offering certificates
of attendance or successful completion. The price of a course,
however, does not seem to be an important selection criterion,
as only one of the participants selected this option. In
addition, none of the participants showed interest in the
cheapest course.
Fig. 5 shows the responses to the question regarding the
data respondents are willing to share with a Chatbot. 73 out
of 89 participants (82%) only want to share personal data that
is relevant to the course search. The remaining 16
participants (18%), however, are open to sharing their
personal data for a better user experience.
To dig deeper into the characteristics of a Chatbot
providing counseling during lifelong learning course
selection, we utilized the conducted Chatbot workshop with
15 experts in the field of technology-enhanced learning. In
groups of five, participants were asked to design a flowchart
for a Chatbot that would help them find valuable courses for
lifelong learning. As a result, all three groups developed a
basic interaction flow with the Chatbot to help them find
suitable courses based on their desires and competencies.
Group one proposed a Chatbot for a system that has access
to the user profile (e.g., educational background, skills),
where the Chatbot is helping people in changing their job
and/or improving or teaching new skills. The Chatbot
analyzes the user profile data and depicts what the user is
good at and what is missing. Then the Chatbot offers a
selection of training/courses based on the user's interests and
profile information.
Fig. 5. What data are you willing to share with a Chatbot? (Single choice
question, n=89).
Group two suggested a Chatbot named ―Learning
Companion‖. In this case, the Chatbot is supposed to help the
user plan their learning, make suggestions based on their
profile, and provide some effective support.
Group three designed a Chatbot that asked the users a few
questions regarding their non-cognitive skills and the
technologies they know. After the user answers the questions,
the Chatbot recommends a course and asks the user if the user
is interested in the course or not. If this is not the case, the
Chatbot shows another course until the user is interested in a
course. After this section, the Chatbot lists all the similar
courses.
V. DISCUSSION
To answer our first RQ, what user profile characteristics
can be used in the context of LA instruments to help find
suitable courses, our results show that profiles should reveal
the user's competency levels. Another important attribute in
the profiles is the desired career path of the user. By knowing
the current competencies of the user and their desired career
path, it is possible to offer visualizations showing how the
user matches with their desired path. Moreover, it enables a
system to suggest courses that can guide users toward their
desired path. Experts in our study suggested providing these
visualizations in the form of a spider chart, which in LA
studies have been shown to support learners by illustrating
their current learning progress and motivating them to learn
and master new skills [16, 35].
When looking at available open options, such as Europass,
to evaluate the utilization of user profiles, we concluded that
the information in Europass only allows the inference of
language-related competencies. Therefore, we do not
consider Europass suitable for this purpose. Without
additional information about the provided information within
the CVs, a useful automatic application is not feasible.
However, we think that having a digital platform is a good
starting point to enable the collection of suitable information.
An important aspect to provide counseling regarding the
course selection process of lifelong learners is to identify
their current competencies. We argue that one way to achieve
this is through some type of competency assessment.
Manually creating assessment questions for all possible
course topics is not feasible. Accordingly, RQ2 concerns the
automatic extraction of questions that can help with a light
assessment of users' competencies. Results from our
consultations show that automatically extracting useful
questions from the course descriptions within the portals is
not feasible, mainly because the information contained in
these descriptions is highly heterogeneous and in many cases
not related to the content or competencies targeted by the
courses. However, we saw that, in the specific case of
―JavaScript‖, it is possible to automatically extract good
enough questions by using the content of textbooks about the
topic. As a limitation, we need to explore to what extent the
NLP techniques used can work for other course topics.
Nonetheless, the obtained results are promising.
To answer our third RQ, we investigated the relevant
characteristics of Chatbots to provide counseling for learners
during the course selection process. Our results from the
online survey revealed that the Chatbot's conversation should
be concise and quickly recommend a suitable course. The
Chatbot should be intelligent enough to answer basic
questions regarding the course, and it must be available 24/7.
The user craves the usability of the Chatbot instead of the
appealing user interface, which is a secondary preference.
Our results further reveal that course prices seem less
relevant, which is possibly due to the group of survey
participants. However, a more in-depth survey is needed with
a wide variety of participants (students, experts, researchers,
and teachers) to validate the outcomes presented in this study.
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
427
As identified by our survey participants, a preselection of
courses is essential. Nonetheless, more research is required
on how to accomplish that. Even in Germany, people seem to
be willing to share data that is relevant for a successful course
search. A more in-depth investigation is needed to identify
(data-saving) information for good search results.
The Chatbot workshop with experts confirmed our
intuition that Chatbots are a good tool to assess the
competency level of users and hence serve in the counseling
aspect of searching for lifelong learning courses. Moreover,
they seem to be a suitable user interface to provide this
counseling by helping people to find suitable courses.
Fig. 6. The proposed structure extends the existing lifelong learning platform
with digital counseling.
By examining the answers to our research questions, we
derive the following design for the development of a lifelong
learning counseling system designed to enhance current
Lifelong Learning platforms. Fig. 6 shows a sketch of our
design. It includes a user profile that stores the competencies
of the user and also enables users to select the desired career
path. The profile can then visualize the strengths and
weaknesses of the user and overlay a projection on how these
important attributes of the user will change by taking
different courses. The whole counseling process can be
moderated through the use of a Chatbot. The Chatbot can dig
deeper into the competency level of the user through the use
of assessment questions that can be automatically extracted
from textbooks via NLP. Finally, suitable courses can be
recommended to users.
VI. CONCLUSION
This paper presents a crucial first step in the iterative
design-based research process aimed at developing lifelong
learning platforms that are enhanced through digital
counseling. It describes a set of formative studies that first
allowed us to extract basic requirements for our proposed
digital counseling. Moreover, the studies allowed us to
examine the feasibility of using current technologies to
develop the proposed digital counseling.
Based on the results from our studies, we consider that the
implementation of the proposed design is feasible with
current technologies. It will be an improvement to current
lifelong learning platforms where users want to find suitable
courses for them through the use of queries. The challenge is
that in many cases users search for topics that they are not
even familiar with and whose keywords they do not even
know. We plan to implement the proposed structure by
starting with the Chatbot as one of the key components, as the
results indicate. Building upon this, additional evaluation and
light assessments will be included. This will help lifelong
learners to navigate the maze of lifelong learning course
catalogs leading them to their desired career path.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
AUTHOR CONTRIBUTIONS
Atezaz Ahmad conducted the study on finding LA
user-profile characteristics that answers our first RQ. Jan
Schneider conducted a study on using NLP to extract
questions from the text that answers our second RQ. Daniel
Schiffner was responsible for leading a survey on finding
relevant characteristics of Chatbots that answers our third RQ.
Natalie Kiesler helped in analyzing the data. Sebastian
Wollny helped in the brainstorming and in preparing the RQs.
Atezaz Ahmad wrote the paper, where all the authors
contributed equally in further extending/rewriting the
sections and preparing/proofreading the final version. All
authors approved the final version.
FUNDING
This work was supported by DIPF | Leibniz Institute for
Research and Information in Education, Germany. Caught In
The Lifelong Learning Maze: Helping People With Learning
Analytics And Chatbots To Find Personal Career Paths.
REFERENCES
[1] M. London, The Oxford Handbook of Lifelong Learning, Oxford
University Press, 2011.
[2] D. Antoci, ―Encarta, Lifelong Learning, 2022.
[3] H. Ates and K. Alsal, ―The importance of lifelong learning has been
increasing,‖ Procedia-Social and Behavioral Sciences, vol. 46, pp.
40924096, 2012.
[4] K. Głabicka, ―The importance of lifelong learning in XXI century,‖
Central European Review of Economics & Finance, vol. 8, no. 2, pp.
5161, 2015.
[5] A. H. Odeh et al., ―Analytical and comparison study of main web
programming languages ASP and PHP,‖ TEM Journal, vol. 8, no. 4,
pp. 15171522, 2019.
[6] P. Chakraborty, R. Shahriyar, A. Iqbal, and G. Uddin, ―How do
developers discuss and support new programming languages in
technical Q&A site? An empirical study of Go, Swift, and Rust in Stack
Overflow,‖ Inf Softw Technol, vol. 137, p. 106603, 2021.
[7] IWWB. (InfoWeb Further Education): About IWWB. [Online].
Available: https://www.iwwb.de/kurssuche/startseite.html
[8] H. Weiterbildungsdatenbank. (Aug. 29, 2022). Education-portal-hesse:
About education-portal-hesse. [Online]. Available:
https://www.bildungsportal-hessen.de/g10341
[9] Weiterbildung.de. (Aug. 29, 2022). Further education: about further
education. [Online]. Available:
https://www.weiterbildung.de/ueber-uns
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
428
[10] S. Zolaktaf and G. C. Murphy, ―What to learn next: Recommending
commands in a feature-rich environment,‖ in Proc. 2015 IEEE 14th
International Conference on Machine Learning and Applications
(ICMLA), 2015, pp. 10381044.
[11] F. E. Weinert, ―Concept of competence: A conceptual clarification,‖
2001.
[12] S.-C. Wong, ―Competency definitions, development and assessment: A
brief review,‖ International Journal of Academic Research in
Progressive Education and Development, vol. 9, no. 3, pp. 95114,
2020.
[13] G. Siemens, ―Call for papers of the 1st international conference on
learning analytics & knowledge (lak 2011),‖ 2010.
[14] S. J. Aguilar, S. A. Karabenick, S. D. Teasley, and C. Baek,
―Associations between learning analytics dashboard exposure and
motivation and self-regulated learning,‖ Comput Educ, vol. 162, p.
104085, 2021.
[15] I. Jivet, M. Scheffel, M. Schmitz, S. Robbers, M. Specht, and H.
Drachsler, ―From students with love: An empirical study on learner
goals, self-regulated learning and sense-making of learning analytics in
higher education,‖ Internet High Educ, vol. 47, p. 100758, 2020.
[16] Y. Park and I.-H. Jo, ―Development of the learning analytics dashboard
to support students’ learning performance,‖ Journal of Universal
Computer Science, vol. 21, no. 1, p. 110, 2015.
[17] S. Wollny, J. Schneider, D. Mitri, J. Weidlich, M. Rittberger, and H.
Drachsler, ―Are we there yet? A systematic literature review on
chatbots in education,‖ Front Artif Intell, vol. 4, 2021.
[18] M. Prensky, ―Khan academy,‖ Educational Technology, vol. 51, no. 5,
p. 64, 2011.
[19] J. Knox, S. Bayne, J. Ross, H. Macleod, and C. Sinclair, ―MOOC
pedagogy: The challenges of developing for Coursera,‖ Association for
Learning Technology (ALT) Online Newsletter, no. 28, 2012.
[20] Iversity. Iversity: About Iversity. [Online]. Available:
https://iversity.org/
[21] FutureLearn. FutureLearn: About FutureLearn. [Online]. Available:
https://www.futurelearn.com/about-futurelearn
[22] M. R. Kennedy, ―LinkedIn learning product review,‖ Journal of the
Canadian Health Libraries Association/Journal de l’Association des
bibliothèques de la santé du Canada, vol. 40, no. 3, pp. 142143, 2019.
[23] P. Carvalho, ―Academic MOOCS providers for learners vs
instructors,‖ in Proc. EDULEARN21 Proceedings, 2021, pp.
94479455.
[24] J. P. Espada, C. C. Rodr’iguez, V. Garc’ia-D’iaz, and R. G. Crespo,
―Method for analysing the user experience in MOOC platforms,‖ in
Proc. 2014 International Symposium on Computers in Education
(SIIE), 2014, pp. 157162.
[25] R. S. Adham and K. O. Lundqvist, ―MOOCS as a method of distance
education in the arab world A review paper,‖ European Journal of
Open, Distance and E-learning, vol. 18, no. 1, pp. 123139, 2015.
[26] F. Wang and M. J. Hannafin, ―Design-based research and
technology-enhanced learning environments,‖ Educational
Technology Research and Development, vol. 53, no. 4, pp. 523, 2005.
[27] GoogleDrawing. (Aug. 29, 2022). Google Drawings. [Online].
Available: https://docs.google.com/drawings
[28] A. Ahmad et al., ―What indicators can i serve you with? An evaluation
of a research-driven learning analytics indicator repository,‖ in Proc.
the 14th International Conference on Computer Supported Education -
Volume 1: CSEDU, 2022, pp. 5868.
[29] Europass. Europass (free and personal tool for learning and working in
Europe): About Europass. [Online]. Available:
https://europa.eu/europass/en/about-europass
[30] J. L. Zuckarelli, Wie stelle ich sicher, dass ich (und andere) mein
Programm später noch verstehe?‖Programmieren lernen mit Python
und JavaScript, Springer, 2021, pp. 7585.
[31] J. L. Zuckarelli, ―Was muss ich tun, um ein Programm zum Laufen zu
bringen?‖Programmieren lernen mit Python und JavaScript, Springer,
2021, pp. 441450.
[32] J. L. Zuckarelli, Programmieren lernen mit Python und JavaScript:
Eine praxisorientierte Einführung für Einsteiger, Springer, 2021.
[33] T. Steuer, A. Filighera, and C. Rensing, ―Exploring artificial jabbering
for automatic text comprehension question generation,‖ in Proc.
European Conference on Technology Enhanced Learning, 2020, pp.
114.
[34] A. Ahmad et al., ―Connecting the dots A literature review on
learning analytics indicators from a learning design perspective,‖ J
Comput Assist Learn, 2022.
[35] J. Kim, I.-H. Jo, and Y. Park, ―Effects of learning analytics dashboard:
analyzing the relations among dashboard utilization, satisfaction, and
learning achievement,‖ Asia Pacific Education Review, vol. 17, no. 1,
pp. 1324, 2016.
Copyright © 2023 by the authors. This is an open access article distributed
under the Creative Commons Attribution License which permits unrestricted
use, distribution, and reproduction in any medium, provided the original
work is properly cited (CC BY 4.0).
Atezaz Ahmad is a research assistant and doctoral
candidate within the Leibniz Institute for Educational
Research and Information and Goethe University in
Frankfurt am Main, Germany. His PhD thesis is about
learning analytics indicators and dashboards, where he
is working, developing and evaluating a LA tool called
OpenLAIR that helps course designers, teachers,
students, and educational researchers to make
informed decisions about the selection of learning design activities, LA
indicators, and metrics for their course design or LA dashboard. He is further
working on NLP to automize the process of harvesting LA indicators and
their metrics from the LA literature and uploading them to OpenLAIR to
keep the tool up to date.
He graduated with a master's in software system engineering in learning
analytics from RWTH Aachen University, Germany. His research interests
are learning analytics indicators and dashboard, visual analytics, NLP, and
text data mining.
Natalie Kiesler completed her doctorate in computer
science at Goethe University Frankfurt, Germany in
2022. The focus of her thesis was on modeling
competencies and tutoring feedback for novice learners
of programming. She is currently a senior researcher at
the DIPF Leibniz Institute for Educational Research
and Information in Frankfurt, Germany where she
contributes to the development of an open research data
infrastructure for qualitative research data. Dr. Kiesler's
research interests include computing education, competency-based learning
and teaching, feedback, and open science.
Daniel Schiffner is the head of the Educational
Computer Science group at the DIPF. After his
studies in Computer Graphics, he received his PhD in
2012 at the Goethe University Frankfurt, Germany.
He then began research in the field of usability and
educational technologies. In 2014 he took the
position of head of research and development at the
central e-learning institution studiumdigitale at
Goethe University. He switched to his current position in 2019, where he
researches and maintains information platforms, usability of those as well as
new technologies in the context of education.
Jan Schneider is a postdoctoral researcher at the
Educational Computer Science group of the DIPF. He
started his career as a researcher in 2008 working as a
Human Computer Interaction researcher at the
Expertise Centre for Digital Media at Hasselt
University in Belgium. There he worked for several
research projects in the areas of Multi-Touch and
mobile interactions. On December 2017 he received
his PhD from the Open University of the Netherlands
on the topic of ―Sensor based Learning support‖. During his PhD he worked
on two European projects (Metalogue and Wekit) and his developed research
prototypes got awarded in three different international conferences (EC-TEL
2014, EC-TEL 2015, ICMI 2015). Dr. Schneider's current research focus is
in the area of Multimodal Learning Analytics (MMLA), where he is
investigating the creation of generic frameworks and solutions designed to
support the learning process with the use of multimodal data.
Sebastian Wollny is a PhD student at the DIPF
Leibniz Institute for Research and Information in
Education, based in Frankfurt am Main, Germany.
Since his bachelor's degree in Electrical Engineering
and Computer Science at the Multimedia
Communications Lab of the TU Darmstadt, he has
been working on the processing of information in
learning environments. Sebastian focuses in his PhD
on dialog-based learning, which a special focus on
Chatbots in education, Natural Language Processing, and AI in education.
International Journal of Information and Education Technology, Vol. 13, No. 3, March 2023
429
... Other studies in this theme focus on the development and use of AI-powered recommender systems to assist individuals navigate the vast array of potential job opportunities (Parida et al., 2022) and learning pathways (Ahmad et al., 2023;Kokkodis & Ipeirotis, 2021). These systems filter and target job and learning options based on individual competencies and desired career paths (Ahmad et al., 2023;Parida et al., 2022). ...
... Other studies in this theme focus on the development and use of AI-powered recommender systems to assist individuals navigate the vast array of potential job opportunities (Parida et al., 2022) and learning pathways (Ahmad et al., 2023;Kokkodis & Ipeirotis, 2021). These systems filter and target job and learning options based on individual competencies and desired career paths (Ahmad et al., 2023;Parida et al., 2022). They also account for contextual factors such as company recruitment needs (Parida et al., 2022) and current and future market trends in skill demands (Kokkodis & Ipeirotis, 2021). ...
... They also account for contextual factors such as company recruitment needs (Parida et al., 2022) and current and future market trends in skill demands (Kokkodis & Ipeirotis, 2021). Additionally, chatbot interfaces are proposed to facilitate digital counselling in the process of course selection for skill development (Ahmad et al., 2023). Some research within this theme also adopts an organizational perspective, focusing on the development of models that optimize staff training in alignment with job requirements and career paths (Karamalak et al., 2014). ...
Article
Full-text available
As artificial intelligence (AI) use expands within organizations, its influence is increasingly permeating careers and vocational domains. However, there is a notable lack of structured insights regarding AI's role in shaping individual career paths across career stages. To address this gap, we undertook a systematic literature review of 104 empirical articles, aiming to synthesize the scholarship on AI in the context of careers. Drawing upon career stage theory, we examine the implications of AI on careers, identify key barriers and enablers of AI use in this area, and reveal how the utilization of AI impacts individuals' career competencies. In doing so, we illustrate how AI actively shapes individuals' career trajectories and we dissect these effects both within and across various career stages to situate AI within the broader context of careers research. Adopting a sustainable career lens, we conclude by outlining a future research agenda that advocates for the design and adoption of AI systems that promote sustainable and equitable careers.
... One is brainstorming sessions to identify and recommend meaningful learning activities in Moodle courses. Another example is workshops tailored to designing and recommending LA indicators and their metrics for LA dashboards (Ahmad et al., 2023;Karademir et al., 2022). ...
Conference Paper
Full-text available
Each year, the amount of research publications is increasing. Staying on top of the state of the art is a pressing issue. The field of Learning Analytics (LA) is no exception, with the rise of digital education systems that are used broadly these days from K12 up to Higher Education. Keeping track of the advances in LA is challenging. This is especially the case for newcomers to the field, as well as for the increasing number of LA units that consult their teachers and scholars on applying evidence-based research outcomes in their lectures. To keep an overview of the rapidly growing research findings on LA, we developed LAxplore, a tool that uses NLP to extract relevant information from the LA literature. In this article, we present the evaluation of LAxplore. Results from the evaluation show that LAxplore can significantly support researchers in extracting information from relevant LA publications as it reduces the time of searching and retrieving the knowledge by a factor of six. H owever, the accurate extraction of relevant information from LA literature is not yet ready to be fully automatized and some manual work is still required.
Article
Full-text available
Lay Description What is already known about this topic? Learning design (LD) is the pedagogic process used in teaching/learning that leads to the creation and sequencing of learning activities and the environment in which it occurs. Learning analytics (LA) is the measurement, collection, analysis & reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. There are multiple studies on the alignment of LA and LD but research shows that there is still room for improvement. What this paper adds? To achieve better alignment between LD and LA. We address this aim by proposing a framework, where we connect the LA indicators with the activity outcomes from the LD. To demonstrate how learning events/objectives and learning activities are associated with LA indicators and how an indicator is formed/created by (several) LA metrics. We address this aim in our review. This article also aims to assist the LA research community in the identification of commonly used concepts and terminologies; what to measure, and how to measure. Implications for practice and/or policy This article can help course designers, teachers, students, and educational researchers to get a better understanding on the application of LA. This study can further help LA researchers to connect their research with LD.
Conference Paper
Full-text available
In recent years, Learning Analytics (LA) has become a very heterogeneous research field due to the diversity in the data generated by the Learning Management Systems (LMS) as well as the researchers in a variety of disciplines, who analyze this data from a range of perspectives. In this paper, we present the evaluation of a LA tool that helps course designers, teachers, students and educational researchers to make informed decisions about the selection of learning activities and LA indicators for their course design or LA dashboard. The aim of this paper is to present Open Learning Analytics Indicator Repository (OpenLAIR) and provide a first evaluation with key stakeholders (N=41). Moreover, it presents the results of the prevalence of indicators that have been used over the past ten years in LA. Our results show that OpenLAIR can support course designers in designing LA-based learning activities and courses. Furthermore, we found a significant difference between the relevance and usage of LA indicators between educators and learners. The top rated LA indicators by researchers and educators were not perceived as equally important from students' perspectives.
Article
Full-text available
Chatbots are a promising technology with the potential to enhance workplaces and everyday life. In terms of scalability and accessibility, they also offer unique possibilities as communication and information tools for digital learning. In this paper, we present a systematic literature review investigating the areas of education where chatbots have already been applied, explore the pedagogical roles of chatbots, the use of chatbots for mentoring purposes, and their potential to personalize education. We conducted a preliminary analysis of 2,678 publications to perform this literature review, which allowed us to identify 74 relevant publications for chatbots’ application in education. Through this, we address five research questions that, together, allow us to explore the current state-of-the-art of this educational technology. We conclude our systematic review by pointing to three main research challenges: 1) Aligning chatbot evaluations with implementation objectives, 2) Exploring the potential of chatbots for mentoring students, and 3) Exploring and leveraging adaptation capabilities of chatbots. For all three challenges, we discuss opportunities for future research.
Article
Full-text available
Context New programming languages (e.g., Swift, Go, Rust, etc.) are being introduced to provide a better opportunity for the developers to make software development robust and easy. At the early stage, a programming language is likely to have resource constraints that encourage the developers to seek help frequently from experienced peers active in Question–Answering (QA) sites such as Stack Overflow (SO). Objective In this study, we have formally studied the discussions on three popular new languages introduced after the inception of SO (2008) and match those with the relevant activities in GitHub whenever appropriate. For that purpose, we have mined 4,17,82,536 questions and answers from SO and 7,846 issue information along with 6,60,965 repository information from Github. Initially, the development of new languages is relatively slow compared to mature languages (e.g., C, C++, Java). The expected outcome of this study is to reveal the difficulties and challenges faced by the developers working with these languages so that appropriate measures can be taken to expedite the generation of relevant resources. Method We have used the Latent Dirichlet Allocation (LDA) method on SO’s questions and answers to identify different topics of new languages. We have extracted several features of the answer pattern of the new languages from SO (e.g., time to get an accepted answer, time to get an answer, etc.) to study their characteristics. These attributes were used to identify difficult topics. We explored the background of developers who are contributing to these languages. We have created a model by combining Stack Overflow data and issues, repository, user data of Github. Finally, we have used that model to identify factors that affect language evolution. Results The major findings of the study are: (i) migration, data and data structure are generally the difficult topics of new languages, (ii) the time when adequate resources are expected to be available vary from language to language, (iii) the unanswered question ratio increases regardless of the age of the language, and (iv) there is a relationship between developers’ activity pattern and the growth of a language. Conclusion We believe that the outcome of our study is likely to help the owner/sponsor of these languages to design better features and documentation. It will also help the software developers or students to prepare themselves to work on these languages in an informed way.
Article
Full-text available
Competencies have been used as valid predictors of superior on-the-job performance in business organizations over the last 40 years. An abundant of empirical evidence has suggested that competencies play important roles in human resource management (HRM) practices for different types of organizations. However, the important roles of competency have not been fully integrated into the business world, creating a gap between theory and practice. There seems to also appear a number of misperceptions with regards to the use of the term 'competency' in the literature. This paper intends to trace the origins of competency and document various definitions and concepts of competency proposed by different scholars. In addition, the paper also aims to shed light on the processes, guidelines, and techniques for developing competencies pragmatically for a particular job or profession as well as the importance of competency-based assessment in organizations for today's HRM practices.
Article
Full-text available
Unequal stakeholder engagement is a common pitfall of adoption approaches of learning analytics in higher education leading to lower buy-in and flawed tools that fail to meet the needs of their target groups. With each design decision, we make assumptions on how learners will make sense of the visualisations, but we know very little about how students make sense of dashboard and which aspects influence their sense-making. We investigated how learner goals and self-regulated learning (SRL) skills influence dashboard sense-making following a mixed-methods research methodology: a qualitative pre-study followed-up with an extensive quantitative study with 247 university students. We uncovered three latent variables for sense-making: transparency of design, reference frames and support for action. SRL skills are predictors for how relevant students find these constructs. Learner goals have a significant effect only on the perceived relevance of reference frames. Knowing which factors influence students' sense-making will lead to more inclusive and flexible designs that will cater to the needs of both novice and expert learners.
Book
Ob auf dem Computer, Tablet, Handy, im Auto oder in der Kaffeemaschine – Computerprogramme bestimmen unseren Alltag. Software wird immer wichtiger, kaum noch etwas funktioniert ohne die geheimnisvolle Macht der Algorithmen. Aber wie funktionieren Programme? Und wie entwickelt man sie? Dieses Buch vermittelt Ihnen anschaulich das Einmaleins des Programmierens. Anhand von Alltagsbeispielen lernen Sie zunächst die Grundkonzepte des Programmierens kennen, die in allen Programmiersprachen ähnlich sind. Auf Basis dieser Grundideen erlernen Sie dann auf systematische Weise und mit vielen praktischen Übungen zwei populäre und sehr nützliche Programmiersprachen, Python und JavaScript, die Sie für ein breites Spektrum an unterschiedlichen Aufgaben einsetzen können. Das Buch richtet sich an Programmieranfänger aller Altersklassen (vom Schüler bis zum Berufstätigen), die bisher keinerlei Programmiererfahrung besitzen. Über die Springer Nature Flashcards-App stehen für Leserinnen und Leser digitale Lernkarten zum Buch zur Wissenskontrolle und -vertiefung kostenlos zur Verfügung. Der Inhalt • Über das Programmieren • Die Grundkonzepte des Programmierens • Einführung in Python • Einführung in JavaScript Der Autor Joachim Zuckarelli hat Volkswirtschaftslehre an der Universität Mannheim studiert. Nach Stationen in der Unternehmensberatung und der Labordiagnostik-Branche verantwortet er heute den Bereich Business Intelligence bei einer Tierklinik-Kette. Bereits seit seiner Schulzeit programmiert er in verschiedenen Programmiersprachen. Zuletzt hat er eine Einführung in die Statistik-Programmiersprache R veröffentlicht. Er lebt und arbeitet in München.
Chapter
Bevor wir richtig in die Python-Programmierung einsteigen, werden wir uns in diesem Kapitel zunächst damit beschäftigen, wie Python-Code und seine wichtigsten Grundelemente eigentlich aussehen und welche fundamentalen Konventionen es bei der Code-Gestaltung einzuhalten gilt. Python weist im Vergleich zu vielen anderen Programmiersprachen eine Besonderheit in der Code-Gestaltung auf, die uns als Programmierern zwar auf den ersten Blick etwas Gestaltungsfreiheit wegzunehmen scheint, aber unser Leben zugleich auch einfacher macht.
Article
Learning analytics dashboards (LADs) are intended to give relevant information to students and other stakeholders to inform potential next steps in the learning process. The current study examines the relationship between information indirectly presented through academic advisors' use of LADs, and college students' academic motivation, self-regulated learning, and academic achievement. We modeled how changes in student motivation and self-regulated learning (SRL) were related to what occurred during 1-on-1 meetings with academic advisors during which students had the potential to view representations of their achievement embedded within an Early Warning System (EWS) that visually represented aspects of their academic performance referenced with course averages. Constructs associated with SRL were moderated by advisor-advisee meetings. Results indicated that advisors' use of EWS while they met with students was negatively associated with the rate of decrease of students' reporting of using memorizing strategies but positively related when students' performance was compared to that of their peers. We discuss the moderating effects of students’ exposure to visualizations of academic performance on their SRL strategies and academic motivation. This study points to the importance of monitoring the effects of information presented via EWS on motivation and SRL.