Content uploaded by Roland Happ
Author content
All content in this area was uploaded by Roland Happ on Jul 12, 2024
Content may be subject to copyright.
EP
Empirische Pädagogik – 2024 – 38 (1)
Roland Happ, Kristina Kögler, Jacqueline
Schmidt & Marc Egloffstein (Hrsg.)
Lehren und Lernen mit und über Künstliche
Intelligenz in der Aus- und Weiterbildung
Empirische Pädagogik
______________________
38. Jahrgang 2024
1. Heft
Herausgeber
Zentrum für Empirische Pädagogische Forschung (zepf)
Bürgerstraße 23, 76829 Landau/Pfalz
Telefon: +49 6341 280 32165, Telefax: +49 6341 280 32166
E-Mail: sekretariat@zepf.uni-landau.de
Homepage: www.zepf.eu
Verlag
Empirische Pädagogik e. V.
Bürgerstraße 23, 76829 Landau/Pfalz
Telefon: +49 6341 280 32180, Telefax: +49 6341 280 32166
E-Mail: info@vep-landau.de
Homepage: www.vep-landau.de
Druck
Sowa Piaseczno
Zitiervorschlag
Happ, R., Kögler, K., Schmidt, J. & Egloffstein, M. (Hrsg.). (2024). Lehren und Lernen
mit und über Künstliche Intelligenz in der Aus- und Weiterbildung (Empirische
Pädagogik, 38 (1), Themenheft). Landau: Verlag Empirische Pädagogik.
Alle Rechte, insbesondere das Recht der Vervielfältigung und Verbreitung sowie der
Übersetzung, werden vorbehalten. Kein Teil des Werks darf in irgendeiner Form
(durch Fotografie, Mikrofilm oder ein anderes Verfahren) ohne schriftliche
Genehmigung des Verlags reproduziert oder unter Verwendung elektronischer
Systeme verbreitet werden.
ISSN 0931-5020
ISBN 978-3-944996-97-4
Verlag Empirische Pädagogik, Landau 2024
https://doi.org/10.62350/RZIP6202
Empirische Pädagogik © Verlag Empirische Pädagogik
2024, 38. Jahrgang, Heft 1
Inhalt
Editorial
Roland Happ, Kristina Kögler, Jacqueline Schmidt & Marc Egloffstein
Lehren und Lernen mit und über Künstliche Intelligenz in der Aus- und
Weiterbildung ................................................................................................................................... 6
Thementeil
Jacqueline Schmidt & Roland Happ
Validierung eines Tests zu Grundlagenwissen über Künstliche Intelligenz
von angehenden Lehrkräften ..................................................................................................... 9
Jule Hangen & Eveline Wuttke
Künstliche Intelligenz verstehen lernen – Ergebnisse eines
Trainingsprogramms zur Förderung von Startup-Fähigkeiten im Bereich
KI ..........................................................................................................................................................26
Sabine Seufert, Lukas Spirgi, Jan Delcker, Joana Heil & Dirk Ifenthaler
Umgang mit KI-Robotern: maschinelle Übersetzer, Textgeneratoren,
Chatbots & Co – Eine empirische Studie bei Erstsemester-Studierenden ............47
Josef Guggemos, Jacqueline Schmidt & Roland Happ
Eine Frage von Macht: Einstellungen angehender Lehrerkräfte zu den
ethischen Grundsätzen des Einsatzes künstlicher Intelligenz in der
Bildung...............................................................................................................................................73
Marc Egloffstein, Kristina Kögler & Dirk Ifenthaler
Evidenzgestützte Entwicklung von onlinebasierten Lernangeboten zu
Künstlicher Intelligenz in der beruflichen Bildung: Stakeholder-
Perspektiven und Implementierung ......................................................................................98
Julia Pargmann, Anna Leube, Florian Berding, Elisabeth Riebenbauer, Karin
Rebmann, Andreas Slopinski & Michael Gillert
Die elektronisch-didaktische Assistenz (EDDA) zur Unterrichtsplanung
auf Basis künstlicher Intelligenz: Funktionsweise, Anwendungsbereiche
und Forschungsperspektiven ................................................................................................ 118
Allgemeiner Teil
Joel Guttke & Raphaela Porsch
Kognitive Interviews als Methode der Inhaltsvalidierung von
Fragebogenitems zur Erfassung kognitiver Aktivierung im
Englischunterricht der Grundschule ................................................................................... 147
Rezension
Wolfgang Klug
Melanie Misamer (2023): Machtsensibilität in der Sozialen Arbeit:
Grundwissen für reflektiertes Handeln ............................................................................. 168
Call for Guest Editors ....................................................................................................................... 173
Impressum ........................................................................................................................................... 175
Contents
Articles
Jacqueline Schmidt & Roland Happ
Validation of a test on prospective teachers’ basic knowledge about
artificial intelligence ....................................................................................................................... 9
Jule Hangen & Eveline Wuttke
Learning about artificial intelligence – results from a training program to
promote startup skills in the AI field .....................................................................................26
Sabine Seufert, Lukas Spirgi, Jan Delcker, Joana Heil & Dirk Ifenthaler
Interaction with AI robots: machine translators, text generators,
chatbots & co - an empirical study among first-year students .................................47
Josef Guggemos, Jacqueline Schmidt & Roland Happ
A matter of power: prospective teachers’ attitudes towards the ethical
principles of artificial intelligence use in education .......................................................73
Marc Egloffstein, Kristina Kögler & Dirk Ifenthaler
Evidence-based development of online learning resources on Artificial
Intelligence in vocational education and training: Stakeholder
perspectives and implementation ..........................................................................................98
Julia Pargmann, Anna Leube, Florian Berding, Elisabeth Riebenbauer, Karin
Rebmann, Andreas Slopinski & Michael Gillert
Electronic didactic assistance (EDDA) for lesson planning based on
artificial intelligence: functionality, areas of application and research
perspectives .................................................................................................................................. 118
Topic-centered
Joel Guttke & Raphaela Porsch
Assessing the content validity of questionnaire items on cognitive
activation in English language teaching at primary level through
cognitive interviewing .............................................................................................................. 147
Empirische Pädagogik © Verlag Empirische Pädagogik
2024, 38. Jahrgang, Heft 1, S. 6-8 https://doi.org/10.62350/QVHQ3765
Editorial
Roland Happ, Kristina Kögler, Jacqueline Schmidt & Marc Egloffstein
Lehren und Lernen mit und über Künstliche
Intelligenz in der Aus- und Weiterbildung
Konzepte und Anwendungen der Künstlichen Intelligenz (KI) haben in jüngerer Zeit
im Bildungskontext und insbesondere auch in der beruflichen Aus- und Weiterbil-
dung immens an Relevanz und Sichtbarkeit gewonnen. Dabei werden der Einfluss
von KI-gestützten Systemen auf zahlreiche Berufsbilder einerseits, andererseits aber
auch die Potenziale und Herausforderungen von KI-Anwendungen für die Gestal-
tung von Lehr- und Lernprozessen intensiv diskutiert. Es lassen sich somit in der
Auseinandersetzung mit KI im beruflichen Bildungskontext mindestens zwei wich-
tige Perspektiven ausmachen, die beide umfassende Fragen auf verschiedenen Ebe-
nen nach sich ziehen: Zum einen geht es um KI als zunehmend bedeutenden Bil-
dungsinhalt, ‚über‘ den es zu lernen gilt, zum anderen um KI als Methodik, ‚mit‘ der
sich Lehr-Lern- und Arbeitsprozesse verändert denken und gestalten lassen. Ent-
sprechende Forschungs- und Entwicklungsvorhaben, die sich mit den KI-induzierten
Veränderungen in Aus- und Weiterbildung befassen, sind angesichts der ausgespro-
chen dynamischen Entwicklungen der jüngeren Zeit zunehmend mit der Herausfor-
derung konfrontiert, anschlussfähig an den jeweils aktuellen Erkenntnis- und Imple-
mentierungsstand zu sein und gleichzeitig mit Blick auf Nachhaltigkeitsfragen auch
Erkenntnisse zu generieren, die zumindest mittelfristig Bestand haben. Dies bedeu-
tet insbesondere für Vorhaben, die über generische Fragen und Zugänge hinausge-
hen, dass sie eher den Charakter einer Momentaufnahme haben. Vor diesem Prob-
lemhintergrund widmet sich die vorliegende Ausgabe der Zeitschrift Empirische
Pädagogik aktuellen Forschungsansätzen zu KI im berufsbildenden Bereich. Mit den
Beiträgen des Themenheftes werden drei thematische Akzente adressiert:
(1) Ansätze zur Erfassung und Förderung KI-bezogener Kompetenzfacetten
Schmidt und Happ berichten von der Validierung eines Testinstruments zur Erfas-
sung des Grundlagenwissens zu KI von angehenden Lehrkräften im berufsbildenden
Bereich. Dabei werden die Analysen zur Beurteilung der Validierungsaspekte "Tes-
tinhalt" und "Beziehung zu anderen Merkmalen" diskutiert. Auf Basis der Befunde
können theoretisch postulierte Annahmen zum KI-Wissen von (angehenden) Lehr-
kräften empirisch untermauert werden, wodurch ein wichtiger Beitrag zur Nutzbar-
keit des entwickelten Instruments geleistet wird.
Editorial 7
Hangen und Wuttke nehmen in ihrer Mixed-Methods Evaluationsstudie Wissen,
Motivation sowie Überzeugungen und Einstellungen von (angehenden) Gründerin-
nen und Gründern von Startups in den Blick. Im Prä-Post-Vergleich nimmt das
selbstberichtete Wissen über Unternehmensgründungen zu. Die Evaluationsergeb-
nisse zeigen allerdings keine signifikante Wissensveränderung über Anwendungen
der KI. In Bezug auf Gründungsaspekte kann das Trainingsprogramm daher als ef-
fektiv angesehen werden, weniger jedoch im Hinblick auf KI.
(2) Ethische Herausforderungen des Einsatzes von KI in Bildungsprozessen.
Den Umgang mit KI-Robotern untersuchen Seufert, Spirgi, Delcker, Heil und
Ifenthaler in einer empirischen Studie mit Studierenden im ersten Semester
(N = 636). In einem weiten Verständnis fassen die Autor*innen darunter Systeme
mit menschenähnlicher Leistung (z. B. Übersetzer, Schreibassistenten) sowie Sys-
teme mit menschenähnlicher Erscheinung (z. B. Chatbots, Avatare). Ausgehend von
sieben Anwendungsfällen werden die Nutzungshäufigkeit sowie die ethische Beur-
teilung in den Blick genommen. In der Gesamtschau überwiegen dabei die wahrge-
nommenen Risiken gegenüber den identifizierten Chancen. Die Nutzungshäufigkeit
von KI-Robotern war zudem im Erhebungszeitraum – kurz vor der Veröffentlichung
von ChatGPT – als eher gering einzustufen.
Guggemos, Schmidt und Happ untersuchen die Einstellungen angehender Lehr-
kräfte zu den ethischen Grundsätzen des Einsatzes von KI im Bildungswesen. In der
Studie mit N = 90 angehenden Lehrkräften erweist sich das postulierte Messmodell
zur Bewertung der Einstellung als reliabel und valide. Eine latente Profilanalyse führt
zu drei von Kontextvariablen unabhängigen Profilen mit jeweils unterschiedlichen
Einstellungsstrukturen, die sich vor allem in der Einschätzung unterscheiden, ob es
der KI erlaubt sein sollte, harte Macht über Lehrkräfte und Lernende auszuüben.
(3) Evidenzgestützte Entwicklung von KI-bezogenen Lehr-Lern-Ressourcen
Egloffstein, Kögler und Ifenthaler beschreiben die evidenzgestützte Entwicklung von
Online-Lernangeboten zu KI in der beruflichen Bildung. Im Zentrum steht dabei eine
qualitative Studie zur Zielgruppen- und Kontextanalyse, in der N = 48 Akteur*innen
aus der beruflichen Bildung zu KI-bezogenen Aspekten befragt wurden. Auf Basis
der Ergebnisse wurden Impulse für die Gestaltung von AI_VET, einer Serie von vier
Online-Kursen auf der Plattform des KI-Campus, abgeleitet. Erste Evaluationsergeb-
nisse deuten auf eine differenzierte Nutzung der implementierten Kursbausteine so-
wie auf Akzeptanz auf Seiten der Lernenden hin.
Pargmann et al. stellen eine KI-Plattform zur Unterstützung der Planungskompetenz
von Unterricht vor. Die elektronisch-didaktische Assistenz (EDDA) kann für die Ana-
lyse von Unterrichtsentwürfen und -materialien im Studium, im Vorbereitungsdienst
und in der schulischen Berufstätigkeit verwendet werden. EDDA stellt Rückmeldun-
gen zur Umsetzung ausgewählter didaktischer Merkmale der Unterrichtsplanung
8 Happ, Kögler, Schmidt & Egloffstein
bereit und liefert dabei Hinweise zur Reflexion und Weiterentwicklung der Entwürfe.
Darüber hinaus kann EDDA als Ausgangsbasis für weitere Forschungen zum Lehren
und Lernen mit KI dienen.
Anhand der drei Themenakzente lassen sich wichtige Spannungsfelder nachzeich-
nen, in denen sich die Diskussion um KI in der beruflichen Bildung aktuell bewegt.
Das Spannungsfeld zwischen einer hohen Entwicklungsdynamik und dem Streben
nach Nachhaltigkeit offenbart sich etwa besonders dann, wenn es darum geht, Wis-
sen über KI in einem Testverfahren zu operationalisieren. Die ersten beiden Artikel
verdeutlichen, wie schnell die Entwicklungen im Themenfeld KI das Design der
Messinstrumente, insbesondere auf der Inhaltsebene, beeinflussen. Es bedarf der
ständigen Anpassung von Testinstrumenten, um valide Testwertinterpretationen zu-
zulassen. Hierbei hat sich die interdisziplinäre Zusammenarbeit (bspw. mit der Wirt-
schaftsinformatik) als hilfreich und notwendig erwiesen. Dass die technischen Mög-
lichkeiten immer auch in Bezug zu ethischen Herausforderungen abgewogen
werden müssen, ist die Quintessenz der Beiträge aus Bereich zwei. Aus beiden Bei-
trägen wird deutlich, dass der stärkere Verbreitungsgrad des Chatbots ChatGPT
auch auf non-kognitive Facetten eine Wirkung haben sollte. Für die Zukunft bieten
sich mit den bestehenden Messinstrumenten zu den non-kognitiven Facetten auch
Kohortenvergleiche an (bspw. Stichprobe 2021 und mögliche Stichprobe 2024), ob
bspw. bei Studierenden aus vergleichbaren Gruppen Veränderungen der non-kog-
nitiven Personenmerkmale zu beobachten sind. Dass im Zuge der Entwicklung von
Bildungsangeboten immer auch die beiden Perspektiven ‚KI als Inhalt‘ und ‚KI als
Werkzeug‘ mitgedacht werden müssen, wird schließlich durch die Beiträge aus dem
dritten Bereich illustriert. Wie die Bildungsangebote ständig aktualisiert werden,
stellt auch bei diesen beiden Beiträgen eine wesentliche Herausforderung dar.
Die Gastherausgeber*innen
Prof. Dr. Roland Happ, Universität Leipzig, Institut für Wirtschaftspädagogik, Univer-
sitätsprofessor für berufliche Bildung mit dem Schwerpunkt Wirtschaft
Prof. Dr. Kristina Kögler, Universität Stuttgart, Institut für Erziehungswissenschaft,
Universitätsprofessorin für Berufspädagogik
Jun.-Prof. Dr. Jacqueline Schmidt, Technische Universität Dresden, Fakultät Wirt-
schaftswissenschaften, Juniorprofessorin für Wirtschaftspädagogik, insbesondere
Digitalisierung in Bildungs- und Arbeitswelten
Dr. Marc Egloffstein, Universität Mannheim, Area Wirtschaftspädagogik, Wissen-
schaftlicher Mitarbeiter
Korrespondenz an: happ@wifa.uni-leipzig.de
Empirische Pädagogik © Verlag Empirische Pädagogik
2024, 38. Jahrgang, Heft 1, S. 9-25 https://doi.org/10.62350/JECI9288
Thementeil
Jacqueline Schmidt & Roland Happ
Validierung eines Tests zu Grundlagenwissen über
Künstliche Intelligenz von angehenden Lehrkräften
Künstliche Intelligenz (KI) hat eine zunehmende Bedeutung für zahlreiche Lebens- und Arbeitsbereiche und
wird als "Treiber der Digitalisierung" in der zweiten Welle der digitalen Transformation verortet. Die berufli-
che Bildung ist durch die KI-bedingten Veränderungen der Tätigkeitsprofile und daraus resultierender Kom-
petenzanforderungen an (zukünftige) Fachkräfte unmittelbar von der dynamischen Entwicklung betroffen.
Für die Ausgestaltung der Qualifikationsprozesse ist das Lehrpersonal zuständig, in dessen Ausbildung KI-
Inhalte bisher nicht verankert sind. Es kann davon ausgegangen werden, dass diese Gruppe über wenig
Vorkenntnisse zu KI verfügt. Um belastbare Befunde über die Ausprägung d es KI-Wissens von (angehenden)
Lehrkräften zu generieren und darauf aufbauend adressatengerechte Aus- und Weiterbildungsangebote zu
schaffen, werden Testinstrumente benötigt, mit denen das Wissen valide gemessen werden kann. Für neu
entwickelte Instrumente sind in Anlehnung an die "Standards for Educational and Psychological Testing" der
American Educational Research Association (AERA) et al. (2014) umfassende Analysen zur Beurteilung der
Validität notwendig. Im Artikel werden die Analysen zur Beurteilung der Aspekte "Testinhalt" und "Bezie-
hung zu anderen Merkmalen" diskutiert. Auf Basis der Befunde können die theoretisch postulierten Annah-
men zum KI-Wissen von (angehenden) Lehrkräften mit empirischer Evidenz untersetzt werden.
Schlagworte: Digitale Transformation – Künstliche Intelligenz – Lehrkräfte – Validierung – Wissenstest
Validation of a test on prospective teachers’ basic
knowledge about artificial intelligence
Artificial intelligence (AI) is becoming increasingly important for numerous areas of life and work and is
considered to be a "driver of digitalisation" in the second wave of digital transformation. Vocational educa-
tion and training is directly affected by this dynamic development, given the AI-related changes to job
profiles and the resulting skills requirements for (future) skilled workers. Responsibility for organising the
qualification processes lies with the teaching staff, whose professional training so far lacks AI content. It can
be assumed that this group has little prior knowledge of AI. In order to generate reliable findings on the
extent of the AI knowledge of (prospective) teachers and to establish appropriate training and further edu-
cation programmes based on this, test instruments are required that can be used to validly measure
knowledge. For newly developed instruments, comprehensive analyses based on the "Standards for Educa-
tional and Psychological Testing" of American Educational Research Association (AERA) et al. (2014) are
required to ensure validity. The article discusses the analyses regarding the assessment of the aspects "test
content" and "relation to other characteristics". Based on the findings, the theoretically postulated as-
sumptions regarding the AI knowledge of (prospective) teachers can be substantiated with empirical evi-
dence.
Keywords: artificial intelligence – digital transformation – knowledge test – validation – vocational teachers
10 Schmidt & Happ
1 Relevance
The impact of digitalization on everyday life and work has been the subject of con-
troversial debates over the past few decades (Schumann et al., 2022). Widespread
use of digital technologies has led to changes in professional practices (Dubs, 2018;
Geiser, 2022) and thus competence requirements for professionals in many fields
(Geiser et al., 2021; Riebenbauer et al., 2022). Although a technology-oriented per-
spective often is taken in these discussions, a practical perspective is emerging re-
garding what knowledge and skills people need to survive in their dynamically
changing world (Müller, 2018; Sczogiel et al., 2019) and how these competences can
be fostered within educational processes. Vocational education plays a crucial role
in creating a well-equipped workforce and thus is directly affected by this digital
transformation (Winkler & Schwarz, 2021). Not only is the content of digital media
as a subject in vocational education programs being discussed, but so is the poten-
tial of digital media in the design and delivery of those programs (Ständige Wissen-
schaftliche Kommission (SWK), 2022). Vocational teachers play a distinct role in
digitization processes at two levels: they must understand and teach digital media
content to students, and they must understand how to employ digital media to en-
hance teaching and learning processes (Gerholz et al., 2022; Meiners et al., 2022).
The increasing importance of digital media in education has led to the establishment
of several conceptual approaches to developing potential vocational teachers’ dig-
ital competences (e. g., TPACK, Mishra & Koehler, 2006; DigCompEdu, Redecker,
2017). Over the past several years, the discussion has shifted towards the develop-
ment and dissemination of artificial intelligence (AI), as it often has been character-
ized as the “driver of digitalization” (Federal Government, 2018, p. 10) initiating the
second wave of digital transformation (Seufert et al., 2020; Winkler & Schwarz, 2021).
In the context of educational processes, the widespread use of AI-based systems
such as ChatGPT (OpenAI, 2023) has been controversial (Steppuhn, 2023). With con-
tinuous and rather easy access to such AI-based systems the question arises as to
what knowledge and skills students and teachers need to be able to use them com-
petently. Given the premise that teachers should be equipped with the same com-
petences they seek to foster in their students (Guggemos & Seufert, 2020), it is es-
sential that teachers' AI-related knowledge and competences be assessed and suit-
able training programs in AI be developed and implemented to meet their profes-
sional development needs. So far, no conceptual framework of AI-related
knowledge that teachers need has been developed, and no instrument that takes
into account crucial quality criteria of test theory has been created to assess that
knowledge, both of which are essential for developing effective learner-centred
training programs (Schmidt, 2024). Findings from current research (for an overview,
see Schmidt, 2024) indicate a deficit in teachers’ knowledge about AI (Hofstetter &
Validation of a test on prospective teachers‘ basic knowledge about AI 11
Massmann, 2021; Lindner & Berges, 2020; Lindner & Romeike, 2019; Nenner et al.,
2021; Pfeiffer et al., 2021; Serholt et al., 2014), which seems to be mainly influenced
by so-called ‘hype topics’ and their representation in the media (Lindner & Berges,
2020; Lindner & Romeike, 2019) and a lack of systematic and standardized proce-
dures for test-based assessment of knowledge about AI. Further, in most of these
studies focus is on (prospective) teachers in the general education sector; thus, pro-
spective vocational teachers’ knowledge of AI has been largely neglected in the lit-
erature (e. g., Roppertz, 2021).
Among other things, this led to the development of a structural model for the cog-
nitive and non-cognitive AI-related competences of prospective vocational teachers
in which knowledge is a central facet. A test based on this model was then designed
to empirically assess the target group's knowledge about AI (Schmidt, 2024). To en-
sure valid testscore interpretations, several analyses considering the Standards for
Educational and Psychological Testing by the American Educational Research Asso-
ciation (AERA et al., 2014) have been conducted (Schmidt, 2024). According to the
AERA et al. (2014) there are several sources of validity evidence. In this article, the
evidence regarding the aspect test content and the test construct’s relationship to
external/other variables are discussed comprehensively as a preliminary step in the
validation process. Once satisfactory findings are obtained on these two elements,
other sources of validity evidence (i. e., response processes and consequences of
testing) must be explored to obtain the differentiated data necessary to ensure va-
lidity of the test as a tool to assess (prospective) vocational teachers’ knowledge
about AI.
In this study, the following two questions concerning the validity of the assessment
tool are addressed:
RQ1: To what extent does the content of this test allow valid interpretation of test
scores?
RQ2: To what extent does the test construct’s relations to other variables allow valid
interpretation of test scores?
In the following, an overview is given of the validation process as set out in the
Standards for Educational and Psychological Testing (AERA et al., 2014). Then, the
procedures for designing and administering the test are outlined. Finally, the results
are presented and ways in which these preliminary insights can be used to revise
the instrument for further validation are discussed.
2 Validation Process
For the interpretation of test scores to be valid (Kane, 2013), an assessment instru-
ment must be examined thoroughly for its validity. In theories on assessment, valid-
12 Schmidt & Happ
ity is considered the most important criterion for quality (e. g., Krohne & Hock, 2007;
Rost, 2004). The validation process is intended to provide information about the
extent to which the theoretical assumptions about the construct can be grounded
with empirical evidence (Förster et al., 2017; Krohne & Hock, 2007; Meinhardt, 2018).
Furthermore, the validation process requires analysis of conceptual assumptions
(Krohne & Hock, 2007), which in this context refers to the substantive modeling of
the measurement object. Thus, aggregation of several analyses allows an evaluation
to be made of the extent to which the instrument allows valid interpretation of test
values (Kane, 2013).
The Standards for Educational and Psychological Testing were established by the
American Educational Research Association (AERA), American Psychological Associ-
ation (APA), and the National Council on Measurement in Education (NCME) to serve
as a guide particularly in empirical research in social and education sciences for de-
veloping and administering tests to ensure their validity (AERA et al., 2014; see, e. g.,
Kuhn, 2014; Happ, 2019; Schmidt, 2024). These standards address issues concerning
test content, response processes, internal structure, relations to other variables, and
consequences of testing (AERA et al., 2014).
To explore the extent to which the content of the test designed for this study reflects
the construct under investigation accurately, the relevance, thoroughness, and
representativeness of each test item were examined (Krohne & Hock, 2007). To ex-
plore the test construct’s relations to other variables, the known groups method
(Schnell et al., 2008) was employed and the test taker’s field of study (contrast
group I) and year of study in the same field (contrast group II) were taken into con-
sideration.
In Table 1 an overview is given of the sources of validity explored in this paper and
the methods employed to gather both qualitative and quantitative data.
Validation of a test on prospective teachers‘ basic knowledge about AI 13
Table 1: Sources of validity explored in the validation process of the test de-
signed to assess prospective vocational teachers’ knowledge about AI
(Schmidt, 2024)
Criteria examined for validity Method
test
content
1) Items as a representative set of the measurement
object
Systematization of
the measurement
object (matrix)
Test development
2) Relevance of the items in terms of content
3) Adequacy of the items
relations
to other
variables
4) Extent of knowledge about AI in a field of study Method of known
groups
5) Extent of knowledge about AI in relation to the
year of study
Method of known
groups
3 Analyses
3.1 Test content
To determine the validity of the content of a test, the relationship between the con-
ceptual considerations of the theoretical construct and its representation in test
items must be examined (AERA et al., 2014). Accordingly, the extent to which the
items on the test in this study adequately represented the content of the theoretical
construct in question was explored (Krohne & Hock, 2007). The relevance and
representativeness of the test items were of central importance in determining
whether the test instrument adequately covered the measurement object in terms
of both content and cognition. In Schmidt (2024), an extensive theoretical model
was created to determine what basic knowledge about AI in terms of content and
cognitive facets are needed to understand and apply basic concepts of AI. This
model served as a basis for analyzing the validity of the test content, as it permitted
consideration of the coherence between the theoretically modeled construct and its
representation in the test items.
The content-related modeling was based on textbook analysis (N = 5)1 and expert
interviews (N = 4) (Meß, 2022). A horizontal perspective (Ernst, 2012) was adopted
to analyze the textbooks. Due to the wide availability of online teaching-learning
resources, online introductory works were considered in addition to print media.
1 The analysis was aligned with the following works: Buxmann and Schmidt (2021); Deckert and Meyer
(2020); Elements of AI (2022); KI-Campus (2020); Reinhart et al. (2021); Taulli (2019); Wittpahl (2019).
14 Schmidt & Happ
Because the target group was assumed to have limited knowledge about AI and due
to a lack of curricular anchoring (Pargmann & Berding, 2022), some widely used
standard works (e. g., Russel & Norvig, 2016) were excluded from analysis because
they were designed for learners who had technical knowledge.2 The experts inter-
viewed were selected according to the criteria outlined by Gläser and Laudel (2010)
and were chosen because they dealt with AI and the mediation of AI basics in the
course of their daily work (e. g., in the context of consulting activities or through the
development of learning opportunities on this topic). During the interviews the ex-
perts were questioned about the depth and scope of their knowledge about, and
experience with, AI (see Bogner et al., 2014).
With these two qualitative approaches to collecting data the following four areas of
knowledge of AI could be distinguished: basics of AI, methods of AI, applications of
AI, and effects of AI. These four content areas are not to be considered in isolation;
rather, they represent a more integrative content model (Meß, 2022) and formed
the basis for specifying the measurement object of basic knowledge about AI in
terms of test content.
Knowledge as a cognitive facet constitutes a fundamental element in models of
teachers' professional competence (Seufert & Guggemos, 2022; Zlatkin-Troitschan-
skaia et al., 2013). It thus provides the foundation for all other cognitive performance
dispositions (Happ, 2017; Minnameier, 2005). According to the understanding of
Beck (1995) and Dubs (1995), basic knowledge about AI can be classified as declar-
ative-static (knowledge of facts) and/or declarative-procedural (knowledge of struc-
tures) (Schmidt, 2024). By narrowing down the cognitive disposition of knowledge
to knowledge of facts and knowledge of structures, restrictions must be imposed
on the content-related modeling. Empirical findings from the modeling process in-
dicate that effects of AI refer mainly to ethical-moral issues (Meß, 2022) that cannot
be covered within the framework of the concept of knowledge specified in this
study. For this reason, the three areas of basics of AI, methods of AI, and applications
of AI were initially considered for the operationalization of knowledge of AI.3 The
cognition-based modeling in this study was aligned with international models of AI
literacy. The cognitive disposition of knowledge, which in this context includes re-
membering and understanding, forms the basis for all other cognitive performance
dispositions (Ng et al., 2021).
2 Although this widely used book essentially contains foundational information, it is aimed at computer
scientists and computer science students. It was assumed that the students in the target group in this
study did not have the technical knowledge that the book's target group is expected to have.
3 Further development of the model to include ethics/morality has been initiated (see Guggemos et al. in
this issue).
Validation of a test on prospective teachers‘ basic knowledge about AI 15
Based on the content-related and cognition-related modeling, the assumptions
made in chapter 2 regarding test content were evaluated. This applied especially to
the relevance and representativeness of the test items. Furthermore, the appropri-
ateness of the items for the intended target group had to be determined partly
because validity must always be assessed in relation to the purpose of the test
(Seufert et al., 2020). Within this framework, appropriateness was assessed by speci-
fying the object of measurement as being knowledge aligned with the cognition-
based modeling. This specification arose from the assumption that the target group
has little to no knowledge about AI, which can be explained by the dynamic nature
of the research topic and the fact that the participants had not received any formal
training in AI as part of their university education. In addition, the participants likely
had little practical teaching experience as they were in their first phase of teacher
education; therefore, it could not be assumed that they had prior experience in using
or even encountering AI in educational processes.
Based on the content- and cognitive-related modeling, a test consisting of 21 items
in a single-choice format was developed (Schmidt, 2024; Schmidt & Happ, 2022).
Considering the content-related modeling, seven items were designed to assess ba-
sics of AI, six to assess methods of AI, and eight to assess applications of AI (Schmidt,
2024). With reference to cognitive differentiation, 12 items were designed to assess
declarative-static knowledge (knowledge of facts) and nine to assess declarative-
procedural knowledge (knowledge of structures) (Schmidt, 2024). By considering
both the content-related and cognition-related modeling, the validation criterion of
test content could be considered fulfilled.
The connection between theoretical modeling and operationalization over the
course of the test development is shown in the following matrix (see Table 2).
Table 2: Mapping of the Ttest items developed with respect to content-related
and cognition-related differentiation (Schmidt, 2024) 4
Cognition-related differentiation
Content-related
differentiation
Declarative-static Declarative-procedural
Basics of AI
W4, W8, W9, W10, W15, W20
W5
Methods of AI
W11, W12, W17
W2, W3, W13
Applications of AI W7, W14, W19 W1, W6, W16, W18, W21
4 The items are included in Schmidt (2024), which is available as open access.
16 Schmidt & Happ
3.2 Relations to other variables
To evaluate a test construct’s relations to other variables, either a convergent or a
discriminant approach can be taken to conduct analyses (AERA et al., 2014). Con-
vergent validation must be excluded in this study due to the lack of comparable test
instruments (Schmidt, 2024). Discriminant validation, however, involves using the
same instrument with different target groups. The known groups method (Schnell
et al., 2008) can be employed to ground theoretically postulated differences in the
manifestation of features using empirical evidence. For this study, a group of stu-
dents in business informatics was considered a suitable contrast group. Due to the
curricular structure of their study program, the students in contrast group I were
expected to have greater knowledge about AI than the target group of prospective
vocational teachers. Discriminant validation with regard to the test taker’s stage of
education (progress of study) also was conducted. The target group of prospective
vocational teachers consisted of master degree students, and contrast group II con-
sisted of bachelor degree students in the same field, which seemed suitable for
comparison because it often is assumed that as one progresses through a course of
study, one gains knowledge. However, neither the bachelor degree program nor the
master degree program offered formal opportunities to learn about AI, which is why
the level of knowledge about AI was not expected to differ significantly between
these groups (Schmidt, 2024).
To obtain the quantitative data needed to examine the underlying assumptions, the
test developed for this study was administered over the Limesurvey online platform
(Limesurvey, 2011) and deployed at nine universities5 where prospective teachers in
the vocational education sector are trained. For the purpose of discriminant valida-
tion, the test also was administered at two universities offering a business informat-
ics program. Data collection took place between April 2021 and July 2023. The
knowledge test was embedded in a larger assessment of cognitive and non-cogni-
tive facets associated with AI (see Schmidt, 2024). In this study, however, the findings
of the knowledge test were analyzed to determine the validity of the test items in
terms of content and the construct’s relations to other variables. As shown in
Table 3, N = 191 prospective vocational teachers6 in master degree programs were
5 The test was administered in German to students at universities in Leipzig, Mainz, Hamburg,
Kaiserslautern, Freiburg, Stuttgart, Göttingen, Schwäbisch Gmünd (Germany) and at a university in Zürich
(Switzerland). Conducting the survey at numerous universities made it possible to cover not only the
commercial/administrative area, but also the industrial/technical field and the area of healthcare, thus
reflecting the three pillars of vocational and business education (Sektion Berufs- und
Wirtschaftspädagogik, 2014) in the empirical dataset.
6 In this case, prospective teachers were all master degree students who could pursue a teaching career
after completing a teacher traineeship.
Validation of a test on prospective teachers‘ basic knowledge about AI 17
surveyed as the target group, N = 27 students in a master degree program in busi-
ness informatics7 were surveyed as contrast group I, and N = 29 prospective voca-
tional teachers in a bachelor degree program in business and economics education
were surveyed as contrast group II.8
Table 3: Sample
Group
Definition
N
Target group Prospective vocational teachers in a master degree program 191
Contrast group I
Students in a master of business informatics program
27
Contrast group II
Prospective vocational teachers in a bachelor degree
program in business and economics education
29
Notes: N = sample size
In the target group (N = 191), the subjects were on average 28.26 years old
(SD = 5.665) and 56.5 % identified as female. Contrast group I subjects (N = 27) had
a mean age of 23.7 years (SD = 1.938) and 37 % identified as female. The subjects
of contrast group II (N = 29) were on average 22.5 years old (SD = 4.702) and 58.6 %
identified as female. With regard to the fields of study, 71.2 % of the entire sample
were in business and economics education, whereas 16.2 % were in the industrial-
technical field, and 12.6 % were in the field of health and care.9
The quantitative data was analyzed using IBM SPSS software, version 27. For the
calculations, the test scores were coded dichotomously: a correct answer on the
knowledge test was given a score of 1, and an incorrect answer, regardless of which
of the three incorrect answer alternatives was selected, was given a score of 0 in the
data set (Schmidt, 2024). The scores were tallied so that a maximum of 21 points
could be achieved on the knowledge test.
7 To assess this contrast group, the test was administered in German at Leipzig University and in English at
the University of Wroclaw (English version of the test).
8 The second contrast group was assessed at Leipzig University.
9 In the data set no significant differences were found in the relevant variables between the subgroups,
meaning that the target group of prospective vocational teachers relevant for the analyses included all
of these groups.
18 Schmidt & Happ
Table 4: Average test performance of the target group and contrast groups I and
II
N M Correct response rate SD
Target group 191 8.90 0.424 2.61
Contrast group I (business
informatics)
27
10.96
0.522
3.93
Contrast group II (bachelor
students of business and
economics education)
29 8.66 0.412 2.30
Notes: N = sample size, M = mean, SD = standard deviation
The findings indicate that on average, the target group solved fewer than half of the
test items correctly (Table 4), scoring 8.90 (SD = 2.61) points on average, which
equates to a correct response rate of 42.4 %. Contrast group I performed better,
with an average correct response rate of 52.2 %, totalling 10.96 (SD = 3.927) points.
The mean difference was significant on the basis of inferential statistical analyses
(p < .01) 10. According to Cohens' d (0.735), this effect can be considered moderate.
The test values of these two groups also reveal with regard to the standard deviation
that the extent of knowledge about AI ranged more widely among business infor-
matics students than it did among prospective vocational teachers. Contrast group II
achieved an average of 8.66 (SD = 2.30) points on the knowledge test. This value
was only marginally below the average value of the target group. Thus, there was
no significant mean difference between the target group and contrast group of pro-
spective teachers. This is further supported by Cohens' d (-0.093).
More differentiated conclusions about the level of knowledge of the respective
groups can be drawn by analyzing the response patterns with regard to the theo-
retical content-related and cognition-related internal differentiation of the con-
struct. The average correct response rate of the item sets for each content area are
presented in the following (see Table 5).
10 Results of the Levene test of equality of variance showed no homoscedasticity could be assumed for the
two groups. Although the t-test for independent samples is fairly robust to any violation of the
prerequisites, an increased error rate must be assumed for samples of significantly different sizes and
variance inequality (Bortz, 2005). For this reason, non-parametric tests were used in this case and the
general tendency was determined using the Mann-Whitney test based on test scores.
Validation of a test on prospective teachers‘ basic knowledge about AI 19
Table 5: Correct response rates of each group with regard to content areas
Basics of AI
Methods of
AI
Applications
of AI
Target group
0.473 (0.209)
0.462 (0.218)
0.349 (0.156)
Contrast group I (business informatics)
0.640 (0.249)
0.548 (0.244)
0.407 (0.165)
Contrast group II (bachelor degree
program in business and economics
education)
0.429 (0.162) 0.466 (0.191) 0.358 (0.166)
The results indicate that the target group did better on items related to basics of AI
with an average correct response rate of 0.473 than on items related to methods of
AI with a rate of 0.462 and on items related to applications of AI with 0.349. This
reveals that the lowest number of items could be solved correctly in the area
"Applications of AI" among the target group.
Predictably, contrast group I (students of business informatics) outperformed the
target group in all content areas. Moreover, unlike the other two groups, contrast
group I solved more than 50 % of all the items correctly. With an average correct
response rate of 0.640, contrast group I solved almost two thirds of the items related
to basics of AI correctly. This area yielded the highest average correct response rate
for this group. Contrast group II (bachelor degree students in business and econom-
ics education) performed similarly to the target group, which was to be expected
from the global analysis of the test scores. What is noticeable at this point is that
the content area with the highest average correct response rate was methods of AI.
In addition to the analysis of individual content areas, an analysis according to the
cognition-related differentiation of the measurement object was conducted. Based
on the theoretical modeling, the items were assigned to declarative-static
knowledge and declarative-procedural knowledge. The average correct response
rates of the subsets can be found in Table 6.
Table 6: Correct response rates of each groups with regard to the cognition-
related modeling
Declarative-static
Declarative-procedural
Target group
0.381 (0.144)
0.479 (0.163)
Contrast group I (business informatics)
0.512 (0.168)
0.535 (0.249)
Contrast group II (business and
economics education)
0.365 (0.134)
0.475 (0.175)
20 Schmidt & Happ
The findings reveal that all groups performed better in the area of declarative-pro-
cedural knowledge (knowledge of structures) than in the area of declarative-static
knowledge (knowledge of facts). It is striking that the differences in the average
correct response rates between these two facets were only marginally distinct within
the group of business informatics students, whereas these differences were more
prominent within the target group and within contrast group II.
According to these findings, the assumptions derived in chapter 2 regarding the
source of validity evidence relations to other variables can be confirmed.
4 Discussion, limitations, and further steps
The aim of this article was to determine the validity of scores on a test instrument
developed for assessing prospective vocational teachers’ knowledge of AI. The test
was aligned with the Standards for Educational and Psychological Testing (AERA et
al., 2014) and the sources of validity evidence under investigation were test content
and relations to other variables. Both qualitative and quantitative approaches were
taken to gathering data.
According to the results, the measurement object modeled was represented accu-
rately in the test instrument, taking into account the content areas that emerged
from the textbook analyses and expert interviews (Meß, 2022). This conclusion is
drawn from the roughly equal distribution of items over the three content areas
basics of AI, methods of AI, and applications of AI. By taking these content areas
into account, the relevance of the test items were ensured.
In addition, cognition-based modeling of declarative-static knowledge (knowledge
of facts) and declarative-procedural knowledge (knowledge of structures) was also
considered when developing the test. The almost equal division of the test items
into these two categories of knowledge ensured adequacy of the items. Thus, the
assumptions established were sufficiently tested within the scope of the validation
of test content when answering RQ1.
Quantitative analyses were conducted to test the assumptions in the context of the
validation aspect "relations to other variables". For this purpose, two contrast groups
were assessed using a discriminant approach and subsequently analyzed with re-
spect to their mean test performance. The findings indicate that, in line with theo-
retically derived expectations, the students of business informatics (contrast group I)
performed better on the test than the target group, confirming the assumption that
this contrast group has more advanced knowledge about AI than the target group.
Furthermore, the assumption was made that the target group’s knowledge about AI
did not differ significantly from the AI knowledge of bachelor degree students in the
Validation of a test on prospective teachers‘ basic knowledge about AI 21
same subject area. Based on the quantitative findings, this assumption was verified.
By confirming this assumption, RQ2 was answered.
To improve the test instrument developed in this study, further analyses are neces-
sary. According to the Standards for Educational and Psychological Testing (AERA
et al., 2014), response processes and consequences of testing are other important
sources of validity evidence and must be explored. Furthermore, a follow-up inter-
view with experts would be beneficial to assess the relevance of the test items de-
veloped (Beck, 2020). The aim of future research will be to adjust the instrument
more specifically to the intended target group and thereby increase the validity of
test score interpretations. The aim of the analyses conducted in this study was to
assess the validity of test score interpretations based on conceptual considerations
and empirical findings. The findings of this study will improve the further
development of the test so that it will be more effective in assessing the knowledge
of the target group in question. Particular attention will be paid to factors
influencing knowledge, the development of knowledge over time, and possible
reciprocal effects on other facets.
In this study, only prospective vocational teachers’ knowledge of AI was explored;
in-service teachers’ knowledge of AI will be investigated in future research, as this
information is essential for developing effective, research-based professional devel-
opment programs designed to meet the particular needs of the target group. Ad-
dressee-appropriate learning opportunities have great potential, as results of a re-
cent intervention study indicate that (prospective) teachers were able to increase
their AI knowledge significantly within the time span of only one semester (Schmidt,
2024; Schmidt & Happ, 2022). The approach has already been initiated in a large-
scale collaborative project (KIWi-MOOC) within the framework of the WÖRLD Com-
petence Center (Thiel de Gafenco & Klusmeyer, 2023).
References
American Educational Research Association (AERA), American Psychologigal Association (APA) & National
Council of Measurment in Education (NCME) (2014). Standards for educational and psychological
testing. Washington: American Educational Research Association.
Beck, K. (1995). Wirtschaftskundliches Wissen bei kaufmännischen Lehrlingen – Zur Diagnose kognitiver
Strukturen mit dem WBT [Business knowledge in commercial apprentices – On the diagnosis of
cognitive structures with the WBT]. In C. Metzger & H. Seitz (Hrsg.), Wirtschaftliche Bildung: Träger,
Inhalte und Prozesse (S. 375-395). Zürich: Verlag des Schweizerischen Kaufmännischen Verbandes.
Beck, K. (2020). Ensuring content validity of psychological and educational tests – The role of experts.
Frontline Learning Research, 8 (6), 1-37.
Bogner, A., Littig, B. & Menz, W. (2014). Interviews mit Experten. Eine praxisorientierte Einführung [Interviews
with experts: A practice-oriented introduction]. Wiesbaden: Springer.
Bortz, J. (2005). Statistik für Human- und Sozialwissenschaftler [Statistics for human and social scientists]
(6. Aufl.). Berlin: Springer.
22 Schmidt & Happ
Buxmann, P. & Schmidt, H. (2021). Künstliche Intelligenz. Mit Algorithmen zum wirtschaftlichen Erfolg.
[Artificial intelligence. With algorithms to economic success] (2. Aufl.). Berlin: Springer Gabler.
Deckert, R. & Meyer, E. (2020). Digitalisierung und Künstliche Intelligenz. Kooperation von Menschen und
Maschinen aktiv gestalten [Digitalization and artificial intelligence. Actively shaping cooperation
between people and machines]. Wiesbaden: Springer Fachmedien.
Dubs, R. (1995). Lehrerverhalten: Ein Beitrag zur Interaktion von Lehrenden und Lernenden im Unterricht
[Teacher behavior: A contribution to teacher-learner interaction in the classroom]. Zürich: Verlag des
Schweizerischen Kaufmännischen Verbandes.
Dubs, R. (2018). Gedanken zur Zukunft der kaufmännischen Grundbildung [Thoughts on the future of basic
commercial education]. In J. Schlicht & U. Moschner (Hrsg.), Berufliche Bildung an der Grenze zwischen
Wirtschaft und Pädagogik: Reflexionen aus Theorie und Praxis (S. 37-58). Wiesbaden: Springer VS.
Elements of AI. (2022). Einführung in die KI. [Introduction to AI]. Verfügbar unter: https://course.elements
ofai.com/de/ [10.01.2024].
Ernst, F. (2012). Fachdidaktische Analyse von Lehrbüchern für den Rechnungswesenunterricht in Deutsch-
land und den USA [Didactic analysis of accounting textbooks in Germany and the U.S.]. Dissertation.
Friedrich-Schiller-Universität Jena.
Federal Government (2018). Artificial intelligence strategy. Verfügbar unter: https://www.ki-strategie-
deutschland.de/home.html?file=files/downloads/Nationale_KI-Strategie_engl.pdf&cid=729
[15.11.2018].
Förster, M., Happ, R. & Molerov, D. (2017). Using the U. S. test of financial literacy in Germany – Adaptation
and validation. Journal of Economic Education, 48 (2), 123-135. https://www.doi.org/10.1080/00220485.
2017.1285737
Geiser, P. (2022). Lehrerüberzeugungen zur Bedeutung der Digitalisierung: Eine Interviewstudie mit
Lehrkräften zur Ausbildung kaufmännischer Fachkräfte [Teachers' beliefs about the importance of
digitization. An interview study with teachers training commercial specialists]. Bielefeld: wbv.
Geiser, P., Busse, J., Seeber, S., Schumann, S., Weber, S., Zarnow, S., Hiller, F., Hackenberg, T. & Lange, A.
(2021). Kompetenzen in digitalisierten kaufmännischen Arbeitsplatzsituationen [Competencies in
digitalized commercial workplace situations]. Zeitschrift für Berufs- und Wirtschaftspädagogik, 117 (4),
630-657.
Gerholz, K.-H., Schlottmann, P., Slepcevic-Zach, P. & Stock, M. (2022). Digital Literacy in der beruflichen
Lehrerinnen- und Lehrerbildung – Einleitung [Digital literacy in professional teacher education –
Introduction]. In K.-H. Gerholz, P. Schlottmann, P. Slepcevic-Zach & M. Stock (Hrsg.), Digital Literacy in
der beruflichen Lehrer:innenbildung. Didaktik, Empirie und Innovation (S. 11-18). Bielefeld: wbv.
Gläser, J. & Laudel, G. (2010). Experteninterviews und qualitative Inhaltsanalyse [Expert interviews and
qualitative content analysis]. Springer VS.
Guggemos, J. & Seufert, S. (2020). Teaching with and teaching about technology – Evidence for professional
development of in-service teachers. Computers in Human Behavior, 115, 1-11.
Happ, R. (2017). Die Entwicklung des volkswirtschaftlichen Grundlagenwissens im Studienverlauf – Effekte
von Eingangsvoraussetzungen auf den Wissenserwerb [The development of basic economic knowledge
in the course of study – Effects of entry requirements on knowledge acquisition]. Landau: Verlag
Empirische Pädagogik.
Happ, R. (2019). Der Einfluss des Migrationshintergrundes auf unterschiedliche Facetten des ökonomischen
Wissens – Implikationen für die Wirtscha ftsdidaktik [The influence of migration background on different
facets of economic knowledge – Implications for business education]. Kumulative Habilitationsschrift.
Mainz: Johannes-Gutenberg-Universität.
Hofstetter, A. & Massmann, C. (2021). Digital skills gap report 2020 – Skills we need, skills we teach.
Verfügbar unter: https://kohorten.com/2020/05/13/digital-skill-gap-studie/ [05.08.2023].
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational
Measurement, 50 (1), 1-73.
KI-Campus (2020). Einführung in die KI [Introduction to AI]. Verfügbar unter: https://ki-campus.org/
courses/einfuehrungki2020 [05.05.2023].
Validation of a test on prospective teachers‘ basic knowledge about AI 23
Krohne, H.-W. & Hock, M. (2007). Psychologische Diagnostik: Grundlagen und Anwendungsfelder
[Psychological diagnostics: Fundamentals and fields of application]. Stuttgart; Kohlhammer.
Kuhn, C. (2014). Fachdidaktisches Wissen von Lehrkräften im kaufmännisch-verwaltenden Bereich.
Modellbasierte Testentwicklung und Validierung [Subject didactic knowledge of teachers in the
commercial-administrative field. Model-based test development and validation]. Landau: Verlag
Empirische Pädagogik.
Limesurvey. (2011). LimeSurvey: The open source survey application.
Lindner, A, & Berges, M.-P. (2020). Can you explain AI to me? Teachers’ pre-concepts about artificial
intelligence. In 2020 IEEE Frontiers in Education Conference (FIE). Uppsala, SE.
Lindner, A. & Romeike, R. (2019). Teachers’ perspectives on artificial intelligence. Conference Paper. ISSEP
2019. 12th International conference on informatics in schools: Situation, evaluation and perspectives.
Meiners, H., Hartmann, P., Niegemann, H., Seeber, S., Wuttke, E. & Schumann, S. (2022). Digitale
Medienkompetenz als Voraussetzung für die Erstellung von Prüfungsaufgaben [Digital media literacy
as a prerequisite for the creation of examination tasks]. In S. Schumann, S. Seeber & S. Abele (Hrsg.),
Digitale Transformation in der Berufsbildung: Konzepte, Befunde und Herausforderungen (S. 123-144).
Bielefeld: wbv.
Meinhardt, C. (2018). Entwicklung und Validierung eines Testinstruments zu Selbstwirksamkeitserwartungen
von (angehenden) Physiklehrkräften in physikdidaktischen Handlungsfeldern [Development and
validation of a test instrument on self-efficacy expectations of (prospective) physics teachers in physics
didactic fields of action]. Berlin: Logos Verlag. https://www.doi.org/10.30819/4712
Meß, M. (2022). Wissenstests zu Anwendungen der Künstlichen Intelligenz – Validierung von Items eines
Fragebogens durch Experteninterviews [Knowledge tests on artificial intelligence applications –
Validation of items of a questionnaire by expert interviews]. Masterarbeit. Universität Leipzig.
Minnameier, G. (2005). Wissen und inferentielles Denken [Knowledge and inferential thinking]. Berlin: Peter
Lang.
Mishra, P. & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher
knowledge. Teachers College Record, 108 (6), 1017-1054.
Müller, (2018). Berufsbildung 4.0. Was müssen die Fachkräfte der Zukunft können? [Vocational training 4.0
- what do the skilled workers of the future need to master?] In C. Bär, T. Grädler & R. Mayer (Hrsg.),
Digitalisierung im Spannungsfeld von Politik, Wirtschaft, Wissenschaft und Recht (S. 292 – 298). Berlin,
Heidelberg: Springer.
Nenner, C., Damnik, G. & Bergner, N. (2021). Weil informatorische Bildung nicht erst in der Sekundarstufe
beginnen darf: Integration informatorischer Bildung ins Grundschullehramtsstudium [Because
computer literacy should not start in secondary school: Integrating information technology education
into elementary school teacher education]. In L. Humbert (Hrsg.), Informatik – Bildung von Lehrkräften
in allen Phasen (S. 103-112). Gesellschaft für Informatik.
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W. & Shen, M. Q. (2021). Conceptualizing AI literacy: An exploratory
overview. Computers and Education: Artificial Intelligence, 2, 1-11. https://www.doi.org/10.1016/
j.caeai.2021.100041
OpenAI. (2023). GPT-4 Technical Report. https://www.doi.org/10.48550/arXiv.2303.08774
Pargmann, J. & Berding, F. (2022). Künstliche Intelligenz in der beruflichen Bildung – Utopie oder Dystopie?
Chancen und Herausforderungen von KI als Bildungstechnologie [Artificial intelligence in vocational
education – Utopia or dystopia? Opportunities and challenges of AI as an educational technology]. In
M. Eckelt, T. Ketschau, J. Klassen, J. Schauer, J. Schmees & C. Steib (Hrsg.), Berufsbildungspolitik: Struk-
turen – Krise – Perspektiven (S. 215-232). https://www.doi.org/10.3278/9783763972623
Pfeiffer, A., Bezzina, S., Dingli, A., Wernbacher, T., Denk, N. & Fleischhacker, M. (2021). Adaptive learning and
assessment: From the teachers’ perspective. Proceedings of INTED 2021 Conference.
Redecker, C. (2017). European framework for the digital competence of educators: DigCompEdu.
Publications Office of the European Union. Verfügbar unter: https://publications.jrc.ec.europa.eu
/repository/handle/JRC107466 [05.08.2023].
24 Schmidt & Happ
Reinhart, J., Mayer, O. & Greiner, C. (2021). Künstliche Intelligenz. Grundlagen, Anwendungsbeispiele und
Umsatzstrategien für Unternehmen [Artificial intelligence. Basics, application examples and revenue
strategies for companies]. Würzburg: Vogel Buchverlag.
Riebenbauer, E., Berding, F. & Flick-Holtsch, D. (2022). Fachwissenschaftliche Implikationen für die Aus- und
Fortbildung von Lehrkräften zur Digitalisierung im Rechnungswesen [Subject-specific implications for
teacher education and training on digitization in accounting]. In K.-H. Gerholz, P. Schlottmann,
P. Slepcevic-Zach & M. Stock (Hrsg.), Digital Literacy in der beruflichen Lehrer:innenbildung. Didaktik,
Empirie und Innovation. (S. 199-212). Bielefeld: wbv.
Roppertz, S. (2021). Die Rolle und Bedeutung von Künstlicher Intelligenz in der Berufsausbildung –
Implikationen für angehende Berufs- und Wirtschaftspädagogog*innen [The role and importance of
artificial intelligence in vocational education – Implications for prospective vocational and business
educators]. bwp@, 40, 1-23.
Rost, J. (2004). Lehrbuch Testtheorie – Testkonstruktion (2. Vollst. überarb. und erw. Aufl.) [Textbook Test
Theory - Test Construction]. Bern: Verlag Hans Huber.
Russel, S. J. & Norvig, P. (2016). Artificial intelligence: A modern approach. Upper Saddle River: Prentice Hall.
Schmidt, J. (2024). Grundlagenwissen zu Künstlicher Intelligenz von angehenden Lehrkräften –
Modellbasierte Testentwicklung und Validierung [Basic knowledge about artificial intelligence of
(prospective) teachers – Model-based test development and validation]. Bielefeld: Wbv.
https://www.doi.org/ 10.3278/9783763976539
Schmidt, J. & Happ, R. (2022). Fachspezifisches Wissen, Einstellungen und Überzeugungen zu Anwendungen
der Künstlichen Intelligenz bei Studierenden der Wirtschaftspädagogik [Subject-specific knowledge,
attitudes and beliefs about applications of artificial intelligence among students of business and
economics education]. In K. Kögler, U. Weyland & H. H. Kremer (Hrsg.), Jahrbuch der berufs- und
wirtschaftspädagogischen Forschung 2022 (S. 89-102). Leverkusen: Verlag Barbara Budrich.
Schnell, R., Hill, P. B. & Esser, E. (2008). Methoden der empirischen Sozialforschung [Methods of empirical
social research] (8. Aufl.). München: Oldenbourg.
Schumann, S., Seeber, S. & Abele, S. (Hrsg.). (2022). Digitale Transformation in der Berufsbildung: Konzepte,
Befunde und Herausforderungen. [Digital transformation in vocational education: concepts, findings
and challenges] Bielefeld: wbv.
Sczogiel, S., Malapally, A., Schmitt-Rueth, S. & Williger, B. (2019). Future Digital Job Skills: Die Zukunft
kaufmännischer Berufe [Future digital job skills: The future of commercial professions]. Industrie- und
Handelskammer Nürnberg für Mittelfranken.
Sektion Berufs- und Wirtschaftspädagogik [BWP] (2014). Basiscurriculum für das universitäre Studienfach
Berufs- und Wirtschaftspädagogik im Rahmen berufs- und wirtschaftspädagogischer Studiengänge
[Basic curriculum for the university subject of vocational and business education as part of vocational
and business education degree programmes]. Verfügbar unter: https://www.dgfe.de/fileadmin/
OrdnerRedakteure/Sektionen/Sek07_BerWiP/2014_Basiscurriculum_BWP.pdf [16.08.2023].
Serholt, S., Barendregt, W., Leite, I., Hastie, H., Jones, A., Paiva, A., Vasalou, A. & Castellano, G. (2014).
Teachers’ views on the use of empathic robotic tutors in the classroom. The 23rd IEEE International
Symposium on Robot and Human Interactive Communication, 955-960.
Seufert, S. & Guggemos, J. (2022). Digitale Kompetenzen von Lehrpersonen – Ergebnisse einer empirischen
Untersuchung in der Berufsbildung [Digital competencies of teachers – Results of an empirical study in
vocational education]. In K.-H. Gerholz, P. Schlottmann, P. Slepcevic-Zach & M. Stock (Hrsg.), Digital
Literacy in der beruflichen Lehrer:innenbildung. Didaktik, Empirie un d Innovation. (S. 213-226). Bielefeld:
wbv.
Seufert, S., Guggemos, J., & Sailer, M. (2020). Technology-related knowledge, skills and attitudes of pre-
service and in-service teachers: The current situation and emerging trends. Computers in Human
Behavior, 115, 1-7. https://www.doi.org/10.1016/j.chb.2020.106552
Validation of a test on prospective teachers‘ basic knowledge about AI 25
Ständige Wissenschaftliche Kommission (SWK). (2022). Digitalisierung im Bildungssystem: Handlungs-
empfehlungen von der Kita bis zur Hochschule: Gutachten der Ständigen Wissenschaftlichen Kommis-
sion der Kultusministerkonferenz (SWK) [Digitization in the education system: Recommendations for
action from daycare to higher education: Report by the standing scientific commission of the standing
conference of the ministers of education and cultural affairs of the Länder in the federal republic of
Germany (SWK)]. Verfügbar unter: https://www.kmk.org/fileadmin/Dateien/pdf/KMK/SWK/2022/SWK-
2022-Gutachten_Digitalisierung.pdf [05.08.2023].
Steppuhn, D. (2023). Der Einsatz Künstlicher Intelligenz in Schulen [The use of artificial intelligence in
schools]. Bildung und Beruf, 6, 122-125.
Taulli, T. (2019). Artificial intelligence basics: A non-technical introduction. New York: Apress.
Thiel de Gafenco, M. & Klusmeyer, J. (2023). Akteurs-Netzwerk ‚WÖRLD-House‘: Digitalisierungsbezogene
Kompetenzentwicklung in der wirtschaftswissenschaftlichen Lehrkräftebildung [Actor network
'WÖRLD-House': Digitization-related competence development in economics teacher education].
Posterbeitrag: Zukunftswerkstatt Lehrkräftebildung Neu Denken des Stifterverbandes, Berlin, 25. Mai
2023. Verfügbar unter: https://www.stifterverband.org/download/file/fid/12307 [22.08.2023].
Winkler, F. & Schwarz, H. (2021). IT-Berufe im Wandel [IT professions in transition]. Zeitschrift für Berufs-
und Wirtschaftspädagogik, Beiheft 31: Künstliche Intelligenz in der Berufsbildung, 77-102.
Wittpahl, V. (Hrsg.). (2019). Künstliche Intelligenz. Technologie, Anwendung, Gesellschaft [Artificial
intelligence. Technology, application, society]. Berlin: Springer Verlag. https://www.doi.org/10.1007/
978-3-662-58042-4
Zlatkin-Troitschanskaia, O., Happ, R., Förster, M., Preuße, D., Schmidt, S. & Kuhn, C. (2013). Analyse der
Ausprägung und Entwicklung der Fachkompetenz von Studierenden der Wirtschaftswissenschaften und
der Wirtschaftspädagogik [Analysis of the expression and development of professional competence of
students of economics and business education]. In O. Zlatkin-Troitschanskaia, R. Nickolaus & K. Beck
(Hrsg.), Kompetenzmodellierung und Kompetenzmessung bei Studierenden der Wirtschaftswissen-
schaften und der Ingenieurwissenschaften (S. 69-92). Lehrerbildung auf dem Prüfstand (Sonderheft).
Landau: Verlag Empirische Pädagogik.
Autor*innen
Jun.-Prof. Dr. Jacqueline Schmidt, Technische Universität Dresden, Fakultät
Wirtschaftswissenschaften, Juniorprofessur für Wirtschaftspädagogik, insb.
Digitalisierung in Bildungs- und Arbeitswelten
Prof. Dr. Roland Happ, Universität Leipzig, Institut für Wirtschaftspädagogik,
Professur für Berufliche Bildung mit dem Schwerpunkt Wirtschaft
Korrespondenz an: jacqueline.schmidt@tu-dresden.de
Empirische Pädagogik © Verlag Empirische Pädagogik
2024, 38. Jahrgang, Heft 1, S. 26-46 https://doi.org/10.62350/TOEF2442
Jule Hangen & Eveline Wuttke
Künstliche Intelligenz verstehen lernen – Ergebnisse
eines Trainingsprogramms zur Förderung von
Startup-Fähigkeiten im Bereich KI
Künstlicher Intelligenz (KI) wird als Schlüsseltechnologie ein hohes Potential für u. a. Produktivitätszuwächse
zugeschrieben; daher werden Start-ups, die KI-Innovationen anbieten, als entscheidend für Wirtschafts-
wachstum angesehen. Allerdings scheitern viele neue Unternehmen in den ersten Jahren, und die Wirksam-
keit von Trainingsprogrammen für (angehende) Unternehmer*innen wird selten systematisch evaluiert. In
dieser Studie untersuchen wir das Wissen, die Motivation, die Überzeugungen und die Einstellungen von
potenziellen Gründer*innen im Rahmen der Teilnahme an einem Trainingsprogramm. Dazu wurden Frage-
bögen und Interviews eingeset zt. Die Interviews zeigten vielfältige Gründe für die Teilnahme am Programm
auf, u. a. die Suche nach Mitgründer*innen und das Interesse an KI. Darüber hinaus schätzten die Teilneh-
mer*innen den Nutzen und die Relevanz von KI für ihre Arbeit höher ein als die Benutzerfreundlichkeit. Die
Evaluationsergebnisse zeigen keine signifikante Wissensveränderung über Anwendungen der KI, aber das
selbstberichtete Wissen über Unternehmensgründungen nimmt zu. Die intrinsische Motivation war nach
dem Programm signifika nt niedriger. Das Programm kann insg esamt als effektiv hinsichtlich der Vermit tlung
von Wissen über Unternehmensgründungen gesehen werden, in Zukunft sollte jedoch der KI Fokus noch
stärker betont werden, um die Gründung von KI oder data-driven ventures zu unterstützen.
Schlagwörter: Entrepreneurship Education – Gründungen – Künstliche Intelligenz – Trainingsprogramm
Learning about artificial intelligence – results from a
training program to promote startup skills in the AI
field
Artificial intelligence (AI) is key to productivity growth in the 21st century; thus, startups offering AI innova-
tions are considered vital for the economy. However, many new businesses fail in their first few years, and
the effectiveness of entrepreneurial training programs seldom is evaluated systematically. In this study, we
investigate the knowledge, motiva tion, beliefs, and attitudes of potential founders of AI startups upon start-
ing and completing a training program we designed for such potential entrepreneurs. We administered a
questionnaire and conducted interviews with people enrolled in the program. We found participants had
varying and often multiple reasons for enrolling in the training program, for example, to find business part-
ners and get a motivational boost, and they rated the usefulness and job relevance of AI greater than its
ease of use. Although we found no significant change in participants’ knowledge about AI, we noticed a
substantial increase in their knowledge about starting a business. Interestingly, their intrinsic motivation
was significantly lower after the program. Overall, our training program for AI startups was effective in
providing general sta rtup knowledge but needs to emphasize the AI focus to support cr eation of AI or data-
driven ventures.
Keywords: artificial intelligence – entrepreneurship education – startups – training study
Startup skills in the field of artificial intelligence 27
1 Introduction
Artificial intelligence (AI) is considered key to productivity growth in the 21st century
(Bundesregierung, 2020). The economic importance of AI for Germany becomes
clear when considering that an estimated 220.6 billion euros in domestic annual
sales are influenced by AI applications (Brandt, 2019). Although startups in the field
of AI, particularly those involving the transfer of knowledge about AI technologies
from research to practical application, are considered vital for the economy, many
new ventures fail within their first few years. According to Plümer and Niemann
(2016) approximately 50 % of such failures (not only in the domain of AI) are due to
a lack of entrepreneurial skills, and Wang, Yang, Han, Huang and Wu (2022) find
that 95 % are due to developmental problems during the postformation phase
when entrepreneurs have to manage economic crises (see also Heinrichs, 2016,
2021).
It has been argued that for AI or data-driven startups to be successful, innovative
“ecosystems” of AI talent are needed to generate ideas, transform knowledge into
practical applications, and encourage and retain talents (Ecker, Coulon & Kohler,
2021). Ideally, such an ecosystem would involve innovators, investors, and institu-
tions working together to meet the challenge of melding ideas for AI-based startups
with knowledge about AI technologies to implement them in AI- or data-driven
business models. Founders of such startups are expected to have knowledge about
AI, motivation and positive beliefs about AI as well as a positive attitude toward AI.
They furthermore need basic entrepreneurial knowledge to launch the new venture
and sustain its development thereafter (Alberti, 1999). Entrepreneurial knowledge
encompasses understanding of business concepts and skills, and it is key to a suc-
cessful launch of a new venture and its sustained success (Wu, Chang & Chen, 2008).
Through entrepreneurial training with a focus on knowledge, motivation and beliefs,
one can develop the competences necessary to take on the challenges of establish-
ing a business (Maresch, Harms, Kailer & Wimmer-Wurm, 2016), launching it, and
developing it during the post-founding phase. Effective trainings, following
research based standards for trainings, address the aforementioned qualification
issues and provide new perspectives in various areas – in this case AI-related and
data-driven new ventures. When designing entrepreneurial training programs,
established criteria for successful trainings (Cademartori et al., 2017) should be
considered and the impact of the programs should be assessed to determine their
effectiveness. To this end, participants’ perceptions, learning gains, and behavioral
responses should be investigated (Kirkpatrick & Kirkpatrick, 2006, for more details
on design criteria and evaluation levels see chapter 2.3). The effectiveness of
entrepreneurial training programs in general, and startup training programs in
particular, often is not evaluated systematically (Nabi, Holden & Walmsley, 2006).
28 Hangen & Wuttke
We developed, implemented and systematically evaluated a startup training in the
domain of data-driven businesses. Participants intend to form a startup in the field
of data-driven ventures. Knowledge about AI applications or knowledge about AI
models were part of the program content. In each subsequent edition, new devel-
opments (such as ChatGPT) as well as traditional machine learning solutions were
considered. The general research question is, if the training is able to promote the
AI related entrepreneurial competence (knowledge, attitudes, beliefs and motiva-
tion) of potential founders in AI or data driven businesses.
In the following chapters we first explore the theoretical background of entrepre-
neurship education, and we describe the training program we designed accordingly
for potential founders of AI startups. Next, we present our research questions and
explain our method of data collection to determine the effectiveness of the pro-
gram. Finally, we present our findings and discuss their implications.
2 Theoretical background
2.1 Relevance and effects of entrepreneurship education
Entrepreneurship education has become increasingly popular over the past few
decades (Sreenivasan & Suresh, 2023) with the growing awareness that the
economy depends to a certain degree on new ventures to stimulate economic
growth (Moberg, 2014). Technologically innovative startups contribute to progress
through market novelties and thus drive structural change (KfW, 2015).
Entrepreneurship is a multifaceted endeavor, and the success of a new venture is
affected by a variety of social, cultural, environmental, demographic and economic
factors (Gaddam, 2007). Stamboulis and Barlas (2014) list economic, psychological
(personal characteristics of the entrepreneur), social (e. g., consumer habits),
environmental (e. g., access to resources, and economic climate), demographic, and
cultural (e. g., shared values and beliefs) factors that influence the success of a new
business. Some factors might hinder potential entrepreneurs from committing
themselves to entrepreneurial ventures. These include negative beliefs about
entrepreneurship (e. g., status and risk) and a lack of competence, experience, and
basic knowledge about business.
Against this background, entrepreneurship education is deemed crucial and thus is
already being promoted and integrated into school curricula in many European
countries (European Commission, 2006) and in the United States (Kuratko, 2005). A
key assumption underlying programs is that entrepreneurial skills can be taught and
the success of a startup is not dependent on personal characteristics (Oosterbeek,
van Praag & Ijsselstein, 2010); therefore, it is expected that through entrepreneur-
ship education individuals will develop the competences necessary to take on the
Startup skills in the field of artificial intelligence 29
challenges of establishing and running a business (Maresch et al., 2016; Roxas,
Cayoca-Panizales & Mae de Jesus., 2008).
In the literature, findings concerning the impact of entrepreneurial training pro-
grams are scarce, and views sometimes are divergent. Of the few investigations into
the effectiveness of such programs, most have been exploratory and descriptive and
have yielded inconsistent findings. In particular, the role of training programs and
their impact on entrepreneurial behavior remains unclear (Roxas et al., 2008).
2.2 Importance of knowledge, motivation, beliefs, and attitudes
regarding entrepreneurship in general and AI-related
entrepreneurship in particular
The success of learning processes has long been defined by output (e. g., Short,
1985; Klieme et al., 2003). The focus is in general not only on the acquisition of
knowledge, but rather on the acquisition of competences (Norris, 1991) understood
in terms of dispositions, usually encompassing knowledge, skills, and attitudes, and
expressed in terms of coping with (occupational) requirements through the use of
those dispositions (Hoffmann, 1999; Hartig, 2008; Weinert, 2001; Zlatkin-
Troitschanskaia & Seidel, 2011). The use of competences in observable action is
further influenced by motivational, volitional, and social factors. Among the most
agreed upon definitions of competence in the German-speaking academic world is
that of Weinert (2001), who describes it as the cognitive abilities and skills available
to an individual or that can be learned by an individual to solve specific problems,
as well as the motivational, volitional, and social readiness associated with them so
that solutions to problems can be used successfully and responsibly in variable sit-
uations.
In entrepreneurship education, it often is assumed that entrepreneurial knowledge
and access to it are a learner’s most important resources (Widding, 2005). Although
this popular view suggests that entrepreneurship education influences behavior and
intentions to start a business, the causal relationship remains unclear. This could be
seen as an indication that developing entrepreneurial knowledge is important for
starting a business but that it is not enough. Beliefs, attitudes, and self-efficacy
might play an important role as well (Roxas et al., 2008). Attitudes are described as
a tendency resulting from the degree of approval or disapproval of something
(Bohner, 2002). Expressions of attitude are influenced in particular by cognitive and
affective dispositions (Siegfried & Wuttke, 2021) and exert an influence on
intentions to act (Ajzen, 1991). Maresch et al. (2016) argue for example, that when
one evaluates the outcome of starting a business as positive, one will have a more
favorable attitude toward starting a business and consequently greater intention to
30 Hangen & Wuttke
do so. They also found that pro-entrepreneurial attitudes correlated positively with
entrepreneurial intentions.
Following the framework of Schmidt and Happ (2022b), the facets of
entrepreneurial competence mentioned above which are deemed necessary to
launch an AI startup successfully, are as follows (see Figure 1):
knowledge about AI applications;
a positive attitude toward AI and positive beliefs about AI;
extrinsic and intrinsic motivation toward AI applications.
Figure 1: Framework for Facets of AI-Related Entrepreneurial Competence
(Adapted from Schmidt & Happ, 2022b)1
1 The term AI literacy often is used interchangeably with AI competence. In a comprehensive literature
review, Laupichler, Aster, Schirch, and Raupach (2022) found that although the term AI literacy is defined
in different ways, one of the most mentioned facets of AI literacy is AI knowledge.
Startup skills in the field of artificial intelligence 31
2.2.1 Content knowledge
Research on vocational competence in general has provided ample evidence that
cognitive factors such as content knowledge are powerful predictors of
achievement (Nitzschke, Velten, Dietzen & Nickolaus, 2019). Therefore, knowledge
about AI and AI applications is a central prerequisite for successful AI startups. Such
knowledge includes understanding of basic functionalities of AI technologies and
AI-based applications (Schmidt & Happ, 2022b).
2.2.2 Attitude toward AI
In a survey on attitudes toward AI conducted in 2021 by the Statista Research De-
partment (2022), 51 % of respondents reported having a positive attitude toward
AI. In 2019, 46 % of respondents answered similarly. Schmidt and Happ (2022b)
point out that attitude toward AI seldom has been addressed in scientific studies
and that instruments for valid measurement have not been developed.
2.2.3 Beliefs about AI
The term “beliefs” goes hand in hand with terms such as values, attitudes, and opin-
ions, a clear definition is missing. According to Pajares (1992) and Richardson (1996),
beliefs are enduring ideas, hypotheses, subjective opinions, and assumptions about
something (Dubberke, Kunter, McElvany, Brunner & Baumert, 2008). Beliefs are seen
as (unconscious) drivers of behavior (Pajares, 1992). There is evidence that beliefs
influence judgements on learning (Mueller & Dunlosky, 2017); therefore, beliefs
about AI are expected to influence learning about AI.
2.2.4 Motivation related to AI and training motivation
Motivation is relevant in the present study in two respects. First, we consider moti-
vational dispositions in relation to AI (intrinsic and extrinsic motivation), second, we
include training motivation. Intrinsic motivation can be expected to influence learn-
ing processes positively and thus lead to sustainable learning success and ultimately
support implementation (intention to found an AI startup, actual founding) of the
content learned (AI and startup knowledge) (Schiefele, 1993, 1996; Schiefele &
Köller, 2001). Extrinsic motivation is more about external reasons for engaging with
AI applications (e. g. to earn a lot of money); it is not expected to influence the
learning processes in the program.
2.3 Criteria for successful training and levels of training evaluation
Criteria for designing and implementing effective training programs have been
identified as follows and were taken into account (Salas & Cannon-Bowers, 2001;
Hochholdinger, Rowold & Schaper, 2008):
32 Hangen & Wuttke
(1) The relevance of the topic was emphasized in the training offer, which is ex-
pected to have a positive effect on the training motivation (Gräsel, Fussangel
& Schellenbach-Zell, 2008; Lipowsky, 2011; Lipowsky & Rzejak, 2021).
(2) The state of research on effective trainings was taken into account, especially
input and application phases were offered alternately (Cademartori et al., 2017;
Krille, 2019). In addition, reflection was repeatedly encouraged (Cademartori et
al., 2017; Lipowsky, 2009).
(3) Previous findings with regard to the duration of trainings do not yet allow any
clear conclusions to be drawn (ibid.), but it is to be expected that very short
measures are unlikely to be effective in the long term (Lipowsky & Rzejak 2021;
Seifried & Wuttke, 2017). Therefore, the training was designed over a period of
several weeks.
The effectiveness of training programs should be evaluated at different levels (Kirk-
patrick, 1998; Kirkpatrick & Kirkpatrick, 2006): participants’ reactions to the program
(e. g., did the participants enjoy the program and find it valuable?), learning gains
(e. g., did the participants increase their knowledge?), the behavior (e. g., did the
participants use their acquired knowledge, e. g., to write a business plan), as well as
the results (e. g., did the participants found a startup and, if so, with what success?)
In this paper we focus on the reaction and the learning success (knowledge about
AI applications, knowledge about startups, motivation toward AI, attitudes toward
AI and beliefs about AI).
3 The present study
To address the need for an ecosystem for AI talents and for effective training for
potential entrepreneurs, we established the H-Ventures training program. This pro-
gram offers first-time entrepreneurs the opportunity to gain essential knowledge
about AI and starting a business, to build valuable networks through mentoring
from business experts and working with peers, to build a viable team, and to build
a foundation for AI- or data-driven businesses (TechQuartier, 2022). The training
program is funded by the state of Hesse,2 Germany (from May 2022 to May 2024)
and offered through an experienced training provider located in Hesse who adver-
tised the program across its networks and through accelerators at universities. The
program, its objectives, learning formats and exact structure will be described in
detail in another article (Hangen, Wuttke & Weber, 2024).
2 “Digitales Hessen” funding in accordance with the guidelines of the state of Hesse on innovation funding
part II, Nr. 1 from December 9, 2016 (StAnz. 52/2016, S. 1676)
Startup skills in the field of artificial intelligence 33
The nine-week program is divided into three phases:
(1) team formation
(2) bootcamp
(3) incubation
During the team formation phase participants are encouraged to validate their busi-
ness idea and build a team. During the bootcamp phase participants become ac-
quainted with other potential founders and mentors. During the incubation phase
participants are mentored and coached by experts in their respective fields. In terms
of instruction, the program is delivered in a variety of (learning) formats that can be
classified as social events (e. g., networking opportunities) and educational sessions
(e. g., workshops and input sessions). During social events, participants can ex-
change ideas and engage in networking; during educational sessions, various issues
are addressed such as effective teamwork, agile frameworks, ethics in AI, data prod-
uct development, business modelling and positioning, and fundraising. Most of the
educational sessions are offered in a workshop format and lectures are comple-
mented with opportunities for direct exchange among participants. To accommo-
date differences in terms of participants’ level of knowledge about fundamentals of
AI and skills in programming, part of the training program is also offered online and
permits participants to work independently at their own pace and at a time that is
convenient to them.
Although applicants do not need to have an idea for a new business, they must go
through an application process to enroll in the program that involves attending an
interview and writing a brief report on their reason for applying to participate in the
program. There were no formal restrictions regarding enrolment. Enrolment is free,
but participants are expected to commit themselves to the program, which means
investing a significant amount of time and energy participating in sessions, contrib-
uting to discussions, and doing evaluations.
Our research questions are as follows:3
(1) What knowledge do participants have about AI applications upon starting the
training program and what knowledge do they gain through the program?
(2) How do the participants rate their knowledge about starting a new business
prior to and after the program?
(3) What are the applicants’ initial beliefs about and attitudes toward AI?
(4) How do intrinsic and extrinsic motivation for AI applications change over the
course of the program?
3 The first four questions are answered with the help of a questionnaire, the last two are answered from
interviews.
34 Hangen & Wuttke
(5) Why did the applicants decide to enroll in the training program and what were
their expectations?
(6) Are the participants interested in AI in general and do they plan to integrate AI
into their startups?
4 Methods
4.1 Measures and data analyses
By addressing the aforementioned research questions, we investigated whether our
training program was able to promote the entrepreneurial competences of
potential founders in AI or data-driven businesses. We collected data in two ways:
with a survey involving administration of a questionnaire before participation in the
program and upon completion of the program, and through semi-structured
interviews. The survey procedure is illustrated in Figure 2.
Figure 2: Survey Outline
Startup skills in the field of artificial intelligence 35
4.1.1 Questionnaire
The questionnaire was administered using the Unipark survey software program
before applicants started the program (pre) and upon completing the program
(post). The questions were designed to gather (1) demographic information as well
as details regarding their (2) knowledge about AI applications (Schmidt & Happ,
2022a; Schmidt & Happ, 2022b), which were presented on a 21-item knowledge
test (single choice format). The assessed knowledge is declarative knowledge in the
form of structural or factual knowledge and is differentiated into declarative-static
and declarative-procedural knowledge (for further details, see Schmidt in
preparation). Motivational scales adapted to the context of AI by Schmidt and Happ
(2022b) were used to gather information on participants’ (3) intrinsic and extrinsic
motivation (Schiefele et al., 2002) (e. g., “I am interested in artificial intelligence
because I enjoy working with content related to artificial intelligence.”), (4) beliefs
about AI (ibid.)(e. g., “There are many scientific findings in the field of artificial
intelligence that will always be valid”), and (5) attitudes toward AI (perceived
usefulness, perceived ease of use, job relevance) (Davis, 1989; Venkatesh & Davis,
2000) (e. g., “I find it easy to learn how to use artificial intelligence applications.”).
To assess participants’ self-reported knowledge about starting a business (6), they
indicated on a 4-point Likert scale (1 = reject the statement/nothing; 4 = fully agree
with the statement/almost everything) how much they knew about various aspects
of starting a business (e. g., “what to consider when starting a business in general”,
Shane, 2000; see also Heinrichs, 2016). Analyses of the responses were conducted
using IBM SPSS software.
4.1.2 Interviews
Semi-structured interviews (i. e., warm-up phase, main part, conclusion) were con-
ducted online via Zoom between the first (team formation) and second part
(bootcamp) of the program by trained interviewers who followed procedural proto-
cols. The interviews were recorded and transcribed verbatim. The answers to the
three interview questions below will be the object of this analysis:
Why did you apply to participate in the program?
What are your expectations of the program?
What is your/your team’s business idea?
Following the principles of qualitative content analysis (e. g., Schreier, 2014a) we
devised a guide for categorizing responses. The main categories aligned with the
research questions. Subcategories were then created based on six interviews as a
result of a generalization process (inductive category formation; Mayring, 2010). The
interview transcripts were segmented, that is, divided into units for coding prior to
initial coding (Schreier, 2014b). To be able to predict the inter coder reliability nine
36 Hangen & Wuttke
interviews were coded by two coders (κ = .77). Categories, subcategories, and the
number of codes across all documents are listed in Table 1 (multiple codes per in-
terview were possible).
Table 1: Category system
Subcategory
Code
quantity
1.
Initial motivation and expectations
1.1
Reasons
for ap-
plying
to the
pro-
gram
1.1.1
timing/optimal time for
respective startup phase
6
1.1.2
location and network area
14
1.1.3
recommended by co-founders
5
1.1.4
searching for new co-founders/team
members
19
1.1.5
finds AI a promising
topic/interested in AI
10
1.1.6
developing new fields or areas
7
1.2
Expec-
tations
of the
pro-
gram
1.2.1
acquisition of knowledge
1.2.1.1
knowledge about successful teams
2
1.2.1.2
unspecified knowledge
15
1.2.1.3
entrepreneurial knowledge (incl.
financing)
17
1.2.2
networking and visibility within the
community
17
1.2.3
continue working on the idea
(motivational boost)
27
1.2.4
obtain feedback on the startup idea
27
1.2.5
exchange with other participants (e. g.,
same problems)
12
2. Startup idea
2.1
Startup
sector
2.1.1
finance/fintech
3
2.1.2
leisure
4
2.1.3
health & pharma
5
2.1.4
human resources
2
2.1.5
sustainability & energy
5
2.1.6
education
3
2.1.7
marketing & marketplace
4
2.1.8
other
4
2.2
AI
2.2.1
no AI reference
6
2.2.2
AI reference
26
Notes: AI = Artificial Intelligence
4.2 Sample
We collected data from two cohorts, which we treated as one sample because they
had completed the same program only at different times. The questionnaire sample
Startup skills in the field of artificial intelligence 37
consisted of applicants to the program; the interview sample consisted of only those
applicants who had an idea for their startup when they applied to the program.
Of the 83 respondents to the questionnaire at the first measurement point
(Mage = 28.94, 52 identified as male) 27 could be merged to complete pre/post ob-
servations. Almost half of the sample reported having a master's degree, one
quarter claimed to have a bachelor's degree, and eight a Ph. D., and 20 stated they
had already started a business.
A total of 32 people (Mage = 31.00, 23 identified as male) were recruited for the
interviews. Of these 32 people, seven had a bachelor’s degree, 18 a master’s degree,
and two a Ph. D., and eight indicated that they had already founded a company.
The first round of data collection ran from September to November 2022; the
second ran from March to May 2023.
5 Results
5.1 Questionnaire
Knowledge of AI applications prior to participation in the program averaged 11.45
points (max = 17; min = 6). Upon completion of the program, this dropped to 10.17
points (max = 17; min = 2). Looking only at those who participated at both meas-
urement points,4 a similar trend emerged (N = 27, Mt0 = 11.26, SD = 2.71;
Mt1 = 10.33, SD = 3.20), but the difference between the two measurement points
was statistically insignificant.
4 In Table 2 the mean values of the total sample are shown. For the pre-post comparisons, however, only
a subsample was considered, namely the people who participated in both questionnaires.
38 Hangen & Wuttke
Table 2: Questionnaire Results
Mean (standard deviation)
Facets
Subscales
t0 (N = 84)
t1 (N = 28)
Concerning AI
knowledge
11.45 (2.73)
10.17 (3.51)
usefulness
3.37 (0.63)
attitudes* ease of use 2.73 (0.63) -
job relevance
3.39 (0.67)
beliefs
*
2.40 (0.44)
-
motivation* intrinsic 3.21 (0.63) 2.86 (0.55)
extrinsic
2.97 (0.84)
2.84 (0.83)
startup knowledge
*
2.61 (0.57)
2.81 (0.45)
Notes: *4-point Likert scale; N = sample size; t0 = questionnaire pre; t1 = questionnaire post
In contrast, self-reported knowledge about how to start a business increased from
M = 2.61 (SD = 0.57) to M = 2.81 (SD = 0.45) when rated on a 4-point Likert scale.
Looking at only those who participated at both measurement points, a similar trend
emerged (N = 27, Mt0 = 2.56 (SD = 0.51); Mt1 = 2.81 (SD = 0.45)). Self-reported
knowledge about how to start a business was significantly higher upon completion
of the program than when starting it (t(26) = 2.32; p = .028; d = .446)).
Likewise, before the program, there was a significant difference in self-reported
knowledge about how to start a business between people who had already founded
a company and those who had not (Mexperienced found er = 2.88 (SD = 0.49); Munexperienced
founder = 2.53 (SD = 0.57); t(80) = 2.40; p = .018; d = .62).
The mean values of attitudes toward AI ranged from M = 2.73 (SD = 0.63) for ease
of use to M = 3.39 (SD = 0.67) for job relevance and could be considered rather
high based on the scale of 1 to 4 used. The mean values of beliefs about AI were M
= 2.40 (SD = 0.44).
In terms of motivation toward AI applications, extrinsic motivation decreased
slightly (but not significantly), while intrinsic motivation decreased significantly
(t(23) = 2.63; p = .015; d = .54). Intrinsic motivation prior to the program did not
differ significantly between people who had already founded a business and those
who had not (Mexperience d founder = 3.18 (SD = 0.69); Munexperienced founder =3.20 (SD =
0.61)). Due to the low number of participants after the program (n = 4 participants
who had already founded a company), the difference could not be calculated, but
descriptive differences became apparent (Mexperienced founder = 2.50 (SD = 0.29);
Munexperienced founder = 2.90 (SD = 0.53)).
Startup skills in the field of artificial intelligence 39
5.2 Interviews
From the 32 interviews we conducted, we found 61 utterances regarding why par-
ticipants chose to enroll in the program. These 61 utterances were grouped into six
different reasons for applying to participate in the program. The most common rea-
son was to find co-founders or team members (N = 19):
“That's why I applied to the program, to find co-founders or like-
minded people who would like to somehow advance my idea.”
(interview 17)
Five people said they had applied because their co-founders had convinced them:
“Because a friend and partner asked me – not pushed me – but asked
me if I would like to do this together with him.” (interview 8)
Another reason for applying was the fit of the program with the respective startup
phase (N = 6, “Then we realized how well the program just fits our next
steps.”; interview 20) and the expectation that this would give them the best
support or encouragement at their current stage of business development.
A frequently cited reason for applying to participate in the program was “location
and network area” (N = 14), meaning the program provided a strong startup eco-
system in which participants hoped to build valuable networks. This response was
accompanied by interest in AI (AI as promising topic/interest in AI; N = 10), and AI
was referred to as “promising topic” and “a field that will grow”. Closely related to
this response was “developing new fields or areas” (N = 7):
“What we expect from this is to see a little bit what makes other
people think, what other ideas there are at the moment. Especially
here in the region. Yes, what people want to come up with and
implement, where the trends are going.” (interview 6)
Reasons for participating in the program were closely aligned with participants’ ex-
pectations of the program. We found 117 comments relating to expectations and
divided them into six categories. The category with the most codings was
acquisition of knowledge (N = 34), which was further divided into three
subcategories: knowledge about successful teams (N = 2), unspecified knowledge
(N = 15), and entrepreneurial knowledge (N = 17). Comments categorized as
unspecified knowledge included all those in which participants stated they wanted
to build knowledge but did not explain precisely what kind of knowledge. This
included statements such as the following:
“Simply first of all, because we always want to take a lot of
knowledge with us.” (interview 20)
“I went in at the beginning with the expectation that I would just
learn a lot.” (interview 31)
The utterances concerning knowledge about entrepreneurship often indicated a
strong interest in gaining business-related knowledge:
40 Hangen & Wuttke
“I want to gain the necessary knowledge, or at least basic
knowledge, in order to approach a startup with a certain degree
of certainty, so, yes.” (interview 13)
“So I'm hoping to get input that will help me when it actually
comes to starting the business, in terms of business model and
idea development.” (interview 14)
In addition to acquiring knowledge, the other main expectation was to receive feed-
back on one's own startup ideas (N = 27). This may have come from new team
members, but the participants mentioned predominantly feedback from mentors
and experts in the field.
“My expectations or our expectations would be first and foremost
that we get good feedback, especially qualified feedback.”
(interview 22)
“I would then like to present my idea and get initial feedback. I
really want to have this expert opinion to then learn what else
do I need to consider? What else do I need to do? Where should I
start my company?” (interview 21)
In addition to obtaining feedback, another expectation of the program was to ex-
change information with other participants (N = 12). Utterances described
obtaining social support, exchanging information about similar problems, and
finding “like-minded people”.
The expectation to network and to generate visibility in the community (N = 17)
describes the participants' expectations of the program’s positive effects on their
professional network of founders of AI or data-driven startups.
“Of course I would like to expand my network!”(interview 21)
Many utterances (N = 27) were assigned to the category “continue working on the
idea”. Here, participants saw the program as an opportunity to work more deeply
and intensively on their idea and, if necessary, to enter into it seriously (e. g., by
giving up permanent employment). The program was also described as a motiva-
tional boost that should lead to investing more time and resources.
“And I think that's always good when you have the pressure to
perform in such a program, so to speak, certain deadlines, certain
deliveries, because I think that's pretty cool.” (interview 16)
“We met briefly and then he said yes, let´s just do it and take
the whole thing as a catalyst to take a few steps forward more
quickly.” (interview 15)
Our sixth research question about whether participants were interested in AI was
answered by looking at the participants’ startup sectors. As can be seen in Table 1,
the startup ideas were spread across a wide range of sectors. Only a small minority
did not relate their idea to AI or a data-driven business regardless of their sector.
Participants’ intentions to incorporate AI into business models varied widely. Some
had already developed an idea and wanted to incorporate AI in a subsequent step;
others had included AI-based or data-driven business models from the beginning.
Startup skills in the field of artificial intelligence 41
Even if the implementation was still in the planning stage for many, ideas were ac-
cumulating for the use of Blockchain together with AI, AI in forecasting models, and
AI in customer service.
6 Discussion
The special issue topic of learning about AI was considered in terms of founders’
competences who intend to have a data-driven startup. We conducted a longitudi-
nal study involving a pre-post survey and semi-structured interviews to track learn-
ing with regard to AI-related facets of entrepreneurial competence (see Figure 1).
Our data indicate that the participants’ knowledge about AI applications did not
change despite completing the training program. Although content knowledge is
considered a powerful predictor of achievement (Nitzschke et al., 2019), it should
be kept in mind that startups are usually founded in teams, and these teams are
usually characterized by their diversity. This means that not every team member
necessarily has to have in-depth knowledge of AI applications; a basic
understanding may be sufficient for some team members. However, our data show
that many of the participants in our training program for potential founders did not
have even a basic understanding of AI.
As self-assessed startup knowledge increased, it is reasonable to assume that the
participants focused more on acquiring general startup knowledge. It is possible
that participants were more interested in exploring general startup issues and there-
fore neglected learning about AI-specific topics addressed in various learning units.
Perhaps acquiring startup knowledge was seen as more "hands on" and easier to
implement immediately while acquiring AI knowledge was seen more as an on-go-
ing endeavor that could be developed at a later time. In addition, the AI knowledge
test focusses on the basics of AI applications and therefore may not capture the
specific knowledge that participants have acquired. Depending on the focus of the
startup, AI knowledge acquisition may have been much more specific, for example,
limited to a particular AI model. Although information is available on which learning
units were offered, we did not investigate whether or how the content of the units
were actually used during or after the training phase.
When looking at the participants’ attitudes toward AI, it is important to note that
the participants rated the benefits of AI and job relevance as greater than the ease
of its use. Thus, they attributed great significance to AI in relation to their job and
expressed awareness of the importance of AI. Motivation was divided into extrinsic
and intrinsic, and intrinsic motivation was significantly lower after the program,
while extrinsic motivation did not decrease significantly. One explanation for the
drop in intrinsic motivation could be that participants were confronted with the
(technical) difficulties in using AI or they could not implement technological ideas
42 Hangen & Wuttke
as they had previously imagined, and so the (expected) fun of working with AI could
have suffered as a result.
Analysis of data collected during the interviews reveals that the reasons for enrolling
in the training program were manifold. At the top of the list was searching for co-
founders or other team members. Further, participants were interested in starting a
new business and learning more about AI. Responses to questions regarding expec-
tations of the training program indicate there may be demand for the development
of future programs: many saw participation in the program mainly as an opportunity
to boost their motivation and their chances for entrepreneurial success. Further-
more, guidance, feedback, and support from experts was another important reason
for enrolling in the program, as was the opportunity to build networks. Concerning
entrepreneurial ideas, the participants emphasized that they wanted to establish AI
in their startups. This finding is consistent with positive attitudes toward AI, beliefs
about AI, and motivation toward AI applications.
There were limitations to our study that should be mentioned. First, the sample that
completed the pre- and post-questionnaire was small and therefore results cannot
be generalized. Second, the sample was drawn from one training program and
therefore results must be interpreted within the regional context and within the
context of the evaluated program only. Nonetheless, the data offer insight into the
AI-related competences of aspiring entrepreneurs and reveal their expectations and
motivations. In future training programs for potential founders of AI or data-driven
startups, more emphasis should be placed on developing AI competences. While
the program seems to have been effective for developing general startup
knowledge, the findings of this research and other studies on training should be
integrated even more strongly in the design of future programs to make the focus
on AI clearer. To this end, (1) the relevance of the topic could be made clearer to
increase motivation and (2) the number of implementation phases could be in-
creased, even if they are associated with more effort and increased budget require-
ments.
In this paper we focused on participants' reactions to their entrepreneurial training
program and their learning gains; in future research we will explore the participants’
post-training behavior (e. g., whether they apply their newly acquired knowledge)
and the results in terms of the impact of their training on their entrepreneurial en-
deavors (e. g., whether they succeed in setting up a profitable venture). Moreover,
we will explore entrepreneurial team building, in particular, the processes of finding
appropriate team members, team dynamics, and teamwork.
Startup skills in the field of artificial intelligence 43
References
Ajzen, I. (1991). The theory of planned behavior. Organizational behavior and human decision processes,
50 (2), 179-211.
Alberti, F. (1999). Entrepreneurship education: scope and theory. In C. Salvate, P. Davidsson & A. Persson
(Eds.), Entrepreneurial knowledge and learning: Conceptual advances and directions for further
research (pp. 64-84). Jonkoping: Jonkoping International Business School.
Bohner, G. (2002). Einstellungen. In W. Stroebe, K. Jonas, M. Hewstone (Hrsg.), Sozialpsychologie. Springer-
Lehrbuch. Berlin: Springer.
Brandt, M. (2019). Künstliche Intelligenz rechnet sich [Artificial intelligence pays off]. Verfügbar unter:
https://tinyurl.com/w7z4wk5 [26.07.2023].
Bundesregierung (2020). Künstliche Intelligenz "Die entscheidende Zukunftstechnologie des 21. Jahrhun-
derts" [Artificial intelligence "The decisive future technology of the 21st century"]. Verfügbar unter:
https://tinyurl.com/ylkj3o82 [26.07.2023].
Cademartori, I., Seifried, J., Wuttke, E., Krille, C., & Salge, B. (2017). Developing a training programme to
promote professional error competence in accounting. In E. Wuttke & J. Seifried, Professional Error
Competence of Preservice Teachers: Evaluation and Support (S. 29-46). SpringerBriefs in Education.
Springer, Cham.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technol-
ogy. MIS Quarterly, 13 (3), 319-340.
Dubberke, T., Kunter, M., McElvany, N., Brunner, M. & Baumert, J. (2008). Lerntheoretische Überzeugungen
von Mathematiklehrkräften: Einflüsse auf die Unterrichtsgestaltung und den Lernerfolg von Schülerin-
nen und Schülern [Learning theory beliefs of maths teachers: influences on lesson planning and stu-
dents' learning success]. Zeitschrift für pädagogische Psychologie, 22 (34), 193-206.
Ecker, W., Coulon, C-H, Kohler, M. (2021). KI in die Anwendung bringen – Eine Gemeinschaftsaufgabe für
Hochschulen, Forschungseinrichtungen, Unternehmen und Politik [Bringing AI into application – a joint
task for universities, research institutions, companies and politics] (Whitepaper). München: Plattform
Lernende Systeme. Verfügbar unter: https://www.acatech.de/publikation/ki-in-die-anwendung-
bringen/ [07.03.2024]
European Commission (2006). Entrepreneurship education in Europe: fostering entrepreneurial mindsets
through education and learning. Final Proceedings of the Conference on Entrepreneurship Education
in Oslo.
Gaddam, S. (2007). A conceptual analysis of factors influencing entrepreneurship behavior and actions. ICFAI
Journal of Management Research, 6 (11), 46-63.
Hangen, Wuttke & Weber (2024). Entwicklung und Evaluation eines Trainingsprogramms für Startups im
Bereich data driven ventures [Development and evaluation of a training program for start-ups in the
field of data driven ventures]. In preparation.
Hartig, J. (2008). Kompetenzen als Ergebnisse von Bildungsprozessen. Kompetenzerfassung in pädagogi-
schen Handlungsfeldern [Competences as results of educational processes. Assessment of compe-
tences in pedagogical fields of action]. Theorien, Konzepte und Methoden, 26, 15-26.
Heinrichs, K. (2016). Dealing with critical incidents in the postformation phase: Design and evaluation of an
entrepreneurship education course. Vocations and Learning, 9 (3), 257-273.
Heinrichs, K. (2021). Assessing critical incidents in the post-formation phase: a matter of entrepreneurial
expertise? Journal of International Education in Business, 14 (2), 161-178.
Hochholdinger, S., Rowold, J. & Schaper, N. (2008). Ansätze zur Trainings- und Transferevaluation. [Ap-
proaches to training and transfer evaluation.] Trainingsevaluation und Transfersicherung: Modelle,
Methoden, Befunde, 30-52.
Hoffmann, T. (1999). The meanings of competency. Journal of European Industrial Training, 23, 275-286.
Kirkpatrick, D. (1998). Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler.
Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating training programs: The four levels. Berrett-Koehler Pub-
lishers.
44 Hangen & Wuttke
Klieme, E., Avenarius, H., Blum, W., Döbrich, P., Gruber, H., Prenzel, M., Reiss, K., Riquarts, K., Rost, J., Tenorth,
H.-E. & Vollmer, H. J. (2003). Zur Entwicklung nationaler Bildungsstandards [Development of national
educational standards] (Expertise). Bonn: Bundesministerium für Bildung und Forschung (BMBF).
Kreditanstalt für Wiederaufbau (KfW) (2015). KfW Gründungsmonitor [KfW Start-up Monitor]. Frankfurt am
Main: Kreditanstalt für Wiederaufbau. Verfügbar unter: https://tinyurl.com/yw7dgz47 [26.07.2023].
Kuratko, D. F. (2005). The emergence of entrepreneurship education: development, trends, and challenges.
Entrepreneurship Theory and Practice, 29 (5), 577-598.
Laupichler, M. C., Aster, A., Schirch, J. & Raupach, T. (2022). Artificial intelligence literacy in higher and adult
education: A scoping literature review. Computers and Education: Artificial Intelligence, 3, 100101.
Maresch, D., Harms, R., Kailer, N. & Wimmer-Wurm, B. (2016). The impact of entrepreneurship education on
the entrepreneurial intention of students in science and engineering versus business studies university
programs. Technological Forecasting & Social Change, 104, 172-179.
Mayring, P. (2010). Qualitative Inhaltsanalyse [Qualitative content analysis]. In G. Mey & K. Mruck (Hrsg.),
Handbuch Qualitative Forschung in der Psychologie. Wiesbaden: VS Verlag für Sozialwissenschaften.
Moberg, K. (2014). Two approaches to entrepreneurship education: The different effects of education for
and through entrepreneurship at the lower secondary level. The International Journal of Management
Education, 12 (3), 512-528.
Mueller, M. L. & Dunlosky, J. (2017). How beliefs can impact judgments of learning: Evaluating analytic pro-
cessing theory with beliefs about fluency. Journal of Memory and Language, 93, 245-258.
Nabi, G., Holden, R. & Walmsley, A. (2006). Graduate career-making and business start-up: a literature re-
view. Education+Training, 48 (5), 373-385.
Nitzschke, A., Velten, S., Dietzen, A. & Nickolaus, R. (2019). Relevance of professional experience and voca-
tional knowledge for solving problems – an analysis of perspective electrical engineering technicians.
Zeitschrift für Weiterbildungsforschung, 43, 117-131.
Norris, N. (1991). The trouble with competence. Cambridge Journal of Education, 21, 331-341.
Oosterbeek, H., van Praag, M. & Ijsselstein, A. (2010). The impact of entrepreneurship education on entre-
preneurship skills and motivation. European Economic Review, 54, 442-454.
Pajares, M. (1992). Teachers’ beliefs and educational research. Cleaning up a messy construct. Review of
Educational Research, 62 (3), 307-332.
Plümer, T. & Niemann, M. (2016). Existenzgründung Schritt für Schritt. Mit 2 ausführlichen Businessplänen
(2. aktual. Aufl.) [Business start-up step by step. With 2 detailed business plans]. Wiesbaden: Gabler.
Richardson, V. (1996). The role of attitudes and beliefs in learnin g to teach. Handbook of research on teacher
education, 2 (102-119), 273-290.
Roxas, B. G., Cayoca-Panizales, R., Mae de Jesus, R. (2008). Entrepreneurial knowledge and its effects on
entrepreneurial intentio ns: Development of a conceptual framework. Asia-Pacific Social Science Review,
8 (2), 61-77.
Salas, E. & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual review of
psychology, 52 (1), 471-499.
Schiefele, H. (1993). Brauchen wir eine Motivationspädagogik? [Do we need a motivational pedagogy?].
Zeitschrift für Pädagogik, 39 (2), 177-186.
Schiefele, U. (1996). Motivation und Lernen mit Texten [Motivation and learning with texts]. Göttingen: Ho-
grefe.
Schiefele, U. & Köller, O. (2001). Intrinsische und extrinsische Motivation [Intrinsic and extrinsic motivation].
In D. H. Rost (Hrsg.), Handwörterbuch Pädagogische Psychologie (S. 304-310). Weinheim: Beltz.
Schiefele, U., Moschner, B. & Husstegge, R. (2002). Skalenhandbuch SMILE-Projekt [Scales manual SMILE
project]. Bielefeld: Universität Bielefeld, Abteilung für Psychologie.
Startup skills in the field of artificial intelligence 45
Schmidt, J. & Happ, R. (2022a). Befunde aus einem Lehr- und Forschungsprojekt zur Integration von Inhalten
zu KI-Anwendungen bei angehenden Berufsschullehrkräften [Findings from a teaching and research
project on the integration of learning on AI applications for prospective vocational school teachers]. In
D.-K. Mah & C. Torner (Hrsg.), Künstliche Intelligenz mit offenen Lernangeboten an Hochschulen lehren.
Erfahrungen und Erkenntnisse aus dem Fellowship-Programm des KI-Campus (S. 57-72). Berlin: KI-
Campus.
Schmidt, J. & Happ, R. (2022b). Fachspezifisches Wissen, Einstellungen und Überzeugungen zu Anwendun-
gen der Künstlichen Intelligenz bei Studierenden der Wirtschaftspädagogik [Subject-specific know-
ledge, attitudes and beliefs about applications of artificial intelligence among students of business
education], Jahrbuch der berufs-und wirtschaftspädagogischen Forschung, 2022, 89-102.
Schreier, M. (2014a). Varianten qualitativer Inhaltsanalyse: Ein Wegweiser im Dickicht der Begrifflichkeiten.
[Variants of qualitative content analysis: A guide through the jungle of terminology.] Forum Qualitative
Sozialforschung/Forum: Qualitative Social Research, 15 (1).
Schreier, M. (2014b). Qualitative content analysis. In U. Flick (Ed.), The SAGE Handbook of qualitative data
analysis. London: SAGE.
Shane, S. (2000). Prior knowledge and the discovery of entrepreneurial opportunities. Organ Sci, 11 (4),
448-469.
Short, E. C. (1985). The concept of competence: Its use and misuse in education. Journal of Teacher
Education, 36, 2-6.
Siegfried, C., & Wuttke, E. (2021). Development of economic competence of students at general grammar
schools through a non-formal opportunity to learn with experts. Citizenship, Social and Economics
Education, 20 (2), 122-142.
Stamboulis, Y. & Barlas, A. (2014). Entrepreneurship education impact on student attitudes. The International
Journal of Management Education, 12, 365-373.
Statista Research Department (2022). Umfrage zur Einstellung gegenüber Künstlicher Intelligenz 2021
[Survey on attitudes towards artificial intelligence 2021]. Verfügbar unter: https://tinyurl.com/ys4grohd
[26.07.2023].
Sreenivasan, A. & Suresh, M. (2023). Twenty years of entrepreneurship education: a bibliometric analysis.
Entrepreneurship Education, 6 (1), 45-68.
TechQuartier (2022). Find your co-founder and launch your data-driven venture. Retrieved from: https://h-
ventures.studio/ [26.07.2023].
Venkatesh, V. & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four
longitudinal field studies. Management Science, 46 (2), 186-204.
Wang, X., Yang, H., Han, H., Huang, Y. & Wu, X. (2022). Explore the entrepreneurial process of AI start-ups
from the perspective of opportunity. Systems Research on Artificial Intelligence, May/June 2022,
569-580.
Wu, W. Y., Chang, M. L. & Chen, C. W. (2008). Promoting innovation through the accumulation of intellectual
capital, social capital, and entrepreneurial orientation. R&D Management, 38 (3), 265-277.
Weinert, F. E. (2001). Vergleichende Leistungsmessung in Schulen – eine umstrittene Selbstverständlichkeit
[Comparative performance assessments in schools – a controversial matter of course]. In F. E. Weinert
(Hrsg.), Leistungsmessungen in Schulen (S. 17-32). Weinheim: Beltz.
Widding, L. (2005). Building entrepreneurial knowledge reservoirs. Journal of Small Business and Enterprise
Development, 12 (4), 595-615.
Zlatkin-Troitschanskaia, O. & Seidel, J. (2011). Kompetenz und ihre Erfassung. Das neue „Theorie-Empirie-
Problem“ der empirischen Bildungsforschung? [Competence and its assessment. The new "theory-em-
piricism problem" of empirical educational research?] In O. Zlatkin-Troitschanskaia (Hrsg.), Stationen
empirischer Bildungsforschung. Traditionslinien und Perspektiven [Klaus Beck zum 70. Geburtstag ge-
widmet] (S. 218-233). Wiesbaden: VS, Verlag für Sozialwissenschaften.
46 Hangen & Wuttke
Autorinnen
Jule Hangen, Wirtschaftspädagogik, insb. empirische Lehr-Lern-Forschung, Goethe
Universität Frankfurt
Prof. Dr. Eveline Wuttke, Wirtschaftspädagogik, insb. empirische Lehr-Lern-
Forschung, Goethe Universität Frankfurt
Korrespondenz an: hangen@econ.uni-frankfurt.de
Empirische Pädagogik © Verlag Empirische Pädagogik
2024, 38. Jahrgang, Heft 1, S. 47-72 https://doi.org/10.62350/QPIZ5920
Sabine Seufert, Lukas Spirgi, Jan Delcker, Joana Heil & Dirk Ifenthaler
Umgang mit KI-Robotern: maschinelle Übersetzer,
Textgeneratoren, Chatbots & Co – Eine empirische
Studie bei Erstsemester-Studierenden
Künstliche Intelligenz (KI) wirkt sich zunehmend auf das Leben der Menschen aus, indem sie zu einer
verstärkten Interaktion zwischen Menschen und Maschine führt (Kim, 2022). KI-Roboter werden als Agenten
betrachtet, die durch KI-Programmierung Aufgaben übernehmen, die traditionell von Menschen ausgeführt
wurden. Es gibt zwei Kategorien von KI-Robotern: solche mit menschenähnlicher Leistung und solche mit
menschenähnlicher Erscheinung zur sozialen Interaktion. Basierend auf diesem Konzept wurden für diese
Studie sieben konkrete Anwendungsfälle entwickelt. Eine Online-Umfrage unter Studierenden (N = 636) im
September 2022 (kurz vor der Veröffentlichung von ChatGPT) untersuchte sowohl die Nutzungshäufigkeit
als auch die ethische Bewertung der Studierenden für jeden der sieben Anwendungsfälle. Zudem wurde
sowohl ein Risiko- als auch ein Chancenindex gebildet, welche die generelle Wahrnehmung der
Studierenden gegenüber KI erheben. Die wahrgenommenen Risiken übersteigen die erkannten Chancen.
Die Nutzungshäufigkeit von KI-Roboter-Typen durch Studierende ist insgesamt als eher gering einzustufen.
Die Akzeptanz ethischer Grundsätze unterscheidet sich abhängig vom Typ des KI-Roboters, wobei
Studierende insbesondere bei KI-Robotern zur (menschenähnlichen) Textgenerierung eher Wert auf
Gerechtigkeit und bei KI-Robotern für menschenähnliche, soziale Interaktionen auf Transparenz legen.
Schlagwörter: KI-Ethik – KI-Kompetenz – KI-Nutzung – KI-Roboter – Künstliche Intelligenz (KI)
Interaction with AI robots: machine translators, text
generators, chatbots & co - an empirical study
among first-year students
Artificial Intelligence (AI) increasingly influences people's lives by fostering enhanced interaction between
humans and machines (Kim, 2022). AI robots are considered agents that undertake tasks traditionally per-
formed by humans through AI programming. There are two categories of AI robots: those with human-like
performance and those with human-like appearance for social interaction. Based on this concept, seven
specific use cases were developed for this study. An online survey conducted among students (N = 636) in
September 2022 (shortly before the release of ChatGPT) examined both the frequency of usage and the
ethical evaluation of each of the seven use cases by students. Additionally, a risk and an opportunity index
were formed to assess students' general perception of AI. The perceived risks outweigh the recognized
opportunities. The frequency of students' usage of AI robot types is generally considered relatively low. The
acceptance of ethical principles varies depending on the kind of AI robot, with students particularly valuing
fairness for AI robots involved in (human-like) text generation and transparency for AI robots engaged in
human-like social interactions.
Keywords: AI Ethics – AI Literacy– AI Robots – AI Usage – Artificial Intelligence (AI)
48 Seufert, Spirgi, Delcker, Heil & Ifenthaler
1 Einleitung
Die Künstliche Intelligenz (KI) scheint zunehmend das Leben der Menschen durch
mehr Interaktionen zwischen Menschen und Robotern zu beeinflussen (Kim, 2022).
KI bezeichnet Systeme, die auf Verfahren maschinellen Lernens, logischer Program-
mierung oder statistischer Verfahren basieren und ausgehend von der Analyse von
Daten Inhalte, Vorhersagen, Empfehlungen, Entscheidungen hervorbringen oder
Handlungen ausführen können (Europäische Kommission, 2022).
Seit der Einführung von ChatGPT im November 2022 hat die KI in Bereichen wie
Arbeit und Bildung Einzug gehalten (Garrel, Mayer & Mühlfeld, 2023). Die Leistungs-
fähigkeit dieser KI hat in der Hochschullehre zu verbreiteten Bedenken geführt, ins-
besondere hinsichtlich der potenziellen Nutzung durch Studierende zum Plagiieren,
indem sie KI-generierte Texte für unkontrolliert erstellte Arbeiten verwenden
(Cotton, Cotton & Shipway, 2023). In öffentlichen Diskussionen dominiert daher oft
die Perspektive der Lehrenden und der Hochschulverwaltung (Alshami, Elsayed, Ali,
Eltoukhy & Zayed, 2023). Vor diesem Hintergrund verfolgt dieser Beitrag das Ziel,
den Umgang mit KI in der Hochschullehre aus der Sicht der Studierenden zu erfor-
schen. Dabei ist es zunächst von zentraler Bedeutung, wie Studierende die Möglich-
keiten und Gefahren der KI in der Gesellschaft einschätzen, was für einen verantwor-
tungsbewussten Einsatz von KI entscheidend zu sein scheint (Kong, Man-Yin Cheung
& Zhang, 2021). Bisher existiert eine begrenzte Anzahl empirischer Studien zur Nut-
zung von KI in der Hochschullehre (Garrel, et al., 2023; Kim, 2022; Lim, Gunasekara,
Pallant, Pallant & Pechenkina, 2023). Darüber hinaus existieren bisher keine konzep-
tionellen Rahmenwerke, um KI-Roboter zu definieren, die sowohl hinsichtlich ihrer
Leistung als auch in ihrer Interaktionsform immer menschenähnlicher werden (Dang
& Liu, 2022; Roesler, Manzey & Onnasch, 2021). Durch diesen Ansatz wird es mög-
lich, die Nutzungshäufigkeit sowie die Einstellungen zur ethischen Nutzung von KI
anhand spezifischer Anwendungsfälle genauer zu analysieren – ein Aspekt, der bis-
her noch unerforscht ist und von Forschenden im Bereich der KI-Ethik zunehmend
gefordert wird (Jobin, Ienca & Vayena, 2019). An dieser Forschungslücke setzt daher
der vorliegende Beitrag an.
Der Beitrag ist folgendermaßen aufgebaut: Zuerst wird in Kapitel 2 der Begriff ‚KI-
Roboter‘ definiert und die verschiedenen Typen von KI-Robotern kategorisiert, um
die Formen der Menschenähnlichkeit von KI hervorzuheben. Zudem wird dargelegt,
nach welchen Kriterien eine ethische Nutzung von KI beurteilt werden kann. Im
nachfolgenden Kapitel 3 werden sowohl die Forschungsfragen als auch das Konzept
der durchgeführten Studie erörtert. Kapitel 4 befasst sich mit dem methodischen
Ansatz der empirischen Untersuchung. Die Präsentation der Untersuchungsergeb-
nisse erfolgt in Kapitel 5. In Kapitel 6 findet eine detaillierte Diskussion der Ergeb-
Umgang mit KI-Robotern – eine empirische Studie 49
nisse statt. Den Abschluss bildet Kapitel 7 mit Implikationen für die Hochschullehre
und einem Ausblick auf zukünftige Forschungen.
2 Theoretischer Hintergrund
2.1 KI-Roboter und neue Mensch-Maschine Interaktionen
Die Begriffe ‚KI-Technologien‘, ‚KI-Tools‘ und ‚KI-Systeme‘ werden oft synonym ver-
wendet und dienen als Sammelbezeichnungen für Softwareanwendungen, die auf
Künstlicher Intelligenz basieren, um spezifische Aufgaben zu automatisieren, zu
optimieren oder zu vereinfachen. Im Kontext der KI hat sich zudem der Begriff des
Agenten (Akteur) als zentral erwiesen (Murphy, 2019). Ein ‚Intelligenter Agent‘ wird
dabei als ein System definiert, das seine Umgebung wahrnimmt und Handlungen
durchführt, mit dem Ziel, seine Erfolgswahrscheinlichkeit zu maximieren (Murphy,
2019). Es kann somit auch Veränderungen in der Umgebung bewirken. Die Betrach-
tung der KI als (teil-)autonome, lernfähige Agenten rückt ethische Aspekte und die
Notwendigkeit einer verantwortungsbewussten Nutzung dieser Technologie in den
Fokus (Floridi & Cowls, 2019).
Ein KI-basierter oder intelligenter Roboter wird ursprünglich in der Robotikfor-
schung als ein physisch situierter Agent in der realen Welt definiert (Murphy, 2019).
Vor dem Hintergrund der neuesten Entwicklungen im Bereich des Natural Language
Processing (NLP) der KI gewinnen Conversational Agents zunehmend an Bedeutung,
die eine Interaktion über Text- oder Sprachschnittstellen mittels natürlicher Sprache
ermöglichen und menschliche Gesprächspartner simulieren (Winkler, Söllner &
Leimeister, 2021). Conversational Agents sind Chatbots und können in Software-
Anwendungen wie Online-Plattformen oder digitale Assistenten integriert oder über
Schnittstellen in die Hardware von Robotern eingebunden werden (Wollny et al.,
2021). Infolge dieser Fortschritte wird der Begriff des KI-Roboters in neueren For-
schungsstudien umfassender definiert: als eine Künstliche Intelligenz, die entweder
eine menschenähnliche Erscheinung aufweist oder menschenähnliche Leistungen
erbringt (Dang & Liu, 2022; Roesler et al., 2021).
Menschenähnliche Leistungen können hinsichtlich des Umfangs und der Komplexi-
tät ihrer Funktionalitäten charakterisiert werden (Dang & Liu, 2022). Im Bereich der
KI-basierten Textgenerierung im akademischen Kontext kann das Stufenmodell von
Boyd-Graber, Okazaki und Rogers (2023) als Referenzrahmen dienen. Für die Erstel-
lung wissenschaftlicher Arbeiten hat die Association for Computational Linguistics,
eine internationale Forschungsgemeinschaft, die sich mit Sprachmodellen wie
ChatGPT beschäftigt, Richtlinien für den ethischen Umgang mit KI-basierten
Schreibtools herausgegeben (Boyd-Graber et al., 2023). Innerhalb dieses Orientie-
rungsrahmens lassen sich verschiedene Stufen definieren, die einen zunehmenden
50 Seufert, Spirgi, Delcker, Heil & Ifenthaler
Leistungsumfang der KI in der Texterstellung aufzeigen, die den Neuigkeitsgehalt
der generierten Inhalte bestimmen:
1) Übersetzungsleistung, reine Sprachunterstützung,
2) Unterstützung bei der Eingabe von Kurztexten (Schreibassistenz),
3) Text mit geringem Neuigkeitsgrad (z. B. Zusammenfassen, paraphrasieren) und
4) Text mit neuen Ideen, hoher Neuigkeitswert der Inhalte.
Der Referenzrahmen enthält für jede Stufe Richtlinien, ob der Einsatz von KI bei der
Texterstellung deklariert werden muss. Grundsätzlich gilt: Je höher der Leistungs-
umfang der KI, desto wichtiger ist die Deklaration und die exakte fachliche Prüfung
der von der KI erstellten Texte.
Menschenähnliche Erscheinung von KI-Robotern, die neue Dimensionen der
Mensch-Maschine-Interaktion eröffnen, lassen sich nach ihrer Modalität charakteri-
sieren. Dies definiert die parallele Nutzung verschiedener Sinneskanäle zur Informa-
tionsübermittlung (Minge, 2012). Ein Mediensystem gilt dann als multimodal, wenn
es mehr als einen Sinneskanal für die Interaktion verwendet. Solche Systeme fördern
natürlichere Interaktionsformen, was die Effizienz der Nutzung, damit den Erfolg der
Kommunikation (Weidemann, 2002) und u. U. sogar den Lernerfolg (Wang, Pang,
Wallace, Wang & Chen, 2022) steigert. Während Chatbots die textuelle und auditive
Modalität für ihre digitale Identität bedienen, repräsentieren Avatare ihre virtuelle
Identität, indem sie Gestik, Mimik und Bewegungen nutzen und auch eine reale Per-
son nachbilden können. Im Gegensatz zu Chatbots und Avataren (Software), haben
soziale Roboter auch eine physische Einheit (Software und Hardware), sind somit in
der physischen Welt verankert und bieten auch eine haptische Dialogschnittstelle
(vgl. Abbildung 1). Sie nehmen ihre Umgebung über Sensoren wahr, interagieren
mittels Motorik und können auch Emotionen zeigen (Guggemos, Seufert,
Sonderegger & Burkhard, 2022).
Umgang mit KI-Robotern – eine empirische Studie 51
Abbildung 1: Rahmenkonzept von KI-Robotern (eigene Darstellung)
Zusammenfassend lässt sich festhalten: ‚KI-Technologien‘, ‚KI-Tools‘ und ‚KI-Sys-
teme‘ dienen als allgemeine Oberbegriffe. ‚Intelligente Agenten‘ hingegen bezeich-
net eine spezifischere Kategorie, die Systeme umfasst, welche ihre Umgebung wahr-
nehmen und Handlungen mit dem Ziel ausführen, ihre Erfolgswahrscheinlichkeit zu
maximieren. Der Begriff ‚KI-Roboter‘ betont die Menschenähnlichkeit. Ursprünglich
bezog sich dieser auf das äußere Erscheinungsbild im Sinne einer physischen Prä-
senz, doch zunehmend wird er auch verwendet, um eine menschenähnliche Leistung
und/oder Erscheinungsform zu beschreiben. In der öffentlichen Berichterstattung
wird die Software ChatGPT oft als ‚Roboter‘ bezeichnet, wie zum Beispiel im NZZ-
Artikel von Kaeser (2022), in dem es heißt: „ChatGPT: Der Roboter schreibt nicht, er
schwafelt“.
2.2 Ethischer Umgang mit KI
Die Fortschritte in der Künstlichen Intelligenz bringen sowohl beträchtliche Mög-
lichkeiten als auch ernstzunehmende Herausforderungen mit sich, die insbesondere
eine ethisch verantwortungsvolle Anwendung der KI verlangen. Bao et al. (2022)
entwickelten einen Index zur Bewertung der potenziellen Vorteile und Gefahren der
KI. Zu den Vorteilen gehören unter anderem die Verbesserung der individuellen Ge-
sundheit und die Verringerung von Verzerrungen in menschlichen Entscheidungen.
Die Gefahren beinhalten jedoch Aspekte wie die Verstärkung sozialer Ungleichhei-
ten oder eine Zunahme der Arbeitslosigkeit, bedingt durch den Ersatz menschlicher
Arbeitskräfte durch Maschinen.
52 Seufert, Spirgi, Delcker, Heil & Ifenthaler
Aufgrund des großen Einflusses von KI auf die Gesellschaft haben viele Organisa-
tionen Initiativen gestartet, um ethische Grundsätze für die Anwendung von KI zu
etablieren. Diese ethischen Leitlinien sollen eine verantwortungsvolle Gestaltung
und Anwendung von KI gewährleisten, damit die Technologie positive Auswirkun-
gen auf die Gesellschaft entfaltet (Floridi & Cowls, 2019). Jobin et al. (2019) haben
in ihrer Meta-Studie bestehende Ethikrichtlinien für KI untersucht und miteinander
verglichen. Sie erstellten eine Übersicht über aktuelle Prinzipien und Richtlinien für
eine ethische KI, um zu prüfen, ob eine weltweite Konvergenz sowohl in den Prinzi-
pien der ethischen KI als auch in den Anforderungen für ihre Implementierung er-
kennbar ist. Ihre Analyse zeigte, dass sich eine globale Übereinstimmung hinsichtlich
fünf ethischer Prinzipien abzeichnet:
Transparenz: Es gibt Forderungen nach klarer Offenlegung darüber, wie KI-Sys-
teme Entscheidungen treffen. Solche Systeme sollten so konzipiert sein, dass
ihre Arbeitsweise und Entscheidungsprozesse nachvollziehbar und transparent
sind. Dies fördert das Vertrauen und ermöglicht es den Nutzern und Betroffenen,
das System zu verstehen und gegebenenfalls zu hinterfragen. Zudem ist damit
verbunden, die Nutzung transparent darzustellen und die menschliche Verant-
wortlichkeit in der Nutzung klar zu definieren.
Gerechtigkeit und Fairness: Datenverzerrungen, bei denen bestimmte Muster
aus den Trainingsdaten, die zwar mit dem Ergebnis korrelieren, aber zu Verzer-
rungen oder Voreingenommenheit führen, prägen das Verhalten des Algorith-
mus. Werden verzerrte Datensätze für das Training von KI-Systemen verwendet,
kann das System diese Verzerrungen – beispielsweise Diskriminierung von Min-
derheiten – in seinen Handlungen widerspiegeln oder sogar verstärken. Die ver-
wendeten Algorithmen beruhen auf Regeln, die eine bewusste oder unbewusste
Bevorzugung bestimmter Personen oder Handlungsoptionen zur Folge haben
können. Die Verhinderung von Diskriminierung und die Gewährleistung, dass
der KI-Einsatz nicht zu sozialen Ungleichheiten führt, sind daher zentral.
Datenschutz und Privatsphäre: Der Schutz der Privatsphäre und der persönlichen
Daten von Individuen ist essenziell. Für das Training von KI-Systemen werden
große Mengen an Daten benötigt. Dadurch, dass immer mehr Daten gesammelt
und effizienter miteinander verknüpft werden können, entsteht die Möglichkeit,
aus scheinbar unpersönlichen Daten Rückschlüsse auf einzelne Personen zu zie-
hen. Dies führt zu neuen Herausforderungen im Bereich des Datenschutzes und
der Wahrung der Privatsphäre.
Nicht-Schädigung und Solidarität: Die Entwicklung und der Einsatz von KI-Sys-
temen sollten darauf ausgerichtet sein, Schaden zu vermeiden, das allgemeine
Wohl sowie den gesellschaftlichen Nutzen zu fördern und sicherzustellen, dass
KI-Systeme die menschliche Autonomie nicht beeinträchtigen.
Umgang mit KI-Robotern – eine empirische Studie 53
Verantwortliche Entwicklung von KI: Dieses Prinzip richtet sich an die Gestaltung
und Qualitätsentwicklung von KI-Systemen. Es muss definiert werden, wer für
die Konsequenzen mangelhafter KI-Systeme verantwortlich ist. Damit kann das
Vertrauen in KI-Systeme durch die Entwicklung zuverlässiger und sicherer Tech-
nologien gestärkt werden.
Die Ethikrichtlinien und skizzierten Prinzipen sind in einer sehr allgemeinen Form
gehalten. Die Forschungsgruppe um Jobin et al. (2019) betont deshalb die Notwen-
digkeit, diese ethischen Richtlinien speziell für KI-Systeme und ihre Anwendungsbe-
reiche zu konkretisieren. Die ethischen Aspekte variieren je nach Interessengruppe,
und es ist wichtig, die jeweilige Perspektive einzunehmen, um sich mit diesen ethi-
schen Fragen auseinanderzusetzen. Beispielsweise könnten Bildungseinrichtungen
die Auswirkungen von KI-Technologien auf den Lernprozess und die Gerechtigkeit
im Bildungswesen betrachten, während Softwareentwickler und KI-Forschende sich
mit Themen wie der Transparenz von Algorithmen und der Vermeidung von Vor-
eingenommenheit in KI-Systemen beschäftigen könnten. Datenschutzbeauftragte
würden sich hingegen auf den Schutz der Privatsphäre und den sicheren Umgang
mit Nutzerdaten konzentrieren. Jede dieser Gruppen hat eine eigene Perspektive
auf ethische Fragen, die es zu berücksichtigen gilt (Morley et al., 2023).
3 Die vorliegende Studie
Diese Studie verfolgt das Ziel, den Umgang mit KI-Robotern in der Hochschulbil-
dung aus der Perspektive der Studierenden zu beleuchten. Insbesondere soll unter-
sucht werden, wie häufig die Studierenden KI-Roboter nutzen und wie sie die Chan-
cen und Risiken von KI-Robotern einschätzen. Die Untersuchung der Wahrnehmung
von Studierenden bezüglich der Möglichkeiten und Gefahren, die KI in der Gesell-
schaft mit sich bringt, ist von großer Wichtigkeit, da sie das Fundament für einen
verantwortungsvollen Einsatz von KI bildet (Kong et al., 2021). Es wird davon ausge-
gangen, dass die Fähigkeit der Studierenden, KI aus verschiedenen Blickwinkeln zu
beurteilen, entscheidend für ihren durchdachten Einsatz ist und wesentlich zur
Gestaltung einer von Technologie bestimmten Zukunft beiträgt (Kong et al., 2021).
Individuen, die der Meinung sind, dass ihr Umgang mit KI bedeutsame Auswirkun-
gen hat, zeigen tendenziell eine größere innere Motivation, die erforderlichen
Fähigkeiten zu erlernen (Frymier, Shulman & Houser, 1996). Daher ist es im universi-
tären Kontext wichtig, interdisziplinäre Diskussionen und Reflexionen zu den Aus-
wirkungen von KI zu fördern.
Auf der konkreten Anwendungsebene erscheint es sinnvoll, die Nutzung und ethi-
sche Einstellung nach KI-Roboter Typen in menschenähnliche Leistungsfähigkeiten
und menschenähnliche Erscheinung zu differenzieren: Diese Unterscheidung ist von
Bedeutung, da beide Aspekte unterschiedliche Facetten der Technologieakzeptanz
54 Seufert, Spirgi, Delcker, Heil & Ifenthaler
haben. Die menschenähnliche Leistung von KI bezieht sich auf deren Fähigkeit, Auf-
gaben auf eine Art und Weise zu bewältigen, die menschlicher Kognition,
Problemlösung und Entscheidungsfindung ähnelt. Im Gegensatz dazu beschreibt
die menschenähnliche Erscheinungsform die physische oder visuelle Gestaltung der
KI, die die Wahrnehmung und emotionale Reaktion der Nutzer beeinflussen kann.
Durch das getrennte Erfassen dieser Merkmale kann untersucht werden, inwiefern
jede dieser Eigenschaften die Akzeptanz, das Vertrauen und die Bereitschaft zur In-
teraktion mit KI-Technologien im Hochschulbereich beeinflusst.
Die nachfolgende Abbildung veranschaulicht das Forschungsdesign. Eine Syste-
matisierung der Typen von KI-Robotern wurde nach der menschenähnlichen Leis-
tung sowie menschenähnlichen Erscheinung unterschieden. Für die Studie wurden
insgesamt sieben verschiedene Anwendungsfälle entwickelt. Jeder Anwendungsfall
beschreibt einen Typ von KI-Robotern, mit denen die Studierenden in der Hoch-
schulbildung in Kontakt kommen können. Die Anwendungsfälle 1 bis 4 sind KI-Ro-
boter mit menschenähnlicher Leistung. Die Anwendungsfälle 5 bis 7 sind KI-Roboter
mit menschenähnlicher Erscheinung. Auf Basis der sieben Anwendungsfälle wurde
ein Fragebogen entwickelt (siehe Kapitel 4.2), um die drei definierten Forschungs-
fragen (FF) zu beantworten. Die in der Studie definierten KI-Roboter mit menschen-
ähnlichen Leistungen werden von den Studierenden vor allem zur Textgenerierung
verwendet (FF2) und die definierten KI-Roboter mit menschenähnlicher Erscheinung
werden primär für die soziale Interaktion verwendet (FF3). Aus der übergeordneten
Zielsetzung der Studie ergeben sich somit die folgenden Forschungsfragen:
Wie beurteilen Studierende die potenziellen Chancen und Risiken, die Künstliche
Intelligenz für die Gesellschaft mit sich bringt (FF1)?
In welchem Umfang setzen Studierende KI-Roboter für die Texterstellung ein
und wie beurteilen sie in diesem Kontext die relevanten ethischen Aspekte (FF2)?
In welchem Umfang setzen Studierende KI-Roboter für soziale Interaktionen ein
und wie beurteilen sie in diesem Kontext die relevanten ethischen Aspekte (FF3)?
Umgang mit KI-Robotern – eine empirische Studie