Effects of Remote Learning on Practitioner Integration

Abstract

The widespread inclusion of experts and practitioners in educational settings to teach and collaboratively learn can help alleviate a multitude of systemic problems. A new, inclusive path to teach youth the skills needed to utilise the problem solving approach named computational thinking is explored in this case study. During 2020 remote learning became ubiquitous and after a successful face to face workshop the consequences of a virtual environment were evaluated. This publication answers three questions based on an action research approach: What effect has remote learning on practitioner integration? What learning outcomes does a flipped classroom approach lead to? What lessons can be learned for a post-social-distancing world? Data was gathered during an expert driven virtual workshop, in an Austrian technical school with predominantly male students aged 17 to 18 (K-12). Analysis revealed the benefits of remote expert integration as relatively little overhead can establish practical knowledge and differentiated perspectives in an almost uninterrupted virtual workflow. The integration of practitioners should be made possible within virtual environments to minimise distraction and overhead if applicable. Despite its clear benefits a blended environment with additional face to face settings led to more interaction and excitement from the learners. Easy access to experts and practitioners is key to offer young people the tools necessary to face the challenges of the future.

Full text

Effects of Remote Learning on Practitioner Integration
Michael Pollak
Educational Technology, Graz University of Technology, 8010 Graz, Austria
research@michaelpollak.org
Nanna Nora Sagbauer
Educational Technology, Graz University of Technology, 8010 Graz, Austria
nanna.sagbauer@htl-hl.ac.at
Martin Ebner
Educational Technology, Graz University of Technology, 8010 Graz, Austria
martin.ebner@tugraz.at
Abstract: The widespread inclusion of experts and practitioners in educational settings to teach
and collaboratively learn can help alleviate a multitude of systemic problems. A new, inclusive path
to teach youth the skills needed to utilise the problem solving approach named computational
thinking is explored in this case study. During 2020 remote learning became ubiquitous and after a
successful face to face workshop the consequences of a virtual environment were evaluated. This
publication answers three questions based on an action research approach: What effect has remote
learning on practitioner integration? What learning outcomes does a flipped classroom approach
lead to? What lessons can be learned for a post-social-distancing world? Data was gathered during
an expert driven virtual workshop, in an Austrian technical school with predominantly male
students aged 17 to 18 (K-12). Analysis revealed the benefits of remote expert integration as
relatively little overhead can establish practical knowledge and differentiated perspectives in an
almost uninterrupted virtual workflow. The integration of practitioners should be made possible
within virtual environments to minimise distraction and overhead if applicable. Despite its clear
benefits a blended environment with additional face to face settings led to more interaction and
excitement from the learners. Easy access to experts and practitioners is key to offer young people
the tools necessary to face the challenges of the future.
Introduction
In 2020 everything went online, institutions were forced to implement programs for teaching and learning
remotely overnight. As educators and software developers raced to get a grip on the evolving situation it became
obvious that some elements of learning are very well suited to remote and decentralised systems. Big lectures and
one way communication formats were the first to be translated to the digital realm, formats best suited to higher
education (Ebner et al., 2020). At the same time primary and secondary schools struggled to make the transition to
some form of remote teaching while offering guidance, help and personal connections to the students in this
unprecedented situation.
Figure 1: Usage patterns (clicks and publications) at TU Graz
-389-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021
Originally published in: Pollak, M., Sagbauer, N.N. & Ebner, M. (2021). Effects of Remote Learning on Practitioner Integration. In T. Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp.
389-400). United States: Association for the Advancement of Computing in Education (AACE). Retrieved July 28, 2021 from https://www.learntechlib.org/primary/p/219684/.

In today's educational systems, where the public expects high efficacy of educators despite their below
average funding (4,9 % of Austrias GDP for education in 2018 (Canova, 2018; Gavurova et al., 2017)), the crisis of
2020 has uncovered and magnified a multitude of structural and organisational pitfalls. Society could and should
aim to unburden motivated educators and incorporate them into an interdisciplinary effort to create a future proof
and sustainable educational system (Ingram et al., 2007; Lam et al., 2014; Wentzel, 1998). Based on the success of
previous experiments the need to introduce experts from an industrial, economical and scientific background into
classroom settings was determined (Pollak, 2020; Pollak & Ebner, 2019, 2020). This exploration to find new and
better ways to introduce practitioner based training in computational thinking education saw an unexpected twist this
year. If schools are not allowed to host on site workshops due to the COVID-19 pandemic situation, is it possible to
integrate the knowledge gathered before into a remote setting? By allowing teachers to include and host subject
matter experts in a remote learning environment the paper at hand answers this question and aims to outline a
framework for future installments of resilient remote learning.
About Computational Thinking
The term computational thinking (CT) denotes a problem solving methodology based on the idea, that in
order to solve societal problems technologies need structured and precise input. First popularised by Jeannette M.
Wing 15 years ago (Wing, 2006, 2008, 2011) this approach became prevalent in scientific literature and later in the
educational context of national curricula. The learners in today's schools are growing up in an increasingly complex
societal landscape filled with a multitude of challenges as well as incredible opportunities. The set of skills provided
by CT adds value far outside the technical community and offers a way to make sense of today's uncertain world
that ever more relies on technological progress. The definition of what exactly CT entails varies widely over its
existence in the scientific community (Pollak & Ebner, 2019). This research study chose to adopt the definition of
Csizmadia et al. wherein CT
“[...] is the process of recognizing aspects of computation in the world that surrounds us and applying tools
and techniques from computing to understand and reason about natural, social, and artificial systems and
processes. It allows pupils to tackle problems, to break them down into solvable chunks, and to devise
algorithms to solve them” (Csizmadia et al., 2015, p. 5)
Ultimately CT can and should enable every student to “bend computation to (their) needs” Today's society
does not need more coders and programmers but young people that understand the basic computational principles
and can become interfaces to tech savvy engineers or artificial intelligence systems (GPT-3) thus offering the
profound set of skills needed to reach the Sustainable Development Goals we as a society strive towards (Bocconi et
al., 2016, p. 25; THE 17 GOALS | Sustainable Development, n.d.). Even though CT was included in the Austrian
curriculum for K-12 schools the actual conversion in classrooms around the country is still lacking. This slow
turnaround between educational policy and the situation in schools can also be seen internationally and should be
attributed to a number of factors. One component is the systemically slow teacher education system which makes it
cumbersome to introduce new subjects and topics as knowledge transfer is very laborious. In Austria compulsory
school teachers are obligated to participate in 15 hours of professional training each year (Andreitz & Müller, 2015;
Močinić & Piršl, 2019, p. 2) . With the speed of technological advances around learners and educators this approach
fails noticeably and can only be sustained with the energy and motivation of teachers spending their personal time,
money and energy to close that widening gap. Continued self-directed education outside formally paid functions is
crucial but research shows that this can lead to a significant increase in burned out professionals (García-Carmona et
al., 2019, p. 14; Ingram et al., 2007; Lam et al., 2014; Wentzel, 1998) This research study advocates for a quick
turnover and knowledge transfer between practitioners and learners, to unburden motivated educators and integrate
them into a viable interdisciplinary effort to create a future proof educational system.
Research Design
The study at hand attempts to answer the following research questions:
What effect has remote learning on practitioner integration?
-390-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

What learning outcomes does a flipped classroom approach lead to?
What lessons can be learned for a post-social-distancing world?
In accordance with prior research the approach first introduced by Kurt Lewin (1946) aptly named action
research (AR) was utilised. This research methodology allows for the interactive, collaborative and participatory
exploration of problems within volatile and complex systems. In the setting of K-12 education the students
perspective is a major concern in the evaluation of an intervention. Learning greatly benefits from a functional
interpersonal relationship and engagement between learners and educators, thus it is an important factor to assess
(Gibbs et al., 2017). The AR approach is expressed by four main characteristics, namely the active participation of
research stakeholders, an iterative process, the urge to change a situation to the benefit of all, and a strong focus on
practical exploration (Denscombe, 2014, p. 73). After reflecting on prior workshop settings and evaluating the
results, an updated iterative plan was formed to utilise the current remote learning environment as a base for
exploratory research. In the case study at hand, remote, virtual practitioner integration for a project based CT
workshop was evaluated. After the three sessions concluded data from learners and the participating teacher was
collected in an effort to determine the efficacy of this approach and identify possible shortcomings. The findings are
shared with the academic community and at the end of this paper improved iterations are proposed.
Idea and Framework
Based on the three research questions a workshop was designed that introduced a practical project with the
goal of fostering CT skills and abilities in a K-12 classroom. To test the effects of a flipped classroom environment
two groups of students were working on a similar project. Group one (G1) was taught in a lecture style setting
without preparation while group two (G2) got their tasks in advance and was engaged in a more interactive, flipped
classroom style. Lessons for G1 started at 07:50 and lasted till 09:30 while G2 started the synchronous section of the
workshop at 08:30 and ended also at 09:30.
Lecture Style Flipped Classroom
Idea Collection 2.12.2020 (10 / 3) 9.12.2020 (13 / 0)
Breakout Sessions 16.12.2020 (9 / 3) 23.12.2020 (12 / 0)
Integration and Evaluation 13.1.2021 (11 / 3) 20.1.2021 (14 / 0)
Table 1: Date of sessions as well as the number of male and female participants
Participating students were between 17 and 18 years old and currently attending a college for higher
vocational education (ISCED level 5) with a focus on electrical engineering in Lower Austrias district of Hollabrunn
(htl-hl.ac.at). As this case study is limited to a specific school, broader applicability needs to be assessed and
evaluated on a case to case basis. According to the schools data only 81 (7 %) of the 1.104 students in 2020 were
female. This fact corresponds to the data collected, where only three of the 28 participants were female (11 %). That
constitutes a prominent limitation of this study and is in stark contrast to the partner school in Waidhofen an der
Thaya (hakwt.at) with an economical focus, where workshop participants were split almost evenly and the schools
224 learners are predominantly female (152 / 72). Yet another clear reminder that more female learners should push
Figure 2: Action research as an iterative process
-391-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

into STEAM areas and schools need to tap into this massive unused potential (Moote et al., 2020; Schön et al.,
2020)
Compared to previous iterations a shorter time allowance of three sessions was agreed upon where students
were asked to quickly and efficiently use the learned skills in a practical way. Every group had three scheduled
workshop slots to come up with a suitable solution for a given problem, embedded in the curricular subject of
computer aided project development (CPE). The interactive sessions were held with the schools groupwork tool
Microsoft Teams. In Austria this collaboration suite has become the de facto standard as the regional authorities
(“Bildungsdirektionen”) rolled out Microsoft Teams to consolidate efforts in K-12 education during the second
COVID-19 lockdown.
The sessions
Comparing this workshop's schedule with the prior conducted case studies shows that a much quicker and shorter
approach was used. Instead of seven sessions only three sessions were scheduled in this evolution, to meet the
practical needs and restrictions of the research partners. Prior iterations utilised the chance to talk to more outside
experts and develop ideas independently of predetermined scaffolding.
Based on time restrictions and the more complex virtual setting a condensed format was decided on by the
school, teacher and researcher. With only six workshops a strict scaffolding was prepared and prior work done. This
iteration consisted of a first session to get to know the students, present the problem and collect ideas. A second date
was agreed upon to work in smaller breakout groups and develop three distinct fields, namely technologies used,
graphical representation and textual content. During the third and last session knowledge of the three teams was
integrated into a final product.
Figure 3: Timeline of the previous workshop iteration
Figure 4: Evolution between the first workshop iteration and the current state
-392-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

1 – Collect Ideas
The first session was held on December 2nd for G1 and December 9th 2020 for G2 to introduce the project to the
participating students. As students were only informed that a workshop with an outside expert would occur during
these sessions the pitch event was combined with the collection of ideas and potential project approaches. As a
broad problem was previously agreed upon learners were confronted with the task and queried on the topic.
To gain instant feedback and interact with the students specialised tools were evaluated during the preparation
phase. The online tool “Mentimeter” (mentimeter.com) was utilised, to allow simple and intuitive audience
interaction. One of the first questions asked of the students determined what technology they would like to use for
hosting information for their teachers. Examples given were a website, videos, posters, ticktock, or presentations.
The majority of the learners from G1 as well as G2 preferred to work on a website to gain more skills and
knowledge in this area. During this session the transparent grading scheme was also presented. Due to policy
reasons the in class workshop needed to be graded, so the users were explained why and how this is going to be
done. Three areas were graded independently, being the participation during the course, the quality of the
coursework as well as the handed in learning diaries. Grading in general seems to be fair and transparent at the
partner school as discussed with the groups during a short audience interaction session.
Finally the term CT and the corresponding definitions and usages were introduced to the group. At first the
translated long form definition of Prof. J. Wing (2011, p. 5) as described in the introduction was given to the
students, wherein everybody should be able to:
1. understand which aspects of a problem can be solved with a tool,
2. evaluate if a specific tool can solve a problem,
3. understand the limitations of computational tools,
4. adapt a computational tool to a new use,
5. recognize new ways to use tools, and
6. apply computational strategies in any domain.
After that a more practical and shortened framework was introduced and discussed. This four tier approach
is used for example by the BBCs popular MOOC “Bitesize” in the subject “Introduction to computational thinking“.
The first step named decomposition enables learners to break down a complex task into more manageable, simpler
chunks that can be worked on by a smaller group or an individual. During pattern recognition students were asked to
find similarities with other problems and challenges they already know from their life experience. Abstraction looks
to exclude unimportant sections and focus on the relevant and significant details of a solution. Finally an algorithmic
answer can be developed where simple, repeatable and concise steps or rules lead to a usable solution (What Is
Computational Thinking?, n.d.). To show not merely what elements CT entails but also what it can accomplish
students were given the first challenge to solve. As a simple assessment tool to understand and test the capabilities
of participating students, challenges from the testing suite “Biber der Informatik” were used throughout the
workshop. These exercises and questions are designed to assess CT and informatics skills of young people (K-12)
and are a German translation of the well known Bebras testing suite (bebras.org) featured in the international contest
of the same name (V. Dagienė et al., 2019; Dagiene & Stupuriene, 2016; Valentina Dagienė & Sentance, 2016;
Micheuz et al., 2019).
With five minutes of lead time students were given the first challenge to solve named “Mutation of an
Alien” in the Bebras questions catalogue. After all answers were posted participants discussed their solutions and the
way they solved this challenge. During this part it became obvious that CT is part of learners' everyday problem
Figure 5: Primary steps of computational thinking
-393-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

solving process and developing a clear understanding of its elements is beneficial to future complex problem
solving. A second challenge was offered named “Bikefun” and answered by students. The following discussion and
comparison of approaches made the aforementioned point even clearer. Students at this point got an understanding
of the importance of clear and concise problem solving approaches as well as communicating their methodologies.
After establishing a shared baseline the practical tasks for this workshop iteration were introduced. The given
problem of remote teaching and learning, students role as subject matter experts and a path to support their teachers
was showcased.
In a final interactive session students were asked to identify the most problematic elements of a virtual
classroom, what challenges and problems they face during their remote learning experience. This simply was
discussed with the groups as interaction at this point became easy and seamless. All mentions were grouped and
categorised by the participants and a final roster of eight main problems was decided on.
2 – Breakout Sessions
The established problems were identified by the groups and after a short reiteration and discussion the eight
problematic fields were ranked as follows.
The first and second most significant problems are the technical issues of “Is the sound quality good” and
“Is the internet connection good”. Simple technical problems still seem to be the main challenge after six months of
remote learning. Also issues concerning virtual teaching style and the specific needs of online learning have a focus.
The questions “Do I ask enough questions”, “Is my writing legible and well scaled” and “Are examples slowly built
and explained” came up in this research project. The technical need of entering lectures and appointments in a
shared calendar was explained where else homework and exams seem to have a lower ranked importance.
Three teams were established with a focus on technology, graphics as well as texts to tackle problems in
separate breakout sessions before combining their efforts. The tasks for the flipped classroom group - G2 - were
handed out with prepared worksheets. Room for questions was offered but not used by the students. Given tasks
were split up to facilitate teamwork and collaboration. The technology team was asked to research software tools to
build a simple and free website. Three possible solutions were given with “sites.google.com”, “webnode.at” and
“wix.com”. The graphics team was tasked to find one animation or image for each of the eight main problems.
Lastly the team working on textual content collaboratively wrote a paragraph for every issue to explain their need in
detail and to a broader audience. The teams were given 30 minutes to work on their work packages and prepare a
short presentation for the whole group, which ended the second session.
Figure 6: Checklist for remote teaching ranked by importance
-394-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

3 - Integration and Evaluation
The primary goals during the last session were integration and implementation. After a short reiteration the
team focusing on technological aspects presented the drag and drop editor to the entire group. G1 started to
implement the first four guidelines collaboratively, with students from the texts team adding their paragraphs as well
as learners from the graphics team adding fun animations and graphics. The second group was tasked with
reviewing the previous work and adding the missing key points teaching staff should focus on. The finished product
was then published with permission here: https://1028164.wixsite.com/gutevirtuellelehre
Outcomes and Lessons Learned
Reception by Students and Teachers
To gather participants feedback the students were tasked to write a short learning diary after every session.
The completion of these learning diaries was mandatory for the students and contributed to their grading in this
curricular course. An optional template including the following headlines was provided:
What did I learn today?
What did I not understand?
What will I ask in the next session?
The goal was to foster reflection from the learners and start a discussion about understanding of a concept
like CT and integral knowledge brought from prior learning experience. From the 28 students 26 handed in their
learning diaries and allowed them to be anonymously analysed. The questions posted by the learners revealed a wide
spread of interests. For example the desire to learn more about the audience interaction tools and how students can
use these capabilities was often referenced. Learning diaries should be parsed and checked after each workshop
session as most students wrote their ideas down but did not feel comfortable speaking up during the online lessons.
Students overall took significant time to work on their learning diaries and obviously felt pride in doing so.
Workload for Teachers
In-class time is a scarce good in technical subjects in a college for higher vocational education. A lot of
new and trend-setting technologies and knowledge push into the schools and adaptations of curricula can only
happen delayed. According to the participating teacher it is often a struggle to fit all curricular content and state-of-
the-art developments into the scheduled time frame. The teachers need to choose the focus and the depth of their
lessons wisely. Educators abilities to offer technological up-to-date knowledge and profound expertise demand
constant on-the-job training and professional development in addition to regular teaching. The Austrian school
system permits a secondary school teacher a sum of three days absence from classes to participate in training
sessions without financial loss. The attendance of specialised technical courses outside the teacher training programs
of University Colleges of Teacher Education (“Pädagogische Hochschule”) is not stipulated. These facts illustrate
the need of external professionals and experts to provide students and teachers with glimpses of real-world problems
and solutions, and enthralling state-of-the-art topics. Together with an educator the main obstacles for teachers when
inviting experts were analysed as follows.
1. Choosing experts and practitioners
Finding experts that are willing to share their knowledge and real-world experiences, who are able to break them
down to a level the students can understand, who prepare a workshop, lecture or talk, and last but not least, without
financial incentive is no trivial task.
2. Class integration
Every teacher in Austria is obligated to provide a timetable, where the content of the curriculum is assigned to a
class´schedule in the beginning of each school year (“Lehrfächerverteilung”). Events which take up time originally
planned for curricular content can cause problems, especially if the content is a requirement for a following topic.
-395-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

3. Grading
Teachers are obligated to grade students in class. There are punctual examinations like written exams and oral
examinations, and there is continuous observation of class participation. When an expert workshop like the one at
hand takes up a couple of weeks assessment of participating students and grading is necessary.
4. Organizational time and effort
The class teacher was required to put in some organizational effort by inviting the expert to the group meeting via
email. The presenter options needed adaptation by setting the expert as presenter in order to allow the expert to share
the screen or show a presentation. Initial issues could be solved by adding the expert to the class team. Overall the
teacher´s preparation requirements for the workshop were more time-consuming than preparing usual lessons for
this class. During the workshops the teacher inherited an observing position to keep records of absences, to assess
student's involvement, and support the practitioner if required. From a teacher's point of view the intensity of labor
was significantly lower for assisting the expert workshop than for teaching a regular class. So, it can be summarised
that a practitioner workshop consumed more of the class teacher´s time due to additional organizational issues
regarding online remote access of the expert, which was countered by a lower intensity or attention level during
workshops.
Grading and Assessment
The Austrian curricula changed slightly from a focus on knowledge toward competencies over the last
years. In technical education in general and in HTL Hollabrunn in particular the ability for utilizing gained skill sets
and a capability to adapt to changing (work-) environments are, and have always been, major goals for students and
highly regarded by businesses in the schools´ alumni. Grading students is always a very sensitive task for a teacher.
It is easy to measure repeatable facts in a written or oral exam while it is much more difficult to assess the level of
skill and know-how a student acquired. It is especially challenging for a teacher to assess and grade a particular
student in a collaborative group workshop, where personal learning effects cannot be spotted easily or sometimes
cannot be externally observed at all.
To assess students' efficacy and learning progress challenges based on the “Biber der
Informatik” framework were posed during the sessions. In a playful and fun way the groups participants can review
their thinking progress and discuss problem solving approaches. The yearly competition “Biber der Informatik”, a
German version of the well known Bebras testing suite offers CT challenges and riddles that can be entertaining to
answer and think about. Students' feedback was positive and the mode of individual thinking to solve the riddle
alone before discussing approaches and problems was well received. Evaluating CT learning progress with a simple,
quick and playful methodology is a topic for another publication but from this initial response it seems to be a
promising start. As the group size is small these findings are very limited and should be deemed inconclusive. The
assessment of complex skill sets like CT is challenging and only long term studies on a more involved scale can
show significant impact in learners' thinking styles and problem solving abilities (Grover, 2017; Hadad et al., 2019).
Despite that limitation the results suggest that lecture style practitioner integration leads to a better and more direct
communication pattern, more interpersonal contact and ultimately to more understanding of the CT principles.
Figure 7: Percentages of correct answers per group
-396-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Questionnaires
After the last sessions students were asked to answer three questions in a short questionnaire. The precise
wording of the questions was “What elements of this workshop would you like to integrate in the regular lessons?”
and “What are your thoughts about the integration of practitioners in regular lessons?”. The posted answers (26)
were categorised and visualised as seen in Figure 8.
Clearly the chance to utilize variation in schools is a vital factor to enhance learners' experience. The
possibility to exchange information, knowledge and approaches is key to student motivation and interaction, leading
to a more powerful and sustainable learning experience. A number of students (3) argue on the other hand that this
form of project based learning should be held only scarcely as not to take time out of their normal obligations. This
may be connected to the hope that full day workshops would be beneficial as the routine school schedule can be
interrupted.
Online Environment
It became clear that after months of virtual learning students are used to and really good at virtual
communication and learning. Every school has a specified and trusted set of tools established at this time, for the
participants the proprietary communication platform named Microsoft Teams was the de facto standard, probably
based on the countrywide Campus License available to schools (“Microsoft Austrian College and High School
Agreement”). Practitioners and outside experts have to take into account what software tools are regularly used by
the students as well as the teaching staff, to avoid unnecessary complexity and overhead. Changing tools and
environments should be done very carefully and deliberately as it means a noticeable change in the learners
attention. Basing the workshops on the commonly used and well known tools and only afterwards introducing new
tools for audience participation was very well received during this experimental period.
Ultimately practitioners are involved in a very different environment usually and do have to cope with
unexpected issues in a classroom setting. Young students benefit from informal discussions and talks a lot, these
settings are very hard to imitate in a virtual environment. Expectation management is vital for a workshop where
worlds collide and distinct mindsets collaborate. Keeping an open mind makes online interactions engaging for all
participants, having a plan B and scheduling accordingly is important. Together with the students the researchers
experienced some fun anecdotes during this project. For example one group did not manage to determine a team to
create text content, because of some confusion and a simple lack of motivation. The workshop's hosts only noticed
after the 30 minutes breakout session and a short discussion later plans were changed and amended. This is what
happens in the real world, we highly recommend to enjoy the show.
“The best laid plans of mice and men often go awry” - Robert Burns
Figure 8: Learners responses to a practitioner lead workshop
-397-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Flipped Classroom
As found in previous studies the success of a flipped classroom approach is highly dependent on the
knowledge and self motivation of the participating students as well as the personal prepositions, as reproduced in the
study at hand. Lecturing practitioners have to work closely with professional teaching staff to get feedback on the
given tasks and try to understand what elements learners will focus on. The tasks need to be very precise and clear,
taking into account the prior knowledge and experiences of learners. As students are used to working in the morning
most did not engage with the given tasks until the usual school timeframe. One of the data points examined was at
what time a given question was answered by students. Despite the availability of the task days prior, the posed
question was answered mainly on the morning of the workshop, only two of the 13 answers were registered on the
evening before. These results match prior studies of flipped classroom approaches. Some learners work well with a
varying degree of freedom but ultimately preparation of the cohort was lacking and not done as precisely as needed
to gain the potential benefits.
For a short workshop with little face to face and in person time flipped classroom seems to be not the right
choice. Interpersonal contact, group dynamics and a feeling of responsibility are key elements for a successful
flipped classroom session, a task that can not be accomplished within a six hour window. For longer workshops or
ongoing collaboration this can be beneficial but as far as the paper at hand is concerned it is not a recommended
approach.
Conclusion and Future Work
It seems obvious that based on our findings remote learning is an ideal way to introduce experts and
practitioners without much overhead into the classroom environments of our schools. This can lead to further
interaction and new patterns of learning, by forcing learners outside of their comfort zone and into new experiences
and environments. Although this method of practitioner integration into CT education seems viable within our case
study environment, face to face sessions are in most circumstances preferred by all parties. To answer the primary
research question posed in this publication it is clear that remote learning offers new possibilities and potential to
integrate practitioners from other regions and niche disciplines effectively. Despite its benefits the increased
workload for teaching staff is not negligible, this should be balanced with the advantages of remote teaching and
learning.
A short workshop project featuring an outside expert seems to be unsuitable for a flipped classroom setting.
Unfavorable group dynamics and missing interpersonal relationships lead to a lack of commitment in the flipped
classroom test group. In a world where the COVID-19 pandemic is a distant memory and forced social distancing
belongs to the past, remote workshops with experts and practitioners can become a format to diversify learning and
explore complex topics with curious learners. The chance to interact with and learn from experts in the field with an
intrinsic knowledge of the up-to-date scientific work can lead learners to new pathways and conclusions. For a post-
social-distancing world, as queried in the third research question, schools and institutions should foster a new,
collaborative understanding of seamless integration and interaction over remote learning environments. Teaching
and learning is evolving, virtual interaction can offer a pathway to a more open, more current and more collaborative
vision of schools and universities.
To create a contrast point to the widely available and completely asynchronous virtual courses known as
massive open online courses (MOOCs) like the Khan Academy or iMooX, a blended environment can offer all
positive attributes of an online environment with all the benefits of face to face interaction and learning. As schools
progress in a more blended teaching setup, virtual as well as physical spaces to interact with subject matter experts
and the tools that enable holistic learning need to be developed and integrated into the available spectrum of learning
spaces. Makerspaces around the world already allow for this sort of interaction, where the public sees a big spread of
expertise and knowledge accessible with a low border of entry and a wish to share and interact with diverse
communities (Grandl et al., 2021; Kjartansdottir et al., 2017; Schön et al., 2020). From FabLabs and Otelos (https://
otelo.or.at) to little library Makerspace, offering the tools and knowledgeable people to learn, experiment and master
a skill is key. CT education thrives in these environments, where makers and craftspeople regularly and intuitively
determine the most important tasks of a project and collaborate with each other. Problem solving skills are ingrained
in the process of making. With the help of a new networking platform, hosted and maintained by local Makerspaces,
-398-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

schools and teachers can gain access to practitioners and experts and lower the overhead of interdisciplinary
education. A future case study will explore new pathways of interaction between Makerspaces and schools, fostering
a relationship of learning and growing from each other. Makerspaces can connect and network people, either during
in-person workshops or in virtual environments. Easy access for students as well as teachers to experts and
practitioners is key to give young people the opportunity and tools to face the challenges of the future.
References
Andreitz, I., & Müller, F. (2015). In-service teacher training in Austria. In K. G. Karras & C. C. Wolhuter (Eds.), International
handbook of teacher education training and retraining systems in modern world (pp. 25–41). Studies and Publishing.
https://www.researchgate.net/publication/283328501_In-service_teacher_training_in_Austria
Bocconi, S., Chioccariello, A., Dettori, G., Ferrari, A., & Engelhardt, K. (2016). Developing Computational Thinking in
Compulsory Education—Implications for policy and practice (No. JRC104188; JRC Working Papers). Joint Research
Centre (Seville site). https://ideas.repec.org/p/ipt/iptwpa/jrc104188.html
Canova, P. (2018, November 21). Education and Training Monitor 2018 Austria Report [Text]. Education and Training -
European Commission. https://ec.europa.eu/education/resources-and-tools/document-library/education-and-
training-monitor-2018-austria-report_en
Csizmadia, A., Curzon, P., Dorling, M., Humphreys, S., Ng, T., Selby, C., & Woollard, J. (2015). Computational thinking—A
guide for teachers.
Dagienė, V., Futschek, G., & Stupurienė, G. (2019). Creativity in Solving Short Tasks for Learning Computational Thinking.
Constructivist Foundations, 14(3), 382–396.
Dagiene, V., & Stupuriene, G. (2016). Bebras—A Sustainable Community Building Model for the Concept Based Learning of
Informatics and Computational Thinking. Informatics in Education, 15(1), 25–44.
https://doi.org/10.15388/infedu.2016.02
Dagienė, Valentina, & Sentance, S. (2016). It’s Computational Thinking! Bebras Tasks in the Curriculum. In A. Brodnik & F.
Tort (Eds.), Informatics in Schools: Improvement of Informatics Knowledge and Perception (pp. 28–39). Springer
International Publishing. https://doi.org/10.1007/978-3-319-46747-4_3
Denscombe, M. (2014). The Good Research Guide: For Small-Scale Social Research Projects. McGraw-Hill Education (UK).
Ebner, M., Schön, S., Braun, C., Ebner, M., Grigoriadis, Y., Haas, M., Leitner, P., & Taraghi, B. (2020). COVID-19 Epidemic as
E-Learning Boost? Chronological Development and Effects at an Austrian University against the Background of the
Concept of “E-Learning Readiness”. Future Internet, 12(6), 94. https://doi.org/10.3390/fi12060094
García-Carmona, M., Marín, M. D., & Aguayo, R. (2019). Burnout syndrome in secondary school teachers: A systematic review
and meta-analysis. Social Psychology of Education, 22(1), 189–208. https://doi.org/10.1007/s11218-018-9471-9
Gavurova, B., Kocisova, K., Belas, L., & Krajcik, V. (2017). Relative efficiency of government expenditure on secondary
education. Journal of International Studies, 10(2), 329–343. https://doi.org/10.14254/2071-8330.2017/10-2/23
Gibbs, P., Cartney, P., Wilkinson, K., Parkinson, J., Cunningham, S., James-Reynolds, C., Zoubir, T., Brown, V., Barter, P.,
Sumner, P., MacDonald, A., Dayananda, A., & Pitt, A. (2017). Literature review on the use of action research in higher
education. Educational Action Research, 25(1), 3–22. https://doi.org/10.1080/09650792.2015.1124046
Grandl, M., Ebner, M., Schön, S., & Brünner, B. (2021). MAKER DAYS for Kids: Learnings from a Pop-up Makerspace. In W.
Lepuschitz, M. Merdan, G. Koppensteiner, R. Balogh, & D. Obdržálek (Eds.), Robotics in Education (pp. 360–365).
Springer International Publishing. https://doi.org/10.1007/978-3-030-67411-3_33
Grover, S. (2017). Assessing Algorithmic and Computational Thinking in K-12: Lessons from a Middle School Classroom (pp.
269–288).
Hadad, R., Thomas, K., Kachovska, M., & Yin, Y. (2019). Practicing Formative Assessment for Computational Thinking in
Making Environments. Journal Science Education and Technology. https://doi.org/10.1007/s10956-019-09796-6
-399-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021

Ingram, M., Wolfe, R. B., & Lieberman, J. M. (2007). The Role of Parents in High-Achieving Schools Serving Low-Income, At-
Risk Populations. Education and Urban Society, 39(4), 479–497. https://doi.org/10.1177/0013124507302120
Kjartansdottir, S. H., Marusteru, G., Pétursdóttir, S., Ólafsdóttir, M. E., Blum-Ross, A., Scott, F., Hyatt, D., & Little, S. (2017).
Makerspaces in the Early Years: A Literature Review (2017). Makerspaces in the Early Years: A Literature Review.
University of Sheffield: MakEY Project.
https://www.academia.edu/34084704/Makerspaces_in_the_Early_Years_A_Literature_Review_2017_
KS3 Computer Science. (n.d.). BBC Bitesize. Retrieved 24 February 2021, from
https://www.bbc.co.uk/bitesize/guides/zp92mp3/revision/1
Lam, S.-F., Jimerson, S., Wong, B. P. H., Kikas, E., Shin, H., Veiga, F. H., Hatzichristou, C., Polychroni, F., Cefai, C., Negovan,
V., Stanculescu, E., Yang, H., Liu, Y., Basnett, J., Duck, R., Farrell, P., Nelson, B., & Zollneritsch, J. (2014).
Understanding and measuring student engagement in school: The results of an international study from 12 countries.
School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological
Association, 29(2), 213–232. https://doi.org/10.1037/spq0000057
Lewin, K. (1946). Action Research and Minority Problems. Journal of Social Issues, 2(4), 34–46.
https://doi.org/10.1111/j.1540-4560.1946.tb02295.x
Micheuz, P., Pasterk, S., & Bollin, A. (2019). Facts, Figures, Remarks and Conclusions about the Austrian Bebras Challenge
with Regard to Computational Thinking and Computer Science Domains. 13.
Močinić, S., & Piršl, E. (2019). Initial Teacher Education: Appropriate Models for a Knowledge Society? European Journal of
Education, 2(1), 6–15. https://doi.org/10.26417/ejed-2019.v2i1-48
Moote, J., Archer, L., DeWitt, J., & MacLeod, E. (2020). Comparing students’ engineering and science aspirations from age 10
to 16: Investigating the role of gender, ethnicity, cultural capital, and attitudinal factors. Journal of Engineering
Education, 109(1), 34–51. https://doi.org/10.1002/jee.20302
Pollak, M. (2020). Praktikerinnen und Praktiker im Computational Thinking Unterricht.: Medienimpulse, 58(1), 21 Seiten-21
Seiten. https://doi.org/10.21243/mi-01-20-17
Pollak, M., & Ebner, M. (2019). The Missing Link to Computational Thinking. Future Internet, 11(12), 263.
https://doi.org/10.3390/fi11120263
Pollak, M., & Ebner, M. (2020). Practitioner Integration in Computational Thinking Education. 570–580.
https://www.learntechlib.org/primary/p/217354/
Schön, S., Rosenova, M., Ebner, M., & Grandl, M. (2020). How to Support Girls’ Participation at Projects in Makerspace
Settings. Overview on Current Recommendations (pp. 193–196). https://doi.org/10.1007/978-3-030-18141-3_15
THE 17 GOALS | Sustainable Development. (n.d.). Retrieved 17 February 2021, from https://sdgs.un.org/goals
Wentzel, K. (1998). Social Relationships and Motivation in Middle School: The Role of Parents, Teachers, and Peers. Journal of
Educational Psychology, 90, 202–209. https://doi.org/10.1037/0022-0663.90.2.202
Wing, J. (2006). Computational Thinking. Communications of the ACM, 49, 33–35.
https://doi.org/10.1145/1118178.1118215
Wing, J. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A:
Mathematical, Physical and Engineering Sciences, 366(1881), 3717–3725. https://doi.org/10.1098/rsta.2008.0118
Wing, J. (2011). Research Notebook: Computational Thinking—What and Why? 8.
Acknowledgements
We thank HTL Hollabrunn for providing a space for learning and collaboration.
-400-
EdMedia + Innovate Learning 2021 - Online, United States, July 6-8, 2021