Journal of Information Technology Education: Innovations in Practice

Electronic portfolios are a student-centered outcomes-based assessment regime involving learners in the gathering, selection, and organization of artifacts synthesized into a compilation purposed to demonstrate knowledge, skills, and/or achievements supported by reflections that articulate the relevance, credibility, and meaning of the artifacts being presented. Electronic portfolios have been found to be a valid way to document student progress, encourage student involvement in assessment, showcase student work samples, promote students professionally, and provide a method of student learning outcomes and curriculum evaluation. However, electronic portfolio adoption represents a sizable commitment that is influenced by a number of variables and that requires foresight as well as a thoughtful strategy. This paper presents a model for selecting, designing, and implementing an electronic portfolio project and illustrates its application through the presentation of a detailed case study of a suc- cessfully implemented and ongoing electronic portfolio project used as a comprehensive assess- ment measure to determine degree mastery in the Department of Business, Management, and Ac- counting at the University of Maryland Eastern Shore. The model introduced in this paper is known as the Pentagonal E-Portfolio Model, named such for its five levels: 1) Level 1 - Identification of Needs; 2) Level 2 - Determination, Assessment, & Budgeting; 3) Level 3 - System Selection and Strategic Planning; 4) Level 4 - Development; and 5) Level 5 - Implementation and Continuation.

Future Information Systems (IS) professionals need to be equipped with a range of analytical and critical thinking skills that will enable them to solve business problems. At the University of Cape Town (UCT), the capstone course of the IS undergraduate curriculum is a group systems development project that requires students to apply these skills in a real-world context. However, students generally experience difficulty in the areas of problem-solving, coding and testing, all of which are required for successful systems development. A reflective approach has been applied to the course design, resulting in the development of a framework that implements the action learning cycle to encourage deep learning. This paper describes how the course has evolved through four phases, culminating in an approach that guides students in transcending from the basic level of following, through detachment towards fluency.

Higher education programs need to prepare their graduates for the practical challenges they can expect to face upon entering the workforce. Students can be better prepared if their academic learning is reinforced through authentic workplace experience, where the link between theory and professional practice can be realized. Increasingly, such learning in the workplace is being seen as an integral part of the university curricula as evidenced through the implementation of the Learn- ing the Workplace & Community (LiWC) Policy at Victoria University, Australia. This policy mandates a minimum of 25% content and assessment of all academic programs be related to work-integrated learning. Recognizing the need for authentic workplace experience in the IT undergraduate program, a re- view found that the existing work-related learning component accounted for only half the re- quired 25% LiWC commitment. Currently, the LiWC component is an industry-based capstone project that spans two semesters in the final year of study. These projects allow students to work on real-life software development tasks where they experience the practical challenges of build- ing software systems whilst appreciating the needs of a business client. In a search of the litera- ture, campus-located industry projects were identified as one of the two most common work- related learning experiences in IT programs, the other being internships sited in the workplace. By retaining the current project-based component, it was decided to add an internship to the pro- gram to further bolster the student learning experience and graduate outcomes. This paper details the existing program structure and explores two possible implementations for the achievement of the LiWC policy. The first approach necessitates the addition of one academic year of coopera- tive education internship to be placed strategically between the current second and third years. Alternatively, the second proposal sacrifices several elective units to accommodate a final semes- ter internship experience. The paper discusses both alternatives against various issues under con- sideration: staffing and administration, assessment, industry partnerships, professional accredita- tion and its impact upon differing cohorts of students.

Deep and shallow learner approaches are useful for different purposes. Shallow learning can be good where fact memorization is appropriate, learning how to swim or play the guitar for exam- ple. Deep learning is much more appropriate when the learning material present involves going beyond simple facts and into what lies below the surface. When students are asked to think about how facts are created and what they mean, then deep learning is needed. Deep learning requires students to think about the conceptual material used to construct a theory and to reflect on its meaning until they understand and can reconceptualise the item under study. Some forms of learning are more conducive to approaches that do not need deep reflection, although this process invariably brings greater learning potential. This paper outlines a course that was considered by the authors to be 'scrugged'. The word scrugged is defined as 'rough' as in “it's been a scrugged day.” This word arose in a teaching and learning environment in South Australia and was shared on a social networking site. Use of this term seems appropriate in a discipline based on continual change. The rough 'scrugged' approach of the standard information systems fare presents a real problem for Information Systems (IS) Academics because it gives IS the 'shallow' treatment. It is at best a loosely joined mix of concepts coming from multiple directions which does not present a useful framework for theory and instead presents a very thinly constructed grouping of concepts that are superficially treated. The shallow conceptual structure leaves no room for reflective thinking, learning or critical thinking. What results is a good understanding of what kinds of in- formation systems exist, but a very shallow understanding of disciplinary themes and meaning beyond simple artefacts. In this paper, we outline an approach to a course which moved students from shallow repetitive tasks to deep reflective learning around the concepts of Information Sys- tems and discuss the long term implications for Information Systems teaching. Yes Yes

Table of Contents of Volume 16 of the Journal of Information Technology Education: Innovations in Practice, 2017.

Table of Contents of the Journal of Information Technology Education: Innovations in Practice, Volume 17, 2018

Table of Contents of the Journal of Information Technology Education: Innovations in Practice, Volume 18, 2019

Table of Contents of the Journal of Information Technology Education: Innovations in Practice, Volume 19, 2020

Table of Contents of the Journal of Information Technology Education: Innovations in Practice, Volume 20, 2021

Table of Contents of the Journal of Information Technology Education: Innovations in Practice, Volume 21, 2022

Table of Contents of the Journal of Information Technology Education: Innovations in Practice, Volume 22, 2023

Aim/Purpose: The goal of the study was to examine the perceptions of senior academic staff who also serve as policymakers in Israeli colleges of education, regarding the integration of technology in teacher education, and the shift to online learning during the Covid-19 pandemic. There is little research on this issue and consequently, the aim of the present study is to fill this lacuna. Background: In Israel, senior academic staff in colleges of education play a particularly important role in formulating institutional policies and vision regarding the training of preservice teachers. They fulfil administrative functions, teach, and engage in research as part of their academic position. During the Covid-19, they led the shift to online learning. However, there is little research on their perceptions of technology integration in teacher education in general, and during the Covid-19, in particular. Methodology: This qualitative study conducted semi-structured interviews with 25 senior academic staff from 13 academic colleges of education in Israel. Contribution: The study has practical implications for the implementation of technology in teacher education, suggesting the importance of establishing open discourse and collaboration between college stakeholders to enable enactment of a vision for equity-that allows programs to move swiftly from crisis-management to innovation and transformation during the Covid-19 pandemic. Findings: The findings obtained from content analysis of the interviews reveals a central concept: “On both sides of the divide”, and points of intersection in the perceptions of the senior academic staff. The central concept encompassed three themes: (1) centralization - between top-down and bottom-up policies, (2) between innovation and conservation, and (3) between crisis and growth. The findings indicate that in times of crisis, the polarity surrounding issues essential to the organisation’s operation is reduced, and a blend is formed to create a new reality in which the various dichotomies merge. Recommendations for Practitioners: The study has practical implications for the scope of discussions on the implementation of technology in teacher education (formulating a vision and policies, and their translation into practice), suggesting that such discussions should consider the perceptions of policymakers. Recommendation for Researchers: The findings reflect the challenges faced by senior academic staff at colleges of education that reflect the ongoing attempts to negotiate and reconcile different concerns. Impact on Society: The findings have implications for colleges of education that are responsible for pre-service teachers' teaching practices. Future Research: An enacted vision for equity-based educator preparation that allows programs to move swiftly from crisis-management to innovation and transformation. Future research might reveal a more complete picture by investigating a broader spectrum of stakeholders both in Israel and elsewhere. Hence, future research should examine the power relations between senior college staff and external bodies such as the Higher Education Council (which determines higher education policies in Israel).

Aim/Purpose: One of the main objectives of higher education institutions is to provide a high-quality education to their students and reduce dropout rates. This can be achieved by predicting students’ academic achievement early using Educational Data Mining (EDM). This study aims to predict students’ final grades and identify honorary students at an early stage. Background: EDM research has emerged as an exciting research area, which can unfold valuable knowledge from educational databases for many purposes, such as identifying the dropouts and students who need special attention and discovering honorary students for allocating scholarships. Methodology: In this work, we have collected 300 undergraduate students’ records from three departments of a Computer and Information Science College at a university located in Saudi Arabia. We compared the performance of six data mining methods in predicting academic achievement. Those methods are C4.5, Simple CART, LADTree, Naïve Bayes, Bayes Net with ADTree, and Random Forest. Contribution: We tested the significance of correlation attribute predictors using four different methods. We found 9 out of 18 proposed features with a significant correlation for predicting students’ academic achievement after their 4th semester. Those features are student GPA during the first four semesters, the number of failed courses during the first four semesters, and the grades of three core courses, i.e., database fundamentals, programming language (1), and computer network fundamentals. Findings: The empirical results show the following: (i) the main features that can predict students’ academic achievement are the student GPA during the first four semesters, the number of failed courses during the first four semesters, and the grades of three core courses; (ii) Naïve Bayes classifier performed better than Tree-based Models in predicting students’ academic achievement in general, however, Random Forest outperformed Naïve Bayes in predicting honorary students; (iii) English language skills do not play an essential role in students’ success at the college of Computer and Information Sciences; and (iv) studying an orientation year does not contribute to students’ success. Recommendations for Practitioners: We would recommend instructors to consider using EDM in predicting students’ academic achievement and benefit from that in customizing students’ learning experience based on their different needs. Recommendation for Researchers: We would highly endorse that researchers apply more EDM studies across various universities and compare between them. For example, future research could investigate the effects of offering tutoring sessions for students who fail core courses in their first semesters, examine the role of language skills in social science programs, and examine the role of the orientation year in other programs. Impact on Society: The prediction of academic performance can help both teachers and students in many ways. It also enables the early discovery of honorary students. Thus, well-deserved opportunities can be offered; for example, scholarships, internships, and workshops. It can also help identify students who require special attention to take an appropriate intervention at the earliest stage possible. Moreover, instructors can be aware of each student’s capability and customize the teaching tasks based on students’ needs. Future Research: For future work, the experiment can be repeated with a larger dataset. It could also be extended with more distinctive attributes to reach more accurate results that are useful for improving the students’ learning outcomes. Moreover, experiments could be done using other data mining algorithms to get a broader approach and more valuable and accurate outputs.

Aim/Purpose: To examine the early perceptions (acceptability) and usability of EASEL (Education through Application-Supported Experiential Learning), a mobile platform that delivers reflection prompts and content before, during, and after an experiential learning activity. Background: Experiential learning is an active learning approach in which students learn by doing and by reflecting on the experience. This approach to teaching is often used in disciplines such as humanities, business, and medicine. Reflection before, during, and after an experience allows the student to analyze what they learn and why it is important, which is vital in helping them to understand the relevance of the experience. A just-in-time tool (EASEL) was needed to facilitate this. Methodology: To inform the development of a mobile application that facilitates real-time guided reflection and to determine the relevant feature set, we conducted a needs analysis with both students and faculty members. Data collected during this stage of the evaluation helped guide the creation of a prototype. The user experience of the prototype and interface interactions were evaluated during the usability phase of the evaluation study. Contribution: Both the needs analysis and usability assessment provided justification for continued development of EASEL as well as insight that guides current development. Findings: The interaction design of EASEL is understandable and usable. Both students and teachers value an application that facilitates real-time guided reflection. Recommendations for Practitioners: The use of a system such as EASEL can leverage time and location-based services to support students in field experiences. This technology aligns with evidence that guided reflection provides opportunities for metacognition. Recommendation for Researchers: Iterative prototyping, testing, and refinement can lead to a deliberate and effective app development process. Impact on Society: The EASEL platform leverages inherent functionality of mobile devices, such as GPS and persistent network connectivity, to adapt reflection tasks based on lo-cation or time. Students using EASEL will engage in guided reflection, which leads to metacognition and can help instructors scaffold learning Future Research: We will continue to advance the application through iterative testing and development. When ready, the application will be vetted in larger studies across varied disciplines and contexts.

Aim/Purpose: Business analytics is a cross-functional field that is important to implement for a college and has emerged as a critically important core component of the business curriculum. It is a difficult task due to scheduling concerns and limits to faculty and student resources. This paper describes the process of creating a central video repository to serve as a platform for just in time teaching and the impact on student learning outcomes. Background: Industry demand for employees with analytical knowledge, skills, and abilities requires additional analytical content throughout the college of business curriculum. This demand needs other content to be added to ensure that students have the prerequisite skills to complete assignments. Two pedagogical approaches to address this issue are Just-in-Time Teaching (JiTT) and scaffolding, grounded in the Vygoskian concept of “Zone of Proximal Development. Methodology: This paper presents a case study that applies scaffolding and JiTT teaching to create a video repository to add business analytics instruction to a curriculum. The California Critical Thinking Skills Test (CCTST) and Major Field Test (MFT) scores were analyzed to assess learning outcomes. Student and faculty comments were considered to inform the results of the review. Contribution: This paper demonstrates a practical application of scaffolding and JiTT theory by outlining the process of using a video library to provide valuable instructional resources that support meaningful learning, promote student academic achievement, and improve program flexibility. Findings: A centrally created library is a simple and inexpensive way to provide business analytics course content, augmenting standard content delivery. Assessment of learning scores showed an improvement, and a summary of lessons learned is provided to guide implications. Recommendations for Practitioners: Pedagogical implications of this research include the observation that producing a central library of instructor created videos and assignments can help address knowledge and skills gaps, augment the learning of business analytics content, and provide a valuable educational resource throughout the college of business curriculum. Recommendation for Researchers: This paper examines the use of scaffolding and JiTT theories. Additional examination of these theories may improve the understanding and limits of these concepts as higher education evolves due to the combination of market forces changing the execution of course delivery. Impact on Society: Universities are tasked with providing new and increasing skills to students while controlling the costs. A centrally created library of instructional videos provides a means of delivering meaningful content while controlling costs. Future Research: Future research may examine student success, including the immediate impact of videos and longitudinally using video repositories throughout the curriculum. Studies examining the approach across multiple institutions may help to evaluate the success of video repositories. Faculty acceptance of centrally created video libraries and assignments should be considered for the value of faculty recruiting and use in the classroom. The economic impact on both the university and students should be evaluated.

Aim/Purpose: During the education of future engineers and experts in the field of computer science and information communication technology, the achievement of learning outcomes related to different levels of cognitive ability and knowledge dimensions can be a challenge. Background: Teachers need to design an appropriate set of activities for students and combine theory-based knowledge acquisition with practical training in technical skills. Including various activities for formative assessment during the course can positively affect students’ motivation for learning and ensure appropriate and timely feedback that will guide students in further learning. Methodology: The aim of the research presented in this paper is to propose an approach for course delivery in the field of software engineering and to determine whether the use of the approach increases student’s academic achievement. Using the proposed approach, the course Process Modeling for undergraduate students was redesigned and experimental study was conducted. Course results of the students (N=82) who took the new version of the course (experimental group) were compared to the results of the students from the control group (N=66). Contribution: An approach for a blended learning course in the field of software engineering was developed. This approach is based on the formative assessment activities that promote collaboration and the use of digital tools. Newly designed activities are used to encourage a greater level of acquired theoretical content and enhance the acquisition of subject-specific skills needed for practical tasks. Findings: The results showed that students who participated in the formative assessment activities achieved significantly better results. They had significantly higher scores in the main components of assessment compared to the students from the control group. In addition, students from the experimental group expressed positive views about the effectiveness of the used approach. Recommendations for Practitioners: The proposed approach has potential to increase students’ motivation and academic achievements so practitioners should consider to apply it in their own context. Recommendation for Researchers: Researchers are encouraged to conduct additional studies to explore the effectiveness of the approach with different courses and participants as well as to provide further insights regarding its applicability and acceptance by students. Impact on Society: The paper provides an approach and an example of good practice that may be beneficial for the university teachers in the field of computer science, information-communication technology, and engineering. Future Research: In the future, face-to-face activities will be adapted for performance in an online environment. Future work will also include a research on the possibilities of personalization of activities in accordance with the students’ characteristics.

Teaching advanced programming can be a challenge, especially when the students are pursuing different majors with diverse analytical and problem-solving capabilities. The purpose of this paper is to explore the efficacy of using a particular problem as a vehicle for imparting a broad set of programming concepts and problem-solving techniques. We present a classic brain teaser that is used to communicate and demonstrate advanced software development concepts and techniques. Our results show that students with varied academic experiences and goals, assuming at least one procedural/structured programming pre-requisite, can benefit from and also be challenged by such an exercise. Although this problem has been used by others in the classroom, we believe that our use of this problem in imparting such a broad range of topics to a diverse student population is unique.

We describe the ways in which an approach to constructive shape modelling based on the Function Representation (FRep) can be used to facilitate creative thinking in artistic and technical education. This approach assumes the use of a simple programming language or interactive software tools for creating a shape model, generating its images and finally fabricating a real object of that model. It can be considered an educational technology suitable not only for children and students but also for researchers, artists, and designers. The corresponding modelling language and software tools are being developed within an international HyperFun Project. We applied the theoretical framework and software tools on different levels of education starting from elementary schools to doctoral thesis research in various areas related to artistic design and animation, computer graphics, programming languages, software development and experimental physics. Several application case studies in various areas of art, design, and technical education from different educational institutions and countries are presented.

Aim/Purpose: To identify positive and negative aspects for learning of interactive tablet technology learning activities that promote student engagement and learning. Background: Engaging students in mathematics classes is an on-going challenge for teachers. In 2008 we were offered the opportunity to run interactive activities with a class set of tablet PCs that had just been released on to the market. Since then, we have run these interactive activities continuously with mathematics classes for computing students, albeit with two changes in hardware. Methodology: In the interactive activities, students submit full worked solutions to various problem types (classified as table, text, open or multi-choice) which can then be displayed to the class anonymously, discussed and annotated by the teacher. We surveyed student and staff perceptions and monitored academic performance. Contribution: We have over 10 years of results, observations, and experience from 2008, when tablet technologies were new and expensive, to the current time, when modern tablets with styli are now affordable. Findings: There was a significant increase in higher grades although pass rates did not increase significantly. Over the ten year period of the study, perceptions of students and staff about how this technology impacted on student learning were consistently positive. The majority of students found all problem types useful for learning even those they rated “too hard” or “too easy”. Benefits included increased feedback, peer learning and engagement. Recommendations for Practitioners: We recommend using tablet learning activities to engage students and teachers and to contribute to learning. Impact on Society: This study shows how using tablet technologies for interactive classroom activities can enable and enhance known pedagogies of feedback, peer instruction, and student engagement for mathematics classes. Future Research: We recommend extending this study to include larger classes, and other technical subjects that use symbols and diagrams. In addition, we suggest considering control groups.

Michael.kyobe@uct.ac.za Executive Summary As students in computing disciplines are introduced to modern information technologies, numer-ous unethical practices also escalate. With the increase in stringent legislations on use of IT, users of technology could easily be held liable for violation of this legislation. There is however lack of understanding of social aspects of computing, and institutions have not integrated many of these aspects into their curricula. The present study investigates the extent to which legal and ethical issues relating to IT are integrated in the undergraduate curricula in South African Universities. It also examines the factors that impede such integration. A review of literature on ethics, computing legislation, and curriculum development was con-ducted. Research questions were formulated and mixed methods consisting of the study of hand-books, a survey, and follow-up interviews were employed to collect data from different universi-ties in South Africa offering computing disciplines. The main findings suggest low levels of knowledge of especially legal aspects which constrains their incorporation into the curriculum. The process of integration is mainly influenced by de-partmental policies and politics. Most institutions have, however, integrated legal and ethical as-pects as add-ons topics to existing computing curriculum courses as opposed to running separate courses focusing on the subjects. A guide is being developed to facilitate the incorporation of le-gal content elements into undergraduate computing curricula.

Executive Summary Active learning has been championed in academic circles as the pedagogical fix to boring lectures typically found in introduction to information systems courses. However, the literature on active learning is mixed. In this paper, we critically examine active learning research and discover a misplaced emphasis leading to paradoxical findings in four areas. First, creating activities for cognitive engagement is not unique to active learning. Second, the amount of instructor led con-trol and direction is often glossed over, leaving a vague impression as to how much is necessary. Third, out of class activities are often ignored when they also accomplish the same effects for the same reasons. Fourth, an over-emphasis on techniques rather than outcomes renders active learn-ing bound to means and not to ends. The proper end should be the meaningful learning of the course objectives. We offer an alternative pedagogical framework for evaluating classroom techniques based on philosophic, psychological, and pedagogical research. Ausubel's Assimilation Learning Theory fits this research with a focus on meaningful learning of classroom objectives. In order to achieve meaningful learning of new concepts, an instructor must accomplish four things: (1) clearly de-fine the concepts, (2) provide proto-typical examples, (3) integrate the concepts within the stu-dents' knowledge, and (4) motivate the students to want to learn. Application of this pedagogical framework to introductory information systems classes provides a basis for evaluating classroom techniques and increasing meaningful learning. Turning the theory into practice, we propose three different ways to teach a Decision Support System module within an information systems course. Each approach is consistent with the meaningful learning framework. Those three approaches include the traditional lecture format, an activity based format consistent with active learning, and a case study based format which serves as a hybrid between passive and active learning tech-niques. For each approach, we highlight how the class structure, examples, and resources can help instructors create a meaningful learning experience for their students.

Executive Summary Text analytics refers to the process of analyzing unstructured data from documented sources, in-cluding open-ended surveys, blogs, and other types of web dialog. Text analytics has enveloped the concept of text mining, an analysis approach influenced heavily from data mining. While text mining has been covered extensively in various computer science curriculums, text mining or text analytics has appeared only briefly in information systems curriculums from business schools, generally within the context of data mining or other quantitatively-oriented analysis techniques. With the emergence of text analytic commercial technology products and the business opportuni-ties afforded to organizations, a full course addressing text analytics is an opportunity for busi-ness schools to support industry and innovate offerings within their curricula. This paper provides an experience report as an Innovation in Practice on the introduction of Text Analytics into the graduate curriculum of a business school in the United States. This experience introduces text analytics as a full semester, business school course. Its orientation is distinctly different from Computer Science where the focus is solely technical; here, the emphasis is analysis, its validity, and the use of technology. The course was introduced in the spring of 2010 as a graduate level course to a controlled registration of seven MSIS and MBA students. Highly positive student feedback supports the success of the course in terms of structure, objectives, interest, and appli-cability to immediate student interest and toward perceived future career opportunities. In order to instill practical as well as conceptual knowledge, the course incorporates both traditional and con-structivist learning techniques to enhance learning objectives. The Introduction section relates to the emerging interest of text analysis and brief assessment of the overall market. Included is a brief discussion of the growth of the technology and how it devi-ates from data mining. Teaching objectives are discussed in the second section and details of the class activities can be found in the third section. It is not in the scope of this paper to provide a tutorial on how to conduct text analytics research, but to demonstrate how the course can be man-aged as a semester-long course. A number of formal feedback sessions were conducted with the students and these results are reported in the fourth section. Lessons learned and Limitations are discussed in the fifth and sixth sections.

Aim/Purpose: As smartphones proliferate, many different platforms begin to emerge. The challenge to developers as well as IS educators and students is how to learn the skills to design and develop apps to run on cross-platforms. Background: For developers, the purpose of this paper is to describe an alternative to the complex native app development. For IS educators and students, the paper provides a feasible way to learn and develop fully functional mobile apps without technical burdens. Methodology: The methods used in the development of browser-based apps is prototyping. Our proposed approach is browser-based, supports cross-platforms, uses open-source standards, and takes advantage of “write-once-and-run-anywhere” (WORA) concept. Contribution: The paper illustrates the application of the browser-based approach to create a series of browser apps without high learning curve. Findings: The results show the potentials for using browser app approach to teach as well as to create new apps. Recommendations for Practitioners : Our proposed browser app development approach and example would be useful to mobile app developers/IS educators and non-technical students because the source code as well as documentations in this project are available for downloading. Future Research: For further work, we discuss the use of hybrid development framework to enhance browser apps.

An important aspect of education is to promote higher-order thinking skills to learners. However, in the lecture environment, learners are passively engaged and it is unlikely for higher-order thinking to occur. Although interventions such as "clickers" can be used to increase engagement in lectures, this does not necessarily promote higher-order thinking. Approaches such as collabo-rative learning are better suited for this but there is little room to use such methods in the short time frame of a lecture. With recent advances in the capabilities of smart mobile devices and their growing penetration rate among the student cohort, it is possible to take advantage of these devices to design a system to promote higher-order thinking skills in the lecture environment. We present the design of a mobile-app-based collaborative learning system named myVote and a process for its usage. Our aim is to present a theoretical paper that discusses the relevant learning theories used in designing the system as well as describe a process to use the system to achieve collaborative learning at varying levels of thinking. We demonstrate the usefulness and flexibility of the system through three scenarios involving different levels of thinking, ranging from lower-to higher-order. Although the scenarios are in the context of IT education, the system is versatile enough to be adapted for education in general and also non-educational settings, such as business-like environments. Our contribution is a framework for using mobile apps and collaborative learning theories within a lecture environment to encourage higher-order thinking in learners. Although a potential limitation of the system is that it may not be appropriate for teaching more technical IT materials, such as programming and SQL code snippets, the problem can be re-casted in a different format such as pseudocode in order to facilitate teaching in these areas.

As mobile devices become prevalent, there is always a need for apps. How hard is it to develop an app, especially a cross-platform app? The paper shares an experience in a project that involved the development of a student services web app that can be run on cross-platform mobile devices. The paper first describes the background of the project, the clients, and the proposed solution. Then, it focuses on the step-by-step development processes and provides the illustration of written codes and techniques used. The goal is for readers to gain an understanding on how to develop a mobile-friendly web app. The paper concludes with teaching implications and offers thoughts for further development.

Since the advent of the iPhone and rise of mobile technologies, educational apps represent one of the fastest growing markets, and both the mobile technology and educational app markets are predicted to continue experiencing growth into the foreseeable future. The irony, however, is that even with a booming market for educational apps, very little research regarding the quality of them has been conducted. Though some instruments have been developed to evaluate apps geared towards student learning, no such instrument has been created for teacher resource apps, which are designed to assist teachers in completing common tasks (e.g., taking attendance, communicating with parents, monitoring student learning and behavior, etc.). Moreover, when teachers visit the App Store or Google Play to learn about apps, the only ratings provided to them are generic, five-point evaluations, which do not provide qualifiers that explain why an app earned three, two, or five points. To address that gap, previously conducted research related to designing instructional technologies coupled with best practices for supporting teachers were first identified. That information was then used to construct a comprehensive rubric for assessing teacher re-source apps. In this article, a discussion that explains the need for such a rubric is offered before describing the process used to create it. The article then presents the rubric and discusses its different components and potential limitations and concludes with suggestions for future research based on the rubric.

Aim/Purpose Improving public schools is a focus of federal legislation in the United States with much of the burden placed on principals. However, preparing principals for this task has proven elusive despite many changes in programming by insti-tutions of higher learning. Emerging technologies that rely on augmented and virtual realities are posited to be powerful pedagogical tools for closing this gap. Background This study investigated the effects of immersive simulation technologies on principals' self-efficacy after treatment and the perceived significance of the design of the immersive simulation experience as an effective tool for adult learners. Methodology The investigator employed a multiple-methods study that relied on a purposive sample of graduate students enrolled in educational leadership programs at two small universities in the southeastern United States. Participants completed a two-hour module of immersive simulation designed to facilitate transfer of knowledge to skills thereby increasing their self-efficacy. Contribution This paper contributes to a small body of literature that examines the use of immersive simulation to prepare aspiring principals. Findings The findings indicate moderate effect sizes in changes in self-efficacy, positive attitudes toward immersive simulation as a pedagogical tool, and significance in the design of immersive simulation modules. This suggests that immersive sim-ulation, when properly designed, aids principals in taking action to improve schools. Recommendations for Practitioners Educational leadership programs might consider the use of immersive simula-tions to enhance principals' ability to meet the complex demands of leading in the 21st century. Impact on Society Principals may be more adept at improving schools if preparation programs provided consistent opportunities to engage in immersive simulations.Future Research Future research should be conducted with larger sample sizes and longitudinally to determine the effectiveness of this treatment.

Aim/Purpose: The purpose of this paper is to develop and evaluate a debiasing-based approach to assessing the learning materials in problem-based learning (PBL) environments. Background: Research in cognitive debiasing suggests nine debiasing strategies improve decision-making. Given the large number of decisions made in semester-long, problem-based learning projects, multiple tools and techniques help students make decisions. However, instructors may struggle to identify the specific tools or techniques that could be modified to best improve students’ decision-making in the project. Furthermore, a structured approach for identifying these modifications is lacking. Such an approach would match the debiasing strategies with the tools and techniques. Methodology: This debiasing framework for the PBL environment is developed through a study of debiasing literature and applied within an e-commerce course using the Model for Improvement, continuous improvement process, as an illustrative case to show its potential. In addition, a survey of the students, archival information, and participant observation provided feedback on the debiasing framework and its ability to assess the tools and techniques within the PBL environment. Contribution: This paper demonstrates how debiasing theory can be used within a continuous improvement process for PBL courses. By focusing on a cognitive debiasing-based approach, this debiasing framework helps instructors 1) identify what tools and techniques to change in an PBL environment, and 2) assess which tools and techniques failed to debias the students adequately, providing potential changes for future cycles. Findings: Using the debiasing framework in an e-commerce course with significant PBL elements provides evidence that this framework can be used within IS courses and more broadly. In this particular case, the change identified in a prior cycle proved effective and additional issues were identified for improvement. Recommendations for Practitioners: With the growing usage of semester-long PBL projects in business schools, instructors need to ensure that their design of the projects incorporates techniques that improve student learning and decision making. This approach provides a means for assessing the quality of that design. Recommendation for Researchers: This study uses debiasing theory to improve course techniques. Researchers interested in assessment, course improvement, and program improvement should incorporate debiasing theory within PBL environments or other types of decision-making scenarios. Impact on Society: Increased awareness of cognitive biases can help instructors, students, and professionals make better decisions and recommendations. By developing a framework for evaluating cognitive debiasing strategies, we help instructors improve projects that prepare students for complex and multifaceted real-world projects. Future Research: The approach could be applied to multiple contexts, within other courses, and more widely within information systems to extend this research. The framework might also be refined to make it more concise, integrated with assessment, or usable in more contexts.

In this paper we discuss the use of automated assessment in a variety of computer science courses that have been taught at Israel Academic College by the authors. The course assignments were assessed entirely automatically using Checkpoint, a web-based automated assessment framework. The assignments all used free-text questions (where the students type in their own answers). Stu-dents were allowed to correct errors based on feedback provided by the system and resubmit their answers. A total of 141 students were surveyed to assess their opinions of this approach, and we analysed their responses. Analysis of the questionnaire showed a low correlation between ques-tions, indicating the statistical independence of the individual questions. As a whole, student feedback on using Checkpoint was very positive, emphasizing the benefits of multiple attempts, impartial marking, and a quick turnaround time for submissions. Many students said that Check-point gave them confidence in learning and motivation to practise. Students also said that the de-tailed feedback that Checkpoint generated when their programs failed helped them understand their mistakes and how to correct them.

Aim/Purpose: The aims of this study were to investigate the feasibility of automatic assessment of programming tasks and to compare manual assessment with automatic assessment in terms of the effect of the different assessment methods on the marks of the students. Background: Manual assessment of programs written by students can be tedious. The assistance of automatic assessment methods might possibly assist in reducing the assessment burden, but there may be drawbacks diminishing the benefits of applying automatic assessment. The paper reports on the experience of a lecturer trying to introduce automated grading. Students’ solutions to a practical Java programming test were assessed both manually and automatically and the lecturer tied the experience to the unified theory of acceptance and use of technology (UTAUT). Methodology: The participants were 226 first-year students registered for a Java programming course. Of the tests the participants submitted, 214 were assessed both manually and automatically. Various statistical methods were used to compare the manual assessment of student’s solutions with the automatic assessment of the same solutions. A detailed investigation of reasons for differences was also carried out. A further data collection method was the lecturer’s reflection on the feasibility of automatic assessment of programming tasks based on the UTAUT. Contribution: This study enhances the knowledge regarding benefits and drawbacks of automatic assessment of students’ programming tasks. The research contributes to the UTAUT by applying it in a context where it has hardly been used. Furthermore, the study is a confirmation of previous work stating that automatic assessment may be less reliable for students with lower marks, but more trustworthy for the high achieving students. Findings: An automatic assessment tool verifying functional correctness might be feasible for assessment of programs written during practical lab sessions but could be less useful for practical tests and exams where functional, conceptual and structural correctness should be evaluated. In addition, the researchers found that automatic assessment seemed to be more suitable for assessing high achieving students. Recommendations for Practitioners: This paper makes it clear that lecturers should know what assessment goals they want to achieve. The appropriate method of assessment should be chosen wisely. In addition, practitioners should be aware of the drawbacks of automatic assessment before choosing it. Recommendation for Researchers: This work serves as an example of how researchers can apply the UTAUT theory when conducting qualitative research in different contexts. Impact on Society: The study would be of interest to lecturers considering automated assessment. The two assessments used in the study are typical of the way grading takes place in practice and may help lecturers understand what could happen if they switch from manual to automatic assessment. Future Research: Investigate the feasibility of automatic assessment of students’ programming tasks in a practical lab environment while accounting for structural, functional and conceptual assessment goals.

Aim/Purpose: To encourage students’ engagement in peer assessments and provide students with better-quality feedback, this paper describes a technique for author-reviewer matching in peer assessment systems – a Balanced Allocation algorithm. Background: Peer assessment concerns evaluating the work of colleagues and providing feedback on their work. This process is widely applied as a learning method to involve students in the progress of their learning. However, as students have different ability levels, the efficacy of the peer feedback differs from case to case. Thus, peer assessment may not provide satisfactory results for students. In order to mitigate this issue, this paper explains and evaluates an algorithm that matches the author to a set of reviewers. The technique matches authors and reviewers based on how difficult the authors perceived the assignment to be, and the algorithm then matches the selected author to a group of reviewers who may meet the author’s needs in regard to the selected assignment. Methodology: This study used the Multiple Criteria Decision-Making methodology (MCDM) to determine a set of reviewers from among the many available options. The weighted sum method was used because the data that have been collected in user profiles are expressed in the same unit. This study produced an experimental result, examining the algorithm with a real collected dataset and mock-up dataset. In total, there were 240 students in the real dataset, and it contained self-assessment scores, peer scores, and instructor scores for the same assignment. The mock-up dataset created 1000 records for self-assessment scores. The algorithm was evaluated using focus group discussions with 29 programming students and interviews with seven programming instructors. Contribution: This paper contributes to the field in the following two ways. First, an algorithm using a MCDM methodology was proposed to match authors and reviewers in order to facilitate the peer assessment process. In addition, the algorithm used self-assessment as an initial data source to match users, rather than randomly creating reviewer – author pairs. Findings: The findings show the accurate results of the algorithm in matching three reviewers for each author. Furthermore, the algorithm was evaluated based on students’ and instructors’ perspectives. The results are very promising, as they depict a high level of satisfaction for the Balanced Allocation algorithm. Recommendations for Practitioners: We recommend instructors to consider using the Balanced Allocation algorithm to match students in peer assessments, and consequently to benefit from personalizing peer assessment based on students' needs. Recommendation for Researchers: Several MCDM methods could be expanded upon, such as the analytic hierarchy process (AHP) if different attributes are collected, or the artificial neural network (ANN) if fuzzy data is available in the user profile. Each method is suitable for special cases depending on the data available for decision-making. Impact on Society: Suitable pairing in peer assessment would increase the credibility of the peer assessment process and encourage students’ engagement in peer assessments. Future Research: The Balanced Allocation algorithm could be applied using a single group, and a peer assessment with random matching with another group may also be conducted, followed by performing a t-test to determine the impact of matching on students’ performances in the peer assessment activity.

Aim/Purpose: This paper focuses on designing and implementing the rubric for objective JAVA programming assessments. An unsupervised learning approach was used to group learners based on their performance in the results obtained from the rubric, reflecting their learning ability. Background: Students' learning outcomes have been evaluated subjectively using a rubric for years. Subjective assessments are simple to construct yet inconsistent and biased to evaluate. Objective assessments are stable, reliable, and easy to conduct. However, they usually lack rubrics. Methodology: In this study, a Top-Down assessment approach is followed, i.e., a rubric focused on the learning outcome of the subject is designed, and the proficiency of learners is judged by their performance in conducting the task given. A JAVA rubric is proposed based on the learning outcomes like syntactical, logical, conceptual, and advanced JAVA skills. A JAVA objective quiz (with multiple correct options) is prepared based on the rubric criteria, comprising five questions per criterion. The examination was conducted for 209 students (100 from the MCA course and 109 from B.Tech. course). The suggested rubric was used to compute the results. K-means clustering was applied to the results to classify the students according to their learning preferences and abilities. Contribution: This work contributes to the field of rubric designing by creating an objective programming assessment and analyzing the learners’ performance using machine learning techniques. It also facilitates a reliable feedback approach offering various possibilities in student learning analytics. Findings: The designed rubric, partial scoring, and cluster analysis of the results help us to provide individual feedback and also, group the students based on their learning skills. Like on average, learners are good at remembering the syntax and concepts, mediocre in logical and critical thinking, and need more practice in code optimization and designing applications. Recommendations for Practitioners: The practical implications of this work include rubric designing for objective assessments and building an informative feedback process. Faculty can use this approach as an alternative assessment measure. They are the strong pillars of e-assessments and virtual learning platforms. Recommendation for Researchers: This research presents a novel approach to rubric-based objective assessments. Thus, it provides a fresh perspective to the researchers promising enough opportunities in the current era of digital education. Impact on Society: In order to accomplish the shared objective of reflective learning, the grading rubric and its accompanying analysis can be utilized by both instructors and students. As an instructional assessment tool, the rubric helps instructors to align their pedagogies with the students’ learning levels and assists students in updating their learning paths based on the informative topic-wise scores generated with the help of the rubric. Future Research: The designed rubric in this study can be extended to other programming languages and subjects. Further, an adaptable weighted rubric can be created to execute a flexible and reflective learning process. In addition, outcome-based learning can be achieved by measuring and analyzing student improvements after rubric evaluation.

Executive Summary This article discusses an exploratory study that examined whether a wiki-based project could fos-ter student-centered learning. Student teachers were divided into five groups to tackle a group project which involved creating digital learning materials on wiki that could teach information technology (IT) to secondary school students. As assessment is a part of learning, they were also required to develop an assessment rubric to assess the wiki pages created by themselves and their peers. It was observed that a variety of learning resources, such as videos, newspaper clippings, and cartoons, were included in the groups' wiki pages. The student teachers also came up with assessment rubrics for self and peer assessment of their wiki pages by adopting or adapting as-sessment rubrics available on the Internet. However, the assessment rubrics did not seem to pro-vide clear assessment guidelines as there were large differences among the groups in terms of assessing the wiki pages. In sum, it appears that wiki can facilitate student-centered activities as the statistics logs gathered from the wiki site indicated that the student teachers often revised their wiki project materials, especially after receiving feedback from their peers and course instructors.

An online Moodle Workshop was evaluated for peer assessment effectiveness. A quasiexperiment was designed using a Seminar in Professionalism course taught in face-to-face mode to undergraduate students across two campuses. The first goal was to determine if Moodle Workshop awarded a fair peer grader grade. The second objective was to estimate if students were consistent and reliable in performing their peer assessments. Statistical techniques were used to answer the research hypotheses. Although Workshop Moodle did not have a built-in measure for peer assessment validity, t-tests and reliability estimates were calculated to demonstrate that the grades were consistent with what faculty expected. Implications were asserted to improve teaching and recommendations were provided to enhance Moodle.

Aim/Purpose: The purpose of this study is to develop an efficient methodology that can assist the evaluators in assessing a variable number of individuals that are working in groups and guarantee that the assessment is dependent on the group members’ performance and contribution to the work developed. Background: Collaborative work has been gaining more popularity in academic settings. However, group assessment needs to be performed according to each individual’s performance. The problem rests on the need to distinguish each member of the group in order to provide fair and unbiased assessments. Methodology: Design Science Research methodology supported the design of a framework able to provide the evaluator with the means to distinguish individuals in a workgroup and deliver fair results. Hevner’s DSR guidelines were fulfilled in order to describe WebAVALIA. To evaluate the framework, a quantitative study was performed and the first results are presented. Contribution: This paper provides a methodological solution regarding a fair evaluation of collaborative work through a tool that allows its users to perform their own assessment and peer assessment. These are made accordingly to the user’s perspectives on the performance of each group member throughout the work development. Findings: The first analysis of the results indicates that the developed method provides fairness in the assessment of group members, delivering a distinction amongst individuals. Therefore, each group member obtains a mark that corresponds to their specific contribution to the workgroup. Recommendations for Practitioners: For those who intend to apply this workgroup assessment method, it is relevant to raise student awareness about the methodology that is going to be used. That is, all the functionalities and steps in WebAVALIA have to be thoroughly explained before beginning of the project. Then, the evaluators have to decide about the students’ intermediate voting, namely if the evaluator chooses or not to publish student results throughout the project’s development. If there is the decision to display these intermediate results, the evaluator must try to encourage collaboration among workgroup members, instead of competition. Recommendation for Researchers: This study explores the design and development of an e-assessment tool – WebAVALIA. In order to assess its feasibility, its use in other institutions or contexts is recommended. The gathering of user opinions is suggested as well. It would then be interesting to compare the findings of this study with the results from other experimentations Impact on Society: Sometimes, people develop a rejection of collaborative work because they feel exploited due to the biased evaluation results. However, the group members assessment distinction, according to each one’s performance, may give each individual a sense of fairness and reward, leading to an openness/willingness towards collaborative work. Future Research: As future work, there are plans to implement the method in other group assessment contexts – such as sports and business environments, other higher education institutions, technical training students – in other cultures and countries. From this myriad of contexts, satisfaction results would be compared. Other future plans are to further explore the mathematical formulations and the respective WebAVALIA supporting algorithms.