No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Supporting Bachelor Thesis Management in Computer Science: A Comparative Study of Large Language Models in Academic Advising
Abstract
Writing a successful computer science bachelor’s thesis requires extensive advisor support. This need for guidance is especially pronounced given the complexity and technical nature of computer science, where students struggle with scope definition and task planning for these first-time major projects. Large language models (LLMs) like GPT-4 and Claude might help by giving personalized advice. This study tested these two AIs with a sample thesis project to see how well they could plan the work and a 300-h schedule. Four people reviewed the AIs’ recommendations for accuracy, relevance, clarity, and practical utility. The results showed that Claude is better at making accurate schedules, while GPT-4 is good at conceptualizing larger ideas. However, this was a small test, so we cannot be certain this will always be true. The initial results look promising for using AI to help advise students, but more research is needed. Future work should focus on how to best combine human and AI advice in computer science bachelor thesis management.
ResearchGate has not been able to resolve any citations for this publication.
The release of the large language model-based chatbot ChatGPT 3.5 in November 2022 has brought considerable attention to the subject of artificial intelligence, not only to the public. From the perspective of higher education, ChatGPT challenges various learning and assessment formats as it significantly reduces the effectiveness of their learning and assessment functionalities. In particular, ChatGPT might be applied to formats that require learners to generate text, such as bachelor theses or student research papers. Accordingly, the research question arises to what extent writing of bachelor theses is still a valid learning and assessment format. Correspondingly, in this exploratory study, the first author was asked to write his bachelor’s thesis exploiting ChatGPT. For tracing the impact of ChatGPT methodically, an autoethnographic approach was used. First, all considerations on the potential use of ChatGPT were documented in logs, and second, all ChatGPT chats were logged. Both logs and chat histories were analyzed and are presented along with the recommendations for students regarding the use of ChatGPT suggested by a common framework. In conclusion, ChatGPT is beneficial for thesis writing during various activities, such as brainstorming, structuring, and text revision. However, there are limitations that arise, e.g., in referencing. Thus, ChatGPT requires continuous validation of the outcomes generated and thus fosters learning. Currently, ChatGPT is valued as a beneficial tool in thesis writing. However, writing a conclusive thesis still requires the learner’s meaningful engagement. Accordingly, writing a thesis is still a valid learning and assessment format. With further releases of ChatGPT, an increase in capabilities is to be expected, and the research question needs to be reevaluated from time to time.
Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (eg, question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs‐based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer‐reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state‐of‐the‐art models (eg, GPT‐3/4), embracing the initiative of open‐sourcing models/systems, and adopting a human‐centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models.
Practitioner notes
What is currently known about this topic Generating and analysing text‐based content are time‐consuming and laborious tasks.
Large language models are capable of efficiently analysing an unprecedented amount of textual content and completing complex natural language processing and generation tasks.
Large language models have been increasingly used to develop educational technologies that aim to automate the generation and analysis of textual content, such as automated question generation and essay scoring.
What this paper adds A comprehensive list of different educational tasks that could potentially benefit from LLMs‐based innovations through automation.
A structured assessment of the practicality and ethicality of existing LLMs‐based innovations from seven important aspects using established frameworks.
Three recommendations that could potentially support future studies to develop LLMs‐based innovations that are practical and ethical to implement in authentic educational contexts.
Implications for practice and/or policy Updating existing innovations with state‐of‐the‐art models may further reduce the amount of manual effort required for adapting existing models to different educational tasks.
The reporting standards of empirical research that aims to develop educational technologies using large language models need to be improved.
Adopting a human‐centred approach throughout the developmental process could contribute to resolving the practical and ethical challenges of large language models in education.
The recent progress in generative AI models, particularly large language models (LLMs), has brought about a transformation in the field of education. Conversational LLM services, such as Google's Bard and OpenAI's ChatGPT, offer students access to many abilities such as summarization and generation of text and code, and on-demand replies to questions on expert topics. In this paper, we observe ChatGPT to explore how LLM services impact learning and instruction in higher education. First, we mapped the capabilities of the system by reviewing the grey literature on ChatGPT and using the system ourselves for two months. Second, we selected a Bachelor level computer science curriculum from a Finnish university, and examined the impact of ChatGPT on the offered courses. As an outcome of this study, we highlight 13 implications for students' learning in higher education, and discuss the contemporary future of AI-assisted learning in universities and beyond.
Large language models represent a significant advancement in the field of AI. The underlying technology is key to further innovations and, despite critical views and even bans within communities and regions, large language models are here to stay. This commentary presents the potential benefits and challenges of educational applications of large language models, from student and teacher perspectives. We briefly discuss the current state of large language models and their applications. We then highlight how these models can be used to create educational content, improve student engagement and interaction, and personalize learning experiences. With regard to challenges, we argue that large language models in education require teachers and learners to develop sets of competencies and literacies necessary to both understand the technology as well as their limitations and unexpected brittleness of such systems. In addition, a clear strategy within educational systems and a clear pedagogical approach with a strong focus on critical thinking and strategies for fact checking are required to integrate and take full advantage of large language models in learning settings and teaching curricula. Other challenges such as the potential bias in the output, the need for continuous human oversight, and the potential for misuse are not unique to the application of AI in education. But we believe that, if handled sensibly, these challenges can offer insights and opportunities in education scenarios to acquaint students early on with potential societal biases, criticalities, and risks of AI applications. We conclude with recommendations for how to address these challenges and ensure that such models are used in a responsible and ethical manner in education.
Bachelor thesis supervision involves a supporting goal and an assessment goal, requiring more guidance versus more autonomy and freedom for students, respectively. This paper tests the hypotheses that the final grade of undergraduate bachelor thesis is positively related to a supervisor’s (a) guidance given to and (b) affiliation with the bachelor thesis student/project. To assess these hypotheses, undergraduate students at the FHNW School of Business have been surveyed for three years about their perception and satisfaction with bachelor thesis supervision. This data (n = 189) was combined with student grades before the thesis and their final thesis grades. Our results show supervision to have a measurable impact on the bachelor thesis outcome and most effective when focused on guidance related to goal definition and methodological support. In other areas supervision can, to a certain degree, follow a more laissez-faire supervision style.
Effective time management is associated with greater academic performance and lower levels of anxiety in students; however many students find it hard to find a balance between their studies and their day-to-day lives. This article examines the self reported time management behaviors of undergraduate engineering students using the Time Management Behavior Scale. Correlation analysis, regression analysis, and model reduction are used to attempt to determine which aspects of time management the students practiced, which time management behaviors were more strongly associated with higher grades within the program, and whether or not those students who self-identified with specific time management behaviors achieved better grades in the program. It was found that students’ perceived control of time was the factor that correlated significantly with cumulative grade point average. On average, it was found that time management behaviors were not significantly different across gender, age, entry qualification, and time already spent in the program.
Conducting research and writing a thesis about it is regarded as a distinctive pedagogy (learning through inquiry) within higher education which brings new challenges to all parties involved. To complete a thesis, students should select a problem, make a systematic plan, implement the plan and, finally, write a report of the process and findings. Students do have a supervisor to guide and support them, but it is the student who plays the key role in the whole research process. The present study is a qualitative, explorative case study to understand the challenges related to research projects within undergraduate management degree programmes in a Sri Lankan national University. Data have been collected using interviews and focus group discussions in six-degree programmes, with around 40 participants in total. The study focuses on identifying problematic areas and creating a general picture of why students' research projects are not progressing favourably. Six main challenges were identified: student motivation, student-supervisor relationships, skills and knowledge, students' workload, the structure of the research project course, and resources and ICT tools. These problematic areas are complex and multidimensional. Therefore, further studies are required to truly understand the complex interrelatedness of these areas.
This second edition guides the reader through successful planning and implementation of a thesis project and provides students in computer science and information systems with all the advice they need. The reader-friendly text offers a simple step-by-step guide to the key processes involved using an approach that has been tried and tested by the authors over a number of years. A new chapter on Information-seeking and use and a subsection entitled Improve your learning (and grade) have been added to strengthen the material on how to search for relevant literature and also how to validate it. In addition, this comprehensive and easy-to-follow text has been fine tuned and updated wherever appropriate. Features and topics: Developing your project proposal Developing the problem description Following objectives Presenting and analysing your data Drawing conclusions Presenting and defending your work Preparing the final version of your report As well as being an essential purchase for students in computer science and information systems, this clear and concise aid also contains lots of useful information for supervisors and examiners, including guidelines on how to review a thesis project. "...mandatory reading for all students embarking on their final year project. Easy to read, loaded with good advice and filled with to-the-point examples, it covers in a step-by-step fashion each and every aspect that readers might be looking for. Whether you have a bachelor, master or Ph.D thesis ahead of you, I strongly recommend you first study this excellent text." Prof. Dr. Klaus Dittrich, Department of Information Technology, University of Zurich, Switzerland " This book... presents a very clear exposition of basic research concepts that are often taken for granted and assumed to be part of common knowledge. It will be very useful reading for anyone undertaking research." Prof. Brian Fitzgerald, University of Limerick, Ireland.
This study is about what supervisors perceive as crucial aspects of quality in supervising students writing their bachelor theses. The questions on quality are related to a scientific perspective, a learning perspective, a societal perspective and a social perspective. The study demonstrates that the criteria on science varied considerably, despite the supervisors being in a single discipline area. There is also marked variation in views of the relation between science and practice and different supervisors expressed variations in their conception of competence. Furthermore, supervisors perceive their own role as senior researchers or senior consultants, and sometimes also in the supporting role of ‘deputy mother’. The conclusion is that the supervisors seem to have a marked lack of understanding of the differences between them in undertaking their role. A main reason for this seems to be the lack of a common theoretical frame of reference. Due to a lack of communication, possible problems related to different perceptions of understanding the task may remain hidden or at least be neglected and underestimated.
Unlocking the power of generative AI models and systems such as GPT-4 and ChatGPT for higher education: A guide for students and lecturers
- H Gimpel
- K Hall
- S Decker
- T Eymann
- L Lämmermann
- A Mädche
- M Röglinger
- C Ruiner
- M Schoch
- M Schoop
Auto-evaluation of anthropic 100k context window. LangChain Blog
- L Martin
Application of generative AI to improve the process of drafting and evaluating final year projects (TFGs)
- J Pereira
- M Azanza
- J M López
- X Garmendia