No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Leveraging on Generative AI to Teach Undergraduate Software Engineering: A Collaborative Prompt Catalogue
ResearchGate has not been able to resolve any citations for this publication.
Context: Recently, many illustrative examples have shown ChatGPT's impressive ability to perform programming tasks and answer general domain questions. Objective: We empirically evaluate how ChatGPT performs on requirements analysis tasks to derive insights into how generative large language model, represented by ChatGPT, influence the research and practice of natural language processing for requirements engineering. Method: We design an evaluation pipeline including two common requirements information retrieval tasks, four public datasets involving two typical requirements artifacts, querying ChatGPT with fixed task prompts, and quantitative and qualitative results analysis. Results: Quantitative results show that ChatGPT achieves comparable or better values in all datasets under a zero-shot setting. Qualitative analysis further illustrates ChatGPT's powerful natural language processing ability and limited requirements engineering domain knowledge. Conclusion: The evaluation results demonstrate ChatGPT' impressive ability to retrieve requirements information from different types artifacts involving multiple languages under a zero-shot setting. It is worthy for the research and industry communities to study generative large language model based requirements retrieval models and to develop corresponding tools.
ChatGPT has revolutionized many research and industrial fields. ChatGPT has shown great potential in software engineering to boost various traditional tasks such as program repair, code understanding, and code generation. However, whether automatic program repair (APR) applies to deep learning (DL) programs is still unknown. DL programs, whose decision logic is not explicitly encoded in the source code, have posed unique challenges to APR. While to repair DL programs, an APR approach needs to not only parse the source code syntactically but also needs to understand the code intention. With the best prior work, the performance of fault localization is still far less than satisfactory (only about 30\%). Therefore, in this paper, we explore ChatGPT's capability for DL program repair by asking three research questions. (1) Can ChatGPT debug DL programs effectively? (2) How can ChatGPT's repair performance be improved by prompting? (3) In which way can dialogue help facilitate the repair? On top of that, we categorize the common aspects useful for prompt design for DL program repair. Also, we propose various prompt templates to facilitate the performance and summarize the advantages and disadvantages of ChatGPT's abilities such as detecting bad code smell, code refactoring, and detecting API misuse/deprecation.
In this technical report, we evaluated the performance of the ChatGPT and GPT-3 models for the task of vulnerability detection in code. Our evaluation was conducted on our real-world dataset, using binary and multi-label classification tasks on CWE vulnerabilities. We decided to evaluate the model because it has shown good performance on other code-based tasks, such as solving programming challenges and understanding code at a high level. However, we found that the ChatGPT model performed no better than a dummy classifier for both binary and multi-label classification tasks for code vulnerability detection.
Chat GPT, powered by the advanced GPT-3 language model from OpenAI, has reached 1 million users only in 5 days. Facebook, Netflix, Instagram, and Twitter could reach this number in 300, 1200, 75, and 720 days. With 175 billion parameters, GPT-3 can generate writing that closely resembles human language. Chat GPT can engage in multiple ongoing conversations, understand and respond to natural language input, and offer customized and interactive assistance. This makes Chat GPT a promising tool for open education, as it can improve the independence and autonomy of autodidactic learners, while also being both practical and adaptable. By providing personalized support, direction, and feedback, Chat GPT has the potential to increase motivation and engagement among autodidactic learners.
This chapter presents design techniques for software engineering, in the form of prompt patterns, to solve common problems that arise when using large language models (LLMs) to automate common software engineering activities, such as ensuring code is decoupled from third-party libraries and creating API specifications from lists of requirements. This chapter provides two contributions to research on using LLMs for software engineering. First, it provides a catalog of patterns for software engineering that classifies patterns according to the types of problems they solve. Second, it explores several prompt patterns that have been applied to improve requirements elicitation, rapid prototyping, code quality, deployment, and testing.
ChatGPT-a blessing or a curse for undergraduate computer science students and instructors?
- I Joshi
GPTutor: a ChatGPT-powered programming tool for code explanation
- E Chen
- R Huang
- H S Chen
- Y H Tseng
- L Y Li
Chen, E., Huang, R., Chen, H.S., Tseng, Y.H., Li, L.Y.: GPTutor: a ChatGPT-powered programming tool for code explanation (2023). DOI 10.48550/arXiv.2304.07232
Difficulty of Software Engineering and Ways to Overcome Common Challenges
- S Murthy
Murthy, S.: Difficulty of Software Engineering and Ways to Overcome Common Challenges
(2023). URL https://tinyurl.com/wguedublog
A prompt pattern catalog to enhance prompt engineering with ChatGPT (2023) Google Scholar
- J White
How chat gpt can transform autodidactic experiences and open education. Department of Distance Education, Open Education Faculty
- M Firat
Firat, M.: How chat gpt can transform autodidactic experiences and open education. Department of Distance Education, Open Education Faculty, Anadolu Unive (2023)
Chatgpt-a blessing or a curse for undergraduate computer science students and instructors?
- I Joshi
- R Budhiraja
- H Dev
- J Kadia
- M O Ataullah
- S Mitra
- D Kumar
- H D Akolekar
Joshi, I., Budhiraja, R., Dev, H., Kadia, J., Ataullah, M.O., Mitra, S., Kumar, D., Akolekar, H.D.:
Chatgpt-a blessing or a curse for undergraduate computer science students and instructors?
arXiv preprint arXiv:2304.14993 (2023)
- J White
- S Hays
- Q Fu
- J Spencer-Smith
- D C Schmidt
White, J., Hays, S., Fu, Q., Spencer-Smith, J., Schmidt, D.C.: Chatgpt prompt patterns for
improving code quality, refactoring, requirements elicitation, and software design. arXiv
preprint arXiv:2303.07839 (2023)