No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Teaching Network Security Through Live Exercises.
Abstract
Live exercises represent a valuable tool to teach the practical aspects of security and the dynamics of network-based attack and defense techniques. However, these exercises are very difficult to organize and execute. For this reason, there are very few courses that offer live exercise as an integral part of the class work. This paper describes a series of live exercises that have been used in a graduate-level Computer Science course on network security. For each exercise, the setup, execution, and lessons learned are discussed The intended audience of this paper is represented by instructors - especially in colleges and universities - who want to start using this type of instructional tools but have no experience and are unsure of the possible pitfalls in their implementation.
... These approaches provide information on structuring red-teaming and general cyber-kill chains. A large number of survey articles have been identified that focus on existing approaches to red-teaming and investigate existing research into cybersecurity-related exercises and hands-on labs (Mirkovic & Peterson, 2014;Vigna, 2003). However, the exercises do not clearly describe the relationship of other exercises or platforms with blue and red teaming, nor the design principles that must be followed for both teams to be included in cybersecurity exercises. ...
... CTF challenges and virtual labs have recently become very popular for organizing hands-on cybersecurity exercises around the world (Taylor, Arias, Klopchic, Matarazzo, & Dube, 2017a;Davis, Leek, Zhivich, Gwinnup, & Leonard, 2014). Although presented mainly at hack conferences such as DEF CON (Taylor et al., 2017a) or Black Hat, CTF challenges are often used by educators and other educators (Schreuders et al., 2017b;Vykopal, Svabensky, & Chang, 2020;Mirkovic & Peterson, 2014;Vigna, 2003;K. S. ...
... Using CTF challenges, the skills of the participants are tested on various security topics such as cryptography, steganography, web or binary exploitation, and reverse engineering, among others. Previous work has shown that CTF challenges are mostly used for bug hunting, usually without including real-case scenarios and without having specific learning objectives (Vigna, 2003; L. C. Mirkovic & Peterson, n.d.;Werther, Zhivich, Leek, & Zeldovich, 2011). On the other hand, CTF challenges maintain the option of customization and might offer high levels of interactivity, thus enhancing the learning experience (Trickel et al., 2017;Schreuders et al., 2017a;Schreuders, Shaw, Muireadhaigh, & Staniforth, 2018). ...
This research aims to validate the learning theory of constructivism and identify the possible systematic approaches to design, deploy, and evaluate cybersecurity learning environments using Capture the Flag (CTF) challenges and Cyber Ranges to provide a significant learning impact. Capture the Flag (CTF) challenges and Cyber Ranges are among the most widely used approaches to provide technical exercises in cybersecurity, mostly as an assessment method to evaluate skills. Despite interest in CTF challenges and Cyber Ranges, few studies have considered systematic approaches to relate technical exercises to the cybersecurity curriculum. Furthermore, research on gamification or game-based learning in cybersecurity has not included these approaches until now. This research contributes to understanding of the design and methodological development to improve approaches, including but not limited to CTF challenges and Cyber Ranges. The research is methodologically based on Action Design Research (ADR) that is used to provide empirical data on the design and development of digital cybersecurity learning environments. Furthermore, the scope of research of this thesis includes a critical evaluation of cyber learning environments, taking into account the concepts of gamification and game-based learning within the scope of this research. This research provides the foundation for a new method to design, develop, and deploy cybersecurity learning environments and related cybersecurity scenarios, as a baseline to acquire technical skills in IT.
... Inserting games to raise awareness in Cybersecurity is a practice that has been gaining traction [11,12]. Regarding the pedagogical value, it is not in the competitions that most of the skills are acquired [8], but in preparation time through training, studies and exchange of experiences. ...
... • attack and defense competitions [8,11,14]; and • challenges (treasure hunt) [12]. ...
... For example, players/teams that first accomplish a task receive more points than those that finish later, following a decreasing scale [8], or everyone that solves a problem inside a given time limit is awarded the same points. Establishing fixed targets for all teams promotes competition in a healthier way than occurs in games in which teams attack each other [12]. In addition, challenges are flexible enough to enable individual practice as well. ...
Computer Security is an increasingly important area, considering the sophistication and increase of threats present in the digital world. The need for information protection contrasts with the lack of professionals and the limited space dedicated to the area in Information Technology (IT) courses. Games and competitions have been used to motivate Computing students to improve their practical knowledge on the subject and also to foster the interest of potential students and professionals in Security. The creation of these games requires specialized knowledge to develop new problems, since the novelty of these games is important to reach the desired level of difficulty and to ensure competitiveness. This work proposes the use of randomization to generate problems and entire competitions in an automated way, obtaining exclusive instances of problems for each player. A tool for generating challenges was developed as proof of concept to evaluate the proposal. Competitions with automatically generated problems were held with undergraduate and continuing education students, at two different institutions. The performance in the competitions and the perception of satisfaction, interest and learning of the involved students were analyzed. The results show that the automatic generation of challenges is feasible and the use of competitions to teach Computer Security is motivating and effective for didactic purposes.
... Gamification in education has already been advocated as a means to enrich learning experiences [15]. In particular, within IT security, the development of Capture-The-Flag like competitions have been argued to be advantageous for education and training [30]. Inspired by the gamified nature of CTF, we propose to address the issues of ICS security education and research with the SWaT Security Showdown (S3) competition. ...
... Several other similar CTF competitions are listed in [10]. In [30] Vigna proposes to use gamified live exercises to teach network security. The motivations and philosophy of this work are similar to ours. ...
... However the focus of the paper is on IT network security (e. g., gain root privileges on a webserver or steal data from a SQL database) and not on OT network security (industrial network devices and protocols). Inspired by [30], in [8] authors of the iCTF event presented two novel, live, and large-scale security competitions. The first is called "treasure hunt" and it exercises network mapping and multi-step network attacks. ...
Our work considers the challenges related to education and research about the security of industrial control systems (ICS). We propose to address those challenges through gamified security competitions. Those competitions should target a broad range of security professionals (e. g., from academia and industry). Furthermore, they should involve both attack and defense components. This could include the development of new attack techniques and evaluation of novel countermeasures. Our gamification idea resulted in the design and implementation of the SWaT Security Showdown (S3). S3 is a Capture-The-Flag event specifically targeted at Industrial Control Systems security. We developed ICS-specific challenges involving both theoretical and applied ICS security concepts. The participants had access to a real water treatment facility and they interacted with simulated components and ICS honeypots.
S3 includes international teams of attackers and defenders both from academia and industry. It was conducted in two phases. The online phase (a jeopardy-style capture the flag event) served as a training session and presented novel categories not found in traditional information security CTFs. The live phase (an attack-defense CTF) involved teams testing new attack and defense techniques on SWaT: our water treatment testbed. During the competition we acted as judges, and we assigned points to the attacker teams according to a scoring system that we developed internally. Our scoring system is based on multiple factors, including realistic ICS attacker models and effectiveness of the detection mechanisms of the defenders. For each phase of the S3 we present the results and relevant statistics derived from the data that we collected during the event.
... CTF-like gamified security competitions are expected to help the ICS security community in many ways [15,30,40]. A CTF is an hands-on learning experience and it can be used as an educational tool, research tool, and as an assessment tool. ...
... The gamification aspect of a CTF allows the participant to express his or her full potential, e. g., attack/defend without fear of consequences or bad marks. CTF events have already been proposed as a means to enhance security education and awareness [15,30,40]. Although such events cover a wide range of security domains, to the best of our knowledge they do not include so far the security of ICS. ...
... Gamification in education has been advocated as a means to en-rich the learning experience [22]. In particular, within IT security, the implementation of CTF-like competitions have been argued to be advantageous for education and training [40]. Inspired by the gamified nature of CTF, we propose the following approach. ...
In this work, we consider challenges relating to security for Industrial Control Systems (ICS) in the context of ICS security education and research targeted both to academia and industry. We propose to address those challenges through gamified attack training and countermeasure evaluation. We tested our proposed ICS security gamification idea in the context of the (to the best of our knowledge) first Capture-The-Flag (CTF) event targeted to ICS security called SWaT Security Showdown (S3). Six teams acted as attackers in a security competition leveraging an ICS testbed, with several academic defense systems attempting to detect the ongoing attacks. The event was conducted in two phases. The online phase (a jeopardy-style CTF) served as a training session. The live phase was structured as an attack-defense CTF. We acted as judges and we assigned points to the attacker teams according to a scoring system that we developed internally based on multiple factors, including realistic attacker models. We conclude the paper with an evaluation and discussion of the S3, including statistics derived from the data collected in each phase of S3.
... Using CTF challenges, the skills of contesters are tested in various security topics such as cryptography, steganography, Web or binary exploitation and reverse engineering among others. Previous work has shown concerns that CTF challenges are mostly used for bug hunting, usually without including real-case scenarios and without having specific learning objectives (Vigna, 2003;Eagle & Clark, 2004;Mirkovic & Peterson, 2014;Werther et al., 2011). On the other hand, CTF challenges maintain the option for customization and might offer high interactivity levels, thus enhancing the learning experience (Trickel et al., 2017;Schreuders et al., 2017;Schreuders et al., 2018). ...
... A number of previous works mentioned the importance of maintaining live exercises and of using CTF challenges as a necessary component of the computer security curriculum (Vigna, 2003;Antonioli et al., 2017). Works such as the above outline the high difficulty and the pitfalls in the implementation and deployment of such approaches. ...
Purpose
This paper aims to highlight the potential of using capture the flag (CTF) challenges, as part of an engaging cybersecurity learning experience for enhancing skills and knowledge acquirement of undergraduate students in academic programs.
Design/methodology/approach
The approach involves integrating interactivity, gamification, self-directed and collaborative learning attributes using a CTF hosting platform for cybersecurity education. The proposed methodology includes the deployment of a pre-engagement survey for selecting the appropriate CTF challenges in accordance with the skills and preferences of the participants. During the learning phase, storytelling elements were presented, while a behavior rubric was constructed to observe the participants’ behavior and responses during a five-week lab. Finally, a survey was created for getting feedback from the students and for extracting quantitative results based on the attention, relevance, confidence and satisfaction (ARCS) model of motivational design.
Findings
Students felt more confident about their skills and were highly engaged to the learning process. The outcomes in terms of technical skills and knowledge acquisition were shown to be positive.
Research limitations/implications
As the number of participants was small, the results and information retrieved from applying the ARCS model only have an indicative value; however, specific challenges to overcome are highlighted which are important for the future deployments.
Practical implications
Educators could use the proposed approach for deploying an engaging cybersecurity learning experience in an academic program, emphasizing on providing hands-on practice labs and featuring topics from real-world cybersecurity cases. Using the proposed approach, an educator could also monitor the progress of the participants and get qualitative and quantitative statistics regarding the learning impact for each exercise.
Social implications
Educators could demonstrate modern cybersecurity topics in the classroom, closing further the gap between theory and practice. As a result, students from academia will benefit from the proposed approach by acquiring technical skills, knowledge and experience through hands-on practice in real-world cases.
Originality/value
This paper intends to bridge the existing gap between theory and practice in the topics of cybersecurity by using CTF challenges for learning purposes and not only for testing the participants’ skills. This paper offers important knowledge for enhancing cybersecurity education programs and for educators to use CTF challenges for conducting cybersecurity exercises in academia, extracting meaningful statistics regarding the learning impact.
... These methods are central to better understanding the ways security systems may fail. Teaching students offensive skills, as opposed to defensive techniques, yields better security professionals [6][7][8][9][10]. Many academics and industry practitioners feel that the best way to prepare system defenses is to understand the attacks that the systems will face [11]. ...
... The threshold should be specified based on the firewall environement and the usage of the ICMP protocol. As an example, the threshold in this experiment has been set to 10. This means that the firewall will accept only 10 ICMP packets per second. ...
... A general description of different approaches to teach information security is given in [1]. Many papers focus on approaches to give the students hand-on experience [2] [2] describes the details of three live exercises that were used as a part of a graduate level course on network security to teach the practical aspects of network security. A description of course that utilizes an isolated lab infrastructure to teach security is described in [3]. ...
... A general description of different approaches to teach information security is given in [1]. Many papers focus on approaches to give the students hand-on experience [2] [2] describes the details of three live exercises that were used as a part of a graduate level course on network security to teach the practical aspects of network security. A description of course that utilizes an isolated lab infrastructure to teach security is described in [3]. ...
With the ever-increasing dependence on e-services, an understanding of network security issues is a vital component to secure such e-services. Due to the breadth of the topic, traditional network security courses tend to focus on few aspects, which may undermine the “big picture” view of the security scene and hence results in lower overall understanding of the topic by students. This paper presents a holistic approach to teaching network security that aims to provide the students with wide perspective of the network security issues. The approach features the use of many hand-on exercises to deepen the understanding of the topics covered and increase the students’ interest. Moreover, professional skills (e.g. communication skills and impact of technical solution on local and global levels) are also incorporated in the course. Tackling and developing these professional skills is a requirement for accreditation for some programs that undergo through accreditation process (e.g. ABET). The paper provides a blueprint of undergraduate course on network security that outlines topical coverage, described hand-on experiments to enhance students understanding of theory portion, provides suggestion of suitable educational software to carry these experiments, and discusses ways to integrate professional skills into the course delivery.
... Is this approach suitable in depth and breadth? [13] and [14]. Because computer security in general covers a wide range of technology, careful selection of topics and particular attention given towards presenting it to the students is vital to ensure that instructors are not lost in the details of each technology. ...
... Other than that, hardware and software resource & funding must be taken into account, whether it is obtained internally or externally [14]. Student to instructor ratio must be adhered to, to enable successful implementation of this approach. ...
This paper discusses a curriculum approach that will give emphasis on practical portions of teaching network security subjects in information and communication technology courses. As we are well aware, the need to use a practice and application oriented approach in education is paramount. Research on active learning and cooperative groups have shown that students grasps more and have more tendency towards obtaining and realizing soft skills like leadership, communication and team work as opposed to the more traditional theory and exam based teaching and learning. While this teaching and learning paradigm is relatively new in Malaysia, it has been practiced widely in the West. This paper examines a certain approach whereby students learning wireless security are divided into and work in small and manageable groups where there will be 2 teams which consist of black hat and white hat teams. The former will try to find and expose vulnerabilities in a wireless network while the latter will try their best to prevent such attacks on their wireless networks using hardware, software, design and enforcement of security policy and etc. This paper will try to show that the approach taken plus the use of relevant and up to date software and hardware and with suitable environment setting will hopefully expose students to a more fruitful outcome in terms of understanding of concepts, theories and their motivation to learn.
... G. Vigna and his team Shellphish, one of the long-lived pioneers in the CTF scene, have been actively involved in organizing and participating in CTFs since the early 2000s. They have made significant contributions to the field, not only by designing new competition formats, but also by incorporating the philosophy of these competitions in the academia by developing a framework to help in organizing attack and defense style CTFs that educational institutions and other organizations can benefit from [6,14]. C. Nelson and Y. Shoshitaishvili developed a platform based on CTFd [1] for their computer systems security class at Arizona State University [4], it's a completely free and open source project that can be adapted to other academic institutions as well. ...
CTF (Capture the Flag) competitions are increasingly being used in information technology and software engineering education to teach students about secure coding best practices. This paper presents an approach to improve engagement of software engineering students during software security laboratory assessments by using CTF-style challenges that are fun to solve and reflect modern security threats. The approach involves gamifying laboratory exercises by providing CTF-like challenges that are not only enjoyable but also reflect real-world security vulnerabilities. Additionally, students are given a single virtual machine bundled with everything needed for the entire semester, eliminating unnecessary logistic problems. Feedback from students confirms the effectiveness of this approach in reaching course objectives. Students were keen to learn more about the topic and showed greater interest in the laboratories. This approach provides a more dynamic and interactive learning experience that fosters critical thinking skills and encourages students to pursue careers in software security.
... The knowledge-sharing aspect of Red and Blue team competitions is equally valuable. Professionals have the opportunity to learn from their peers, gain insights into different perspectives and approaches, and discover innovative solutions to cybersecurity challenges (Vigna, 2003). This collaborative environment encourages participants to continuously refine their skills, stay updated with the latest industry trends, and remain at the forefront of cybersecurity practices. ...
Cybersecurity threats are evolving rapidly, necessitating effective strategies to combat them. Red and Blue team training is a valuable approach to address this challenge. It simulates real-world attack scenarios, with the Red team acting as attackers and the Blue team as defenders. This training helps organizations identify vulnerabilities and trains employees to respond effectively to security incidents. Introducing competition further enhances this training by motivating participants to excel and stay updated with evolving threats. This paper proposes a combined Red and Blue team approach to improve communication and understanding between teams. The findings indicate that this approach enhances capabilities in reacting to real attacks. By fostering better team understanding, participants effectively identify and mitigate vulnerabilities. These results highlight the potential value of a combined Red and Blue team approach for enhancing cybersecurity readiness. Further research is needed to fully explore its benefits and limitations.
... In addition, efficient communication and collaboration between different teams and stakeholders are necessary to mitigate the impact of cyber attacks. In this context, the development of training and simulation exercises can provide a valuable means of preparing individuals and organizations to respond optimally to cyber attacks [3]. Such exercises can enhance technical skills, promote teamwork and collaboration, and facilitate a better understanding of the threat landscape. ...
... Previous research in the field of cybersecurity exercises [8]- [10] has highlighted the importance of cyber drills [11] in aiding teams in designing, implementing, managing, and defending a computer network [3], [12]- [17]. Patriciu and Furtuna [18], [19] propose various processes and criteria for developing a cybersecurity exercise [20]. ...
... Previous work in the CSE domain [40] has highlighted the use of cyber defence competitions or live-attack exercises as a very effective way of teaching information security [10,19], helping teams design, implement, manage and defend a network of computers [1,6,7,30,31]. Vigna [46] and Mink [27] further support these findings. ...
Content generation that is both relevant and up to date with the current threats of the target audience is a critical element in the success of any cyber security exercise (CSE). Through this work, we explore the results of applying machine learning techniques to unstructured information sources to generate structured CSE content. The corpus of our work is a large dataset of publicly available cyber security articles that have been used to predict future threats and to form the skeleton for new exercise scenarios. Machine learning techniques, like named entity recognition and topic extraction, have been utilised to structure the information based on a novel ontology we developed, named Cyber Exercise Scenario Ontology (CESO). Moreover, we used clustering with outliers to classify the generated extracted data into objects of our ontology. Graph comparison methodologies were used to match generated scenario fragments to known threat actors’ tactics and help enrich the proposed scenario accordingly with the help of synthetic text generators. CESO has also been chosen as the prominent way to express both fragments and the final proposed scenario content by our AI-assisted Cyber Exercise Framework. Our methodology was assessed by providing a set of generated scenarios for evaluation to a group of experts to be used as part of a real-world awareness tabletop exercise.
... Previous work in the CSE domain [39] has highlighted the use of cyber defence competitions or liveattack exercises as a very effective way of teaching information security [9,18], helping teams design, implement, manage and defend a network of computers [1,5,6,29,30]. Vigna [45] and Mink [26] further support these findings. ...
Content generation that is both relevant and up to date with the current threats of the target audience is a critical element in the success of any Cyber Security Exercise (CSE). Through this work, we explore the results of applying machine learning techniques to unstructured information sources to generate structured CSE content. The corpus of our work is a large dataset of publicly available cyber security articles that have been used to predict future threats and to form the skeleton for new exercise scenarios. Machine learning techniques, like named entity recognition (NER) and topic extraction, have been utilised to structure the information based on a novel ontology we developed, named Cyber Exercise Scenario Ontology (CESO). Moreover, we used clustering with outliers to classify the generated extracted data into objects of our ontology. Graph comparison methodologies were used to match generated scenario fragments to known threat actors' tactics and help enrich the proposed scenario accordingly with the help of synthetic text generators. CESO has also been chosen as the prominent way to express both fragments and the final proposed scenario content by our AI-assisted Cyber Exercise Framework (AiCEF). Our methodology was put to test by providing a set of generated scenarios for evaluation to a group of experts to be used as part of a real-world awareness tabletop exercise.
... The goal of this competition is to hack the network with a series of attacks without being detected. The Treasure Hunt [180] dataset is the result of a payroll simulator hack competition that aims to execute unauthorized money transfer transactions. Such a set contains a sequence of attacks that are part of an overall plan to achieve a specific goal. ...
Security event correlation approaches are necessary to detect and predict incremental threats such as multi-step or targeted attacks (advanced persistent threats) and other causal sequences of abnormal events. The use of security event correlation techniques also makes it possible to reduce the volume of the original data stream by grouping the events and eliminating their redundancy. The variety of event correlation methods, in turn, requires choosing the most appropriate way to handle security events, depending on the purpose and available resources. This paper presents a systematization of security event correlation methods into several categories, such as publication year, applied correlation methods, knowledge extraction methods, used data sources, architectural solutions, and quality evaluation of correlation methods. The research method is a systematic literature review, which includes the formulation of research questions, the choice of keywords and criteria for inclusion and exclusion. The review corpus is formed by using search queries in Google Scholar, IEEE Xplore, ACM Digital Library, ScienceDirect, and selection criteria. The final review corpus includes 127 publications from the existing literature for 2010–2021 and reflects the current state of research in the security event correlation field. The results of the analysis include the main directions of research in the field of event correlation and methods used for correlation both single events and their sequences in attack scenarios. The review also describes the datasets and metrics used to evaluate security event correlation approaches. In conclusion, the existing problems and possible ways to overcome them are identified. The main contribution of the review is the most complete classification and comparison of existing approaches to the security event correlation, considered not only from the point of view of the algorithm, but also the possibility of unknown attack detection, architectural solutions and the use of event initial data.
... The ability to understand how a cyber defender can detect and mitigate their attack is critical to their training. Interactive exercises which incorporate this type of feedback have been shown to be extremely important in the confirmation of skills in the cyber domain [2], [3], [6]. ...
... It provides an excellent opportunity and ultimate learning experience for the students to improve their skills in protecting and defending information systems are assessed in the context of realistic, true-tolife scenario. On the other side, as discussed by Vigna [1] and Mink [2] , the offensive security training is also an effective way to learn information security. The previous works in this area examined the structure and how to use of cyber defense competitions, overall effectiveness of live-attack exercises in teaching information security, curriculum and course format at CDXs in which teams design, implement, manage and defend a network of computers. ...
Cyber defense exercises (CDXs) are excellent testbed platforms to test and assess IT and OT systems. They (CDXs) are also very important tools when it comes to enhancing the safety awareness of cyberspace, testing an organization"s ability to put up resistance and respond to different cyber events to establish the secure environment, gathering empirical data related to security, and looking at the practical training of experts on this subject. The exercises can give ideas to the decision makers about the precautions in the cybersecurity area and to the officials, institutions, organizations, and staff who are responsible on the cyber tools, techniques, and procedures that can be developed for this field. In the cyber defense exercises, the scenarios that are simulated closest to reality which provides very important contributions by bringing together the necessity of making the best decisions and management capabilities under the cyber crisis by handling stress and coordinated movement as a team. The objective of this paper is to address the issue from a scientific point of view by taking CDXs as a testbed and lesson learned platforms to be able to create better and safer cyber environment.
... It provides an excellent opportunity and ultimate learning experience [4,5] for the students to improve their skills in protecting and defending information systems are assessed in the context of realistic, true-to-life scenario [6]. On the other side, as discussed by Vigna [7] and Mink [8], the offensive security training is also an effective way to learn information security. The previous works in this area examined the structure [9] and how to use of cyber defence competitions, overall effectiveness of live-attack exercises in teaching information security [10], curriculum and course format at CDX in which teams design, implement, manage and defend a network of computers [11][12][13][14][15]. ...
This paper discusses the concept of cyber defence exercises -CDX- that are very important tool when it comes to enhancing the safety awareness of cyberspace, testing an organization's ability to put up resistance and respond to different cyber events to establish the secure environment, gathering empirical data related to security, and looking at the practical training of experts on this subject. The exercises can give ideas to the decision makers about the precautions in the cybersecurity area and to the officials, institutions, organizations, and staff who are responsible on the cyber tools, techniques, and procedures that can be developed for this field. In the cyber defense exercises, the scenarios that are simulated closest to reality which provides very important contributions by bringing together the necessity of making the best decisions and management capabilities under the cyber crisis by handling stress and coordinated movement as a team. The objective of this paper is to address the issue from a scientific point of view by setting out the stages of planning, implementation, and evaluation of these exercises, taking into account and comparing international firefighting exercises. Another aim of the work is to be able to reveal the necessary processes that are required for all kind of cyber exercises, regardless of the type, although the processes involved vary according to the target mass of the planned exercise.
... It provides an excellent opportunity and ultimate learning experience [4,5] for the students to improve their skills in protecting and defending information systems are assessed in the context of realistic, true-to-life scenario [6]. On the other side, as discussed by Vigna [7] and Mink [8], the offensive security training is also an effective way to learn information security. The previous works in this area examined the structure [9] and how to use of cyber defence competitions, overall effectiveness of live-attack exercises in teaching information security [10], curriculum and course format at CDX in which teams design, implement, manage and defend a network of computers [11][12][13][14][15]. ...
This paper discusses the concept of cyber defence exercises (CDX) that are very important tool when it comes to enhancing the safety awareness of cyberspace, testing an organization's ability to put up resistance and respond to different cyber events to establish the secure environment, gathering empirical data related to security, and looking at the practical training of experts on this subject. The exercises can give ideas to the decision makers about the precautions in the cybersecurity area and to the officials, institutions, organizations, and staff who are responsible on the cyber tools, techniques, and procedures that can be developed for this field. In the cyber defense exercises, the scenarios that are simulated closest to reality which provides very important contributions by bringing together the necessity of making the best decisions and management capabilities under the cyber crisis by handling stress and coordinated movement as a team. The objective of this paper is to address the issue from a scientific point of view by setting out the stages of planning, implementation, and evaluation of these exercises, taking into account and comparing international firefighting exercises. Another aim of the work is to be able to reveal the necessary processes that are required for all kind of cyber exercises, regardless of the type, although the processes involved vary according to the target mass of the planned exercise.
... The number of such events is gradually increasing [13], [16]. Such events aid in learning about security vulnerabilities, how these could be exploited, nature of attacks, and strength of the deployed [18], [33], [45] defense mechanisms. To the best of our knowledge, S 3 is the first CTF style event of its kind in ICS that involves participants from the industry and academia, and focuses on an operational water treatment testbed. ...
A hackfest named SWaT Security Showdown (S3) has been organized consecutively for two years. S3 has enabled researchers and practitioners to assess the effectiveness of methods and products aimed at detecting cyber attacks launched in real-time on an operational water treatment plant, namely, Secure Water Treatment (SWaT). In S3 independent attack teams design and launch attacks on SWaT while defence teams protect the plant passively and raise alarms upon attack detection. Attack teams are scored according to how successful they are in performing attacks based on specific intents while the defense teams are scored based on the effectiveness of their methods to detect the attacks. This paper focuses on the first two instances of S3 and summarizes the benefits of hackfest and the performance of an attack detection mechanism, named Water Defense, that was exposed to attackers during S3.
... The number of such events is gradually increasing [13], [16]. Such events aid in learning about security vulnerabilities, how these could be exploited, nature of attacks, and strength of the deployed [18], [33], [45] defense mechanisms. To the best of our knowledge, S 3 is the first CTF style event of its kind in ICS that involves participants from the industry and academia, and focuses on an operational water treatment testbed. ...
A hackfest named SWaT Security Showdown (S 3) has been organized consecutively for two years. S 3 has enabled researchers and practitioners to assess the effectiveness of methods and products aimed at detecting cyber attacks launched in real-time on an operational water treatment plant, namely, Secure Water Treatment (SWaT). In S 3 independent attack teams design and launch attacks on SWaT while defence teams protect the plant passively and raise alarms upon attack detection. Attack teams are scored according to how successful they are in performing attacks based on specific intents while the defense teams are scored based on the effectiveness of their methods to detect the attacks. This paper focuses on the first two instances of S 3 and summarizes the benefits of hackfest and the performance of an attack detection mechanism, named Water Defense, that was exposed to attackers during S 3 .
... Generally, CTF game is divided into three-frequentlyused-scenarios: Jeopardy, Attack-Defense and Mixed [6]. In Jeopardy scenario, the players are asked to solve several tasks -to gain and collect points. ...
... We have continued to host the iCTF every year since then (the most recent edition was in March of 2017). Each year, we experiment with various designs and approaches to the game [12,15,31,34,36]. ...
Although we are facing a shortage of cybersecurity professionals , the shortage can be reduced by using technology to empower all security educators to efficiently and effectively educate the professionals of tomorrow. One powerful tool in some educators' toolboxes are Capture the Flag (CTF) competitions. Although participants in all the different types of CTF competitions learn and grow their security skills, Attack/Defense CTF competitions offer a more engaging and interactive environment where participants learn both offensive and defensive skills, and, as a result, they develop their skills even faster. However, the substantial time and skills required to host a CTF, especially an Attack/Defense CTF, is a huge barrier for anyone wanting to organize one. Therefore, we created an on-demand Attack/Defense tool via an easy-to-use website that makes the creation of an Attack/Defense CTF as simple as clicking a few buttons. In this paper, we describe the design and implementation of our system, along with lessons learned from using the system to host a 24-hour 317 team Attack/Defense CTF.
... Players have to deal with real security problems and come acquainted with the terms used in this field. Giovanni Vigna [3] has also presented an approach using a Treasure Hunt game, where two teams have to complete a number of tasks in order to break into a simulated payroll system and perform a money transfer transaction. By playing this Treasure Hunt game, the player approaches the network security from the perspective of the attacker. ...
... To raise defender vigilance against deceptive threats, a different way of thinking is required-one that adopts the thinking process of the adversary [17,19,23]. Modern defenders must understand the psychology of attackers, and be aware of their strategies and techniques in order to anticipate their actions. ...
Modern cyber security educational programs that emphasize technical skills often omit or struggle to effectively teach the increasingly important science of cyber deception. A strategy for effectively communicating deceptive technical skills by leveraging the new paradigm of honey-patching is discussed and evaluated. Honey-patches mislead attackers into believing that failed attacks against software systems were successful. This facilitates a new form of penetration testing and capture-the-flag style exercise in which students must uncover and outwit the deception in order to successfully bypass the defense. Experiences creating and running the first educational lab to employ this new technique are discussed, and educational outcomes are examined.
... We started in 2003 and have since designed, implemented, and run 11 security competitions (see Table 1), and, to this day, the iCTF has been consistently the world's largest interactive CTF focused on computer security education. In more than ten years of competitions, we have experimented with different designs, scoring systems, combinations of challengebased and interactive competitions, and ways to collect interesting datasets that might support research into security education in particular, and system security in general [5,7,3,2,9,8]. ...
Security competitions have become a popular way to foster security education by creating a competitive environment in which participants go beyond the effort usually required in traditional security courses. Live security competitions (also called " Capture The Flag, " or CTF competitions) are particularly well-suited to support hands-on experience, as they usually have both an attack and a defense component. Unfortunately, because these competitions put several (possibly many) teams against one another, they are difficult to design, implement, and run. This paper presents a framework that is based on the lessons learned in running, for more than 10 years, the largest educational CTF in the world, called iCTF. The framework's goal is to provide educational institutions and other organizations with the ability to run customiz-able CTF competitions. The framework is open and leverages the security community for the creation of a corpus of educational security challenges.
... Treasure Hunt [14] is an event organized as part of the graduate-level security course at the University of California at Santa Barbara. The class was divided into two teams: Alpha and Omega and the goal was to compete against each other in breaking into a payroll system and performing a money transaction. ...
A significant challenge in applying IDS alert correlation in today's dynamic threat environment is the labor and expertise needed in constructing the correlation model, or the knowledge base, for the correlation process. New IDS signatures capturing emerging threats are generated on a daily basis, and the attack scenarios each captured activity may be involved in are also multitude. Thus it becomes hard to build and maintain IDS alert correlation models based on a set of known scenarios. Learning IDS correlation models face the same challenge caused by the dy-namism of cyber threats, compounded by the inherent difficulty in applying learning algorithms in an adversarial environment. We propose a new method for conducting alert correlation based on a simple and direct semantic model for IDS alerts. The correlation model is separate from the semantic model and can be constructed on various granularities. The semantic model only maps an alert to its potential meanings, without any reference to what types of attack scenarios the activity may be involved in. We show that such a correlation model can effectively capture attack scenarios from data sets that are not used at all in the model construction process, illustrating the power of such correlation methods in detecting novel, new attack scenarios. We rigorously evaluate our prototype on a number of publicly available data sets and a production system, and the result shows that our correlation engine can correctly capture almost all the attack scenarios in the data sets.
... In Vigna,15 using a distributed client/server architecture, the simulation of cyber attacks is performed with the help of the red and blue teams (i.e., real attackers and defenders are involved in the simulation process). A similar work is Brown et al., 16 where HLA is used to create a human-inthe-loop simulation. ...
The aim of this work is to propose a framework for the distributed simulation of cyber attacks based on high-level architecture (HLA), which is a commonly used standard for distributed simulations. The proposed framework and the corresponding simulator, which is called the distributed cyber attack simulator (abbreviated by DCAS), help administrators to model and evaluate the security measures of the networks. At the core of the DCAS is a simulation engine based on Portico, which is an open source HLA run-time infrastructure. The DCAS works in two modes: interactive and automated. Three types of simulation components (which are called federates in HLA terminology) are considered in the framework: the (1) network federate, (2) attacker federate and (3) defender federate. The simulator provides features for graphical design of the network models, animated traffic simulation, data collection, statistical analysis and different consoles for attacking and defending elements (e.g., intrusion detection systems, intrusion prevention systems). To increase the fidelity of the simulation outputs, real-world payloads are used by the DCAS. All the exploits information and the parameters of various network elements are automatically extracted from the open source vulnerability database. Also, the Snort rule-set is used as the signature database of the defending elements. The architecture and algorithms of the DCAS and the corresponding underlying simulation engine plus the security evaluation results of two illustrative examples are presented in this paper.
... However, others claim that teaching offensive techniques yields better security professionals than those that are taught only defensive techniques Freiling 2006, Arce andMcGraw 2004, Arnett and Schmidt 2005, Dornseif, Holz, and Mink 2005, Vigna 2003, Yuan, Matthews, Wright, Xu, and Yu 2010, Livermore 2007. It is important to note that the corporate businesses employ experts that use offensive techniques for penetration testing, to ensure their security. ...
Teaching offensive techniques is a necessary component of a computer security education and yields better security professionals than teaching defensive techniques alone. In this paper, we describe a case study of the implementation of comprehensive hands-on lab exercises that are essential to security education. The first hands-on lab exercise is about how to perform a Denial of Service (DoS) attack based on the poisoning of the CAM tables (Content Access Memory) of Local Area Network (LAN) switches. The second exercise is about how to prevent CAM table poisoning attack. The hands-on labs confirmed further the ethical and legal concerns regarding the teaching of offensive techniques in the academic environment. In fact, the number of injected malicious traffic targeting the university switches' CAM tables, increased considerably each time the students experiment the DoS attack. That is why every course in IT security should be accompanied by a basic discussion of legal implications and ethics.
... Alternatively, a dedicated lab/network may be constructed that will be cut-off from the rest of the network (Hill et al. 2001, Mateti 2003, Tikekar 2003, Vigna 2003a, Vigna 2003b, Wagner & Wudi 2004. Under such circumstances the restoration of its normal operation is both easier and quicker. ...
Purpose
– Teaching information systems security features some peculiarities, compared to other scientific fields, as the trainees have to design and protect systems against both known and unknown attacks. Moreover, the so far established stereotypes present the potential intruders as being ingenious and able to penetrate almost every system. The paper aims to discuss these issues.
Design/methodology/approach
– Within the scope of two different modules in higher education institutes, the students' involvement into practical pre-designed scenarios was attempted, in order for them to understand the way intruders think, the methodologies they follow and the liabilities one may face for the flawed security of network applications and/or the supporting infrastructure. For this reason, an educational software tool was developed (named “Hackademic Challenges”), which comprised a variety of realistic scenarios, where the student had to locate and exploit various vulnerabilities, in order to successfully complete the challenge. Evaluation of the developed tool was attempted through an online, anonymous questionnaire.
Findings
– The results show that the students embraced this approach and have benefited significantly from going through these exercises.
Originality/value
– The contribution consists of findings that may be useful to other instructors teaching similar subjects.
The Russian-Ukrainian conflict has led to a significant increase in cyber attacks on critical infrastructure in Ukraine, with the energy sector being a primary target. The goal of these cyber attacks is to support military operations on the battlefield. Enhancing the resilience of the energy sector is a primary and urgent assignment for the security and defence sector of Ukraine.
The objectives of this study are to identify the critical energy infrastructure’s cyber resilience factors, and their possible dependencies, and analyze the causes of their occurrence.
Accordingly, an analysis of the problems of the resilience of the critical energy infrastructure in Ukraine has been carried out. Based on this analysis, we’ve identified and studied some dependencies between power energy infrastructure cybersecurity and other sectors, so-called cascade effects. By analyzing cause-and-effect relationships in power outages, the prerequisites for the emergence of negative factors affecting the resilience of critical infrastructure in the conditions of war have been determined.
Using the obtained information about cascade effects, procedures have been proposed to enhance resilience. These include implementing processes for collecting and processing big data on cyber statistics, optimizing public-private cooperation, and organizing cyber training.
The goal of these processes is to increase the level of cybersecurity of critical infrastructure. These processes are aimed at increasing the effectiveness of responding to cybersecurity crises in conditions of limited time and material resources.
The experience of Ukraine in conducting such research is unique. This can become the basis for the development of models and architectures for the resilience of electric power systems in other countries.
Designing engaging exercises when students do not yet possess a lot of knowledge can be difficult. We show how we draw on students’ prior knowledge, along with basic introductory concepts, to design an elemental (but fun) port scan exercise in an introductory security testing module. While “capture the flag” is a security industry standard for exercises, it can require a lot of in-depth knowledge to properly implement and complete. Using basic computer science concepts such as ports and ASCII values, we design a simplified capture the flag exercise where students can make use of deductive reasoning to complete the game. Overall, the exercise was received favourably by the students who found it challenging but enriching.
Designing a cybersecurity course for a big cohort of students from the different educational background is a challenging job. Examined in this study are the perceptions, preferences and performance of students who have participated in a strategic blended learning initiative aimed at preparing students for their working lives. For this purpose, both self-reported and observational data were collected from 115 students who voluntarily registered for the pilot run of the course. Self-reported data was used to measure students’ preferences as well as perceptions related to satisfaction, engagement, convenience, interaction and views on learning. Observational data measuring students’ performance was directly extracted from the collaborative learning platform on which the course was hosted. The results show that overall students liked the blended design of the course. They were satisfied with the format of the course, they felt engaged, and most of them secured good grades. Moreover, no significant difference in perceptions and preferences were found when controlled for gender, educational discipline, and overall performance, showing that the blended design of the course was accepted across the board.
Many people know how to compromise existing systems, and capture-the-flag contests are increasing this number. There is a dearth of people who know how to design and build secure systems. A collaborative contest to build secure systems to meet specific goals—a “make-the-flag” exercise—could encourage more people to participate in cybersecurity exercises, and learn how to design and build secure systems. This paper presents a generic design for such an exercise. It explores the goals, organization, constraints, and rules. It also discusses preparations and how to run the exercise and evaluate the results. Several variations are also presented.
Recently, smartphones have been growing increasingly in popularity within the student community. Hence, novel educational activities and tools, as well as learning approaches can be developed to get benefit from the prevalence of smartphones (e.g. mobility and closeness to students' daily lives). This paper discusses an Android mobile app, called Packets Generator, that aims at taking advantages of the benefits of smartphones and the best practices in learning information security, as well as promoting students' interests and increasing their self-efficacy. Packets Generator app allows students to further enhance their hands-on skills on network traffic and Denial of Service (DoS) attacks generation, using their smartphones, by practicing inside as well as outside the traditional desktop based laboratories, in the real-world environment; i.e., anywhere and anytime, at the students' convenience. Packets Generator app is freely available at Google Play Store. Based on statistics from Google Play Store, in about two years, the app turned popular with more than 50,000 downloads worldwide and around 3.73/5.0 users' rating. The impact of the app on the students' performance in terms of achieving the course outcomes is discussed.
This paper describes an exercise that combines the business case for penetration testing with the application of the testing and subsequent management reporting. The exercise was designed for students enrolled in information systems and computer science courses to present a more holistic understanding of network and system security within an organization. This paper explains the objectives and structure of the exercise and its planned execution by two groups of students, the first group being information systems students in Australia and the second group comprising students enrolled in a computer security course in the United States.
As mobile devices grow increasingly in popularity within the student community, novel educational activities and tools, as well as learning approaches can be developed to get benefit from this prevalence of mobile devices (e.g. mobility and closeness to students’ daily lives). Particularly, information security education should reflect the current trend in computing platforms away from the desktop and towards mobile devices. This paper discusses a case study of a learning approach that aims at taking advantages of the benefits of mobile devices and the best practices in learning information security, as well as promoting students’ interests and increasing their self-efficacy. The learning approach uses two Android learning apps to enhance students’ hands-on skills on firewall filtering rules implementation, by practicing network traffic filtering outside the traditional laboratory activities, in the real-world environment; i.e., anywhere and anytime, at the students’ convenience. Practically, the two Android apps are a firewall app and a packet generator app; both apps are freely available at Google Play Store. Based on statistics from the Google Play Store, in about one and a half years, the packet generator app turned popular with over 20,000 downloads worldwide and a 3.75 users’ rating. A comparative analysis of various existing Android firewall apps with the proposed firewall app emphasizes its significance. The impact of the Android apps on the students’ performance in terms of achieving the course outcomes is also discussed.
This paper reports on the experience of using the EDURange framework, a cloud-based resource for hosting on-demand interactive cybersecurity scenarios. Our framework is designed especially for the needs of teaching faculty. The scenarios we have implemented each are designed specifically to nurture the development of analysis skills in students as a complement to both theoretical security concepts and specific software tools.
Our infrastructure has two features that make it unique compared to other cybersecurity educational frameworks. First, EDURange is scalable because it is hosted on a commercial, large-scale cloud environment. Second, EDURange supplies instructors with the ability to dynamically change the parameters and characteristics of exercises so they can be replayed and adapted to multiple classes. Our framework has been used successfully in classes and workshops for students and faculty. We present our experiences building the system, testing it, and using feedback from surveys to improve the system and boost user interest.
Denial of Service (DoS) attacks are important topics for security courses that teach ethical hacking techniques and intrusion detection. This paper presents a case study of the implementation of comprehensive offensive hands-on lab exercises about three common DoS attacks. The exercises teach students how to perform practically the DoS attacks in an isolated network laboratory environment. The paper discuses also some ethical and legal issues related to teaching ethical hacking, and then lists steps that schools and educators should take to improve the chances of having a successful and problem free information security programs.
This paper discusses a curriculum approach that will give emphasis on practical sessions of teaching network security subjects in information and communication technology courses. As we are well aware, the need to use a practice and application oriented approach in education is paramount [1]. Research on active learning and cooperative groups showed that students grasps and have more tendency towards obtaining and realizing soft skills like leadership, communication and team work as opposed to learning using the traditional theory and exam based method. While this teaching and learning paradigm is relatively new in Malaysia, it has been practiced widely in the West. This paper examines a particular approach whereby students learning wireless security are divided into small manageable groups consisting of black hat and white hat team. The former will try to find and expose vulnerabilities in a wireless network while the latter will try to prevent such attacks on their wireless networks using hardware, software, design and enforcement of security policy and etc. This paper will try to demonstrate whether this approach will result in a more fruitful outcome in terms of students concept and theory understandings and motivation to learn.
Higher education is facing a paradigm shift in the ownership and use of computer hardware. The school computer lab is no longer the primary place of student computer use. Instead, students increasingly expect to use their own hardware to complete their school assignments. This creates a challenge for computer science educators: we must now support a wide range of heterogeneous hardware without the benefits of tight control over its use. To address this ``Bring-Your-Own-Device'' (BYOD) challenge, we leverage virtualization and software packaging systems to gracefully deploy and support a standardized development environment for all core CS courses across a range of both school-owned and student-owned computing devices. We have deployed and evaluated our system for the previous two years at scale and continue to actively use and develop it. It has effectively helped us support multiple classes comprising hundreds of students with very limited IT staffing. We describe the design and management of our system, present our experience using our system, and discuss the lessons we've learned. We also provide data reflecting current student user experience with our system. Our system has proven very effective in addressing the student BYOD challenge in a manageable, cost-efficient, and easy-to-use manner.
Games have a long tradition in teaching IT security: Ranging from international capture-the-flag competitions played by multiple teams to educational simulation games where individual students can get a feeling for the effects of security decisions. All these games have in common, that the game's main goal is keeping up the security. In this paper, we propose another kind of educational security games which feature a game goal unrelated to IT security. However, during the game session gradually more and more attacks on the underlying infrastructure disturb the game play. Such a scenario is very close to the reality of an IT security expert, where establishing security is just a necessary requirement to reach the company's goals. By preparing and analyzing the game sessions, the students learn how to develop a security policy for a simplified scenario. Additionally, the students learn to decide when to apply technical security measures, when to establish emergency plans, and which risks cannot be covered economically. As an example for such a disturbed playing game, we present our distributed air traffic control scenario. The game play is disturbed by attacking the integrity and availability of the underlying network in a coordinated manner, i.e., all student teams experience the same failures at the same state of the game. Beside presenting the technical aspects of the setup, we are also discussing the didactic approach and the experiences made in the last years.
The field of academic security education today is dominated by defensive techniques. However, recently, offensive techniques which were originally developed by hackers, are gaining widespread approval. Many information security educators believe that teaching offensive methods yields better security professionals than teaching defensive techniques alone. In addition, every course in IT security should be accompanied by a basic discussion of legal implications and ethics.
In this paper, we describe a case study of the implementation of comprehensive hands-on lab exercises that are essential to security education. The lab exercises are about how to perform Denial of Service (DoS) and Man-in-the-Middle (MiM) attacks using ARP (Address Resolution Protocol) cache poisoning. The available defense techniques for detecting and preventing malicious ARP cache poisoning activities are also presented. The consequence of offering offensive lab exercises is that the overall students performance improved; but a major ethical concern has been identified. That is, the number of injected malicious ARP packets in the university network, from the students' laptops, increases considerably each time the students experiment the attacks in an isolated network laboratory environment.
Zusammenfassung Offensive Methoden der IT-Sicherheit haben das Ziel, Schutzvorkehrungen eines IT-Systems zu überwinden und somit dessen Integrität,
Vertraulichkeit oder Verfügbarkeit zu verletzen. Obwohl sie mittlerweile zum Standardwissen von Sicherheitsfachleuten gehören,
ist das eigentliche Potential offensiver Methoden für die IT-Sicherheit noch weitgehend unentdeckt.
The role of the network security administrator is continually morphing to keep pace with the ever-changing area of computer and network security. These changes are due in part to both the continual development of new security exploits by attackers as well as improvements in network security products available for use. One area which has garnered much research in the past decade is the use of visualization to ease the strain on network security administrators. Visualization mechanisms utilize the parallel processing power of the human visual system to allow for the identification of possible nefarious network activity. This research details the development and use of a visualization system for network security. The manuscript is composed of four papers which provide a progression of research pertaining to the system. The first paper utilizes research in the area of information visualization to develop a new framework for designing visualization systems for network security. Next, a visualization system is developed in the second paper which has been utilized during multiple cyber defense competitions to aid in competition performance. The last two papers deal with evaluating the developed system. First, an exploratory analysis provides an initial assessment using participant interviews during one cyber defense competition. Second, a quasi field experiment explores the intention of subjects to use the system based on the type of visualization being viewed.
: This paper presents experience from laboratory projects performed by students in Applied Computer Security, a course given at Chalmers University of Technology. From this, we conclude that the combination of security research and education results in a synergy that can be very beneficial to both fields. The paper describes in detail three different types of laboratory projects: intrusion experiments, intrusion analysis and remediation, and intrusion detection. Furthermore, we discuss the outcome and impact of the student projects with respect to education, research, and synergy between the two, as well as ethical issues. Among the benefits of the close connection between research and education in the projects, we find that the students were very motivated by this research connection and that they produced unique research data with natural diversity. 1 The author is also with the Department of Computer Science, Karlstad University, SE-651 88 Karlstad, Sweden. 2 Stefan Lindskog, Ulf Lindqvist, and Erland Jonsson 1.
ECE 297—Special Topics Network Security: Honeypots
- R Daniel
[Daniel, 2002] Daniel, R. (2002). ECE 297-Special Topics. Network Security: Honeypots.
The George Washington University, School of Engineering and Applied Science,
Department of Electrical and Computer Engineering.
- D Boneh
, 2002] Boneh, D. (2002). CS 155: Computer and Network Security. Stanford
University.
What Do We Mean By “Computer Security Education
- M Bishop
Academia and Education in Information Security: Four Years Later
- M Bishop
, 1999] Bishop, M. (1999). What Do We Mean By "Computer Security Education"?
In Proceedings of the 22nd National Information Systems Security Conference.
[Bishop, 2000] Bishop, M. (2000). Academia and Education in Information Security: Four
Years Later. In Proceedings of the Fourth National Colloquium on Information System
Security Education.