No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Characteristics of Internet Background Radiation
Abstract
Monitoring any portion of the Internet address space reveals incessant activity. This holds even when monitoring traffic sent to unused addresses, which we term "background radiation." Background radiation reflects fundamentally nonproductive traffic, either malicious (flooding backscatter, scans for vulnerabilities, worms) or benign (misconfigurations). While the general presence of background radiation is well known to the network operator community, its nature has yet to be broadly characterized. We develop such a characterization based on data collected from four unused networks in the Internet. Two key elements of our methodology are (i) the use of filtering to reduce load on the measurement system, and (ii) the use of active responders to elicit further activity from scanners in order to differentiate different types of background radiation. We break down the components of background radiation by protocol, application, and often specific exploit; analyze temporal patterns and correlated activity; and assess variations across different networks and over time. While we find a menagerie of activity, probes from worms and autorooters heavily dominate. We conclude with considerations of how to incorporate our characterizations into monitoring and detection activities.
... This refers to misconfigured systems and network services that attempt to initiate connection to non-existent hosts. It also refers to faulty hardware and software that sends packets to arbitrary destinations or to bad Network Address Translation (NAT) implementations incorrectly mapping RFC 3330 [7] addresses into public ones [8]. ...
... • Pang et al. [8]: internet background radiation, network telescope; • Wustrow et al. [6]: darknet, internet background traffic, network pollution; • Benson et al. [26]: internet background radiation; network telescope; opportunistic network analysis; and • Chindipha et al. [18]: internet background radiation, network telescope, threat intelligence, network scanning. The result is the following main search string: "internet background radiation" OR "internet background noise" OR "network telescope" OR "network telescopes" OR "darknet" OR "darknets" ...
... • Technical report by Moore [1]; • Primary research work Pang [8]; • Long-term IBR analysis by Wustrow [6]; and • Long-term IBR analysis by Irwin [2]. Forward snowballing was performed upon the four seminal works, which enabled the identification of a range of relevant, interconnected research contributions. ...
For two decades, cyber security researchers have been looking to answer one major question: what threats affect the Internet at large? In addition, what malicious traffic patterns would emerge if we could sample the unsolicited traffic - termed Internet Background Radiation (IBR) - arriving at devices directly connected to the Internet? The standard approach to collecting malicious traffic is the Network Telescope: a computer device assigned with a public IP address range, configured to passively listen to incoming packets. The deployment of Network Telescopes has helped to detect and quantify major cyberspace outbreaks, from the rise of the Conficker malware, to uncovering massive botnet propagation activity, such as performed by Mirai and its variants, against the Internet-of-Things. This paper introduces the Cloud Telescope: an ephemeral, cloud-native architecture, described as Infrastructure-as-Code, enabling for geographically distributed capture of the IBR, along with a discussion of a 5-month-long validation experiment, in which a sensor fleet comprising 130 cloud instances was launched across twenty-six regions of the world. The result is a quantitative and qualitative analysis of 530 million captured packets. This includes traffic breakdown by protocol: TCP (80%), UDP (3%), ICMP (17%), and by source country. We also discuss traffic aggregation by destination country and by affected cloud region, enabling novel forms of geopolitical influence analysis.
... Internet Background Radiation (IBR) is defined as non-productive data packets on the Internet, which target unused IP addresses, or ports where there is no network device set up to receive them (Cooke et al., 2004;Pang et al., 2004;Wustrow et al., 2010;Guillot et al., 2019). In theory, no traffic should ever arrive at such an IPv4 address, and so such traffic is marked as an anomaly and thus recorded and analysed (Hunter et al., 2013;Guillot et al., 2019;Richter and Berger, 2019). ...
... The value of network telescopes has been dealt with thoroughly by other researchers to the point that its significance to cybersecurity research cannot be overemphasised Irwin, 2013;Bou-Harb et al., 2018). Often this traffic shows evidence of either malicious activity or poor configuration (Pang et al., 2004;Nkhumeleni, 2014;Bou-Harb et al., 2014;Richter and Berger, 2019). The poor configuration could either be temporary or permanent (Nkhumeleni, 2014;Fachkha et al., 2017). ...
... The depletion of the availability of large IPv4 network blocks is of great concern, particularly in the cybersecurity field that often relies on acquiring large network blocks for its threat intelligence gathering (Pang et al., 2004;Bailey et al., 2005). Large net-blocks are significant because they give a broad spectrum from which to observe threats and thus are better placed to make a more informed decision than what one would get if a smaller network telescope or data were used (Atifi and Bou-Harb, 2017;Piotr et al., 2019). ...
As the Internet has grown in popularity, there has been an increasing demand for addresses used to connect devices. This has placed pressure on the use of addresses for purposes such as security monitoring. The author investigated the viability of using established methods for gathering cyber threat intelligence using much smaller network sensors and datasets. Mathematical models were created to quantify the accuracy of the techniques. The techniques evaluated were found to maintain a high degree of accuracy despite a reduction in both the range and volume of data collected.
... Unsurprisingly there is a fair amount of traffic that is observablegenerally produced though a combination of misconfiguration, active scanning and reflected denial of service activity. This traffic was termed by Pang et al. (2004) as Internet Background Radiation (IBR). IBR provides a valuable proxy by which to gauge the degree of unsolicited traffic traversing the internet, and from which other activities can be inferred. ...
... Traditionally major contributors to IBR have been in the East, China, Korea, Russia and in the west the USA, Canada and Brazil. The degree of contribution is largely linked to costs of internet connectivity, internet adoption, and most significantly the number of IP addresses in use (Irwin, 2013;Pang et al., 2004;Wustrow et al., 2010). These factors together with national legislation around Internet Security, all correlate with the general volume of scanning, and potentially malicious traffic emanating from an IP range (and ultimately country). ...
... This data was used as the basis for the remaining analysis presented in this paper. The examination of this data was guided by prior work such as Benson et al. (2015), Iglesias and Zseby (2019), Irwin (2013), Pang et al. (2004), and Wustrow et al. (2010). The results of this are discussed in the section following. ...
This paper explores the contribution made by IPv4 address space attributable to Mauritian organisations to the Internet Background Radiation (IBR). Data spanning a duration of 19 months starting in January 2021, from six discrete network telescopes is used as the basis for the analysis. A decomposition of the traffic is presented considering top origins by both ASN and netblock. An analysis is presented on the top 10 targeted TCP ports across the data. Alongside this an exploration is done into some of the more unusual probing for known vulnerable services that was observed. A determination of the reflected traffic and consideration of identified anomalies concludes the analysis. Mauritian IP address space is found to be relatively well regulated, and not have a large population of contributors to IBR either via active scanning or via reflected traffic.
... Unsurprisingly there is a fair amount of traffic that is observablegenerally produced though a combination of misconfiguration, active scanning and reflected denial of service activity. This traffic was termed by Pang et al. (2004) as Internet Background Radiation (IBR). IBR provides a valuable proxy by which to gauge the degree of unsolicited traffic traversing the internet, and from which other activities can be inferred. ...
... Traditionally major contributors to IBR have been in the East, China, Korea, Russia and in the west the USA, Canada and Brazil. The degree of contribution is largely linked to costs of internet connectivity, internet adoption, and most significantly the number of IP addresses in use (Irwin, 2013;Pang et al., 2004;Wustrow et al., 2010). These factors together with national legislation around Internet Security, all correlate with the general volume of scanning, and potentially malicious traffic emanating from an IP range (and ultimately country). ...
... This data was used as the basis for the remaining analysis presented in this paper. The examination of this data was guided by prior work such as Benson et al. (2015), Iglesias and Zseby (2019), Irwin (2013), Pang et al. (2004), and Wustrow et al. (2010). The results of this are discussed in the section following. ...
This paper explores the contribution made by IPv4 address space attributable to Mauritian organisations to the Internet Background Radiation (IBR). Data spanning a duration of 19 months starting in January 2021, from six discrete network telescopes is used as the basis for the analysis. A decomposition of the traffic is presented considering top origins by both ASN and netblock. An analysis is presented on the top 10 targeted TCP ports across the data. Alongside this an exploration is done into some of the more unusual probing for known vulnerable services that was observed. A determination of the reflected traffic and consideration of identified anomalies concludes the analysis. Mauritian IP address space is found to be relatively well regulated, and not have a large population of contributors to IBR either via active scanning or via reflected traffic.
... Bailey et al. [4] collected incoming payloads, before stateless scanning was introduced. Pang et al. [45] monitored IBR and built application-level responders. Their results underscore the importance of analyzing unsolicited traffic to identify new breeds of malicious activities. ...
... In contrast to honeypots [47], Spoki does not emulate any services or complete network stacks. It runs on a given IP prefix without knowledge US --'04 [9] US --'04 [45] () US -'06 [1] US, PL -() '06-'10 [58] US -'08-'15 [5] US -'11 [13] US () '13-'14 [15] US --'15 [21] ----'16 [26] KR -'18 [48] () 156 cou. --'20-'21 Spoki US, EU of individual IP addresses or sockets. ...
... Pre-ZMap era ZMap release in 2013 enabled easy two-phase scanning 2004 [45] 2010 [58] 2014 [15] 2018 [26] 2020 (two-phase) 2020 (one-phase) HTTP ( ...
Large-scale Internet scans are a common method to identify victims of a specific attack. Stateless scanning like in ZMap has been established as an efficient approach to probing at Internet scale. Stateless scans, however, need a second phase to perform the attack, which remains invisible to network telescopes that only capture the first incoming packet and is not observed as a related event by honeypots. In this work, we examine Internet-wide scan traffic through Spoki, a reactive network telescope operating in real-time that we design and implement. Spoki responds to asynchronous TCP SYN packets and engages in TCP handshakes initiated in the second phase of two-phase scans. Because it is extremely lightweight it scales to large prefixes where it has the unique opportunity to record the first data sequence submitted within the TCP handshake ACK. We analyze two-phase scanners during a three months period using globally deployed Spoki reactive telescopes as well as flow data sets from IXPs and ISPs. We find that a predominant fraction of TCP SYNs on the Internet has irregular characteristics. Our findings also provide a clear signature of today's scans as: (i) highly targeted, (ii) scanning activities notably vary between regional vantage points, and (iii) a significant share originates from malicious sources.
... That figure is slightly higher than in other modern studies [4][5][6] but is still in the same range. As a matter of fact, in the research conducted in 2004, a background noise packet would be detected once in 10 minutes [7]. ...
... Judging by that, it can be said that the classification of UDP-ports is often compromised because the rules of assigning ports are often broken. Nevertheless, it is fair to say that for protocols such as DNS, NTP, SNMP and similar ones the packet contents correlates with the port, even though there are some exceptions (in the study [7] there is an example of illegal usage of port 53 in order to bypass the firewall). ...
... The results for AS activity are significantly different from our previous [7] and other [6] similarly publications. To figure out the reasons of that longer observations are required to monitor the changing in the list of the most noise-active AS in the timeline. ...
Internet background noise (IBN, also known as Internet background radiation) is unsolicited network packets. For example, packets addressed to non-existing host. There are several reasons for background noises: untargeted scanning of global network in search of vulnerable hosts, network devices misconfiguration, backscatters of DDoS attacks with IP-spoofing technology, traces of worms propagation etc. We study background noises with «dark» collector – it is the trap that records all incoming packets and does not respond to them or do any other activity (term constructed in opposite to «honeypot»). In this research, we captured about ten millions packets of background noise. All captured packets was divided into groups according to character combination of flags and parameters in network, transport and application headers. For each group of packets the reason of appearing in the background noise was suggested and statistical characteristic was given. We suppose that different kinds of noises should be study separately. It will allow to study global networks threats trends by changing frequency of appearing packets from such groups. Based on the given classification, what may be developed are the dashboards for global network monitoring as well as the triggers of new kinds of attacks.
... It is this legitimate traffic which adds additional overheads in terms of processing and analysing. For certain types of research, researchers have developed telescopes to separate user requests from illegitimate traffic such as malicious network scanning, and backscatter (Pang et al., 2004, Moore et al., 2006, Irwin, 2011. Telescopes are discussed further in Section 3.3. ...
... Backscatter is Internet traffic originating from hosts on the Internet who are responding to spoofed IP addresses (Pang et al., 2004, Moore et al., 2006, Wustrow et al., 2010, Irwin, 2011. These hosts are likely the victim of a Denial of Service attack (Section 2.7.2). Backscatter does not present any threat to an organisation receiving the traffic. ...
... Typically telescopes do not respond to any requests, they simply receive and log them (Benson et al., 2015). The general term for the traffic recorded on these telescopes is Internet Background Radiation (IBR) (Pang et al., 2004). IBR includes traffic such as Backscatter, Scans, vulnerabilities, and worms, even Misconfiguration of devices. ...
Organisations and individuals are facing increasing persistent threats on the Internet from
worms, port scanners, and malicious software (malware). These threats are constantly
evolving as attack techniques are discovered. To aid in the detection and prevention
of such threats, and to stay ahead of the adversaries conducting the attacks, security
specialists are utilising Threat Intelligence (TI) data in their defense strategies.
TI data can be obtained from a variety of different sources such as private routers, firewall
logs, public archives, and public or private network telescopes. However, at the rate and
ease at which TI is produced and published, specifically Open Source Threat Intelligence
(OSINT), the quality is dropping, resulting in fragmented, context-less and variable data.
This research utilised two sets of TI data, a collection of OSINT and active Internet
Background Radiation (IBR). The data was collected over a period of 12 months, from 37
publicly available OSINT datasets and five IBR datasets. Through the identification and
analysis of common data between the OSINT and IBR datasets, this research was able
to gain insight into how effective OSINT is at detecting and potentially reducing ongoing
malicious Internet traffic. As part of this research, a minimal framework for the collection,
processing/analysis, and distribution of OSINT was developed and tested. The research
focused on exploring areas in common between the two datasets, with the intention of
creating an enriched, contextualised, and reduced set of malicious source IP addresses
that could be published for consumers to use in their own environment.
The findings of this research pointed towards a persistent group of IP addresses observed
on both datasets, over the period under research. Using these persistent IP addresses, the
research was able to identify specific services being targeted. Amongst these persistent
IP addresses were significant packets from Mirai like IoT Malware on port 23/tcp and
2323/tcp as well as general scanning activity on port 445/TCP.
... The diminishing availability of IPv4 is of great concern within the cybersecurity field that has traditionally relied on large network blocks for performing Internet Background Radiation (IBR) monitoring and research [2,6,22,23,30]. Large net-blocks have been significant because of the large 'lens' from which to observe threats [24]. This has been better than observations conducted on smaller network telescopes such as in [1,12,26,30]. ...
... This has been better than observations conducted on smaller network telescopes such as in [1,12,26,30]. This technique monitors IPv4 addresses which have no services running on them [13,22]. In practice, no traffic should ever arrive at such an IPv4 address, and so is marked as an anomaly, and thus recorded and analysed [10,19]. ...
... In monitoring unsolicited traffic received by network telescopes, traditionally contiguous blocks were used as they offer ease of monitoring and continuity [19,22]. Distributed network telescopes have also been used with small sized network telescopes for observing different segments of the network with its output combined into one [13,14,19]. ...
Network telescopes have been used for over a decade to aid in identifying threats by gathering unsolicited network traffic. This Internet Background Radiation (IBR) data has proved to be a significant source of intelligence in combating emerging threats on the Internet at large. Traditionally, operation has required a significant contiguous block of IP addresses. Continued operation of such sensors by researchers and adoption by organisations as part of its operation intelligence is becoming a challenge due to the global shortage of IPv4 addresses. The pressure is on to use allocated IP addresses for operational purposes. Future use of IBR collection methods is likely to be limited to smaller IP address pools, which may not be contiguous. This paper offers a first step towards evaluating the feasibility of such small sensors. An evaluation is conducted of the random sampling of various subnet sized equivalents. The accuracy of observable data is compared against a traditional 'small' IPv4 network telescope using a /24 net-block. Results show that for much of the IBR data, sensors consisting of smaller, non-contiguous blocks of addresses are able to achieve high accuracy rates vs. the base case. While the results obtained given the current nature of IBR, it proves the viability for organisations to utilise free IP addresses within their networks for IBR collection and ultimately the production of Threat intelligence.
... As such, since this network scanning is an indispensable process for cyber attacks, attention should still be paid to it, even though it has been studied, investigated, and monitored for a long time. Indeed, researchers and practitioners have already deeply surveyed, analyzed, and measured this behavior [3,8,26,34,35]. However, it should be kept in mind that the characteristics of network scanning (e.g., main target services and scan origins) are quite sensitive to the trends of popular network services and popular malware, and thus they have been actively changed to match these trends. ...
... Our scanning measurement has a number of advantages over the previous ones with respect to the data set. First, many previous measurement studies [8,26,34] have collected packets destined to unused IP address spaces. It is known that the target selection of distributed scanning by Internet worms is often not random intentionally [30] (for efficient infection) or unexpectedly [21] (for implementation issues). ...
... One limitation of our data set is the exclusion of protocols other than TCP and UDP. Even though they are small in volume (see Table 1), a large number of ICMP packets are observed in some network in 2004 [26]. However, it is reported that the volume has become very small in 2010 [34] and 2014 [8]. ...
Network scanning is the primary procedure preceding many network attacks. Until recently, network scanning has been widely studied to report a continued growth in volume and Internet-wide trends including the underpinning of distributed scannings by lingering Internet worms. It is, nevertheless, imperative to keep us informed with the current state of network scanning, for factual and comprehensive understanding of the security threats we are facing, and new trends to serve as the presage of imminent threats.
In this paper, we analyze the up-to-date connection-level log data of a large-scale campus network to study the recent scanning trends in breadth. We find, most importantly, the scanning landscape is greatly shifted, predominantly by an unprecedented rise in Telnet service scannings. Furthermore, not only are the scan sources comprehensively identified in terms of targeted services and geographical/network locations, but also their characteristics, such as being responsible in scanning and their connection-level behavior, are studied.
... The internet threat landscape is constantly changing and evolving, and threat actors are finding novel ways of identifying and compromising potential targets. Network telescopes (NT) provide a different perspective and insight into the general level of "noise" and background activity occurring on the Internet [1,2]. This can include "echoes" of DDoS attacks and malware events [3,4,5], the data can aid in identifying the sources, likely targets, and possibly what type of techniques are used. ...
... Network telescopes occupy and monitor unused IP address spaces that have no active hosts on them. The network traffic that arrives at these unassigned IP addresses is referred to as Internet Background Radiation (IBR) and can be classified as unwanted traffic [1,7,8]. Network telescopes are completely passive sensors; their primary objective is to detect unwanted traffic arriving at the monitored address space and provide insight into remote events such as various malware attacks, distributed denial-of-service attacks, and network scanning [9]. ...
This research investigates changes in Internet Background Radiation (IBR) by analysing data captured from network telescopes. Network telescopes, which provide a unique insight into unsolicited network traffic and can be indicative of widespread malicious activity. The primary focus of the study is on a comparative analysis between network data from 2017 and 2023, captured from the same IP netblock. The methodology is grounded in descriptive statistical analysis. Among our findings were changes in protocol distribution, with an increase in TCP traffic, a decrease in UDP traffic, and a substantial increase in ICMP traffic, primarily from Russia, while observing a notable decrease in the Russian overall attributed traffic. A sharp decrease in specific destination port targeting for both TCP and UDP traffic suggests broader scanning activity than before.
... A typical scenario is the temporary allocation of an unused CIDR block for which traffic is forwarded to a capture-enabled device. A Network Telescope is configured to record to all inbound traffic (with ideally no upstream filtering) as PCAP files for later analysis [1], [3]. For nearly 20 years, research groups have been analysing samples of the IBR captured by Network Telescopes to better understand cyber attacks, target acquisition methods, and attacker behaviour. ...
... This work focused on backscatter analysis to identify flooding-style denial-of-service attacks (SYN and ICMP), while observing the presence of unsolicited responses across one monitored IP address block. Seminal work by Pang et al. [3] formally characterised IBR. The three-month traffic capture monitored a set of IP addresses comprised of one full /8 block, a couple of non-contiguous /19 networks, and ten /24 address pools. ...
Internet Background Radiation (IBR) comprises a range of unsolicited traffic directed towards Internet hosts. In general, this type of traffic is characterised by high levels of port scanning activity, malware propagation, application exploits, system misconfiguration and denial-of-service attacks. IBR capture is typically undertaken by a system termed a network telescope. This records unfiltered incoming internet traffic for a specific CIDR block in the form of a packet capture (PCAP) file for analysis. This work proposes a novel, cloud-native approach to capturing IBR by the deployment of an ephemeral and reproducible architecture, described as code, distributed across all regions of a cloud service provider. In this paper we discuss the technical and financial viability of using a fleet of small-sized compute instances, in a spot price auction model, to maximise platform collection, capillarity and duration. We also present an overview analysis of the primary characteristics of IBR as collected during a month long proof-of-concept experiment across 26 regions of a cloud service provider in May 2023. Our analysis discusses the aspects of the dataset in quantitative terms: traffic aggregation per protocol, top TCP and UDP ports, top radiation sources and radiation distribution per cloud region. We also provide an overview of the most relevant threats detected. Our results include a formalisation and validation of the cloud telescope, with the corresponding supporting architecture described in Terraform and Ansible. The aggregate dataset amounted to 2.2 GB, and 21.8 million packets. Composition by protocol was 78% TCP, 14% ICMP and 8% UDP.
... A typical scenario is the temporary allocation of an unused CIDR block for which traffic is forwarded to a capture-enabled device. A Network Telescope is configured to record to all inbound traffic (with ideally no upstream filtering) as PCAP files for later analysis [1], [3]. For nearly 20 years, research groups have been analysing samples of the IBR captured by Network Telescopes to better understand cyber attacks, target acquisition methods, and attacker behaviour. ...
... This work focused on backscatter analysis to identify flooding-style denial-of-service attacks (SYN and ICMP), while observing the presence of unsolicited responses across one monitored IP address block. Seminal work by Pang et al. [3] formally characterised IBR. The three-month traffic capture monitored a set of IP addresses comprised of one full /8 block, a couple of non-contiguous /19 networks, and ten /24 address pools. ...
Internet Background Radiation (IBR) comprises a
range of unsolicited traffic directed towards Internet hosts.
In general, this type of traffic is characterised by high levels
of port scanning activity, malware propagation, application
exploits, system misconfiguration and denial-of-service attacks.
IBR capture is typically undertaken by a system termed a
network telescope. This records unfiltered incoming internet
traffic for a specific CIDR block in the form of a packet capture
(PCAP) file for analysis. This work proposes a novel, cloudnative approach to capturing IBR by the deployment of an
ephemeral and reproducible architecture, described as code,
distributed across all regions of a cloud service provider. In
this paper we discuss the technical and financial viability of
using a fleet of small-sized compute instances, in a spot price
auction model, to maximise platform collection, capillarity and
duration. We also present an overview analysis of the primary
characteristics of IBR as collected during a month long proof-ofconcept experiment across 26 regions of a cloud service provider
in May 2023. Our analysis discusses the aspects of the dataset
in quantitative terms: traffic aggregation per protocol, top TCP
and UDP ports, top radiation sources and radiation distribution
per cloud region. We also provide an overview of the most
relevant threats detected. Our results include a formalisation
and validation of the cloud telescope, with the corresponding
supporting architecture described in Terraform and Ansible. The
aggregate dataset amounted to 2.2 GB, and 21.8 million packets.
Composition by protocol was 78% TCP, 14% ICMP and 8%
UDP
... Prior work has used Domain Name System (DNS) traffic to assess misbehavior seen in specific networks or resolvers [58], [27], [41], [57], [2], but this work does not generalize to network-wide activity. Darknets [38], [35], [56], [13], [14], [17] and honeypots (for example, [42]) are effective at understanding network-wide activity, but they miss targeted scans (scanning only Alexa top websites [17]), and new large darknets are unlikely given IPv4 full allocation and the huge IPv6 space. Search engines gather information about activity that appears in the public web, but information is unstructured and may be delayed by indexing [50]. ...
... Non-DNS passive sensors: Darknets (or network telescopes) are a commonly used passive technique to characterize large-scale network activity [38], [35], [56], [13], [14], [17]. By monitoring a large, unoccupied blocks of addresses, darknets see active probes from viruses and scanners, queries from misconfiguration, and backscatter from spoofed traffic; traffic that can predict global malware, and its absence, network outages. ...
... A darknet may be called black hole monitors, dark space, network telescopes (Pang et al. 2004), and spurious traffic. In addition to being smaller than real traffic, darknet traffic contains malicious activity traces. ...
Darknet, a source of cyber intelligence, refers to the internet’s unused address space, which people do not expect to interact with their computers. The establishment of security requires analyses of the threats characterizing the network. New machine learning classifiers known as stacking ensemble learning are proposed in this paper to analyze and classify darknet traffic. In dealing with darknet attack problems, this new system uses predictions formed by 3 base learning techniques. The system was tested on a dataset comprising more than 141,000 records analyzed from CIC-Darknet 2020. The experiment results demonstrated the study’s classifiers’ ability to distinguish between the malignant traffic and benign traffic easily. The classifiers can effectively detect known and unknown threats with high precision and accuracy greater than 99% in the training and 97% in the testing phases, with increments ranging from 4 to 64% by current algorithms. As a result, the proposed system becomes more robust and accurate as data grows. Also, the proposed system has the best standard deviation compared with current A.I. algorithms.
... If a device is subject to anomalous harmless noise [48], over time, we expect the values of the columns (i.e., features) in its MRT feed to show little variation. The number of clusters will remain stable, and the percentage of mutual matches will stay close to 100%. ...
Besides coming with unprecedented benefits, the Internet of Things (IoT) suffers deficits in security measures, leading to attacks increasing every year. In particular, network environments such as smart homes lack managed security capabilities to detect IoT-related attacks; IoT devices hosted therein are thus more easily targeted by threats. As such, context awareness of IoT infections is hard to achieve, preventing prompt response. In this work, we propose MUDscope, an approach to monitor malicious network activities affecting IoT systems in real-world consumer environments. We leverage the recent Manufacturer Usage Description (MUD) specification, which defines networking allow-lists for IoT devices in MUD profiles, to reflect consistent and necessarily-anomalous activities from smart things. Our approach characterizes this traffic and extracts signatures for given attacks. By analyzing attack signatures for multiple devices, we gather insights into emerging attack patterns. We evaluate our approach on both an existing dataset and a new, openly available dataset created for this research. We show that MUDscope detects several attacks targeting IoT devices with an F1-score of 95.77% and correctly identifies signatures for specific attacks with an F1-score of 87.72%.
... Network telescopes consist of monitoring infrastructure that receives and records unsolicited traffic destined to vast swaths of unused but routed Internet address spaces (i.e., millions of IPs). This traffic, coined as "Internet Background Radiation" [13], [14], captures traffic from nefarious actors that perform Internetwide scanning activities, malware and botnets that aim to infect other victims, "backscatter" activities that denote DoS attacks [14] and network misconfigurations. Thus, Darknets offer a unique lens into macroscopic Internet activities and timely detection of new abnormal Darknet behaviors is extremely important. ...
Network operators and system administrators are increasingly overwhelmed with incessant cyber-security threats ranging from malicious network reconnaissance to attacks such as distributed denial of service and data breaches. A large number of these attacks could be prevented if the network operators were better equipped with threat intelligence information that would allow them to block or throttle nefarious scanning activities. Network telescopes or "darknets" offer a unique window into observing Internet-wide scanners and other malicious entities, and they could offer early warning signals to operators that would be critical for infrastructure protection and/or attack mitigation. A network telescope consists of unused or "dark" IP spaces that serve no users, and solely passively observes any Internet traffic destined to the "telescope sensor" in an attempt to record ubiquitous network scanners, malware that forage for vulnerable devices, and other dubious activities. Hence, monitoring network telescopes for timely detection of coordinated and heavy scanning activities is an important, albeit challenging, task. The challenges mainly arise due to the non-stationarity and the dynamic nature of Internet traffic and, more importantly, the fact that one needs to monitor high-dimensional signals (e.g., all TCP/UDP ports) to search for "sparse" anomalies. We propose statistical methods to address both challenges in an efficient and "online" manner; our work is validated both with synthetic data as well as real-world data from a large network telescope.
... In fact, this traffic type is considered a problem, as the efficiency and usefulness of the network, as well as the end users' equipment, are reduced. There are other definitions [4] that focus mainly on malicious traffic in its narrow sense, that is, it is always associated with criminal activities such as launching intrusion attacks or sustaining a black economy. Thus, according to RFC 4948 [5], unsolicited traffic is divided into three types: nuisance, malicious and unknown. ...
... Furthermore, regardless of the type of analysis (binary or multiclass), traffic classes are commonly retrieved from Intrusion Detection System (IDS) datasets without proper discussion about the class meaning, the label assignment, the nature of attacks, and the attack deployment within the tested data. To give an example, in binary classification (attack/non-attack), it is commonly not clear which binary label should correspond to backscatter traffic (for a detailed description of backscatter traffic, see [120]). Backscatter traffic is not formed by attack packets, but indirectly caused by malicious activities, which provoke that vulnerable servers to generate such spurious traffic. ...
The increased interest in secure and reliable communications has turned the analysis of network traffic data into a predominant topic. A high number of research papers propose methods to classify traffic, detect anomalies, or identify attacks. Although the goals and methodologies are commonly similar, we lack initiatives to categorize the data, methods, and findings systematically. In this paper, we present Network Traffic Analysis Research Curation (NTARC), a data model to store key information about network traffic analysis research. We additionally use NTARC to perform a critical review of the field of research conducted in the last two decades. The collection of descriptive research summaries enables the easy retrieval of relevant information and a better reuse of past studies by the application of quantitative analysis. Among others benefits, it enables the critical review of methodologies, the detection of common flaws, the obtaining of baselines, and the consolidation of best practices. Furthermore, it provides a basis to achieve reproducibility, a key requirement that has long been undervalued in the area of traffic analysis. Thus, besides reading hard copies of papers, with NTARC, researchers can make use of a digital environment that facilitates queries and reviews over a comprehensive field corpus.
... RTBH was designed to prevent forwarding of unwanted traffic [21,46], e.g., (i) attack traffic (DoS), (ii) incoming scan traffic [7], or (iii) Internet background radiation [34]. For the latter two, the traffic volume is comparatively small and operational best practices such as firewalls and static ACL filters [8] are adequate solutions. ...
Large Distributed Denial-of-Service (DDoS) attacks pose a major threat not only to end systems but also to the Internet infrastructure as a whole. Remote Triggered Black Hole filtering (RTBH) has been established as a tool to mitigate inter-domain DDoS attacks by discarding unwanted traffic early in the network, e.g., at Internet eXchange Points (IXPs). As of today, little is known about the kind and effectiveness of its use, and about the need for more fine-grained filtering.
In this paper, we present the first in-depth statistical analysis of all RTBH events at a large European IXP by correlating measurements of the data and the control plane for a period of 104 days. We identify a surprising practice that significantly deviates from the expected mitigation use patterns. First, we show that only one third of all 34k visible RTBH events correlate with indicators of DDoS attacks. Second, we witness over 2000 blackhole events announced for prefixes not of servers but of clients situated in DSL networks. Third, we find that blackholing on average causes dropping of only 50% of the unwanted traffic and is hence a much less reliable tool for mitigating DDoS attacks than expected. Our analysis gives also rise to first estimates of the collateral damage caused by RTBH-based DDoS mitigation.
... One of the Cyber security strategies is to analyze the darknet traffic. Darknet and darknet traffic are often referred to as darkspace, black hole monitors, network telescopes, unsolicited network traffic, Internet Background Radiation (IBR) [2], spurious traffic, etc. ...
The cyberspace continues to evolve more complex than ever anticipated, and same is the case with security dynamics there. As our dependence on cyberspace is increasing day-by-day, regular and systematic monitoring of cyberspace security has become very essential. A darknet is one such monitoring framework for deducing malicious activities and the attack patterns in the cyberspace. Darknet traffic is the spurious traffic observed in the empty address space, i.e., a set of globally valid Internet Protocol (IP) addresses which are not assigned to any hosts or devices. In an ideal secure network system, no traffic is expected to arrive on such a darknet IP space. However, in reality, noticeable amount of traffic is observed in this space primarily due to the Internet wide malicious activities, attacks and sometimes due to the network level misconfigurations. Analyzing such traffic and finding distinct attack patterns present in them can be a potential mechanism to infer the attack trends in the real network. In this paper, the existing Basic and Extended AGgregate and Mode (AGM) data formats for darknet traffic analysis is studied and an efficient 29-tuple Numerical AGM data format suitable for analyzing the source IP address validated TCP connections (three-way handshake) is proposed to find attack patterns in this traffic using Mean Shift clustering algorithm. Analyzing the patterns detected from the clusters results in providing the traces of various attacks such as Mirai bot, SQL attack, and brute force. Analyzing the source IP validated TCP, darknet traffic is a potential technique in Cyber security to find the attack trends in the network.
... However, large scale scans of the IP address space are also conducted for research and survey purposes by systems such as Censys [6] and Shodan [7], while network misconfigurations may cause similar effects. This incessant non-productive traffic termed Internet Background Radiation may mask malicious traffic sources when studying isolated traffic snapshots [8]. However, traffic analysis across large time scales and across multiple vantage points allows the extraction of unique patterns and characteristics for malicious scanners. ...
The 2016 Mirai outbreak established an entirely new mindset in the history of large-scale Internet attacks. A plethora of Mirai-like variants have emerged in the last two years that are capable to infiltrate any type of device. In this paper we provide a 7-month retrospective analysis of Internet-connected energy systems that are infected by Mirai-like malware variants. By utilizing network measurements from several Internet vantage points, we demonstrate that a number of energy systems on a global scale were infected during the period of our observation. While past works have studied vulnerabilities and patching practises of ICS and energy systems, little information has been available on actual exploits of such vulnerabilities. Hence, we provide evidence that energy systems relying on ICS networks are often compromised by vulnerabilities in non-ICS devices (routers, servers and IoT devices) which provide foothold for lateral network attacks. Our work offers a first look in compromised energy systems by malware infections, and offers insights on the lack of proper security practices for systems that are increasingly dependent on internet services and more recently the IoT. In addition, we indicate that such systems were infected for relatively large periods, thus potentially remaining undetected by their corresponding organizational units.
... Several works [22]- [24] are on radiation and port scanning measurements. However, most of them are concerned with a single network (Tier-1 ISP [24], sinkhole traffic [23]). ...
Internet distributed denial of service (DDoS) attacks are prevalent but hard to defend against, partially due to the volatility of the attacking methods and patterns used by attackers. Understanding the latest DDoS attacks can provide new insights for effective defense. But most of existing understandings are based on indirect traffic measures (e.g., backscatters) or traffic seen locally. In this paper, we present an in-depth analysis based on 50,704 different Internet DDoS attacks directly observed in a seven-month period. These attacks were launched by 674 botnets from 23 different botnet families with a total of 9026 victim IPs belonging to 1074 organizations in 186 countries. Our analysis reveals several interesting findings about today's Internet DDoS attacks. Some highlights include: 1) geolocation analysis shows that the geospatial distribution of the attacking sources follows certain patterns, which enables very accurate source prediction of future attacks for most active botnet families; 2) from the target perspective, multiple attacks to the same target also exhibit strong patterns of inter-attack time interval, allowing accurate start time prediction of the next anticipated attacks from certain botnet families; and 3) there is a trend for different botnets to launch DDoS attacks targeting the same victim, simultaneously or in turn. These findings add to the existing literature on the understanding of today's Internet DDoS attacks and offer new insights for designing new defense schemes at different levels.
... The Spoofer project 2 shows that, at the start of 2017, close to a fifth of Internet addresses, and a quarter of autonomous systems, allow their hosts to spoof. The analysis of backscatter presented in [8] also suggests that, despite the proliferation of NAT devices, spoofing is still widespread, and the next generation of attacks may intelligently probe networks and adapt their behavior based on the ability to spoof [9], [10]. When attacks involve NTP or DNS(SEC) reflection, source spoofing is used to engineer them, but the packets that contribute to saturation do not have spoofed source addresses, suggesting that spoofing need not be considered during mitigation. ...
Distributed Denial-of-Service (DDoS) attacks continue to trouble network operators and service providers, and with increasing intensity. Effective response to DDoS can be slow (because of manual diagnosis and interaction) and potentially self-defeating (as indiscriminate filtering accomplishes a likely goal of the attacker), and this is the result of the discrepancy between the service provider’s flow-based, application-level view of traffic and the network operator’s packet-based, network-level view and limited functionality. Furthermore, a network required to take action may be in an Autonomous System (AS) several AS-hops away from the service, so it has no direct relationship with the service on whose behalf it acts. This paper presents Antidose, a means of interaction between a vulnerable peripheral service and an indirectly related AS that allows the AS to confidently deploy local filtering with discrimination under the control of the remote service. We implement the core filtering mechanism of Antidose, and provide an analysis of it to demonstrate that conscious attacks against the mechanism will not expose the AS to additional attacks. We present a performance evaluation to show that the mechanism is operationally feasible in the emerging trend of operators’ willingness to increase the programmability of their hardware with SDN technologies such as OpenFlow, as well as to act to mitigate attacks on downstream customers.
... iSinks [25] uses a filtering strategy consisting of analyzing the connections established with the first N destination IPs per every source IP. Pang et al. [63] improved the filtering mechanisms taking into account, for example, the source port, destination and connection. Bailey et al. [64] improved the source-destination based filtering mechanism through expanding the individual darknets into multiple darknets for observing the global behavior and the source distribution. ...
A honeypot is a type of security facility deliberately created to be probed, attacked, and compromised. It is often used for protecting production systems by detecting and deflecting unauthorized accesses. It is also useful for investigating the behavior of attackers, and in particular, unknown attacks. For the past 17 years plenty of effort has been invested in the research and development of honeypot techniques, and they have evolved to be an increasingly powerful means of defending against the creations of the blackhat community. In this paper, by studying a wide set of honeypots, the two essential elements of honeypots—the decoy and the captor— are captured and presented, together with two abstract organizational forms—independent and cooperative—where these two elements can be integrated. A novel decoy and captor (D-C) based taxonomy is proposed for the purpose of studying and classifying the various honeypot techniques. An extensive set of independent and cooperative honeypot projects and research that cover these techniques is surveyed under the taxonomy framework. Furthermore, two subsets of features from the taxonomy are identified, which can greatly influence the honeypot performances. These two subsets of features are applied to a number of typical independent and cooperative honeypots separately in order to validate the taxonomy and predict the honeypot development trends.
Internet-wide scanners can efficiently scan the expansive IPv6 network by targeting the active prefixes and responsive addresses on the hitlists. However, it is not clear enough how scanners discover fresh prefixes, which include newly assigned or deployed prefixes, as well as previously unused ones. This paper studies the whole discovery process of fresh prefixes by scanners. We implement four DNS-based address-exposing methods, analyze the arrival sequence of scans from distinct ASes, and examine the temporal and spatial scan patterns, with darknet and honeynet. Over six months, our custom-made darknet and probabilistic responsive honeynet collected 33 M packets (1.8 M sessions) of scans from 116 distinct ASes and 18.8 K unique source IP addresses. We investigate the whole process of fresh prefix discovery, including address-exposing, initial probing, hitlist registration, and large-scale scan campaigns. Furthermore, we analyze the difference in scanning behavior by ASes, and categorize the scanners into three types, honeynet-exclusive, honeynet-predominant and balanced, based on the respective ratio of scans to darknet and honeynet. Besides, we analyze the intentions of scanners, such as network reconnaissance or scanning responsive targets, and the methods they used to obtain potential targets, such as by sending DNS queries or using public hitlist. These findings bring insights into the process of fresh prefixes attracting scanners and highlight the vital role of responsive honeynet in analyzing scanner behaviors.
Network telescopes or “Darknets” received unsolicited Internet-wide traffic, thus providing a unique window into macroscopic Internet activities associated with malware propagation, denial of service attacks, network reconnaissance, misconfigurations and network outages. Analysis of the resulting data can provide actionable insights to security analysts that can be used to prevent or mitigate cyber-threats. Large network telescopes, however, observe millions of nefarious scanning activities on a daily basis which makes the transformation of the captured information into meaningful threat intelligence challenging. To address this challenge, we present a novel framework for characterizing the structure and temporal evolution of scanning behaviors observed in network telescopes. The proposed framework includes four components. It (i) extracts a rich, high-dimensional representation of
scanning profiles
composed of features distilled from network telescope data; (ii) learns, in an unsupervised fashion, information-preserving
succinct representations
of these scanning behaviors using
deep representation learning
that is amenable to clustering; (iii) performs
clustering
of the scanner profiles in the resulting latent representation space on daily Darknet data, and (iv)
detects temporal changes
in scanning behavior using techniques from
optimal mass transport
. We robustly evaluate the proposed system using both synthetic data and real-world Darknet data. We demonstrate its ability to detect real-world, high-impact cybersecurity incidents such as the onset of the Mirai botnet in late 2016 and several interesting cluster formations in early 2022 (e.g., heavy scanners, evolved Mirai variants, Darknet “backscatter” activities, etc.). Comparisons with state-of-the-art methods showcase that the integration of the proposed features with the deep representation learning scheme leads to better classification performance of Darknet scanners.
This paper describes BigBen, a network telemetry processing system designed to enable accurate and timely reporting of Internet events (e.g., outages, attacks and configuration changes). BigBen is distinct from other Internet-wide event detection systems in its use of passive measurements of Network Time Protocol (NTP) traffic. We describe the architecture of BigBen, and a cloud-based implementation developed to process large NTP data sets and provide accurate daily event reporting. We demonstrate BigBen on a 15.5TB corpus of NTP data. We show that BigBen identifies a wide range of Internet events characterized by their location, scope and duration. We compare the events detected by BigBen vs. events detected by a large active probe-based detection system. We find only modest overlap between the two datasets and show how BigBen provides details on events that are not available from active measurements. Finally, we report on the perspective that BigBen provides on Internet events that were reported by third parties. In each case, BigBen confirms the event and provides details that were not available in prior reports, highlighting the utility of the passive, NTP-based approach.
Network behavior analysis provides critical insights and visibility on what is happening to millions of networked systems and thousands of Internet applications in a variety of network environments. A number of studies have demonstrated the benefits and feasibility of these behavioral insights and visibility in a wide spectrum of applications such as traffic profiling, cybersecurity, and network forensics. This chapter first presents the applications of network behavior analysis in profiling Internet traffic and discovering server and service behavior profiles, heavy-hitter host behavior profiles, scan and exploit profiles, and deviant or rare behavior profiles. Subsequently, this chapter discusses how network behavior analysis enhances cybersecurity by discovering and stopping scanning and exploit traffic from the Internet. Finally, this chapter sheds light on the applications of cluster-aware network behavior analysis by exploring the benefits of end-host behavior clusters and application behavior clusters, particularly in characterizing traffic patterns of network prefixes and detecting emerging applications and threats, which share strong similarities with existing and known applications and threats.
As the Internet continues to bring innovative applications and services to the broad society, making sense of behavioral objects on the Internet with network behavior analysis will remain an important technique for understanding and characterizing novel network environments, emerging network applications, and new networked systems. This chapter presents the research frontiers of network behavior analysis in cloud computing, smart home networks, and the Internet of Things (IoT) paradigms. This chapter first discusses network behavior analysis as a service (NBA-as-a-service) in cloud computing environments for monitoring and securing large-scale Internet data centers. Subsequently, this chapter presents how network behavior analysis provides new traffic and behavior insights into Internet-connected devices in distributed smart home networks. Finally this chapter introduces a multidimensional network behavior analysis framework to characterize behavioral patterns of heterogeneous IoT devices in edge networks.
The state of the network can be reflected by the background traffic. Negative network measurements can be a very important way to understand the Internet. I would like to express appreciation to CERNET, who provided us with an IPv6 address space allocated but not a fully used network. By announcing a large /20 covering prefixes on this address, we have published routing information on China's domestic education network, business network, and foreign education network. Based on the honeypot method, we collect relative traffic at the last hop router of the experiment network. Thus, we make our experiment environment a network telescope. We discover that background radiation traffic grew more rapidly than it was years ago under the current ipv6 network situation. Moreover, suspicious IPv6 address scanning traffic shows up. We classify and analyze the traffic and classify all the source addresses and destination addresses. We found that the source addresses are mainly from Asian countries. In particular, we conduct further detection and monitor on the suspicious source addresses. We analyze the time when it appears and what it scans, including the destination address and the port type. The most interesting destination ports to the outside world are mainly 80, 8080, 443, 53, 21, 22, 23, and 25, which are related to web services and host system applications. We explain most of the data and highlight the significant attributes of the data. We found several special addresses scanning our address segment periodically. Our work reveals the situation and the problem under the current IPv6 network situation.
Despite society’s strong dependence on electricity, power outages remain prevalent. Standard methods for directly measuring power availability are complex, often inaccurate, and prone to attack. This paper explores an alternative approach to identifying power outages through intelligent monitoring of IP address availability. In finding these outages, we explore the trade-off between the accuracy of detection and false alarms.
We begin by experimentally demonstrating that static, residential Internet connections serve as good indicators of power, as they are mostly active unless power fails and rarely have battery backups. We construct metrics that dynamically score the reliability of each residential IP, where a higher score indicates a higher correlation between the IP’s availability and its regional power. We then select and monitor subsets of residential IPs and evaluate the exactitude with which they indicate current county power status.
Using data gathered during the power outages caused by Hurricane Florence, we demonstrate that we can track power outages at different granularities (state and county), in both sparse and dense regions. By comparing our detections with the reports gathered from power utility companies, we achieve an average detection accuracy of 90%, where we also show some of our false alarms and missed outage events could be due to imperfect ground truth data. Therefore, our method can be used as a complementary technique for power outage detection.
Scanning of hosts on the Internet to identify vulnerable devices and services is a key component in many of today's cyberattacks. Tracking this scanning activity, in turn, provides an excellent signal to assess the current state-of-affairs for many vulnerabilities and their exploitation. So far, studies tracking scanning activity have relied on unsolicited traffic captured in darknets, focusing on random scans of the address space. In this work, we track scanning activity through the lens of unsolicited traffic captured at the firewalls of some 89,000 hosts of a major Content Distribution Network (CDN). Our vantage point has two distinguishing features compared to darknets: (i) it is distributed across some 1,300 networks, and (ii) its servers are live, offering services and thus emitting traffic. While all servers receive a baseline level of probing from Internet-wide scans, i.e., scans targeting random subsets of or the entire IPv4 space, we show that some 30% of all logged scan traffic is the result of localized scans. We find that localized scanning campaigns often target narrow regions in the address space, and that their characteristics in terms of target selection strategy and scanned services differ vastly from the more widely known Internet-wide scans. Our observations imply that conventional darknets can only partially illuminate scanning activity, and may severely underestimate widespread attempts to scan and exploit individual services in specific prefixes or networks. Our methods can be adapted for individual network operators to assess if they are subjected to targeted scanning activity.
Despite continuous defense efforts, DDoS attacks are still very prevalent on the Internet. In such arms races, attackers are becoming more agile and their strategies are more sophisticated to escape from detection. Effective defenses demand in-depth understanding of such strategies. In this paper, we set to investigate the DDoS landscape from the perspective of the attackers. We focus on the dynamics of the attacking force, aiming to explore the strategies behind the scenes, if any. Our study is based on 50,704 different Internet DDoS attacks across the globe in a seven-month period. Our results indicate that attackers deliberately schedule their controlled bots in a dynamic fashion, and such dynamics can be well captured by statistical distributions. Furthermore, different botnet families exhibit similar scheduling patterns, strongly suggesting their close relationship and potential collaborations. Such collaborations are further confirmed by bots rotating in multiple families, and such rotation patterns are examined and confirmed at various levels. These findings lay a promising foundation for predicting DDoS attacks in the future and aid mitigation efforts.
Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.
This paper proposes a traffic decomposition approach called UnitecDEAMP based on flow feature profiling to distinct groups of significant malicious events from background noise in massive historical darknet traffic. Specifically, we segment and extract traffic flows from captured darknet data, categorize the flows according to sets of criteria derived from our traffic behavior assessments. Those criteria will be validated through the followed correlation analysis to guarantee that any redundant criteria be eliminated. Significant events are appraised by combined criteria filtering, including significance regarding volume, significance in terms of time series occurrence and significance regarding variation. To demonstrate the effectiveness of our UnitecDEAMP, real world darknet traffic data sets with twelve months are used for conducting our empirical study. The experimental results show that UnitecDEAMP can effectively select the most significant malicious events.
Internet DDoS attacks are prevalent but hard to defend against, partially due to the volatility of the attacking methods and patterns used by attackers. Understanding the latest of DDoS attacks can provide new insights for effective defense. But most of existing understandings are based on indirect traffic measures (e.g., backscatters) or traffic seen locally (e.g., in an ISP or from a botnet). In this study, we present an in-depth study based on 50,704 different Internet DDoS attacks directly observed in a seven-month period. These attacks were launched by 674 botnets from 23 different botnet families with a total of 9026 victim IPs belonging to 1074 organizations in 186 countries. In this study, we conduct some initial analysis mainly from the perspectives of these attacks’ targets and sources. Our analysis reveals several interesting findings about today’s Internet DDoS attacks. Some highlights include: (1) while 40% of the targets were attacked only once, 20% of the targets were attacked more than 100 times (2) most of the attacks are not massive in terms of number of participating nodes but they often last long, (3) most of these attacks are not widely distributed, but rather being highly regionalized. These findings add to the existing literature on the understanding of today’s Internet DDoS attacks, and offer new insights for designing effective defense schemes at different levels.
Attackers routinely perform random portscans of IP addresses to find vulnerable servers to compromise. Network intrusion detection systems (NIDS) attempt to detect such behavior and flag these portscanners as malicious. An important need in such systems is prompt response: the sooner a NIDS detects malice, the lower the resulting damage. At the same time, a NIDS should not falsely implicate benign remote hosts as malicious. Balancing the goals of promptness and accuracy in detecting malicious scanners is a delicate and difficult task. We develop a connection between this problem and the theory of sequential hypothesis testing and show that one can model accesses to local IP addresses as a random walk on one of two stochastic processes, corresponding respectively to the access patterns of benign remote hosts and malicious ones. The detection problem then becomes one of observing a particular trajectory and inferring from it the most likely classification for the remote host. We use this insight to develop TRW (Threshold Random Walk), an online detection algorithm that identifies malicious remote hosts. Using an analysis of traces from two qualitatively different sites, we show that TRW requires a much smaller number of connection attempts (4 or 5 in practice) to detect malicious activity compared to previous schemes, while also providing theoretical bounds on the low (and configurable) probabilities of missed detection and false alarms. In summary, TRW performs significantly faster and also more accurately than other current solutions.
Network intrusions have been a fact of life in the Internet for many years. However, as is the case with many other types of Internet-wide phenomena, gaining insight into the global characteristics of intrusions is challenging. In this paper we address this problem by systematically analyzing a set of firewall logs collected over four months from over 1600 different networks world wide. The first part of our study is a general analysis focused on the issues of distribution, categorization and prevalence of intrusions. Our data shows both a large quantity and wide variety of intrusion attempts on a daily basis. We also find that worms like CodeRed, Nimda and SQL Snake persist long after their original release. By projecting intrusion activity as seen in our data sets to the entire Internet we determine that there are typically on the order of 25B intrusion attempts per day and that there is an increasing trend over our measurement period. We further find that sources of intrusions are uniformly spread across the Autonomous System space. However, deeper investigation reveals that a very small collection of sources are responsible for a significant fraction of intrusion attempts in any given month and their on/off patterns exhibit cliques of correlated behavior. We show that the distribution of source IP addresses of the non-worm intrusions as a function of the number of attempts follows Zipf's law. We also find that at daily timescales, intrusion targets often depict significant spatial trends that blur patterns observed from individual "IP telescopes"; this underscores the necessity for a more global approach to intrusion detection. Finally, we investigate the benefits of shared information, and the potential for using this as a foundation for an automated, global intrus...
We present an artificial workload model of wide-area internetwork traffic. The model can be used to drive simulation experiments of communication protocols and flow and congestion control experiments. The model is based on analysis of wide-area TCP/IP traffic collected from one industrial and two academic networks. The artificial workload model uses both detailed knowledge and measured characteristics of the user application programs responsible for the traffic. Observations drawn from our measurements contradict some commonly held beliefs regarding wide-area TCP/IP network traffic.
The phenomenal growth in popularity of the World Wide Web (WWW, or the Web) has made WWW traffic the largest contributor to packet and byte traffic on the NSFNET backbone. This growth has triggered recent research aimed at reducing the volume of network traffic produced by Web clients and servers, by using caching, and reducing the latency for WWW users, by using improved protocols for Web interaction.Fundamental to the goal of improving WWW performance is an understanding of WWW workloads. This paper presents a workload characterization study for Internet Web servers. Six different data sets are used in this study: three from academic (i.e., university) environments, two from scientific research organizations, and one from a commercial Internet provider. These data sets represent three different orders of magnitude in server activity, and two different orders of magnitude in time duration, ranging from one week of activity to one year of activity.Throughout the study, emphasis is placed on finding workload invariants: observations that apply across all the data sets studied. Ten invariants are identified. These invariants are deemed important since they (potentially) represent universal truths for all Internet Web servers. The paper concludes with a discussion of caching and performance issues, using the invariants to suggest performance enhancements that seem most promising for Internet Web servers.
Network intrusions have been a fact of life in the Internet for many years. However, as is the case with many other types of Internet-wide phenomena, gaining insight into the global characteristics of intrusions is challenging. In this paper we address this problem by systematically analyzing a set of firewall logs collected over four months from over 1600 different networks world wide. The first part of our study is a general analysis focused on the issues of distribution, categorization and prevalence of intrusions. Our data shows both a large quantity and wide variety of intrusion attempts on a daily basis. We also find that worms like CodeRed, Nimda and SQL Snake persist long after their original release. By projecting intrusion activity as seen in our data sets to the entire Internet we determine that there are typically on the order of 25B intrusion attempts per day and that there is an increasing trend over our measurement period. We further find that sources of intrusions are uniformly spread across the Autonomous System space. However, deeper investigation reveals that a very small collection of sources are responsible for a significant fraction of intrusion attempts in any given month and their on/off patterns exhibit cliques of correlated behavior. We show that the distribution of source IP addresses of the non-worm intrusions as a function of the number of attempts follows Zipf's law. We also find that at daily timescales, intrusion targets often depict significant spatial trends that blur patterns observed from individual "IP telescopes"; this underscores the necessity for a more global approach to intrusion detection. Finally, we investigate the benefits of shared information, and the potential for using this as a foundation for an automated, global intrusion detection framework that would identify and isolate intrusions with greater precision and robustness than systems with limited perspective.
Tcpreplay 1 was designed to replay traffic previously captured in the pcap format back onto the wire for testing NIDS and other passive devices. Over time, it was enhanced to be able to test in-line network devices. However, a re-occurring feature request for tcpreplay is to connect to a server in order to test applications and host TCP/IP stacks. It was determined early on, that adding this feature to tcpreplay was far too complex, so I decided to create a new tool specifically designed for this. Flowreplay is designed to replay traffic at Layer 4 or 7 depending on the protocol rather then at Layer 2 like tcpreplay does. This allows flowreplay to connect to one or more servers using a pcap savefile as the basis of the connections. Hence, flowreplay allows the testing of applications running on real servers rather then passive devices.
Network worms are a major threat to the security of today's Internet-connected hosts and networks. The combination of unmitigated connectivity and widespread software homogene-ity allows worms to exploit tremendous parallelism in propaga-tion. Modern worms spread so quickly that no human-mediated reaction to the outbreak of a new worm can hope to prevent a widespread epidemic. In this paper we propose an automated method for detecting new worms based on traffic characteristics common to most of them: highly repetitive packet content, an increasing population of sources generating infections and an in-creasing number of destinations being targeted. Our method gen-erates content signatures for the worm without any human inter-vention. Preliminary results on a small network show promising results: we have identified three confirmed worms with a low per-centage of false positives. This gives us reason to believe that our method could form the core of an effective network-level worm de-tection and countermeasure system capable of substantially slow-ing down the spread of new worms.
We describe Bro, a stand-alone system for detecting network intruders in real-time by passively monitoring a network link over which the intruder's traffic transits. We give an overview of the system's design, which emphasizes high-speed (FDDI-rate) monitoring, real-time notification, clear separation between mechanism and policy, and extensibility. To achieve these ends, Bro is divided into an `event engine' that reduces a kernel-filtered network traffic stream into a series of higher-level events, and a `policy script interpreter' that interprets event handlers written in a specialized language used to express a site's security policy. Event handlers can update state information, synthesize new events, record information to disk, and generate real-time notifications via syslog. We also discuss a number of attacks that attempt to subvert passive monitoring systems and defenses against these, and give particulars of how Bro analyzes the six applications integrated into it so far: Finger, FTP, Portmapper, Ident, Telnet and Rlogin. The system is publicly available in source code form.
Monitoring unused or dark IP addresses offers opportunities to significantly improve and expand knowledge of abuse activity without many of the problems
associated with typical network intrusion detection and firewall systems. In this paper, we address the problem of designing
and deploying a system for monitoring large unused address spaces such as class A telescopes with 16M IP addresses. We describe
the architecture and implementation of the Internet Sink (iSink) system which measures packet traffic on unused IP addresses
in an efficient, extensible and scalable fashion. In contrast to traditional intrusion detection systems or firewalls, iSink
includes an active component that generates response packets to incoming traffic. This gives the iSink an important advantage in discriminating
between different types of attacks (through examination of the response payloads). The key feature of iSink’s design that
distinguishes it from other unused address space monitors is that its active response component is stateless and thus highly scalable. We report performance results of our iSink implementation in both controlled laboratory experiments
and from a case study of a live deployment. Our results demonstrate the efficiency and scalability of our implementation as
well as the important perspective on abuse activity that is afforded by its use.
KeywordsIntrusion Detection-Honeypots-Deception Systems
Today's Internet intrusion detection systems (IDSes) moni - tor edge networks' DMZs to identify and/or filter malicious flows. While an IDS helps protect the hosts on its local edge network from compromise and denial of service, it cannot alone effectively intervene to halt and reverse the spreadi ng of novel Internet worms. Generation of the worm signatures required by an IDS—the byte patterns sought in monitored traffic to identify worms—today entails non-trivial human la - bor, and thus significant delay: as network operators detect anomalous behavior, they communicate with one another and manually study packet traces to produce a worm signature. Yet intervention must occur early in an epidemic to halt a worm's spread. In this paper, we describe Autograph, a sys- tem that automatically generates signatures for novel Internet worms that propagate using TCP transport. Autograph gen- erates signatures by analyzing the prevalence of portions of flow payloads, and thus uses no knowledge of protocol se- mantics above the TCP level. It is designed to produce sig- natures that exhibit high sensitivity (high true positives) and high specificity (low false positives); our evaluation of the system on real DMZ traces validates that it achieves these goals. We extend Autograph to share port scan reports among distributed monitor instances, and using trace-driven sim ula- tion, demonstrate the value of this technique in speeding the generation of signatures for novel worms. Our results elu- cidate the fundamental trade-off between early generation of signatures for novel worms and the specificity of these gener - ated signatures.
The ability of attackers to rapidly gain control of vast numbers of Internet hosts poses an immense risk to the overall security of the Internet. Once subverted, these hosts can not only be used to launch massive denial of service floods, but also to steal or corrupt great quantities of sensitive information, and confuse and disrupt use of the network in more subtle ways.
ABSTRACT Network intrusion detection systems (NIDS) are an important part of any network security architecture They provide a layer of defense which monitors network traffic for predefined suspicious activity or patterns, and alert system administrators when potential hostile traffic is detected Commercial NIDS have many differences, but Information Systems departments must face the commonalities that they share such as significant system footprint, complex deployment and high monetary cost Snort was designed to address these issues
On July 19, 2001, more than 359,000 computers connected to the Internet were infected with the Code-Red (CRv2) worm in less than 14 hours. The cost of this epidemic, including subsequent strains of Code-Red, is estimated to be in excess of $2.6 billion. Despite the global damage caused by this attack, there have been few serious attempts to characterize the spread of the worm, partly due to the challenge of collecting global information about worms. Using a technique that enables global detection of worm spread, we collected and analyzed data over a period of 45 days beginning July 2nd, 2001 to determine the characteristics of the spread of Code-Red throughout the Internet.In this paper, we describe the methodology we use to trace the spread of Code-Red, and then describe the results of our trace analyses. We first detail the spread of the Code-Red and CodeRedII worms in terms of infection and deactivation rates. Even without being optimized for spread of infection, Code-Red infection rates peaked at over 2,000 hosts per minute. We then examine the properties of the infected host population, including geographic location, weekly and diurnal time effects, top-level domains, and ISPs. We demonstrate that the worm was an international event, infection activity exhibited time-of-day effects, and found that, although most attention focused on large corporations, the Code-Red worm primarily preyed upon home and small business users. We also qualified the effects of DHCP on measurements of infected hosts and determined that IP addresses are not an accurate measure of the spread of a worm on timescales longer than 24 hours. Finally, the experience of the Code-Red worm demonstrates that wide-spread vulnerabilities in Internet hosts can be exploited quickly and dramatically, and that techniques other than host patching are required to mitigate Internet worms.
This paper describes a system for automated generation of attack signatures for network intrusion detection systems. Our system applies pattern-matching techniques and protocol conformance checks on multiple levels in the protocol hierarchy to network traffic captured a honeypot system. We present results of running the system on an unprotected cable modem connection for 24 hours. The system successfully created precise traffic signatures that otherwise would have required the skills and time of a security officer to inspect the traffic manually.
The phenomenal growth in popularity of the World Wide Web (WWW, or the Web) has made WWW traffic the largest contributor to packet and byte traffic on the NSFNET backbone. This growth has triggered recent research aimed at reducing the volume of network traffic produced by Web clients and servers, by using caching, and reducing the latency for WWW users, by using improved protocols for Web interaction. Fundamental to the goal of improving WWW performance is an understanding of WWW workloads. This paper presents a workload characterization study for Internet Web servers. Six different data sets are used in this study: three from academic (i.e., university) environments, two from scientific research organizations, and one from a commercial Internet provider. These data sets represent three different orders of magnitude in server activity, and two different orders of magnitude in time duration, ranging from one week of activity to one year of activity. Throughout the study, emphasis is placed on finding workload invariants: observations that apply across all the data sets studied. Ten invariants are identified. These invariants are deemed important since they (potentially) represent universal truths for all Internet Web servers. The paper concludes with a discussion of caching and performance issues, using the invariants to suggest performance enhancements that seem most promising for Internet Web servers.
It has been clear since 1988 that self-propagating code can quickly spread across a network by exploiting homogeneous security vulnerabilities. However, the last few years have seen a dramatic increase in the frequency and virulence of such "worm" outbreaks. For example, the Code-Red worm epidemics of 2001 infected hundreds of thousands of Internet hosts in a very short period - incurring enormous operational expense to track down, contain, and repair each infected machine. In response to this threat, there is considerable effort focused on developing technical means for detecting and containing worm infections before they can cause such damage. This paper does not propose a particular technology to address this problem, but instead focuses on a more basic question: How well will any such approach contain a worm epidemic on the Internet? We describe the design space of worm containment systems using three key parameters - reaction time, containment strategy and deployment scenario. Using a combination of analytic modeling and simulation, we describe how each of these design factors impacts the dynamics of a worm epidemic and, conversely, the minimum engineering requirements necessary to contain the spread of a given worm. While our analysis cannot provide definitive guidance for engineering defenses against all future threats, we demonstrate the lower bounds that any such system must exceed to be useful today. Unfortunately, our results suggest that there are significant technological and administrative gaps to be bridged before an effective defense can be provided in today's Internet.
The characteristic features of spread of Slammer worm are discussed. The worm's spreading strategy uses random scanning which randomly selects IP addresses, eventually finding and infecting all susceptible hosts. Slammer's scanner is limited by each compromised machine's Internet bandwidth. Slammer uses a linear congruent or power residue pseudo random number generation (PRNG) algorithm. The scanner of Slammer produced a heavy load in large traffic volume, lots of packets and large number of new destinations.
Analyzes 3 million TCP connections that occurred during 15 wide-area traffic traces. The traces were gathered at five “stub” networks and two internetwork gateways, providing a diverse look at wide-area traffic. The author derives analytic models describing the random variables associated with TELNET, NNTP, SMTP, and FTP connections. To assess these models the author presents a quantitative methodology for comparing their effectiveness with that of empirical models such as Tcplib [Danzig and Jamin, 1991]. The methodology also allows to determine which random variables show significant variation from site to site, over time, or between stub networks and internetwork gateways. Overall the author finds that the analytic models provide good descriptions, and generally model the various distributions as well as empirical models
The Internet is rapidly growing in number of users, traffic
levels, and topological complexity. At the same time it is increasingly
driven by economic competition. These developments render the
characterization of network usage and workloads more difficult, and yet
more critical. Few recent studies have been published reporting Internet
backbone traffic usage and characteristics. At MCI, we have implemented
a high-performance, low-cost monitoring system that can capture traffic
and perform analyses. We have deployed this monitoring tool on OC-3
trunks within the Internet MCI's backbone and also within the
NSF-sponsored vBNS. This article presents observations on the patterns
and characteristics of wide-area Internet traffic, as recorded by MCI's
OC-3 traffic monitors. We report on measurements from two OC-3 trunks in
MCI's commercial Internet backbone over two time ranges (24-hour and
7-day) in the presence of up to 240,000 flows. We reveal the
characteristics of the traffic in terms of packet sizes, flow duration,
volume, and percentage composition by protocol and application, as well
as patterns seen over the two time scales
2> elements. Individual elements implement simple router functions like packet classification, queueing, scheduling, and interfacing with network devices. A router configuration is a directed graph with elements at the vertices
This paper presents Click, a flexible, modular software architecture for creating routers. Click routers are built from fine-grained components; this supports fine-grained extensions throughout the forwarding path. The components are packet processing modules called elements. The basic element interface is narrow, consisting mostly of functions for initialization and packet handoff, but elements can extend it to support other functions (such as reporting queue lengths). To build a router configuration, the user chooses a collection of elements and connects them into a directed graph. The graph's edges, which are called connections, represent possible paths for packet handoff. To extend a configuration, the user can write new elements or compose existing elements in new ways, much as UNIX allows one to build complex applications directly or by composing simpler ones using pipes
Microsoft IIS 5.0 " translate: f " source disclosure vulnerability
- Security Focus
Security Focus. Microsoft IIS 5.0 " translate: f " source disclosure
vulnerability. http://www.securityfocus.com/bid/1578/discussion/, April
2004.
Attack processes found on the internet
- M Dacier
- F Pouget
- H Debar
M. Dacier, F. Pouget, and H. Debar. Attack processes found on the
internet. In Proceedings of NATO Symposium, 2004.
Code red: A case study on the spread and victims of an internet worm
- D Moore
- C Shannon
- J Brown
D. Moore, C. Shannon, and J. Brown. Code red: A case study on the
spread and victims of an internet worm. In Proceedings of ACM
SIGCOMM Internet Measurement Workshop, November 2002.
Inferring internet denial of service activity
- D Moore
- G Voelker
- S Savage
D. Moore, G. Voelker, and S. Savage. Inferring internet denial of
service activity. In Proceedings of the 2001 USENIX Security
Symposium, Washington D.C., August 2001.
Dame Ware Mini Remote Control Server <= 3. 72 buffer overflow
- Dame Ware Mini