Abstract. Robots, integral to modern automation and services, encounter formidable cybersecurity challenges, primarily due to their inherent complexity and the lack of vendor accountability for security, shifting the burden to end-users. Offensive Robot Cybersecurity, the title of this thesis, introduces a groundbreaking approach by advocating for offensive security methods empowered by automation leveraging modern Artificial Intelligence. It emphasizes the necessity of understanding attackers' tactics and identifying vulnerabilities in advance to develop effective defenses, thereby improving robots' security posture. This endeavor leverages a decade of robotics experience, employing Machine Learning and Game Theory to streamline the vulnerability identification and exploitation process. Intrinsically, the thesis uncovers a profound connection between robotic architecture and cybersecurity, highlighting that the design and creation aspect of robotics deeply intertwines with its protection against attacks. This duality—whereby the architecture that shapes robot behavior and capabilities also necessitates a defense mechanism through offensive and defensive cybersecurity strategies—creates a unique equilibrium. Approaching cybersecurity with a dual perspective of defense and attack, rooted in an understanding of systems architecture, has been pivotal in this research. This synergy affirms the adage that the best defense is a good offense, especially in the complex realm of robotics. Through comprehensive analysis, including ethical considerations, the development of security tools, and executing cyber attacks on robot software, hardware, and industry deployments, this thesis proposes a novel architecture for cybersecurity cognitive engines. These engines, powered by advanced game theory and machine learning, pave the way for autonomous offensive cybersecurity strategies for robots, marking a significant shift towards self-defending robotic systems. This research not only underscores the importance of offensive measures in enhancing robot cybersecurity but also sets the stage for future advancements where robots are not just resilient to cyber threats but are equipped to autonomously safeguard themselves.

Full text transcribed from the original publication source.

Motivation

1Introduction

1.1Why robot cybersecurity?

For the last fifty years, we have been witnessing the dawn of the robotics industry, but robots are not being created with security as a concern. Security in robotics is often mistaken with safety. From industrial to consumer robots, going through professional ones, most of these machines are not prepared for cyber-threats and resilient to security vulnerabilities. Manufacturers’ concerns, as well as existing standards, focus mainly on safety. Security in robotics is still not being considered as a relevant matter. The integration between these two areas from a risk assessment perspective was studied in which resulted in a unified security and safety risk framework. Commonly, robotics safety is understood as developing protective mechanisms against accidents or malfunctions, whilst security is aimed to protect systems against risks posed by malicious actors . A slightly alternative view is the one that considers safety as protecting the environment from a given robot, whereas security is about protecting the robot from a given environment. In this proposal we adopt the later and tackle the emerging need to address security in robotics.

1.2About ROS

Robots are networks of networks, with sensors capturing data, passing to compute technologies, and then on to actuators and back again in a deterministic manner. These networks can be understood as the nervous system of the robot, passing across compute Nodes, that represent neurons. Like the human nervous systems, real-time information across all these computational Nodes is fundamental for the robot to behave coherently.

"Robot brains" are built with this same philosophy. Behaviors take the form of computational graphs, with data flowing between Nodes operating intra-process, inter-process and across physical networks (communication buses), while mapping to underlying sensors, compute technologies and actuators. The Robot Operating System (ROS) is a robotics framework for robot application development. ROS enables a robotics developer to build these computational graphs and create robot behaviors by providing libraries, a communication infrastructure, drivers and tools to put it all together. It provides an open source codebase with a commercially friendly license that helps roboticists reduce the effort to bring up robot behaviors. Widely used to govern interactions across robots, at the time of writing, the original ROS article has been cited about 12000 times, which shows its wide acceptance for research and academic purposes. ROS was born in this environment: its primary goal was to provide the software tools and libraries that users would need to employ to undertake novel robotics research and development. Adoption in industry has also been rapidly increasing over the last few years. According to the latest ROS community metrics sampled every year in July, the number of ROS downloads has increased by over 50%, with about 600 million downloads between July of 2020 and July of 2021 1 . Moreover, based on the download percentages reported from packages.ros.org, we observe a significant increase in adopting ROS 2, which suggests that within 2024 there will be more users using ROS 2 than its predecessor 2 .

Following from the success of ROS, and redesigned for production environments, the second generation of ROS, ROS 2, is rapidly becoming a standard in the robotics industry. Built upon the Data Distribution Service (DDS ) as its default communication middleware (used in safety-critical scenarios), ROS 2 aspiration was to offer a more robust framework that could be integrated into products and industry. Figure 1.1 depicts the ROS 2 robotics stack, relative to the OSI model. From a technical viewpoint, it provides three main components:

1. ROS Plumbing: ROS provides an efficient message-passing infrastructure facilitating seamless communication between different robotic system components. This includes topics for asynchronous message exchange, services for synchronous requests, and actions for handling long-duration tasks. ROS’s process management, or node management, ensures efficient handling of various processes, promoting a modular and scalable design. Additionally, ROS device drivers are integral for interfacing with the robot’s hardware, ensuring effective control and data acquisition from sensors and actuators.

2. ROS Tools: ROS boasts a suite of visualization tools like RViz and rqt, essential for real-time monitoring and understanding of a robot’s operation. CLI tools such as rostopic, rosnode, and rosservice offer convenient system interaction and control, crucial for development and troubleshooting. The simulation environment, provided by tools like Gazebo , enables safe and cost-effective testing in varied scenarios. Moreover, ROS’s data logging capabilities, particularly with rosbag, are vital for data analysis, debugging, and scenario replication.

3. ROS Capabilities: ROS’s control capabilities ensure precise robot movements and actions, encompassing real-time control loops and motion planning for complex task execution. Its perception capabilities, processing data from various sensors, are critical for tasks like navigation and environment mapping. ROS’s manipulation features enable physical interaction with the environment, essential in applications such as assembly and logistics. Furthermore, coordinate transformations, managed by the tf library, maintain spatial relationships crucial for path planning and object manipulation.

ROS 2 stands as a foundational element in the realm of modern robotics, playing a pivotal role in this thesis focused on offensive robot cybersecurity. As the de facto standard for modern robotic software development, ROS offers a comprehensive framework that not only streamlines the creation of complex robotic applications but also inherently shapes the cybersecurity landscape of robotic systems. In the context of offensive cybersecurity, the ubiquity and modular architecture of ROS present both opportunities and challenges. Its widespread adoption makes it a critical target for cybersecurity research, as vulnerabilities within ROS (including any of its higher level abstraction, or underlayers) could potentially impact a vast array of robotic systems. This thesis, therefore, places significant emphasis on ROS, examining its architectures, communication protocols, and tools through the lens of offensive cybersecurity. By doing so, it aims to uncover and address potential security weaknesses, contributing to the robustness and resilience of robotics systems developed with ROS. This exploration not only aids in fortifying ROS-based robots against cyber threats but also provides valuable insights into the broader field of robot cybersecurity.

1.3Research Objectives and Approach

Security is not a product, but a process that needs to be continuously assessed in a periodic manner, as systems evolve and new cyber-threats are discovered. This becomes specially relevant with the increasing complexity of such systems . Current robotic systems are of high complexity, a condition that in most cases leads to wide attack surfaces and a variety of potential attack vectors which makes difficult the use of traditional approaches. Altogether, this leads to the following research question: given the increasing complexity of robotic systems, how can we protect them from cyber-threats? Inspired by the popular adage "the best defense is a good offense", the present work attempts to answer this question by focusing on offensive practices yet, given the time-consuming requirements of traditional offensive security approaches , the answer might not be straightforward and require further automation.

Scalability is one of the core problems of offensive security assessments (red teaming, penetration testing, etc.) as of now . This thesis dives into robot cyber security and study how traditional and modern offensive methods apply to robotics, delivering a new angle to protect such systems. To this research question, the investigation herein proposed reviews and creates new mechanisms and techniques that leverage Machine Learning and Game Theory allowing for the full automation of offensive security assessments.

This research also argues and challenges the underlying ethics of the security community when applied to robotics. Particularly, de facto standards and conventions on security outcomes and disclosures are assessed and challenged, with a critical non-vendor-centered attitude. Ultimately, given the novelty of the topic itself, a relevant number of contributions are expected in the form of blueprints that will empower future research.
In the remainder of this chapter, the research objectives and approach are discussed in more detail. The overall research framework is introduced in Section 1.3, followed by an explanation of each one of the building blocks.

Although security has gained attention over the past years, our computational systems are increasingly vulnerable against cyber attacks. This includes of course all robots. As indicated by Bozic , one reason behind is definitely the system’s complexity, comprising interacting heterogenous systems. Complexity is the enemy of security. It is very difficult to assure no vulnerabilities in a system that an attacker could exploit. In robotics, the reality is even more concerning, as there is no culture of security 3 . As studied in previous work , except few vendors and selected actions, no real concern has been shown. Offensive security methods (such as pentesting or red teaming) play an important part in the security lifecycle. In these engagements, as introduced by Applebaum , (offensive) security teams try to break into an organization’s assets, identifying vulnerabilities along the way. Red teams take this concept even further, trying to fully emulate what real adversaries do: instead of just compromising the network and identifying vulnerabilities, they have a larger goal that requires significant post-compromise work. Offensive security methods are necessary to understand attackers’ behavior, to train defensive mechanisms and ultimately, to help protect existing systems by discovering flaws first. The latter is specially important in highly complex systems, such as robots.
In an attempt to raise awareness and create some base formal studies and tools, this research objective is to study how offensive cyber security methods applied to robotics allow to protect such systems. In particular, this thesis aims to study how traditional methods in offensive security are applied to robotics. Such methods include penetration testing, red teaming, automated attack planning or advanced persistent threats amongst others. The expected outcome is twofold: a) to validate that these practices are applicable to robotics and b) that the inherent complexity of robotics systems, often systems of system at multiple levels, make the manual application of these methods unfeasible in most robotic setups. In addition, this study covers ethics and defensive security mechanisms that protect systems, networks and operations. Such is necessary for implementing offensive methods and understanding the underlying ethical and legal boundaries. Ultimately, this study looks into how to automate and optimize offensive cyber security attacks targeting robots. Preliminary ideas on this direction include the use and extension of prior work (as covered in 2.4) on leveraging Machine Learning for the semi-automation of offensive security assessment. Such machine learning techniques could be trained using popular and widely available Capture The Flag (CTF) environments adapted for robotics. In addition, to optimize the use of resources and define proper learning and exploitation strategies, Game Theoretic approaches are considered.
For coherence, this proposal has been organized in a framework consisting on four building blocks. As depicted in Figure 1.2, from bottom to top, (0) Ethics, (1) Blueprints for researching offensive security for robots, as well as robotics in general, (2) Cyber-attacking robots and (3) Cyber-attacking robots with Machine Learning and Game Theory. Generally, upper layers require of primitives from the underlying ones. The following subsections briefly describe each block.

Ethics

This block treated in Part II aims to study the ethical landscape of cybersecurity in robotics. Particularly, it aims to build some basic knowledge on the ethically accepted situation for robotics in Europe, in main areas of application. Beyond analysing existing vulnerabilities and manufacturer responses to them, this layer also studies robotic users’ perception of cybersecurity by conducting various surveys in popular robotic communities. In addition, this block challenges some of the de facto standards in security when applied to robotics and discuss them with a critical attitude. Including topics such as disclosure policies, the requirement from vendors to freely notify research results and vulnerabilities (so called responsible disclosure) or even, the relevance of disclosing flaws at all, inspired by previous criticism . This block is discussed in more detail in Chapter 3.

Blueprints

Building upon the previous ethical and legal baselines, this block is covered in Part III and develops the tools, libraries and frameworks necessary to perform robotics security research and offensive attacks in robots and their components. Each one of these research elements will be considered as "blueprints". Often considered part of a research continuum, the blueprints produced in this phase include:

• Building upon the DevSecOps in Robotics principles researched in the non peer-reviewed published article , SROS2 proposes a series of developer tools, meant to be usable and that facilitate adding security capabilities to ROS 2 computational graphs (Chapter 4). On top of SROS2, we also discuss the shortcomings of the DDS Security extensions which ROS 2 embraces and how additional layers of security via VPN could help mitigate them.

• Security measures such as those proposed by SROS2 in Chapter 4 add a computational overhead to robotic systems. In order to accurately estimate the performance impact, RobotPerf proposes an open-source and community-driven benchmarking tool designed to assess the performance of robotic computing systems in a standardized, architecture-neutral, and reproducible way, including the security associated aspects of them (Chapter 5).

• Chapter 5 elucidates the computational overhead of integrating security into robotic communications, highlighting the resultant time bottlenecks that diminish system capabilities. To address this, RobotCore is introduced. It aims to empower the development of hardware accelerators that expedite security computations in robotics, thereby enabling the incorporation of security in robotic interactions without compromising capabilities. RobotCore introduces a framework and an open architecture for hardware acceleration that extends ROS to easily develop hardware accelerators in an accelerator-agnostic manner, across GPUs and/or FPGAs (Chapter 6).

Cyber-attacking robots

Using previous blueprint security primitives, tools developed (such as Alurity or ExploitFlow ) and know-how, research produced within this block aims at building basic attacks on robots and robot components targeting both, software and hardware. These attacks’ objectives are threefold: first, they aim to contribute building some basic research ground on offensive cyber security robot practices. Second, these attacks will help raise the awareness of the current practices most vendors are applying. Third and ultimately, these actions help building expertise and develop an intuition on common offensive cyber-behavior, which will later be necessary when automating attacks using Machine Learning and Game Theory.

In Part IV, we present our contributions while cyber-attacking robots. Chapter 7 cares about attacking robot software and investigates the Data Distribution Service (DDS) standard, crucial for robotic operation, uncovering numerous vulnerabilities as detailed in . We highlight the discovery of publicly accessible DDS implementations, stressing the importance of persistent security evaluation and robust integration practices. Chapter 8 explores attacking robot hardware through teardowns, emphasizing the significance of such methods in identifying security flaws. We also address the effects of planned obsolescence in robotics, advocating for a ‘Right to Repair’ stance. The last chapter, 9 focuses on attacking robots in industry and introduces Akerbeltz ransomware, the first instance of ransomware targeting industrial collaborative robots. Akerbeltz demonstrates its impact on industrial robots by encrypting and locking robotic systems, which underscores the critical need for heightened security against potential cyber-attacks.

Automated offensive robot cybersecurity

Bridging the gap between the intricate world of robotic security vulnerabilities and proactive countermeasures, this final block shifts the focus from identification to automated action. It lays the groundwork for systems designed to anticipate and combat threats autonomously, fortifying defenses in an ever-evolving digital battleground where automation play an increasingly pivotal role. This transition encapsulates the move from understanding vulnerabilities and its mitigations (covered in parts III and IV), to implementing advanced strategies for robust and resilient cybersecurity cognitive engines. Constructing upon all previous blocks, in Part V we attempt to design and deploy an autonomous system for cyber-attacking robots using Machine Learning and Game Theory.

Building on the strategic insights provided by game theory, we advance into the domain of proactive cyber defense and offense within automated systems in Chapter 10. This first chapter in Part V tackles the challenge of Determining the attack target with Game Theory, equipping us with a strategic toolset to predict the most vulnerable points in our robot threat landscape and to fortify them accordingly. This chapter describes how to prioritize our defensive efforts and to anticipate the moves of our adversaries, aligning our future actions with the most likely paths they might traverse in the adversary’s illicit endeavors.

Following this strategic planning phase, Guiding attacks with Machine Learning presented in Chapter 11, immerses us into the dynamic process of directing our cyber agents. Here, we leverage the advanced capabilities of modern Machine Learning, particularly the generative power of Large Language Models, to automate the decision-making processes in cyber-attacks. These models act as multi-faceted instruments, capable of parsing vast amounts of data, reasoning through complex scenarios, and generating adaptive attack and defense strategies. With the prototype PentestGPT, we have a glimpse into a future where cybersecurity operations are autonomous, intelligent, and ever-evolving, reflecting the intricate nature of the digital battleground where they operate.

Part V as a whole forms a comprehensive narrative that transitions from a state of awareness and understanding of vulnerabilities to an active, autonomous and intelligent stance against them. They pave the way for automated systems that do not merely react to threats but anticipate and adapt to them, marking a significant leap in the field of offensive robot cybersecurity.

2Background context

2.1Biographical cornerstones in robot cybersecurity

Arguably, the first installation of a cyber-physical system in a manufacturing plant was back in 1962 . The first human death caused by a robotic system is traced back to 1979 and the causes were safety-related according to the reports. From this point on, a series of actions involving agencies and corporations triggered to protect humans and environments from this machines, leading into safety standards.
Security however hasn’t started being addressed scientifically in robotics until recently. One of the first peer-reviewed published articles on the topic goes back to 2016 which already warns about the security dangers of the Robot Operating System (ROS) . Following this first assessment, the same group in Spain authored a series of articles touching into robot security . Contemporary to Lera’s work, White et al. propose an addition to ROS API to support modern cryptography and security measures . This work led by White (UC San Diego, Spain) and Caiazza (Ca’ Foscari University, Italy) delivered a series of articles which focused on proposing defensive blueprints for robot cybersecurity. Starting also in 2016 , Dieber (Joanneum Research, Austria) led a series of publications that researched robot cybersecurity focusing around ROS .
A careful review of the prior art in the last paragraph indicates that there’re three groups, in different points of the world, that almost simultaneously started working in robot cybersecurity by tackling the lack of security primitives and proposing studies and blueprints for securitization. Most of this existing work focuses on defensive approaches and mechanisms. Very few publications touched into offensive approaches to secure robotic systems being one of the first. Some pioneering work on the offensive side focused onto generating primary concerns and social awareness on robot security faults but from this literature review, very few instances of offensive security have been seen so far in robotics.
The following sections shed some additional light into the foundations of this research.

2.2Robotics software quality, safety and security

Quality (Quality Assurance or QA for short) and Security are often misunderstood when it comes to software. Ivers argues that quality "essentially means that the software will execute according to its design and purpose" while "security means that the software will not put data or computing systems at risk of unauthorized access". Within one relevant question that arises is whether the quality problems are also security issues or vice versa. Ivers indicates that quality bugs can turn into security ones provided they’re exploitable, and addresses the question by remarking that quality and security are critical components to a broader notion: software integrity. Depicted in Figure 2.1, software integrity can be represented the union of both software security and quality (Software Integrity = Security \(\cup\) Quality).

Coming from the same group, Vamosi argues that "quality code may not always be secure, but secure code must always be quality code". This somehow conflicts with the previous view and leads one to think that secure software is a subset of quality. The author of this thesis rejects this view and argues instead that Quality and Security remain two separate properties of software that may intersect on certain aspects (e.g. testing) as depicted in Figure 2.2.
While the target of this thesis is Security, Quality is also studied given its intersection. Often, both secure and quality code share several requirements and mechanisms to assess them. This includes testing approaches such as static code testing, dynamic testing, fuzz testing or software component analysis (SCA) among others.

In robotics there is a clear separation between Security and Quality that is best understood with scenarios involving robotic software components. For example, if one was building an industrial Autonomous Guided Vehicle (AGV) or a self-driving car, often, she/he would need to comply with coding standards (e.g. MISRA for developing safety-critical systems). The same system’s communications, however, regardless of its compliance with the coding standards, might rely on a channel that does not provide encryption or authentication and is thereby subject to eavesdropping and man-in-the-middle attacks. Security would be a strong driver in here and as remarked by Vamosi , "neither security nor quality would be mutually exclusive, there will be elements of both".
Quality in robotics, still on its early stages , is often viewed as a pre-condition for Safety-critical systems. Similarly, as argued by several, safety can’t be guaranteed without security . Coding standards such as MISRA C have been extended to become the C coding standard of choice for the automotive industry and for all industries developing embedded systems that are safety-critical and/or security-critical . As introduced by ISO/IEC TS 17961:2013 "in practice, security-critical and safety-critical code have the same requirements". This statement is somehow supported by Goertzel but emphasized the importance of software remaining dependable under extraordinary conditions and the interconnection between safety and security in software. This same argument was later extended by Bagnara who acknowledges that having embedded systems non-isolated anymore plays a key role in the relationship between safety and security. According to Bagnara, "while safety and security are distinct concepts, when it comes to connected software" (non-isolated) "not having one implies not having the other", referring to integrity.
In the opinion of the author of this dissertation, coding standards such as MISRA or ISO/IEC TS 17961:2013 for safety-critical and security-critical software components do not guarantee that the final robotic system will be secure and thereby, safe. As illustrated in the example above, robotics involves a relevant degree of system integration and inter-connectivity (non-isolated embedded systems connected together internally and potentially, externally as well). As such, both secure and ultimately safe robotics systems do not only need to ensure quality by complying against coding standards but also guarantee that they aren’t exploitable by malicious attackers.
In the traditional view of system security, safety in often understood as "nothing bad happens naturally" while security intuitively indicates that "nothing bad happens intentionally". Acknowledging the acceptance of this view in the security community, this thesis puts special focus in the context of robotics To further understand terminology and prior art in a robotics context, Table 2.1 presents a summary of the concepts discussed with their interpretation applied to robotics and the corresponding sources used:

Security, as understood in Table 2.1 shares Integrity with Safety. As discussed in , "the only thing that distinguishes the role of integrity in safety and security is the notion of exceptional condition. This reflects the fact that exception conditions are perceived as accidental (safety hazards) or intentional (security threats)". The later, security threats, are always connected to vulnerabilities. A vulnerability is a mistake in software or hardware that can be directly used by an arbitrary malicious actor or actress to gain access to a system or network, operating it into an undesirable manner . In robotics, security flaws such as vulnerabilities are of special relevance given the physical connection to the world that these systems imply. As discussed in , "Safety cares about the possible damage a robot may cause in its environment, whilst security aims at ensuring that the environment does not disturb the robot operation. Safety and security are connected matters. A security-first approach is now considered as a prerequisite to ensure safe operations". Figure 2.3 depicts the concepts of Safety, Quality and Security representing their relationships. In particular, safety as a super-set of security and quality.

This thesis focuses on security, however, its relationship with quality and safety must be noted. Security intersects quality in the sense that some methods are shared between both (e.g. testing). Moreover, as discussed, a safe system demands first security and quality.
Robot security vulnerabilities are potential attack points in robotic systems that can lead not only to considerable losses of data but also to safety incidents involving humans. Some authors claim that unresolved vulnerabilities are the main cause of loss in cyber incidents. The mitigation and patching of vulnerabilities has been an active area of research in computer science and other technological domains. Unfortunately, even with robotics being an interdisciplinary field composed from a set of heterogeneous disciplines (including computer science), to the best of the knowledge of this thesis’ author and his literature review, not much vulnerability mitigation research related to robotics has been presented so far.

2.3Cybersecurity Across Various Domains: IT, OT, IoT, and Robotics

Cybersecurity is fundamentally about ensuring systems are protected from digital dangers and threats. This concept, while universally applicable, varies in interpretation and implementation across different technological domains such as Information Technology (IT), Operational Technology (OT), Internet of Things (IoT), and robotics. Each domain, due to its unique technological architecture, necessitates a distinct approach to cybersecurity.

Definitions and Differentiations

In the realm of digital security, various terms have evolved to denote domain-specific aspects:

Information Technology (IT):

Involves the use of computers for data management and is typically employed organization-wide .

Operational Technology (OT):

Focuses on managing industrial operations through monitoring and control of devices and processes .

Industrial Control Systems (ICS):

A crucial segment of OT that includes systems used to oversee industrial processes.

Internet of Things (IoT):

Represents the extension of internet connectivity to various sensors and devices.

Industrial IoT (IIoT):

The application of IoT technologies in industrial settings.

Robotics:

An interdisciplinary domain that integrates sensors, actuators, and computation to create autonomous systems.

The application of cybersecurity varies across IT, OT, IoT, IIoT, and robotics. Each technology, while subject to the overarching goal of operational security, requires a tailored approach. For instance, ICS, evolving from isolated systems to more IT-like structures, have increased vulnerability to cyber threats due to increased connectivity . The confluence of IT and OT in IoT necessitates a unique set of security skills, distinct from those required in traditional IT or OT environments . Robotics, due to its inherent complexity and the integration of various systems, demands domain-specific knowledge and a specialized approach to cybersecurity.

Table 2.2 presents a comparative analysis (extended from ) which reveals distinct requirements and challenges in implementing cybersecurity across these technologies. Notably, robotics stands out due to its critical real-time requirements and safety implications, underscoring the need for a comprehensive and dedicated cybersecurity strategy tailored to robotic systems. Such approach should emphasize the relevance of isochronous and deterministic communications, crucial in robotic systems.

Misunderstandings in Robot Security

A prevalent issue in the robotics industry is the conflation of IT security with comprehensive robotics security. Many manufacturers, such as Mobile Industrial Robots (MiR) as depicted in Figure 2.5, focus predominantly on IT security, neglecting the broader spectrum of threats specific to robotics . This oversight can lead to severe safety hazards, as compromised robots pose risks not just to data privacy but also to human safety and environmental integrity. Therefore, it is imperative for manufacturers to adopt a holistic view of security that encompasses all aspects unique to robotic systems.

Given the safety implications of autonomous systems, and given the security precondition of safety, the need for robust cybersecurity measures in robotics surpasses that in IT, OT, or IoT. The complexity and integration of systems in robotics demand a security approach that is both comprehensive and specific to the domain, with careful consideration for robot’s real-time nature. This thesis advocates for the implementation of such dedicated security measures in the field of robotics.

2.4Machine Learning and Game Theoretic approaches to offensive cybersecurity

Offensive security typically involves a malicious attacker performing a series of steps, often connected, and usually requiring a high degree of expertise or knowledge on the target system. Within offensive security practices, Penetration Testing (pentesting) is an active method for assessing and evaluating the security of digital assets by planning, generating and executing all possible attacks that can exploit existing vulnerabilities. Figure 2.6 pictures the pentesting pipeline for a robotics system inspired on prior work .

As described by several authors , current pentesting practices are becoming repetitive, complex and resource consuming despite the use of automated tools. Morever, they generally require a high level of expertise in those systems targeted. Applebaum et al. also underline the importance of pentesting in the security lifecycle and the shortcomings associated with manual execution. These challenges have made several researchers to turn into Machine Learning (ML) to try and automate the process of pentesting or other security assessments 4 .
While several articles describe the growing popularity of the use of machine learning in defensive approaches for security, already report that very few sources cover the use of Machine Learning for offensive approaches to security. While reviewing literature, the author of this thesis found it extremely helpful to step back a few decades to review previous work that attempted to automate pentesting activities using Artificial Intelligence (AI) approaches. A complete and detailed walkthrough on these past efforts is beyond the scope of this thesis but nicely compiled at . For completeness and reader’s conveniece, the most relevant events on this regard prior to the start of this thesis (2020) have been pictured in the diagram below:

Moving past AI and diving more contemporary ML approaches 5 , in 2018 Niculae presents one of the first pieces available where ML, in particular, Reinforcement Learning (RL) is used to tackle the pentesting assessment task while modeling it as a game. Niculae evaluates different RL techniques, namely Q-Learning (QL), Extended Classifier Systems (XCS) or Deep Q-Networks (DQN) while comparing it with Human performance and multiple fixed-strategies including Random, Greedy and a Finite State Machine. During his work and for his particular simplified model, Niculae reports building an agent that can learn to penetrate a network comparable in strength to a human, paving the way for future work in the application of RL in pentesting.

Also in 2018 (though published later), Schwartz argues that current approaches to automated pentesting have relied on methods which require a model of the exploit outcomes, however the cyber security landscape is rapidly changing as new software and attack vectors are developed which makes producing and maintaining up-to-date models a challenge. Instead and similar to Niculae, Schwartz advocates for the application of RL and evaluates QL and DQL obtaining positive results. Interestingly however, Schwartz remarks that the implemented algorithms in the simulated scenarios were only practical for smaller networks and numbers of actions and would not be able to scale to truly large networks.

The most recent studies available on this domain are and . The first one explores the design and development of an ML-based pentesting system while supported by human experts. The article itself provides an interesting motivation for robotics, but does not provide a clear description of the underlying RL techniques, the reward function, or a detailed reasoning on how the state space and the action spaces of the pentesting activity were captured. Moreover, the authors conclude their paper highlighting that the main drawback of their contribution is the need of a high-caliber human expert supervising the initial training. Zhou et al present a new method for pentesting using RL. Particularly, they propose an attack planning algorithm based on RL to automatically discover attack paths without prior knowledge of the scenario network. Opposed to , this last article provides a good review of the pentesting foundations, underlying math formalizing pentesting as a Markov decision process and a detailed description of the assumptions made for representing the problem in reinforcement learning.

Despite the advances on ML, scalability remains a problem for automated pentesting across network layers, even when powered with modern techniques such as the ones described above. Scalability can be understood easily by switching across (attack) subjects and/or determining next steps or actions during a pentesting offensive effort. It is unclear whether the current trend of ML research (mostly driven by increasingly complex RL constructs) can help lead towards an end-to-end ML-driven penetration testing. The research conducted herein should tap into this problem and propose novel approaches. Beyond the use of ML for addressing the scalability problem in pentesting, another interesting issue is selecting the right attack targets. More specifically, selecting which attack vectors (corresponding to individual targets) should be exploited first. When conducting an offensive pentesting exercise, commonly, multiple targets appear over the exercise. For each attack target, multiple attack vectors are often identified. Selecting which attack vector to prioritize first can be somewhat derived by using scoring metrics, like the Common Vulnerability Scoring System (CVSS) . However, in practice, pentesters use this metric as yet one more indicator to build an intuition on where to attack. Developing this intuition is complex and not fully understood. This thesis will also look at this problem by looking at Game Theory (GT). Computational game theory can also help tackle the scalability problem and schedule resources while taking into account the importance of different targets, the responses of the adversaries to the security posture and the potential uncertainties in adversary payoffs and observations. The combination of both, machine learning and game theoretic approaches might eventually lead to results that empower autonomous and versatile red team assessments, first envisioned by Applebaum back in 2016.
A deeper study of the biographical cornerstones for AI in pentesting will be conducted in Chapter 11. To the best of this author’s knowledge and extend of literature review, at the time of the start of this research (2020), no single article covered the use of ML and GT methods to secure robots via an offensive approach 6 . This presents an interesting opportunity which is explored throughout this research.

Ethics

3Ethical Landscape in Robot Cybersecurity

3.1Introduction

This chapter, lying within the larger framework of ethical considerations in cybersecurity for robotics, delves into the complex landscape of this evolving field. The objective is multifaceted: to map out the current ethical standards within the robot cybersecurity domain, dissecting how these standards manifest in key application areas. This exploration goes beyond merely cataloging vulnerabilities and manufacturers’ responses; it also seeks to understand the perceptions and attitudes of robotic users and vendors toward cybersecurity. This understanding is garnered through surveys distributed across popular robotics communities, thus capturing a wide range of insights and sentiments.

An important aspect of this chapter is its critical examination of the established norms and standards in robot cybersecurity. It challenges and scrutinizes these norms, particularly focusing on the ethics of disclosure policies. This includes the expectations placed on vendors regarding the notification of research findings and vulnerabilities—a practice commonly known as responsible disclosure. Furthermore, the chapter probes into the more controversial facets of this topic, such as the ethical implications of disclosing security flaws at all, drawing inspiration from previous critiques like those of Rescorla (2005) .

To achieve a comprehensive understanding of the state of robot cybersecurity, this chapter employs a systematic review approach that unfolds in three distinct phases. Initially, it delves into the existing body of literature, painting a picture of the current cybersecurity landscape in robotics. Following this, the chapter presents findings from surveys conducted within various robotics groups and communities, offering a grassroots perspective on the state of cybersecurity in this field. The final phase of this exploration is anchored in empirical data, drawn from three years of proactive security research in robotics. This phase not only discusses identified vulnerabilities but also reflects on the offensive security exercises undertaken during this period.

The culmination of this chapter is a synthesis of insights and conclusions drawn from these diverse sources, including a discussion of the current ethical standards within the robot cybersecurity domain. It aims to provide actionable guidance on securing robotic systems, understanding their vulnerability to various attack vectors, and strategies to minimize their exposure to potential threats. This chapter thus serves as a critical piece in understanding the ethical underpinnings of cybersecurity in robotics, laying the groundwork for responsible and secure advancements in this dynamic and impactful field.

3.2Background

As introduced before, arguably, the first installation of a cyber-physical system in a manufacturing plant dates back to 1962 . The first human death attributed to a robotic system occurred in 1979 , with safety-related issues cited as the cause. This incident sparked a series of actions by agencies and corporations aimed at protecting humans and environments from these machines, leading to the development of safety standards. However, security in robotics has only started to be addressed more recently. Following after McClean’s (2013) early assessment, one of the initial published articles on the topic by Lera et al. (2016) warned about the security dangers of the Robot Operating System (ROS) . Subsequently, the same group in Spain authored a series of articles focusing on robot cybersecurity . Around the same time, Dieber et al. (2016) led research into cybersecurity in robotics, proposing defensive blueprints for ROS-based robots . Their work included enhancements to ROS APIs to incorporate modern cryptography and security measures. Concurrent with Dieber et al.’s work, White et al. (2016) also began producing a series of articles proposing defensive mechanisms for ROS .

A little over a year later, starting in 2018, more groups began to contribute to the field. Mayoral-Vilches et al. (2018) initiated a series of security research efforts aimed at defining offensive security blueprints and methodologies in robotics, resulting in various contributions . Notably, this group released a framework for conducting security assessments in robotics , a vulnerability scoring mechanism for robots , a CTF environment for robotics to train cybersecurity engineers , and a robot-specific vulnerability database , among other contributions. In 2021, Zhu et al. published a comprehensive introduction to this emerging topic, aiming to foster a sub-community in robotics for cybersecurity . These contributions correspond to the bibliographical milestones in robot cybersecurity presented in Chapter 2.

A careful review of the prior art described in the preceding paragraphs leads to the following observation: based on the literature, robot cybersecurity is still a nascent field that requires further attention, tools, and educational material to train new engineers in security practices for robotics.

3.3Surveying security in robotics communities

During a period of three years (2019 - 2021) various security surveys were conducted in top robotic communities through their corresponding conferences and forums. The following subsections discuss each one of them while attempting to draw some observations:

Surveying the ROS community

Figure 3.1 presents a summarized result of the survey conducted in the ROS community during a period of several months7. The survey received a total of 52 responses, which represented the small interest in security at the time. The largest groups of participants are depicted in Figure 3.1b. The most represented group comes from Universities (30%), followed by Software vendors (18%) and Robot manufacturers (14%) 8 . The majority of the respondents have at last 2 years of experience with ROS and half of them at least 5 (3.1c), most coming from Europe (3.1d). Figure 3.1e present data on security considerations. The data indicates that 73% of the participants think that they have not invested enough to protect their robots from cyber-threats. Coincidentally, the same number of participants indicated that their organizations are open to invest however only 26% acknowledge to actually have invested. This data leads to the following observation:

When considering the mitigation strategies applied by respondents as depicted in Figure 3.1f, it’s important to highlight that most efforts concentrate on perimeter actions (i.e. firewalls, segmentation and segregation) whereas robot-specific defensive solutions are only applied in a 36% of the cases. Similarly, network assessments and security audits are conducted only in one fourth of the cases (26%) which conflicts with the de facto security practices in other industries, wherein assessments are critical to evaluate the resilience of technology.

Figure. Surveying the ROS robotics community (2019), from

Surveying the PX4 community

PX4 is an open source flight control software for drones and other unmanned vehicles. Similar to ROS, its community represents another relevant group in robotics. A security survey was conducted in 2020 and the results are summarized in Figure 3.29. Though the PX4 community is significantly smaller than ROS’s, the sample size obtained (11 respondents) was extremely small to draw major conclusions. Interestingly though, it was observed that the majority of the respondents have yet to see a security issue impacting the community (3.2d), only 27% had seen it.

The majority of the respondents (81%, figure 3.2e) indicated to be willing to invest and more than 90% confirmed that the amount could be 100 USD or above (Figure 3.2f). This aligns nicely with observation 3.3.1 and further hints that growth should be expected in this field.

Figure. Surveying the PX4 robotics community (2020), adapted from .

Surveying the ROS-Industrial community

Also in 2020, a series of security-related surveys were launched as part of the European ROS-Industrial Conference, which happens every year in December. Data collected is presented in Figure 3.3 10 . The majority of the respondents (93%) showed awareness about the threats their robots faced and admitted being aware of their exposure to attackers (Figure 3.3b). Unsurprisingly, as a subset of the overall ROS community, the security mitigation actions in the ROS-I community also concentrate on the perimeter which lead to another observation:

This fact becomes concerning in industrial environments wherein insider threats are as dangerous, and the disruption of ROS could lead to catastrophic consequences for the automation processes , impacting more than 5 robots in 44% of the cases according to respondents (Figure 3.3f). The lack of security measures in ROS are particularly concerning since its distributed communication middleware could be easily used to spread malware across connected robots. Such concept was demonstrated by , which prototyped an instance of ransomware targeting industrial collaborative robots, leaving these machines and its data completely locked until the corresponding ransom is paid.

Figure. Surveying the ROS-Industrial robotics community (2020), adapted from .

Surveying the European robotics community at the European Robotics Forum (ERF) (2020)

As one of the leading geographies in robotics and cybersecurity, the opinion of european robotics experts was sampled during the annual European Robotics Forum (ERF). Figure 3.4 summarizes the most relevant data collected 11 . The most interesting observation relates to the question "Who is the actor to be responsible for cyber-incidents?":

Figure. Surveying the European robotics community (ERF 2020), adapted from .

3.4Security research results in robotics

Figure 3.5 depicts summarized vulnerability research results for three vendors: ABB, Mobile Industrial Robots (MiR) and Universal Robots (UR). The data was collected and archived over a multi-year period. Figures 3.5a, 3.5b and 3.5c illustrate the "days until mitigation" for each vulnerability and according to the public data in the Robot Vulnerability Database (RVD) . The flat line represented by a series of data points in figures 3.5b and 3.5c denotes that the vendor hasn’t reacted yet to any of these flaws and they remain unmitigated (they are zero days). For ABB robots, the scattered plot in Figure 3.5a denotes more security activity. The following observations are drawn from the data:

On top of these, Figures 3.5d to 3.5i enhance previous data with additional private sources of information and consider vulnerabilities that have yet to reach the public domain. It should be noted that the distribution of vulnerabilities signals the security awareness of the manufacturer. Coherently, figure 3.5g shows how for ABB robots, four out of five vulnerabilities considered have been publicly disclosed, triaged and scored. In contrast, for MiR and UR robots the oppositive is observed. Four out of five vulnerabilities have yet to be disclosed publicly.

Figure. Vulnerability data for various robots, adapted from .

3.5Discussion

Ethical Considerations During Surveys

The surveys conducted within various robotics communities were aimed at gathering insights into the state of cybersecurity awareness, practices, and concerns within these communities. While the primary focus was on data collection and analysis, ethical considerations surrounding the conduct of these surveys are essential to address.

1. Ethical Screening: Prior to launching the surveys, ethical screening was performed to ensure that the data collection process adhered to established ethical standards. This screening involved a review of the survey content, methodology, and potential risks to participants.

2. Informed Consent: Participants were provided with clear and comprehensive information regarding the purpose, storage, and use of their personal data. This information was presented at the beginning or within the announcement of each survey (as examples, see announcement for the ROS and PX4 survey), and participants were required to provide informed consent before proceeding.

3. Confidentiality and Anonymity: Measures were implemented to ensure the confidentiality and anonymity of participant responses. Personal identifying information was kept separate from survey responses, and only aggregated data was used for analysis and reporting.

4. Data Storage and Use: Participant data was securely stored and used solely for the purposes outlined in the survey introduction. Data was accessible only to authorized researchers involved in the study and was not shared with third parties.

5. Feedback and Accessibility: Participants were provided with opportunities to provide feedback on the survey process and to access the results of the study upon request. Transparency regarding data handling practices fostered trust and accountability among participants.

6. Conference Context: It’s important to note that some surveys were conducted within the context of conferences, where participants may have been attendees or members of the respective communities. While this facilitated data collection, it also presented unique challenges in terms of informed consent and data privacy.

7. Limitations and Future Considerations: Despite efforts to uphold ethical standards, it’s important to acknowledge the limitations of the survey methodology and consider potential improvements for future research endeavors. This includes exploring alternative data collection methods, such as interviews or focus groups, to ensure a more nuanced understanding of participants’ perspectives.

In summary, ethical considerations were integral to the design and implementation of the surveys conducted within robotics communities. By prioritizing participant welfare, confidentiality, and transparency, the integrity of the research process was upheld, facilitating meaningful insights into cybersecurity practices and concerns within these communities.

Vulnerability Disclosure Policies

Prevailing ethical standards and practices in robot cybersecurity are still on their early stages however, after a few years of research and while interacting with various communities, a few remarks can be made. The subsections below cover some of the common practices observed in robot cybersecurity and discusses their ethical implications.

Evolution of Disclosure Policies in Robotics

The landscape of vulnerability disclosure policies in robotics has undergone notable evolution, particularly influenced by the practices adopted by different organizations and the ethical implications of these choices. This subsection examines three significant contributions in the field, each highlighting different approaches and their resultant impact on the cybersecurity landscape in robotics.

Alias Robotics’ 90-Day Disclosure Deadline

In 2018 and 2019, Alias Robotics pioneered early work on vulnerability disclosure policies in robotics. This was part of their effort to establish the Robot Vulnerability Database (RVD) . This approach adhered to a 90-day disclosure deadline for newly identified vulnerabilities. The rationale behind this timeframe was to balance the urgency for patch development against vendors’ capability to respond effectively. The 90-day deadline aimed to push vendors towards improving their internal processes for vulnerability remediation, which historically had seen delays or even neglect. Alias Robotics’ approach was based on the principle that quicker response times to disclosed vulnerabilities would ultimately result in better security for users.

Open Robotics’ ROS 2 Vulnerability Disclosure Policy

Open Robotics, in contrast to Alias Robotics’ proactive approach, introduced a ROS 2 Vulnerability Disclosure Policy in 2020, which notably did not incorporate relevant feedback from the Security Working Group and derived from Alias Robotics’ initial work. The policy lacked crucial elements like disclosure deadlines, transparent communication, and acknowledgment of the educational value of post-mitigation security research. It diverged from established international security practices, such as assigning CVE IDs, and depended on a non-transparent process for security issue verification. While the policy mentioned a safe harbor, its actual efficiency and effectiveness were questioned. Open Robotics’ approach suggested a missed opportunity to prioritize cybersecurity within the ROS community, reflecting a potential lack of commitment to rigorous security standards.

Security Research Groups’ Direct Coordination with International CERTs

The ROS 2 Vulnerability Disclosure Policy’s limitations led several security research groups to bypass it in favor of direct coordination with international Computer Emergency Response Teams (CERTs). One of such groups included Alias Robotics contributions again and identified relevant vulnerabilities affecting a significant number of ROS robotic endpoints and disclosed them responsibly after meeting reasonable deadlines . The public disclosure got further disseminated in the ROS robotics community (the one owned by Open Robotics) and empowered end-users with knowledge to mitigate outstanding security issues, in contrast to Open Robotics’ response, which downplayed the significance of these disclosures and shifted responsibility to third parties 12 . This direct approach with CERTs highlighted a more efficient and impactful way of handling cybersecurity in the robotics ecosystem, as opposed to working with Open Robotics (the acting vendor, as community responsible).

Ethical Implications of Disclosure Policies

Necessity of Disclosure Deadlines

The implementation of disclosure deadlines by Alias Robotics was driven by ethical considerations related to the timeliness of vulnerability patching. Slow patch timelines pose a significant risk, as attackers could potentially exploit vulnerabilities before they are patched. The 90-day deadline was designed to incentivize vendors to improve their response times, balancing the need for thorough patch development with the urgency to protect users from potential exploits.

Transparency and Educational Value of Disclosures

Open Robotics’ policy, which lacked transparency and discouraged the disclosure of security research for educational purposes, overlooked the long-term benefits of sharing detailed vulnerability information. Disclosing technical details post-patch offers educational value to the security community, helping to inform and strengthen overall cybersecurity practices. This approach contributes to a more informed and capable pool of security professionals and researchers.

Community-Centric versus Vendor-Centric Approaches

The different policies reflect a divide between community-centric and vendor-centric approaches to cybersecurity. Alias Robotics’ policy, with its emphasis on timely public disclosures, aligns with a community-centric approach that prioritizes user security and education. In contrast, Open Robotics’ policy appears more vendor-centric, potentially limiting the flow of critical security information to the wider community and impeding collaborative efforts to address vulnerabilities.

Conclusion: The Ethical Dimension of Cybersecurity in Robotics

The exploration of vulnerability disclosure policies in robotics, particularly those of Alias Robotics and Open Robotics, offers a vital perspective on the ethical dimensions of cybersecurity in this domain. These cases illuminate the divergent paths that organizations can take in addressing security vulnerabilities, each path carrying its own ethical implications and impact on end-users.

User-Centric Approach and Its Importance

Alias Robotics’ approach, with its strict 90-day disclosure deadline, embodies a user-centric methodology. This policy not only impels vendors to act swiftly but also aligns with a broader ethical stance that prioritizes the safety and security of end-users. By setting clear deadlines and making vulnerability information public, Alias Robotics upholds the principle of transparency, empowering users with knowledge and means for protection. This approach is demonstrative of an ethical commitment to the community, where the security and well-being of users are placed at the center of cybersecurity practices.

Vendor-Centric Approach: A Cautionary Tale

On the other hand, Open Robotics’ less prescriptive ROS 2 Vulnerability Disclosure Policy, which lacks clear deadlines and a transparent process, reflects a vendor-centric approach. This policy, while ostensibly designed to manage vulnerabilities, falls short in fostering a proactive security culture within the community. Its lack of urgency and transparency may inadvertently lead to prolonged vulnerability exposure, thereby increasing risks for end-users. This approach can be seen as ethically problematic, as it does not adequately address the immediate security needs of the community and potentially places the interests of vendors above those of the users.

Implications for End-Users

The contrasting policies and their implementation have direct implications for end-users, particularly in terms of their exposure to potential cyber threats. A policy that enforces strict deadlines and encourages open communication about vulnerabilities serves to enhance the overall security posture of the robotics ecosystem. It acknowledges the evolving nature of cyber threats and the importance of keeping users informed and equipped to respond to these threats. In contrast, a policy that lacks these elements may leave end-users vulnerable and uninformed, undermining the trust and reliability essential in the field of robotics.

Security Researcher Perspective: Navigating Ethical and Practical Challenges

The exploration of vulnerability disclosure policies in the context of robotics also brings to the forefront the unique challenges faced by security researchers. These individuals, who play a pivotal role in identifying and reporting vulnerabilities, often navigate a landscape fraught with ethical dilemmas and practical hurdles. A significant concern in this realm is the tendency of some vendor-centric disclosure policies to pressure researchers into revealing their findings without proper recognition, compensation, or, at times, under the threat of legal action.

Compromised Incentives and Recognition

Security researchers invest significant time and effort in uncovering vulnerabilities, with the primary goal of enhancing system security and user safety. However, the expectation set by certain disclosure policies to give away this valuable information, often without compensation or adequate acknowledgment, can be demoralizing. This lack of incentive not only undermines the researchers’ contributions but also potentially devalues the importance of their work. In an environment where recognition and fair compensation are lacking, the motivation to engage in this critical work can diminish, leading to a scarcity of skilled individuals willing to dedicate their efforts to cybersecurity research in robotics.

Legal Threats and the Ethical Dilemma

The situation becomes even more complex when legal threats enter the equation. Researchers, in their pursuit to enhance cybersecurity, may find themselves in a precarious position when their findings are met with hostility or legal intimidation from vendors. This scenario presents an ethical quandary: on the one hand, researchers have a moral obligation to disclose vulnerabilities to protect end-users; on the other hand, they face potential legal repercussions for doing so. Such an environment can create a chilling effect, discouraging researchers from participating in vulnerability discovery and reporting, ultimately hindering the advancement of cybersecurity in the robotics field.

Impact on the Security Landscape

The cumulative effect of these challenges is a security landscape where the vital contributions of researchers are not adequately valued or supported. This situation can lead to a reduction in proactive security research, leaving vulnerabilities undiscovered and unaddressed. Consequently, the overall security posture of robotic systems may be weakened, exposing end-users to increased risks. To foster a more robust and resilient cybersecurity environment, it is essential to recognize and support the critical role of security researchers. This support can manifest in various forms, including fair and transparent policies, legal protections, recognition of their contributions, and avenues for compensation or incentives.

The Way Forward

In conclusion, the ethical landscape of cybersecurity in robotics is one where user safety, transparency, and proactive engagement must be paramount. As robotics continue to integrate more deeply into various aspects of society and industry, the responsibility to ensure robust and ethical cybersecurity practices becomes increasingly critical. The experiences of Alias Robotics and Open Robotics serve as instructive examples, highlighting the necessity of policies that are not only technically sound but also ethically grounded. Such policies should aim to protect users, foster an informed and engaged community, and uphold the highest standards of safety and security in the rapidly evolving world of robotics.

Lack of learning resources

One of the significant challenges in the realm of robot cybersecurity is the lack of comprehensive learning resources. This shortage is exacerbated by the current vulnerability disclosure policies of various actors and groups, such as Open Robotics, as discussed above. These policies, often more vendor-centric than community-driven, do not facilitate the generation of educational content that could foster a deeper understanding of cybersecurity in robotics. This gap in knowledge and resources is particularly concerning given the increasing integration of robotics into various facets of society and industry.

In response to this void, the Robot Hacking Manual (RHM) , authored as part of this research, emerges as a crucial resource. The RHM is an introductory series about cybersecurity for robots, conceived with the intent to raise awareness in the field and underscore the importance of adopting a security-first approach. The manual offers comprehensive case studies and step-by-step tutorials, designed to provide a foundational understanding of robot cybersecurity. The content within the RHM is independent and not aligned with any particular organization, reflecting a genuine effort to democratize knowledge in this critical area.

An integral part of the RHM is the robot hacks list displayed in table 3.1. This list is a non-exhaustive compilation of cybersecurity research in robotics, detailing various robot vulnerabilities and attacks due to cybersecurity issues. The list serves as a testament to the evolving landscape of threats in robot cybersecurity and underscores the necessity for continued vigilance and education in the field.

3.6Conclusion

This chapter, situated within the ethical framework of cybersecurity in robotics, has navigated through the intricate landscape of robot security, revealing its multifaceted nature and current state. The exploration, driven by the goal to understand and critique the prevailing ethical standards and practices in robot cybersecurity, delved deep into various dimensions of this field.

The findings from the systematic review of literature, combined with insights from surveys conducted across prominent robotics communities, have highlighted several key observations. A noticeable gap exists between the anticipated and actual investment in cybersecurity within the robotics domain, particularly in the ROS community (Observation 3.3.1). This gap not only underlines the potential growth area in cybersecurity actions but also mirrors the nascent stage of maturity in this field compared to established sectors like IT or OT (Observation 3.3.2).

A surprising revelation was the majority of the robotics community’s lack of exposure to cyber-attacks (Observation 3.3.3), suggesting a lower perceived threat level. This could potentially lead to a complacency in adopting rigorous cybersecurity measures. Furthermore, the tendency to focus on perimeter defense strategies (Observation 3.3.4), as noted in both ROS and ROS-I communities, signals a possible oversight of insider threats, particularly crucial in industrial environments where robotics systems are integral.

The responsibility for cyber incidents in Europe is largely attributed to the supply chain, with system integrators and robot vendors bearing the brunt of the blame, rather than the end-users (Observation 3.3.5). This points to an ethical standpoint where the onus of security is more on those who manufacture and integrate rather than those who operate.

The situation with collaborative robot manufacturers like MiR and UR is particularly alarming, as they exhibit a pattern of unmitigated zero-day vulnerabilities (Observation 3.4.1). This inaction contrasts sharply with the approach of manufacturers like ABB, who show a varied response to cybersecurity issues (Observation 3.4.2). The ratio of publicly disclosed vulnerabilities to those kept private further serves as an indicator of a manufacturer’s readiness and approach towards cybersecurity (Observation 3.4.3).

Table 3.2 offers a summary of observations from robotics security surveys and research. Overall, this chapter has underscored the importance of re-evaluating and reinforcing cybersecurity measures in the rapidly evolving field of robotics. It advocates for a shift from traditional perimeter defense strategies to more comprehensive approaches that consider insider threats and embrace practices like zero-trust. The observations made here emphasize the need for increased investment and focus on robot-specific security measures and offensive security assessments. This proactive approach is crucial to enhance the resilience of robotic systems and to bridge the gap between current practices and the evolving cybersecurity landscape in robotics.

This critical examination of the ethical standards in robot cybersecurity not only sheds light on the current state of affairs but also sets the stage for future advancements. One of the central themes uncovered is the critical role of vulnerability disclosure policies in shaping the ethical landscape of robot cybersecurity. As highlighted in the discussion, there is a stark contrast between the user-centric approach and the vendor-centric approach. The former underscores the ethical imperative of prioritizing end-user safety and system security through timely and transparent disclosure of vulnerabilities. In contrast, the latter’s approach, characterized by a lack of urgency and transparency, potentially compromises the security and trust in robotic systems. By challenging established norms and scrutinizing practices like responsible disclosure, this chapter contributes to shaping a more ethical landscape and secure future in robotics, where the safety and integrity of both humans and machines are paramount.

Blueprints

4Cybersecurity in ROS

4.1Introduction

A robot is a network of networks . One that is comprised of sensors to perceive the world, actuators to produce a physical change, and computational resources to process it all and respond coherently, in time, and according to its application. Security is of paramount importance in this context, as any disruption of any of these robot networks can cause the complete robotic system to misbehave and compromise the safety of humans, as well as the environment .

The Robot Operating System (ROS) is the de facto framework for robot application development. Widely used to govern interactions across robot networks, at the time of writing, the original ROS article has been cited more than 9300 times, which shows its wide acceptance for research and academic purposes. ROS was born in this environment: its primary goal was to provide the software tools and libraries that users would need to employ to undertake novel robotics research and development. Adoption in industry has also been rapidly increasing over the last few years. According to the latest ROS community metrics sampled every year in July, the number of ROS downloads has increased by over 50%, with about 600 million downloads between July of 2020 and July of 2021. Moreover, based on the download percentages reported from packages.ros.org, one can observe a significant increase in adopting ROS 2, which suggests that by 2023 there would be more users using ROS 2 than its predecessor 13 .

ROS was not designed initially with security in mind, but as it started being adopted and deployed into products or used in government programs, more attention was drawn to security issues. Some of the early work on securing ROS included or , both appearing in the second half of 2016. At the time of writing, none of these efforts remain actively maintained and the community focus on security efforts has switched to ROS 2. A recent study that surveyed the security interests in the ROS community presented data indicating that 73% of the survey participants considered that they had not invested enough to protect their robots from cyber-threats. The same number of participants indicated that their organizations were open to invest, however only 26% acknowledged to actually have invested. This led the original authors to conclude that there is a gap between the security expectations and the actual investment. In this work, it’s argued that this gap is the result not only of the immaturity of security in robotics or the know-how but also by the lack of usability of the available security tools. Being conscious that security in robotics is not a product, but a process that needs to be continuously assessed in a periodic manner , this work advocates for a usable security approach in robotics as the best way to remain secure.

In this chapter we introduce SROS2, a series of developer tools, meant to be usable and that facilitate adding security capabilities to ROS 2 computational graphs. We present in here a security methodology consisting of six steps that allow securing ROS 2 graphs iteratively, with the aid of SROS2. Driven by an application use case, we discuss how SROS2 allows achieving security in complex graphs involving popular ROS 2 packages and analyze the security trade-offs and limitations of the current tooling. The key contributions of this work are:

• Create SROS2, a set of usable tools for adding security to ROS 2 that: (1) help introspect the computational graph by extracting communication middleware-level information; (2) simplify the security operations creating Identity and Permissions Certificate Authorities (CA) that govern the security policies of a ROS 2 graph; (3) help organize all security artifacts in a consistent manner and within a directory tree that is generated within the current ROS 2 workspace overlay; (4) help create a new identity for each enclave, generating a keypair and signing its x.509 certificate using the appropriate CA; (5) create governance files to encrypt all DDS traffic by default; (6) support specifying enclave permissions in familiar ROS 2 terms which are then automatically converted into low-level DDS permissions; (7) support automatic discovery of required permissions from a running ROS 2 system; and (8) dissect communication middleware interactions, to extract key information for the security monitoring of the system.

• Propose a methodology for securing ROS 2 computational graphs that provides roboticists with a structured process to continuously assess their security.

• Expose insights into how to apply SROS2 to real ROS 2 computational graphs by presenting an application case study focused on analyzing the Navigation2 and SLAM Toolbox stacks in a TurtleBot3 robot.

The core components of SROS2 are disclosed under a commercially friendly open-source license and are available and maintained at https://github.com/ros2/sros2.

4.2Background

Considering how ROS was originally intended as a fast prototyping robotic framework, security was not considered a priority feature in its first iteration. As ROS has evolved from the prototyping to the real-world industrial applications the entire stack came to be in dire need of cybersecurity attention .

A first partial analysis, with the goal of understanding what prevented ROS from being used industrially, was conducted by McClean et al. . By means of a ’honeypot’ system, at DEFCON-20, they collected how malicious users would tackle a robot in the wild. Dieber et al. provided a complete and in-depth analysis of the security vulnerabilities and attack surfaces in ROS systems and how to exploit them, highlighting the gaps in the security of the framework. A considerable amount of research has been done as regards the publish-subscribe paradigm reviewing the performance and the techniques to secure it either via the communication channel, and ROS’ internal mechanisms . In the first case, via message authentication , within the later addition of using of encryption and security artifacts . In the latter case, by enhancing the middleware behaviour with some extra, such as a run-time monitor to filter out and log all the requests and operations sent in the graph , an Application Level Gateway - that wraps the existing API calls to enforce authentication and authorization - that exposes a permission token to be evaluated before executing the requested operation ; to the extent of forking the implementation, modifying the transport mode via IPSec , or via a security architecture intended with the addition of x.509 certificates and authorization server . Unfortunately, those approaches suffered to some extent with limitations and downsides, such as a lack of flexibility (e.g. Single Point of Failure (SPOF)) and usability, which were tackled in the Secure ROS (SROS) initiative . With the objective of providing additions to the ROS API and ecosystem to support modern cryptography and security measures, the project introduced new security features to the core of ROS’ codebase and, more importantly, a set of tooling to ease the burden on the developers of correctly implementing security.

With the second iteration of the framework in ROS 2, thanks to the adoption of DDS as the communication middleware 14 , we observed how the inherited security measures and methodologies in the system have evolved the framework. However, we can no longer overlook how its complexity still remains prone to human error in processes such as the access control artifacts distribution , or even to overlooking exposed attack surfaces . Moreover, keeping track of all the new pieces to the ROS puzzle became even more demanding and lengthy procedurally, requiring continuous attention and systematic security analysis–which left usability challenging . The work presented here addresses this challenge with a security toolset (SROS2) and a security methodology for robotics.

4.3Approach

A methodology inspired by the DevSecOps approach is proposed to secure ROS 2 computational graphs: (A) introspect the graph and model its security landscape to determine the necessary security policies and enclaves; implement such policies by (B) defining the authentication and (C) authorization configurations; (D) generate all the required security artifacts; (E) deploy them appropriately across robotic systems; and (F) continuously monitor the network, reverting to (A) modeling when appropriate.

Modeling

Modeling refers to the use of abstractions to aid in a thought process. In security, threat modeling aids in thinking about risks and determines the threat landscape. The output of this effort is often called the threat model. Commonly, a threat model enumerates the potential attackers, their capabilities, resources and their intended targets. In the context of robot cybersecurity, a threat model identifies security threats that apply to the robot and/or its components 15 while providing means to address or mitigate them for a particular use case. A threat model also provides inputs that are used to then determine a set of policy rules (or principles) that direct how ROS 2 should provide security services to protect sensitive and critical graph resources. When put together these policy rules are called the security policy.

SROS2 aims to provide tools to introspect and model the security of ROS 2 computational graphs into the desired security policies. Introspection of the graph can be performed in two ways:

1. By leveraging the ROS 2 API and the framework for ROS 2 command line tools (ros2cli), we can pull ROS Nodes, Topics, Services or Actions information (among others) from the ROS 2 graph and display these in the CLI, see Listing 4.1. This allows us to get a grasp of the computational graph from a ROS 2 perspective. Other tools such as RViz or rqt help get a visual depiction of the graph and its abstractions.

2. Monitoring network interactions at the ROS communication middleware-level can be extremely helpful to model security but incredibly cumbersome from a usability perspective unless the right tooling is provided. ROS 2 uses OMG’s Data Distribution Service (DDS) as its default communication middleware, which is a complex specification. To facilitate introspection of DDS, SROS2 leverages scapy , a powerful interactive packet manipulation library that allows us to forge or decode network packets. Particularly, we contributed an open source scapy dissector 16 that allows us to dissect the wire-level communication protocol that is used by the default ROS 2 communication middleware: the Real-Time Publish Subscribe protocol (RTPS) . Using this, SROS2 provides tooling that allows monitoring network interactions, capturing DDS databus information directly and displaying these for the security analyst’s consumption.

# ROS 2 CLI API allows direct introspection
ros2 topic list
/cmd_vel
/robot_state_publisher
...
ros2 node list
/turtlebot3_diff_drive
...
# SROS2 extensions allow introspecting DDS
ros2 security introspection
DDS endpoint detected (hostId=16974402, appId=2886795267, instanceId=10045242)
    - version: 2.4
    - vendorId: ADLINK - Cyclone DDS
    - IP: 192.168.1.34
    - transport: UDP
DDS endpoint detected (...)

For complete threat modeling, we refer the reader to which discusses details around security modeling ROS 2 computational graphs.

Authentication

Authentication provides proof of a claimed identity (\(\neq\) identification, determination of an unknown entity). ROS 2 offloads authentication to its underlying communication middleware, DDS. By default, DDS allows any arbitrary DomainParticipant to join any Domain without authentication. DDS however provides the means to verify the identity of the application and/or the user that invokes operations on the DDS databus through its DDS Security extensions . With these, for protected DDS Domains, a DomainParticipant that enables authentication will only communicate with other DomainParticipants that also have authentication enabled.

To favour usability and reduce human errors, all implementation details of authentication in ROS 2 through DDS are abstracted away by the SROS2 tools. The appropriate artifacts for enabling authentication capabilities are produced in the Generation step (4.3.4) of the DevSecOps methodology, and default to the security mechanisms specified by OMG’s DDS Security . In particular, each DomainParticipant uses a Public Key Infrastructure (PKI) with a common shared Certificate Authority (CA): Identity CA. All participants interoperating securely must be pre-configured with Identity CA and have a signed certificate from it. Participants are expected to use mutual authentication through a challenge-response mechanism supported by either the Rivest Shamir Adleman (RSA) or the Elliptic Curve Digital Signature Algorithm (ECDSA) asymmetric encryption algorithms. Shared secrets are established using using the Diffie-Hellman (DH) or Elliptic Curve DH (ECDH) (Ephemeral Mode) key agreement protocols.

Listing 4.2 shows an example of how SROS2 tools abstract the complexity of DDS authentication away from ROS developers. The governance.xml policy document is auto-generated by SROS2 and captures domain-wide security settings that include authentication aspects. Additional details about the underlying authentication process and the security artifacts are available in , and .

...
<allow_unauthenticated_participants>false</allow_unauthenticated_participants>
<enable_join_access_control>true</enable_join_access_control>
<discovery_protection_kind>ENCRYPT</discovery_protection_kind>
<liveliness_protection_kind>ENCRYPT</liveliness_protection_kind>
<rtps_protection_kind>SIGN</rtps_protection_kind>
<topic_access_rules>
    <topic_rule>
        <topic_expression>*</topic_expression>
        <enable_discovery_protection>true</enable_discovery_protection>
        <enable_liveliness_protection>true</enable_liveliness_protection>
        <enable_read_access_control>true</enable_read_access_control>
        <enable_write_access_control>true</enable_write_access_control>
        <metadata_protection_kind>ENCRYPT</metadata_protection_kind>
        <data_protection_kind>ENCRYPT</data_protection_kind>
    </topic_rule>
</topic_access_rules>
...

Authorization

Authorization helps define and verify the policies that are assigned to a certain identity. Access control instead –also called permissions or privileges– are the methods used to enforce such policies. While access control is handled by the DDS implementation, authorization policies need to be defined by the developer. SROS2 helps map these policies from the ROS 2 computational graph to the underlying DDS databus abstractions through two resources: the Permissions CA and a permissions.xml policy document. Listing 4.3 shows an extract from one of the policy documents that defines the authorization profile for a particular ROS 2 Node. Details about how access control is implemented by the underlying communication middleware are discussed in and .

<profile node="turtlebot3_diff_drive" ns="/">
    <xi:include href="common/node.xml"
      xpointer="xpointer(/profile/*)"/>
    <topics subscribe="ALLOW">
      <topic>/cmd_vel</topic>
    </topics>
    <topics publish="ALLOW">
      <topic>odom</topic>
      <topic>tf</topic>
    </topics>
</profile>
<profile node="turtlebot3_imu" ns="/">
...
</profile>

Generation

Modeling (4.3.1), Authentication (4.3.2) and Authorization (4.3.3) steps of our methodology (which follows DevSecOps in robotics ) help define one or multiple security policies. To help implement such policies, SROS2 provides means to automate the generation of the corresponding security artifacts and simplify the translation to the underlying DDS implementation. To do so, SROS2 maps a security policy to an enclave: a set of ROS 2 computational graph resources that operate in the same security domain, use particular Identity CA and Permissions CA authorities, and share the protection of a single, common, continuous security perimeter.

All secure interactions in ROS 2 computational graphs must use an enclave that contains the runtime security artifacts unique to that enclave, yet each Node may not necessarily have a unique enclave. Multiple enclaves can be encapsulated in a single security policy to accurately model the information flow control. Users can then tune the fidelity of such models by controlling at what scope enclaves are applied at deployment. For example, one unique enclave per robot, or per swarm, or per network, etc.

Listing 4.4 shows how SROS2 tools help generate all artifacts to implement a new security policy, inferred directly from the running ROS 2 graph. For a more complex policy that involves multiple enclaves, we refer the reader to .

# Generate a new keystore with Identify and Permission CA keys,
# associated certificates and a authentication structure through
# governance.xml file
ros2 security create_keystore new_keystore

# Inspect current ROS graph and produce a security policy
ros2 security generate_policy new_keystore/my_policy.xml

# Generate all security artifacts necessary to enforce the policy,
# this includes enclaves and the access control permission files
ros2 security generate_artifacts \
  -k new_keystore \
  -p new_keystore/my_policy.xml

Deployment

Deployment is a relevant phase in the DevSecOps methodology and must be also exercised securely. We consider three types of deployments of both artifacts and secure information:

1. Distribution of policy artifacts: the resulting artifacts from the Generation phase (4.3.4) must be securely deployed into the targeted robots and related systems. At the time of writing SROS2 does not provide any particular special utilities to deploy security artifacts. We however direct readers to the ongoing efforts to launch ROS 2 graphs remotely and in multi-machine environments for inspiration.

2. Message authentication: verification of the Message Authentication Codes (MAC) is performed using Advanced Encryption Standard (AES) with Galois MAC (AES-GMAC). DDS security extensions abstract this away from the ROS developer.

3. Encryption of secure DDS interactions: authenticated symmetric cryptography governs all DDS interactions within a security policy using also AES in Galois Counter Mode (AES-GCM). Similar to message authentication, DDS abstracts this away from the developer and is enabled automatically provided that the security policy is configured appropriately.

Monitoring and mitigation

The last phase in the DevSecOps methodology leads to a never ending loop of continuous Monitoring, mitigation (4.3.6) and Modeling (4.3.1). This way, security in ROS 2 computational graphs becomes a moving target, a process–one that demands continuous assessments as changes occur in the robots, the network, or as new security flaws are discovered affecting the running systems.

SROS2 provides tools for monitoring running ROS 2 graphs and detecting possible flaws. Listing 4.5 shows an example:

# monitor the network segment for vulnerabilities
# affecting DDS participants
ros2 security monitor
sniffing the DDS network...
Vulnerable DDS endpoint found (hostId=16974402, appId=2886795267, instanceId=10045242)
    - vendorId: Real-Time Innovations, Inc. - Connext DDS
    - version: 6.0.1.25
    - CVE IDs:
        * CVE-2021-38487
        * CVE-2021-38435

4.4Application and analysis

To apply the DevSecOps methodology, as defined originally in , we demonstrate the application of SROS2 using two of the most commonly used frameworks in ROS 2, the navigation2 and slam_toolbox stacks. Particularly, the Navigation2 project 17 provides a software stack including path planning algorithms and behavioral navigation servers that can be seamlessly integrated with existing sensor perception pipelines, localization and mapping services, and drivetrain velocity controllers to support various mobile robotic applications. While Navigation2 remains mostly agnostic of robotic platforms, we selected the widely accessible and community supported TurtleBot3 18 as the target robot for analysis–consisting of a differential drive, circular base footprint, and ground level 2D scanning LIDAR. The application case study is depicted in Figure 4.1.

To start, we begin with the modeling (Subsection 4.3.1), authentication (Subsection 4.3.2) and authorization (Subsection 4.3.3) phases in order to bootstrap an initial security policy that captures the minimal spanning set of security measures required for the nominal function of the application across the distributed computation graph. We can either first bring up the ROS 2 application under a controlled network environment with security mode disabled, or provision an initial keystore enclave with temporary key-material and only access control governance disabled; the first option includes minimal setup while being more transparent to debug, while the later is advantageous in modeling policies directly from field deployments across untrusted networks.

With the ROS 2 application running, SROS2 can capture the topology of the computation graph to populate the permission profiles within the initial policy, registering each active ROS 2 node and respective topic publication and subscription. A limitation in SROS2’s current snapshot approach however is in accurately modeling more ephemeral resource access events, such as service clients or action requests. While ROS 2’s internal graph API (that SROS2 uses to sample topology measurements) provides a middleware agnostic interface, the observation window is only instantaneous and can easily miss asynchronous resource access events.

Given the graph API limitations, it’s often necessary to iteratively test the generated policy by using it to update the signed permission and governance files and relaunch the application with access control enabled. For moderate to advanced applications such as those relying on Navigation2, permission access denied errors may inevitably be encountered. With ROS 2 however, such events can be logged and aggregated into policy refinement, specifying the node and resource namespaces denied.

After iterative policy refinement, once the tested application is fully functional with enforced access control, the policy can then be further optimized. Such policy optimizations include sorting common permission patterns into smaller sub-profiles, being more manageable to audit and modularly reusable across repeating permission sets in a global policy. We demonstrate it in . This auditing process also provides an opportunity to assess the granularity of the policy as well, from both permission Access Control (AC) and Information Flow Control (IFC) perspectives.

While the minimal spanning set of AC permissions may be optimally secure in terms of the Principle of Least Privilege , it may not be optimally usable for a target application domain. Though most computation graphs in ROS 2 are largely static at runtime, cases where resource namespaces change over the application’s lifecycle do exist. For example, multi-robot systems may fluctuate as agents enter or exit networks for missions or maintenance. Additionally, node namespaces sometimes include sequence numbers to ensure namespace uniqueness. To accommodate such scenarios, permissions could be modified to include wildcards as necessary. While static permissions are straightforward to interpret and less likely to inadvertently introduce policy flaws, wildcarding select permissions provides a usable compromise when required.

When auditing from an IFC perspective, optimizing the policy into assorted enclaves becomes a key consideration. As all ROS 2 nodes composed into a shared process share a common DDS context, they subsequently share the same security enclave or set of permission profiles. This of course is inherently coupled with how the application is architected and to be deployed across a distributed system. As such, security requirements for IFC may then instead dictate aspects of the application’s designs. The degree of granularity of IFC sought then dictates the allotment of enclaves used to contain sub-profiles for the application’s policy.

In the case of Navigation2 and its large degree of coupling and composition of nodes, the planning stack derived from a single source tree is perhaps best relegated to its own enclave, while still being readily separable from any other enclave dedicated to perception or control nodes. Admittedly, such auditing procedures in determining the allotment of enclaves remains rather ambiguous for users, and so presents another area of ergonomics for SROS2 to help automate or advise through formal analysis.

The source code of the demonstrated application case study is available at . The resulting security policies of applying the DevSecOps methodology are also available in the same repository and show various profiles that result from a systematic assessment.

4.5ROS 2 over VPN: Tunneling Communications

Challenges in Securing ROS 2 Communications

ROS 2, built upon the Data Distribution Service (DDS), faces significant security challenges. Despite the existence of DDS Security extensions , these measures are still in their infancy concerning robust security implementations. As outlined in upcoming Chapter 7, modern DDS implementations exhibit numerous vulnerabilities, some of which result from collaborative research by security experts. These vulnerabilities open up potential attack vectors for malicious actors, compromising the integrity of ROS 2 systems.

Furthermore, the DDS Security Specification itself is plagued with unresolved security issues, detailed at https://issues.omg.org/issues/spec/DDS-SECURITY. This list highlights various bugs and loopholes that further exacerbate the security risks associated with ROS 2, making it difficult to assert that ROS 2, in its current state, can be deemed secure.

ROS 2 over VPN: A Layered Security Approach

Given the aforementioned security concerns, a practical solution is to encapsulate ROS 2 and DDS communications within an additional security layer. This approach involves tunneling all ROS 2 communications through a secure channel using a Virtual Private Network (VPN). This method, referred to as ROS 2 over VPN, provides an extra layer of security by encrypting the data transmitted over the network, thereby mitigating the risks associated with potential vulnerabilities in DDS implementations and specifications.

Implementation

The concept of ROS 2 over VPN was researched and successfully demonstrated in the FogROS2 project , which builds upon the original contributions and principles presented earlier at . Figure 4.2 illustrates the architecture employed in this implementation. By routing ROS 2 communications through a VPN, data privacy is enhanced, and the network is safeguarded against eavesdropping and other forms of cyberattacks. FogROS2 relies on a series of extensions of the ROS 2 launch system. We implemented the FogROS2 launch sequence, shown in Fig. 4.2 that starts after processing the launch script.

The steps FogROS2 takes are: (1) connect to the cloud provider through its programmatic interface to create and start a new instance along with setting up security groups to isolate from other cloud computers, and generating secure communication key pairs; (2) install the ROS libraries and dependencies on the cloud machine needed for the robot application to run in the cloud; (3) set up virtual private networking (VPN) on robot and cloud machine to secure the ROS 2 DDS communication between them; (4) copy the workspace and all of its node software, from the robot to the cloud machine; (5) configure the DDS provider’s discovery mechanism to work across the VPN; (6) launch docker instances; (7) launch cloud-based nodes; and finally, (8) launch nodes on the robot.

Once the launch process is complete, the nodes running on the robot and on the cloud machine(s) securely communicate and interact with each other—and the only change needed was a few lines of the launch script.

The use of VPN in this context offers several advantages:

1. Enhanced Data Encryption: VPN encrypts all data transmitted between ROS 2 nodes, ensuring that sensitive information remains confidential even if intercepted.

2. Network Isolation: By creating a virtual network overlay, ROS 2 communications are isolated from other network traffic, reducing the exposure to external threats.

3. Access Control: VPNs can be configured to allow access only to authenticated and authorized devices and users, providing a controlled environment for ROS 2 operations.

4. Flexibility and Scalability: VPN solutions can be easily scaled and adapted to various deployment scenarios, from small-scale laboratory setups to large-scale industrial applications.

In summary, while the native security features of ROS 2 and DDS are still maturing, employing a VPN to tunnel communications offers an effective interim solution to address the current security limitations. This approach provides robust encryption, enhanced network control, and improved data privacy, thereby fortifying ROS 2 applications against potential cybersecurity threats.

However, this added security does not come without trade-offs. Notably, the introduction of a VPN can impact the performance of communications in ROS 2 systems. This topic will further expanded in Chapter 5.

4.6Conclusion

In this chapter we presented various approaches to add security to ROS 2 in a scalable manner. Namely, SROS2, a series of developer tools focused on usable security that allow adding security capabilities to ROS 2 computational graphs. We introduce a methodology around these tools consisting of 6 basic steps and aligning to the common DevSecOps flows : (A) introspect the computational graph and model its security to determine the necessary security policies and enclaves; (B) define authentication and (C) authorization configurations; (D) generate all the required security artifacts for implementing such policies; (E) deploy them appropriately across robotic systems; and (F) continuously monitor the network, reverting to (A) modeling when needed. SROS2 facilitates each one of these steps by integrating itself tightly into the usual ROS 2 development flows.

We present an application case study discussing how to propose a secure architecture for the TurtleBot3 robot using the navigation2 and slam_toolbox stacks. This is of special interest since it aligns to the software architecture that many industrial and professional robots are using today, given the popularity of these packages.

We introduce security as a process in robotics and correspondingly, the work herein aims to pave the way for enabling security processes, particularly in ROS 2. Alongside the never-ending reality of security, we acknowledge that SROS2 has various limitations that deserve further attention and improvements. Some of these include the lack of granularity of security configurations in the current abstractions, which makes it difficult to configure encryption and authentication options separately. Others refer to the lifecycle management of security artifacts, including updating certificates and keys, wherein secure deployment plays a key role. We are particularly keen on improving SROS2 mechanisms in the future to ensure secure lifecycles while minimizing the downtime impact in ROS 2 graphs. Promising directions for future work also include the development of more advanced monitoring and introspection capabilities, the extension of SROS2 to other communication middlewares (beyond DDS) and finally, the continuous improvement of the usability of the tools. For this, we believe that the use of Graphical User Interfaces (GUIs) represents an interesting opportunity to further facilitate SROS2 usability to non-roboticists. On top of SROS2, we also discuss the shortcomings of the DDS Security extensions and how additional layers of security via VPN could help mitigate them.

The work presented here aims to inspire groups in robotics to add security to their robotic computational graphs. We look forward to security in robotics becoming more usable and accessible, minimizing the threat landscape that lies before us now, and closing the window of opportunity for bad actors. In the next chapter we will discuss the computational performance impact of adding security to ROS robotic systems and how it can be estimated experimentally with state-of-the-art benchmarking methods.

5The Cost of Security: Benchmarking Robot Computations

5.1Introduction

As discussed in the previous chapter, while the inherent security mechanisms of ROS 2 and DDS are undergoing development and refinement, the strategy of tunneling communications through a VPN emerges as an effective interim measure to mitigate existing security vulnerabilities. This method introduces an additional layer of robust encryption, affording enhanced control over the network and bolstering data privacy. As a result, ROS 2 applications gain a fortified defense against a spectrum of cybersecurity threats. However, this added security does not come without trade-offs. Notably, the introduction of a VPN can impact the performance of communications in ROS 2 systems.

The performance downgrade can manifest in various ways, primarily due to the additional processing required for encrypting and decrypting data, as well as the overhead introduced by the VPN protocol itself. While VPN solutions like WireGuard are designed for efficiency and minimal overhead, they inevitably add latency to communications. In real-time robotic applications, where milliseconds can be critical, this added latency can affect the responsiveness and synchronization of robotic components. Moreover, the throughput of data can be constrained by the VPN, potentially limiting the bandwidth available for intensive data streams, such as video feeds or large sensor data arrays.

Another aspect to consider is the complexity introduced by VPN routing and network management. While VPNs can enhance security by segmenting and controlling network traffic, they also add layers of network configuration that can be challenging to optimize for high-performance robotic applications. This complexity can lead to inefficiencies in network routing, further impacting communication speed and reliability.

Therefore, while employing a VPN provides a strong interim solution for securing ROS 2 communications, it’s important to carefully consider the performance implications, especially in scenarios where real-time processing and high data throughput are essential. Future advancements in ROS 2 and DDS security features may offer more integrated solutions that balance robust security with the performance needs of advanced robotic systems. For now, the choice to use a VPN must be weighed against the specific requirements and constraints of each individual robotic application.

Beyond security itself, more generally in robotics, in order for robotic systems to operate safely and effectively in dynamic real-world environments, their computations must run at real-time rates while meeting power constraints. Towards this end, accelerating robotic kernels on heterogeneous hardware, such as GPUs and FPGAs, is emerging as a crucial tool for enabling such performance . This is particularly important given the impending end of Moore’s Law and the end of Dennard Scaling, which limits single CPU performance .

While hardware-accelerated kernels offer immense potential, they necessitate a reliable and standardized infrastructure to be effectively integrated into robotic systems. As the industry leans more into adopting such standard software infrastructure, ROS has emerged as a favored choice. Serving as an industry-grade middleware, it aids in building robust computational robotics graphs, reinforcing the idea that robotics is more than just individual algorithms. The growing dependency on ROS 2 , combined with the computational improvements offered by hardware acceleration, accentuates the community’s demand for a standardized, industry-grade benchmark to evaluate varied hardware solutions. Recently, there has been a plethora of workshops and tutorials focusing on benchmarking robotics applications , and while benchmarks for specific robotics algorithms and certain end-to-end robotic applications, such as drones , do exist, the nuances of analyzing general ROS 2 computational graphs on heterogeneous hardware is yet to be fully understood.

In this chapter, we introduce RobotPerf, an open-source and community-driven benchmarking tool designed to assess the performance of robotic computing systems in a standardized, architecture-neutral, and reproducible way, accommodating the various combinations of hardware and software in different robotic platforms. Figure 5.1 presents a high level overview of RobotPerf, which targets industry-grade real-time systems with complex and extensible computation graphs using the Robot Operating System (ROS 2) as its common baseline. Emphasizing adaptability, portability, and a community-driven approach, RobotPerf aims to provide fair comparisons of ROS 2 computational graphs across CPUs, GPUs, FPGAs and other accelerators. It focuses on evaluating robotic workloads in the form of ROS 2 computational graphs on a wide array of hardware setups, encompassing a complete robotics pipeline and emphasizing real-time critical metrics. The framework incorporates two distinct benchmarking methodologies that utilize various forms of instrumentation and ROS nodes to capture critical metrics in robotic systems. These approaches are: black-box testing, which measures performance by eliminating upper layers and replacing them with a test application, and grey-box testing, an application-specific measure that observes internal system states with minimal interference. The framework is user-friendly, easily extendable for evaluating custom ROS 2 computational graphs, and collaborates with major hardware acceleration vendors for a standardized benchmarking approach. It aims to foster research and innovation as an open-source project. We validate the framework’s capabilities by conducting benchmarks on diverse hardware platforms, including CPUs, GPUs, and FPGAs, thereby showcasing RobotPerf’s utility in drawing valuable performance insights.

RobotPerf’s source code and documentation are available at https://github.com/robotperf/benchmarks and its methodologies are currently being used in industry to benchmark industry-strength, production-grade systems.

5.2Background

Robotics Benchmarks

There has been much recent development of open-source robotics libraries and associated benchmarks demonstrating their performance as well as a plethora of workshops and tutorials focusing on benchmarking robotics applications . However, most of these robotics benchmarks focus on algorithm correctness (functional testing) in the context of domain specific problems, as well as end-to-end latency on CPUs . A few works also analyze some non-functional metrics, such as CPU performance benchmarks, to explore bottleneck behaviors in selected workloads .

Recent work has also explored the implications of operating systems and task schedulers on ROS 2 computational graph performance through benchmarking as well as by optimizing the scheduling and communication layers of ROS and ROS 2 themselves . These works often focused on a specific context or (set of) performance counter(s).

Finally, previous work has leveraged hardware acceleration for select ROS Nodes and adaptive computing to optimize the ROS computational graphs . However, these works do not provide comprehensive frameworks to quickly analyze and evaluate new heterogeneous computational graphs except for two works that are limited to the context of UAVs .

Research efforts most closely related to our work include ros2_tracing and RobotCore . ros2_tracing provided instrumentation that demonstrated integration with the low-overhead LTTng tracer into ROS 2, while RobotCore illuminates the advantages of using vendor-specific tracing to complement ros2_tracing to assess the performance of hardware-accelerated ROS 2 Nodes. Building on these two specific foundational contributions, RobotPerf offers a comprehensive set of ROS 2 kernels spanning the robotics pipeline and evaluates them on diverse hardware.

Table 5.1 summarizes our unique contributions. It includes a selection of representative benchmarks from above and provides an evaluation of these benchmarks against RobotPerf, focusing on essential characteristics vital for robotic systems. We note that while our current approach focuses only on non-functional performance benchmarking tests, RobotPerf’s architecture and methodology can be extended to also measure functional metrics.

5.3RobotPerf: Principles & Methodology

RobotPerf is an open-source, industry-strength robotics benchmark for portability across heterogeneous hardware platforms. This section outlines the important design principles and describes the implementation methodology.

Non-Functional Performance Testing

Currently, RobotPerf specializes in non-functional performance testing, evaluating the efficiency and operational characteristics of robotic systems. Non-functional performance testing measures those aspects not belonging to the system’s functions, such as computational latency, memory consumption, and CPU usage. In contrast, traditional functional performance testing looks into the system’s specific tasks and function, verifying its effectiveness in its primary goals, like the accuracy of the control algorithm in following a planned robot’s path. While functional testing confirms a system performs its designated tasks correctly, non-functional testing ensures it operates efficiently and reliably.

ROS 2 Integration & Adaptability

RobotPerf is designed specifically to evaluate ROS 2 computational graphs, rather than focusing on independent robotic algorithms. We emphasize benchmarking ROS 2 workloads because the use of ROS 2 as middleware allows for the easy composition of complex robotic systems. This makes the benchmark versatile and well-suited for a wide range of robotic applications and enables industry, which is widely using ROS, to rapidly adopt RobotPerf.

Platform Independence & Portability

RobotPerf allows for the evaluation of benchmarks on a variety of hardware platforms, including general-purpose CPUs and GPUs, reconfigurable FPGAs, and specialized accelerators (e.g., ray tracing accelerators ). Benchmarking robotic workloads on heterogeneous platforms is vital to evaluate their respective capabilities and limitations. This facilitates optimizations for efficiency, speed, and adaptability, as well as fine-tuning of resource allocations, ensuring robust and responsive operation across diverse contexts.

Flexible Methodology

We offer grey-box and black-box testing methods to suit different needs. Black-box testing provides a quick-to-enable external perspective and measures performance by eliminating the layers above the layer-of-interest and replacing those with a specific test application. Grey-box testing provides more granularity and dives deeper into the internal workings of ROS 2, allowing users to generate more accurate measurements at the cost of increased engineering effort. As such, each method has its trade-offs, and providing both options enables users flexibility. We describe each method in more detail below and highlight takeaways in Table 5.2.

Grey-Box Testing

Grey-box testing enables precise probe placement within a robot’s computational graph, generating a chronologically ordered log of critical events using a tracer that could be proprietary or open source, such as LTTng . As this approach is fully integrated with standard ROS 2 layers and tools through ros2_tracing, it incurs a minimal average latency of only 3.3 μs , making it well-suited for real-time systems. With this approach, optionally, RobotPerf offers specialized input and output nodes that are positioned outside the nodes of interest to avoid the need to instrument them. These nodes generate the message tracepoints upon publish and subscribe events which are processed to calculate end-to-end latency.

Black-Box Testing

The black-box methodology utilizes a user-level node called the MonitorNode to evaluate the performance of a ROS 2 node. The MonitorNode subscribes to the target node, recording the timestamp when each message is received. By accessing the propagated ID, the MonitorNode determines the end-to-end latency by comparing its timestamp against the PlaybackNode’s recorded timestamp for each message. While this approach does not need extra instrumentation, and is easier to implement, it offers a less detailed analysis and alters the computational graph by introducing new nodes and dataflow.

Opaque Performance Tests

The requirement for packages to be instrumented directly within the source code poses a challenge to many benchmarking efforts. To overcome this hurdle, for most benchmarks, we refrain from altering the workloads of interest and, instead, utilize specialized input and output nodes positioned outside the primary nodes of concern. This setup allows for benchmarking without the need for direct instrumentation of the target layer. We term this methodology “opaque tests," a concept that RobotPerf adheres to when possible.

Reproducibility & Consistency

To ensure consistent and reproducible evaluations, RobotPerf adheres to specific common robotic dataformats. In particular, it uses ROS 2 rosbags, including our own available at https://github.com/robotperf/rosbags, as well third-party bags (e.g., the r2b dataset ).

To ensure consistent data loading and finer control over message delivery rates, we drew inspiration from . Our computational graphs incorporate modified and improved DataLoaderNode and PlaybackNode implementations, which can be accessed at https://github.com/robotperf/ros2_benchmark. These enhanced nodes offer improvements that report worst-case latency and enable the reporting of maximum latency, introduce the ability to profile power consumption and so forth.

Metrics

We focus on three key metrics: latency, throughput and power consumption including energy efficiency. Latency measures the time between the start and the completion of a task. Throughput measures the total amount of work done in a given time for a task. Power measures the electrical energy per unit of time consumed while executing a given task. Measuring energy efficiency (or performance-per-Watt) captures the total amount of work (relative to either throughput or latency) that can be delivered for every watt of power consumed and is directly related to the runtime of battery powered robots .

Current Benchmarks and Categories

RobotPerf beta introduces benchmarks that cover the robotics pipeline from perception, to localization, to control, as well as dedicated benchmarks for manipulation. The full list of benchmarks in the beta release can be found in Table 5.1. Aligned with our principles defined above, each benchmark is a self-contained ROS 2 package which describes all dependencies (generally other ROS packages). To facilitate reproducibility, all benchmarks are designed to be built and run using the common ROS 2 development flows (ament build tools, colcon meta-build tools, etc.). Finally, so that the benchmarks can be easily consumed by other tools, a description of each benchmark, as well as its results, is defined in a machine-readable format. As such, accompanying the package.xml and CMakeLists.txt files required for all ROS packages, a YAML file named benchmark.yaml is in the root of each benchmark which describes the benchmark and includes accepted results.

Run Rules

To ensure the reliability and reproducibility of the performance data, we adhere to a stringent set of run rules. First, tests are performed in a controlled environment to ensure that performance data is not compromised by fluctuating external parameters. As per best practices recommended by ros2_tracing , we record and report settings like clock frequency and core count. Second, we look forward to the possibility of RobotPerf being embraced by the community and have results undergo peer review, which can contribute to enhancing reproducibility and accuracy. Finally, we aim to avoid overfitting to specific hardware setups or software configurations by encompassing a broad spectrum of test scenarios.

5.4Evaluation

We conduct comprehensive benchmarking using RobotPerf to evaluate its capabilities on three key aspects vital for a robotics-focused computing benchmark. First, we validate the framework’s capacity to provide comparative insights across divergent heterogeneous platforms from edge devices to server-class hardware. Second, we analyze the results to understand RobotPerf’s ability to guide selection of the optimal hardware solution tailored to particular robotic workloads. Finally, we assess how effectively RobotPerf reveals the advantages conferred by hardware and software acceleration techniques relative to general-purpose alternatives. All of our results and source code can be found open-source at: https://github.com/robotperf/benchmarks.

Fair and Representative Assessment of Heterogeneity

Assessing hardware heterogeneity in robotic applications is imperative in the ever-evolving field of robotics. Different robotic workloads demand varying computational resources and efficiency levels. Therefore, comprehensively evaluating performance across diverse hardware platforms is crucial.

We evaluated the RobotPerf benchmarks over a wide list of hardware platforms, including general-purpose CPUs on edge devices (e.g., Qualcomm RB5), server-class CPUs (e.g., Intel i7-8700), and specialized hardware accelerators (e.g., AMD Kria KR260). Figure 5.2 illustrates benchmark performance in robotics per category of workload (perception, localization, control, and manipulation) using radar plots, wherein the different hardware solutions are depicted together alongside different robotic workloads per category. Each hardware solution is presented with a different color. Radar plots illustrate the latency, throughput, and power consumption for each hardware solution and workload, with reported values representing the maximum across a series of runs. Each hardware platform and performance testing procedure is delineated by a separate color, with darker colors representing Black-box testing and lighter colors Grey-box testing. In the figure’s key, the hardware platforms are categorized into four specific types: general-purpose hardware, heterogeneous hardware, reconfigurable hardware, and accelerator hardware. Within each category, the platforms are ranked based on their Thermal Design Power (TDP), which indicates the maximum power they can draw under load. The throughput values for manipulation tasks and power values for localization tasks have not been incorporated into the beta version of RobotPerf. As RobotPerf continues to evolve, more results will be added in subsequent iterations.

Given our ability to benchmark 18 platforms (bottom of Figure 5.2), RobotPerf is capable of benchmarking heterogeneous hardware platforms and workloads, paving the way for community-driven co-design and optimization of hardware and software.

Quantitative Approach to Hardware Selection

The rapid evolution and diversity of tasks in robotics means we need to have a meticulous and context-specific approach to computing hardware selection and optimization. A “one-size-fits-all” hardware strategy would be an easy default selection, but it fails to capitalize on the nuanced differences in workload demands across diverse facets like perception, localization, control, and manipulation, each exhibiting distinctive sensitivities to hardware capabilities. Therefore, a rigorous analysis, guided by tools like RobotPerf, becomes essential to pinpoint the most effective hardware configurations that align well with individual workload requirements.

The results in Figure 5.2 demonstrate the fallacy of a “one-size-fits-all” solution. For example, focusing in on the latency radar plot for control from Figure 5.2 (col 3, row 1), we see that the i7-12700H (I7H) outperforms the NVIDIA AGX Orin Dev. Kit (NO) on benchmarks C1, C3, C4, and C5, but is \(6.5\times\) slower on benchmark C2. As such, by analyzing data from the RobotPerf benchmarks, roboticists can better determine which hardware option best suits their needs given their specific workloads and performance requirements.

One general lesson learned while evaluating the data is that each workload is unique, making it hard to generalize across both benchmarks and categories. To that end, RobotPerf results help us understand how the use of various hardware solutions and dedicated domain-specific hardware accelerators significantly improves the performance.

Rigorous Assessment of Acceleration Benefits

In the rapidly advancing field of computing hardware, the optimization of algorithm implementations is a crucial factor in determining the success and efficiency of robotic applications. The need for an analytical tool, like RobotPerf, that facilitates the comparison of various algorithmic implementations on uniform hardware setups becomes important.

Figure 5.3 is a simplified version of Figure 5.2, depicting AMD’s Kria KR260 hardware solution in two forms: the usual hardware and a variant that leverages a domain-specific hardware accelerator (ROBOTCORE Perception, a soft-core running in the FPGA for accelerating perception robotic computations). The figure demonstrates that hardware acceleration can enable performance gains of as much as 11.5\(\times\) (from 173 ms down to 15 ms for benchmark a5). We stress that the results obtained here should be interpreted according to each end application and do not represent a generic recommendation on which hardware should be used. Other factors, including availability, the form factor, and community support, are relevant aspects to consider when selecting a hardware solution.

Benchmarking the security impact in ROS 2 communications

Building upon the previous RobotPerf work and results, in this subsection, we delve into a series of benchmarks designed to evaluate the latency and performance impacts of implementing security protocols within ROS 2 environments. ROS 2, as a critical framework in the robotics domain, must balance the dual demands of robust security and efficient performance. Our benchmarks focus on various configurations, encompassing communications with no security (ROS 2 Intra-network), internal security measures within the communication middleware, specifically the DDS (Data Distribution Service) Security extensions (ROS 2 Intra-network + DDS Security), as well as the effects of tunneling communications through VPNs (ROS 2 Intra-network + VPN). Additionally, we explore the compounded impact when these two security layers are combined (ROS 2 Intra-network + DDS Security + VPN). Results are depicted in Figure 5.4.

The DDS Security extensions are integral to securing ROS 2 communications, providing mechanisms for authentication, encryption, and access control. However, the computational overhead associated with these security features can potentially affect the system’s responsiveness and throughput. Our communication benchmarks displayed in Figure 5.4 illustrate the DDS Security extensions overhead (ROS 2 Intra-network + DDS Security). When implemented, they lead to a mean latency which is 1.55\(\times\) higher than the no security baseline (ROS 2 Intra-network), offering insights into the trade-offs between security robustness and communication efficiency in ROS 2 environments.

Furthermore, the use of VPNs for remote operation and access of robotic systems introduces another layer of security, albeit with its own performance implications. VPNs can safeguard data transmission across various networks, crucial for operations like teleoperated surgery or remote industrial automation. Yet, the encryption and routing processes inherent in VPNs can introduce additional latency. Our benchmarking efforts demonstrate that tunneling ROS 2 over a modern VPN (ROS 2 Intra-network + VPN) lead to a mean latency which is 3.06\(\times\) higher than the no security baseline (ROS 2 Intra-network), providing a comprehensive understanding of the performance costs associated with secure, remote robotic operations over VPN.

Lastly, we investigate scenarios where DDS Security extensions and VPNs are used concurrently (ROS 2 Intra-network + DDS Security + VPN). This dual-layered security approach represents a robust defense mechanism, aligning with the zero-trust model. However, it also poses the greatest challenge in terms of computational overhead and potential latency increases. Our benchmarks hint towards a mean latency increase 4.19\(\times\) higher than the no security baseline (ROS 2 Intra-network). This result elucidates the cumulative impact of security measures, offering a holistic view of their implications on ROS 2’s latency performance. This analysis will aid stakeholders in making informed decisions about security implementations, balancing the crucial aspects of security and efficiency in robotic systems.

5.5Conclusion

RobotPerf represents an important step towards standardized benchmarking in robotics, including its security capabilities. With its comprehensive evaluation across the hardware/software stack and focus on industry-grade ROS 2 deployments, RobotPerf can pave the way for rigorous co-design of robotic hardware and algorithms. As RobotPerf matures with community involvement, we expect it to compare CPU, GPU and FPGA, exploring their power consumption and flexibility in augmenting real-world robotic computations. With a standardized robotics benchmark as a focal point, the field can make rapid progress in delivering real-time capable systems that will unlock the true potential of robotics in real-world applications.

Through RobotPerf, we observe quantitatively significant overheads due to security implementations. Specifically, the DDS Security extensions increase latency by approximately 1.55\(\times\) compared to a non-secure baseline. When implementing VPNs, this latency escalates to about 3.06\(\times\) the baseline. Moreover, combining DDS Security with VPNs results in a latency increase of around 4.19\(\times\). These insights clearly indicate that VPNs constitute a major bottleneck in secure robotic communications. This finding highlights the urgency of developing solutions that can accelerate these security computations, thereby meeting the real-time demands of advanced robotic systems. Addressing this limitation is crucial, especially for applications requiring real-time responses. The next chapter aims to explore strategies to mitigate these overheads, focusing on the hardware acceleration enablement of ROS computations and communications to align with the real-time requirements of robotic systems.

6Hardware Accelerators in Robotics

In this chapter we embark on a critical exploration of how hardware accelerators and their corresponding acceleration kernels can address the computational and communication bottlenecks identified in robotic systems, specifically those related to security, as revealed in the previous chapter. The focus is on how integrating hardware accelerators—such as GPUs and FPGAs—into robotic architectures can drastically reduce latency and improve efficiency. This chapter delves into the potential of hardware accelerators to revolutionize robotics by enabling real-time processing capabilities, a vital aspect for many robotics applications. We discuss how these accelerators can be optimized for robotic tasks, including but not limited to, security protocol computations, thereby aligning with the real-time performance requirements crucial in robotics. The chapter particularly focuses on highlighting the innovative approaches and challenges involved in integrating these accelerators into the existing ROS 2 framework, shaping the future of robotics towards more efficient and secure systems.

6.1Introduction

Recent work has seen an explosion of specialized robotics acceleration on nontraditional computing platforms such as GPUs, FPGAs, and ASICs . This has been sparked by the decline of Moore’s Law and Dennard Scaling, which limits the performance of traditional CPU computing, positioning hardware acceleration as an emerging solution to achieve high performance and power efficiency in robotics applications.

However, this increased diversity of computing platforms leads to a dramatic growth in design space complexity that makes it difficult for users to easily deploy robotics applications on hardware accelerators without substantial expertise in each specific accelerator platform. The Open Computing Language (OpenCL) is an effort to standardize hardware acceleration under a common language, but its adoption across silicon vendors has been uneven and support for it varies. As a result, current hardware acceleration usage is often tied to a particular vendor’s solutions and platforms. This not only impedes interoperability and reuse of acceleration kernels, but presents yet another layer of complexity that users must overcome while implementing robotic systems that use acceleration kernels. A key obstacle is that each hardware acceleration vendor provides their own framework for development, but these are often disconnected from the common tools and libraries in robotics, and mostly aimed at hardware engineers, not roboticists.

To address this challenge, we present RobotCore, an open architecture for hardware acceleration that extends the Robot Operating System (ROS) . ROS is widely used by academia and industry, and early work has demonstrated its potential for hardware-accelerated robotics applications . We facilitate this emerging direction by implementing a vendor and platform-agnostic abstraction layer for hardware acceleration in robotics. Figure 6.1 depicts the open architecture for hardware acceleration in ROS 2, which extends the ROS 2 build system to support vendor and platform-agnostic deployment of robotics applications on accelerator hardware. The integrated tracing and benchmarking infrastructure enables users to analyze the system and make strategic design improvements to optimize performance. Starting with a popular robotics API as the foundation, our ROS 2-based acceleration architecture provides a common ground for both academic researchers and silicon vendors alike to develop specialized robotics acceleration kernels, and deploy them for easy usage by a large, established user base.

Once roboticists can easily harness hardware acceleration across multiple platforms, the next major challenge is profiling and benchmarking the application. Benchmarking is needed to determine the best mapping of the robotics computational graph to the different hardware resources available to optimize overall robot system performance. This is a difficult task, however, since every application is different and deployment scenarios are widespread. Full end-to-end system analysis is required to understand how different implementation tradeoffs impact overall performance. To enable this analysis, we demonstrate how to leverage prior work to benchmark accelerated ROS 2 kernels with a low-overhead framework for real-time tracing based on the Linux Tracing Toolkit next generation (LTTng) . We demonstrate analysis of a case study deployment using CPU and FPGA nodes for a simple perception pipeline.

Using our framework and benchmarking, we diagnose that substantial latency bottlenecks in this computational graph come from inter-node interactions across ROS 2 layers in the CPU. We recognize this as an opportunity for design optimization in hardware accelerators, because interaction with the CPU should not be necessary for dataflow between nodes co-located on the same non-CPU platform (e.g., FPGA).

Based on the benchmarking analysis, we demonstrate two novel separate paths toward hardware acceleration: (1) kernel fusion, and (2) improved message passing. Kernel fusion results in the highest speedup, an average of \(26.96\%\), but it requires manual redesign of the underlying kernels. To avoid manual redesign entirely and improve design re-use and portability, we alternatively develop an intra-FPGA ROS 2 node communication queue template that leverages AXI4-Stream interfaces and transfers data in a sequential streaming manner directly between acceleration kernels. Using this design pattern improves the overall inter-node performance in our computational graph by \(24.42~\%\) on average, while requiring no change in the accelerated kernels. This template extends to applications beyond our case study, since it can be reused for any ROS 2 inter- or intra-process communication by adapting its data types.

In summary, key contributions of this chapter are that we:

• Create a new open infrastructure to increase the performance of robotics applications by enabling integration of hardware acceleration into ROS 2 that is flexible across accelerator platforms (e.g., FPGAs, GPUs) and system deployments (e.g., edge devices, workstations, data centers, and cloud);

• Expose insights into how to optimize overall system-wide performance by extending and providing a template API for low-overhead tracing and benchmarking framework to analyze application performance across hardware accelerated ROS 2 computational graphs, laying foundation to analyze mixed-platform systems (e.g., combinations of CPU and FPGA-based nodes); and

• Increase ROS 2 node-to-node dataflow performance to achieve an average overall accelerator speedup of \(24.42\%\) over CPU in our experiments by designing a template for intra-FPGA ROS 2 node communication queues, based on insights uncovered using our open acceleration infrastructure and low-overhead benchmarking on a case study of a simple perception graph.

The core components of our architecture are disclosed under a commercially friendly open-source license and are available and maintained at the ROS 2 Hardware Acceleration Working Group GitHub organization: https://github.com/ros-acceleration.

6.2Background

ROS and ROS 2

The Robot Operating System (ROS) is an open-source collection of software frameworks and tools designed to provide a structured communications layer for robotics applications running on heterogenous computer hardware .

ROS applications are designed around event driven graphs of Nodes which communicate through Messages on various Topics, Services, and Actions. Each Node can be thought of as a software process which applies an algorithm to the input message and then broadcasts the resulting output message. By managing all inter-Node communications across abstraction layers (e.g., rclcpp, rcl, rmw), ROS simplifies the robotic system deployment process and enables roboticists to quickly develop and test new algorithms. ROS also provides substantial infrastructure to facilitate the automatic building, evaluation, and deployment of robotic systems, including dependency managers, package managers, build systems and tools, simulators, and visualizers.

ROS 2 is a re-design of ROS that modernizes and updates all of its components while adhering to its core design principles. ROS 2 provides a stronger partitioning of the communication middleware from the robotics logic, enabling more flexibility, scalability, and reliability . ROS 2 also provides an updated build system, ament, and a new universal build tool, colcon. This provides a single simple interface for managing the building and deployment of complete robotics applications. Leveraging these tools, roboticists can write new algorithms and rely on ROS 2 to handle all lower level operations and middleware management.

Hardware Acceleration for ROS and ROS 2

There has been previous work that has focused on ways to accelerate robotics applications by developing tools and methodologies to help roboticists leverage hardware acceleration for select ROS Nodes and to optimize the ROS computational graph through adaptive computing . There has also been some work to accelerate the scheduling and communication layers used by ROS and ROS 2 . Unfortunately, the majority of these efforts assume an end-user has substantial experience with embedded systems and embedded hardware flows, or is customized to a specific hardware acceleration board or deployment scenario.

Our proposed open architecture takes a ROS-centric approach to integrate the hardware and embedded flows directly into the core ROS 2 ecosystem. This enables a separation between those who produce accelerated kernels and those who use them by providing end-users with a build and deployment experience for hardware accelerators similar to the standard, non-accelerated ROS 2 experience.

6.3An Open Architecture for Hardware Acceleration in ROS 2

Our open architecture (Figure 6.2) extends the core ROS 2 build system and tools to provide platform-agnostic (i.e., supports edge, workstation, data center, or cloud targets) and technology-agnostic (i.e., supports FPGAs and GPUs), hardware-accelerated ROS 2 capabilities for roboticists. We: A) extend the ROS 2 build system, ament; B) extend the ROS 2 meta build tool, colcon; and C) develop integrated ROS 2 firmware extensions. We also D) integrate a low-overhead tracing and benchmarking framework to enable the analysis of holistic application performance across ROS graphs. This section describes these extensions in detail.

Extending the ROS 2 Build System

The first pillar of our open architecture, Figure 6.2 , allows roboticists to generate acceleration kernels directly from the ROS 2 build system (ament) in the same way they generate CPU binaries. To do so, the ament_acceleration ROS 2 package and its extensions abstract the ROS build system from vendor-specific accelerators (e.g. FPGAs or GPUs), including their frameworks and software platforms. This allows the build system to easily support hardware acceleration across commercial solutions while using the same syntax, simplifying the work of ROS 2 package maintainers.

Under the hood, ament_acceleration abstracts away the corresponding vendor-specific firmware. For example, ament_vitis 19 relies on the proprietary Xilinx Vitis and on the Xilinx Runtime (XRT) library . This simplifies the creation of acceleration kernels and separates firmware concerns from algorithm development. This way, robotics engineers can focus on improving their computational graphs with a ROS-centric development flow. Separately, hardware experts, potentially sponsored by silicon vendors, can improve acceleration kernels for a particular commercial solution. Overall, these extensions help achieve the objective of simplifying the creation and integration of acceleration kernels from different vendors into ROS 2 computational graphs.

Figure 6.2 depicts the build system extensions showing how ament_acceleration abstracts the build system from vendor-specific solutions. As an example of an alternative acceleration technology supported, ament_jetpack is included and illustrates the integration of Nvidia JetPack .

Extending the ROS 2 Build Tools

The second pillar of our open architecture, Figure 6.2 , extends the colcon ROS 2 meta build tool to integrate hardware acceleration flows into the ROS 2 Command Line Interface (CLI) commands. Examples of these extensions include the selection of the target accelerator and build-time through mixins, emulation capabilities to speed-up the development process and facilitate design without access to the real hardware, raw disk image production tools, and simplified configuration of hypervisors. These extensions are implemented by the colcon-acceleration 20 ROS 2 package. As in Section 6.3.1, colcon_acceleration further enables roboticists to leverage hardware accelerators while using standard ROS 2 commands and flows.

Adding Firmware Extensions

Represented by the abstract acceleration_firmware ROS package and its corresponding specializations (e.g. acceleration_firmware_kv260 21 for the Xilinx Kria KV260 board), the third pillar of our open architecture, Figure 6.2 , firmware extensions, are meant to provide firmware artifacts for each supported technology solution. This again simplifies the process for ROS package consumers and maintainers, and further aligns hardware acceleration workflows with typical ROS development flows. Each ROS 2 workspace can leverage multiple firmware packages, but can only use one at a time. As colcon_acceleration supports the selection of the active firmware in the ROS workspace, by separating the firmware out into their own packages, our open architecture enables silicon vendors to maintain an acceleration_firmware_<solution> package that automatically integrates into standard ROS 2 workflows.

Low-Overhead Real-Time Tracing & Benchmarking

In the context of hardware acceleration in robotics, it is fundamental to be able to inspect performance improvements. To that end, it is important to benchmark and trace the system. Benchmarking is the process of running a computer program to assess its relative performance, whereas tracing is a technique used to understand what is happening in a system while it is running. Tracing helps determine which pieces of a Node are consuming more compute cycles or generating indeterminism, and are thereby good candidates for hardware acceleration. Benchmarking instead helps investigate the relative performance of an acceleration kernel versus its CPU scalar computing baseline. Similarly, benchmarking also helps with comparing acceleration kernels across different hardware acceleration technology solutions (e.g., Kria KV260 vs. Jetson Nano) and across kernel implementations within the same hardware acceleration technology solution.

In order to trace and evaluate the relative performance of both ROS 2 individual Nodes and complete computational graphs, we leverage Linux Tracing Toolkit next generation (LTTng ) for tracing and benchmarking, Figure 6.2 . Building upon prior work , LTTng provides a collection of flexible tracing tools and multipurpose instrumentation for ROS 2 that allow collecting runtime execution information in real-time in distributed systems using low-overhead tracers. For example, when enabling all ROS 2 instrumentation, end-to-end message latency overhead is below \(5.5\)us , making it suitable for a wide variety of hardware acceleration use cases. Building on top of this foundation, we developed a tracing and benchmarking template that enables roboticists to easily instrument both their accelerated and non-accelerated code in a vendor-agnostic manner. This infrastructure also lays a foundation for future integration with platform-specific performance counters and tracing tools that can extend analysis to more fine-grained introspection and profiling of the kernels running onboard an accelerator device.

6.4Case Study: Accelerating ROS 2 Perception

For our case study, we trace, benchmark, and accelerate a subset of image_pipeline , one of the most popular packages in the ROS 2 ecosystem, and a core piece of the ROS perception stack. We compose a simple computational graph consisting of two nodes, resize and rectify, as shown in Figure 6.3. The computational graph of our case study perception application in Figure 6.3 uses the ROS image_pipeline package, containing two ROS 2 nodes: (1) RectifyNode subscribes to the /camera/image_raw and /camera/camera_info topics from Gazebo and publishes a rectified image to (2) ResizeNode, which publishes the final resized image. We leverage our open architecture for hardware acceleration (Section 6.3) to benchmark, trace and accelerate our computational graph, comparing a CPU to an FPGA implementation. In this section we describe the methodology of our approach, and analyze our timing results, presenting a case study for how our open architecture can help enable hardware accelerated applications in ROS 2.

Method

We propose the following steps to analyze a ROS 2 application and design appropriate acceleration: (i) instrument both the core components of ROS 2 and the target kernels; (ii) trace and benchmark the kernels on the CPU to establish a baseline; (iii) develop a hardware accelerated implementation on alternate hardware (e.g., GPU, FPGA); and (iv) trace, benchmark against the CPU baseline, and improve the accelerated implementation.

Following this methodology, in our case study we begin by instrumenting both ROS 2 and our target kernels with LTTng probes. Reusing past work and probes allows us to easily get a grasp of the dataflow interactions within rmw, rcl, and rclcpp ROS 2 layers. We then also instrument the ResizeNode and RectifyNode components of the image_pipeline package used in our case study. The relevant tracepoints placed in our computational graph across ROS 2 stack layers are listed in Figure 6.4 and 6.5 (full list in Pull Request \(717\) in the image_pipeline repository ). On the CPU, these tracepoints enable us to isolate the latency of computation within a node from the time it takes ROS 2 to package and pass information between nodes.

In the following sections we report timing results from using a Xilinx Kria® KV260 Vision AI Starter Kit , which has an onboard integrated Quad-core Arm® Cortex®-A53 CPU and an FPGA containing 256K System Logic Cells and 1.2K DSP Slices. All benchmark results report the mean value obtained from a \(60\) second continuous run of the computational graph. The FPGA kernels are synthesized, placed and routed with a \(250\)MHz clock.

CPU-Only Tracing Results

Figure 6.4 demonstrates the results of instrumenting and tracing our target computational graph (Figure 6.3) across multiple ROS 2 stack layers on the CPU, and Figure 6.5 summarizes the breakdown of timing results across operations, establishing the CPU baseline for our application. The breakdown in Figure 6.5 shows the time taken to do the computations within each node, as well as the time taken by the ROS 2 lower-level message-passing system across the various abstraction layers. We find that the message-passing overhead in our application consumes more than \(73.3\%\) of the total time and is therefore a large bottleneck in the total computation time of the full graph. We next explore FPGA hardware acceleration options, comparing performance to the CPU baseline.

Accelerating and Benchmarking CPU & FPGA

In this section, we explore hardware acceleration options for an FPGA for our case study application (Figure 6.3). In Section 6.4.3.1, we first explore hardware acceleration kernels for the core logic of each of the Nodes (rectify and resize), harnessing our open architecture for implementation. In Section 6.4.3.2, we then explore two different FPGA designs to accelerate the computational graph by optimizing dataflow interactions between FPGA-based nodes, addressing the ROS 2 communication infrastructure performance bottleneck revealed by the CPU baseline in Section 6.4.2.

Accelerating Nodes & Components on an FPGA

We first accelerate the computations at each one of the graph nodes. The RectifyNode and ResizeNode Components of Figure 6.3 are accelerated using Xilinx’s HLS, XRT, and OpenCL targeting the Kria KV260 22 . Each ROS 2 Component has an associated acceleration kernel 23 that leverages the Vitis Vision Library, a computer vision library optimized for Xilinx silicon solutions and based on OpenCV APIs. These accelerated Components and their kernels easily integrate with the rest of the ROS meta-package through our open architecture (Figure 6.2), and are openly available to the public. Building the accelerators is abstracted away from roboticist end-users, and takes no significant additional effort than the standard build of the image_pipeline.

After benchmarking the accelerated Components using the trace points of Section 6.4.2, we observe an average \(6.22\%\) speedup in the total computation time of the perception pipeline when offloading tasks to the FPGA (see Figure 6.6). For this case study example, it is not surprising that accelerating the computational nodes and components alone only gives a modest performance increase because, as we saw in Section 6.4.2, the performance bottleneck in the baseline CPU system was communication overhead, not computation.

Accelerating the Computational Graph on an FPGA

In our case study application, message-passing overheads across the ROS 2 abstraction layers far outweigh other operations, so in this section we focus on optimizing these dataflows. Addressing performance bottlenecks in our system leads to overall lower computational graph latency, and to faster robots. To seize this acceleration opportunity in our case study example, we optimize the dataflow within the computational graph and across ROS 2 Nodes and Components through two different design approaches: (a) kernel fusion, and (b) dedicated streaming queues.

The speedup obtained by integrating both ROS Components on the FPGA into a single unified kernel is shown in Figure 6.6. The benefits of doing this are two-fold. First, we avoid any message-passing between the Rectify and Resize Nodes’ Components. Second, we avoid the compute cycles wasted while memory is mapped back and forth between the host CPU and the FPGA. This results in an overall latency speedup of \(26.96\%\) over the CPU. In addition to speeding up the perception stage, another added benefit of this improvement is that such speedups make room for other robot tasks in a complete end-to-end system. Note, however, that this improvement required the construction of an entirely new ROS Node and unified acceleration kernel on the FPGA.

We then develop a template for an accelerated ROS 2 message passing interface on the FPGA. This interface is Node and Component-agnostic and can be leveraged by roboticists to accelerate the communication channels of any computational graph on an FPGA. This is done by leveraging an AXI4-Stream interface to create an intra-FPGA ROS 2 communication queue template which is then used to pass data across Nodes in the FPGA without sending messages to the CPU 24 . This allows us to completely bypass the original CPU-centric ROS 2 message-passing system and optimizes dataflow, achieving an overall latency improvement of \(24.42\%\) over the CPU in our application (see Figure 6.6).

Based on these results, for this case study, we show that implementing FPGA-accelerated versions of key ROS 2 Components is easily feasible, and that addressing the right bottleneck is key to improving performance. Tracing and benchmarking the CPU baseline suggested that communication is the bottleneck in our case study. In fact, independent examination of, e.g., a single run of the fused-kernel accelerator using the Xilinx Vitis Analyzer, confirms that this is also the case on the FPGA—we note that integrating device-specific profiling tools into our foundational tracing infrastructure in future work can further automate this type of fine-grained introspection of kernels onboard accelerator devices. We can achieve overall performance improvements by either combining Nodes or streamlining intra-FPGA communication. While combining nodes may result in slightly higher performance, it is a much more labor-intensive design effort. By contrast, our accelerated intra-FPGA-Node communication queue template can be applied by any roboticist, to any computational graph.

6.5Conclusion

In this chapter we presented a new open infrastructure to introduce hardware acceleration in ROS 2 in a scalable and technology-agnostic manner. Our architecture allows us to increase the performance of robotics applications through the integration of hardware acceleration with ROS 2 APIs and its conventional flows. We do so by extending ROS 2 in a way that is portable across accelerator platforms (e.g., FPGAs, GPUs) and system deployments (e.g., edge devices, workstations, data centers, and cloud). We also present a template for low-overhead tracing and benchmarking to analyze performance across both hardware accelerated and standard ROS 2 computational graphs.

We use our open architecture and our tracing and benchmarking infrastructure to demonstrate a principled design methodology for ROS 2 hardware acceleration, exposing insights into how to optimize overall system-wide performance by analyzing a CPU baseline, and comparing accelerator design iterations to that original baseline. We examine a case study using the Xilinx Kria KV260 platform to demonstrate FPGA acceleration of one of the most popular packages in the ROS perception pipeline: image_pipeline. We first demonstrate a modest performance speedup of \(6.22\%\) from offloading perception tasks to the FPGA, and then increased speedup by additionally addressing the communication overheads that we identified as bottlenecks by analyzing our CPU baseline. We achieved a speedup of \(26.96\%\) from re-architecting the graph to combine nodes and avoid inter-FPGA-node communication delays inflicted by interactions with the CPU, but this approach requires substantial effort from users to re-architect their graphs. Instead, to avoid this overhead and stay in alignment with the ROS 2 programming model, we then design a novel template for intra-FPGA ROS 2 Node communication queues that allows ROS Nodes and Components to deliver faster dataflows, achieving a \(24.42\%\) speedup over a CPU without excessive manual per-kernel design effort.

We contribute our open architecture to the ROS community, so that future work can use our infrastructure and extend to new applications beyond our case study example. Promising directions for future work include: benchmarking computational graphs with other hardware solutions (e.g., GPUs) to establish consistent cross-accelerator comparisons; extending our tracing and benchmarking approach to include additional tracing information (e.g., profiling within FPGA or GPU devices) for more fine-grained introspection of kernels running onboard accelerators; and applying our open architecture and analysis to other ROS 2 packages.

Our code is disclosed under a commercially friendly open-source license and is available and maintained at the ROS 2 Hardware Acceleration Working Group GitHub organization: https://github.com/ros-acceleration. This work is being further integrated into the ROS ecosystem through a community standardization effort, REP-2008 .

6.6Future work

The promising outcomes of this chapter lay the groundwork for the development of dedicated robot security accelerators. Future initiatives will concentrate on commercializing these innovations. Preliminary findings suggest that our security accelerator prototypes could significantly enhance intra-network ROS 2 communications, potentially reducing average networking latency to single-digit microseconds, which is over 100\(\times\) faster than current software implementations on CPUs. Particularly in worst-case scenarios, these accelerators maintain deterministic and isochronous performance, a stark contrast to the higher latencies observed with CPU-based software implementations. The potential for a maximum latency speedup exceeding 100,000\(\times\) faster communications and security computations marks a significant leap forward in the field of robotics. Due to the commercial potential of these developments, a detailed discussion is outside the scope of this research, but these early results are indicative of the transformative impact these accelerators could have on the future of robotic communications.

Cyber-attacking robots

7Attacking Robot Software

This chapter delves into the security analysis of robot software and particularly, the Data Distribution Service (DDS) standard, a middleware pivotal in operating a wide range of robotic systems and devices, including ROS 2 (DDS is the default communication middleware of ROS 2). The summarized investigation presented herein is the result of a multi-year research effort involving various researchers and led to uncover multiple vulnerabilities in DDS, leading to the issuance of various new CVE IDs for common DDS implementations and even in the standard specifications. An extensive scanning exercise revealed numerous public-facing DDS services across various industries, some affected by these CVEs, highlighting significant security risks due to their unintended public exposure. We emphasize the need for continuous security testing of DDS and similar technologies, offering both immediate and long-term recommendations for secure integration and advocating for robust supply-chain management and ongoing security practices.

7.1Introduction

Even within the industry, a big percentage of practitioners are unaware that the Data Distribution Service (DDS) drives systems such as industrial robots (manipulators), railways, autonomous cars, airports, surgical robots, spacecraft, diagnostic imaging machines, luggage handling, industrial robots, military tanks, and frigates, among others. It has been in use for about a decade, and its adoption continues to steadily increase.

We discovered and reported vulnerabilities in DDS that warranted new CVE IDs: Five with a score of greater or equal than 7.0, four with a score of > 8.5, one vulnerability in the standard specifications, and other deployment issues in the DDS software ecosystem (including a fully open production system).

Successful exploitation of these vulnerabilities can facilitate initial access (MITRE ATT&CK Technique ID TA0108) via exploitation of remote services (T0866, T0886) or supply chain compromise (T0862), and allow the attacker to perform discovery (TA0102, T0856) by abusing the discovery protocol. The consequences of successful exploitation, in any of the critical sectors where DDS is used, range from denial of service (T0814) via brute forcing (T0806), to loss of control (T0827), or loss of safety (T0880). The DDS protocol itself can also be abused to create an efficient command and control channel (T0869). Based on our analysis, we recommend mitigations such as vulnerability scanning (ATT&CK Mitigation ID M1016), network intrusion prevention (M1031), network segmentation (M1030), filter network traffic (M1037), execution prevention (M1038), and auditing (M1047).

Findings in Brief

Given this technology’s versatility, we analyzed and discovered multiple security vulnerabilities, resulting in 13 new CVE IDs for the six most common DDS implementations. This includes one vulnerability in the standard specifications and other deployment issues in the DDS software ecosystem (including a fully open production system). Some of these vulnerabilities have been patched or mitigated by the vendors since we reported them. Others remain exposed and can still be abused.

By measuring the exposure of DDS services, in one month we found over 600 distinct public-facing DDS services in 34 countries affecting 100 organizations via 89 internet service providers (ISPs). Of the DDS implementations by seven distinct vendors (one of which we were initially unaware of), 202 leaked private IP addresses (referring to internal network architecture details), and seven supposedly secret URLs. Some of these IP addresses expose unpatched or outdated DDS implementations, which are affected by some of the vulnerabilities that we’ve discovered and disclosed in November 2021.

During our research, we interviewed key DDS users and system integrators to collect their feedback on our findings and the importance of DDS for innovation in their respective sectors. In this chapter, we analyze and discuss the specifications of DDS and the six most actively developed implementations maintained by certified vendors and with millions of deployments worldwide. We also released an open-source software: a Scapy-based dissector and several fuzzing harnesses for three open DDS implementations.

7.2Background

DDS is a standardized middleware software based on the publish-subscribe paradigm, helping the development of middleware layers for machine-to-machine communication. This software is integral especially to embedded systems or applications with real-time6 requirements. Maintained by the Object Management Group (OMG), DDS is used in all classes of critical applications to implement a reliable communication layer between sensors, controllers, and actuators.

DDS is at the beginning of the software supply chain, making it easy to lose track of and is an attractive target for attackers. Between 2020 and 2021, 66% of attacks focused on the suppliers’ codes . While we were in the process of doing this research, we encountered an exposed source-code repository hosting a proprietary implementation of DDS. Left open, this would have let an attacker infect the source code (MITRE ATT&CK T0873, T0839).

Notably, the following companies and agencies use DDS (note that this is not an exhaustive list of currently using the technology):

• Military robots (e.g. legged robots from Ghost Robotics)

• National Aeronautics and Space Administration (NASA) at the Kennedy Space Center

• Siemens in wind power plants

• Volkswagen and Bosch for autonomous valet parking systems

• Nav Canada and European CoFlight for air-traffic control

DDS is the foundation of other industry standards such as OpenFMB for smart-grid applications and Adaptive AUTOSAR , among other sectors that we identify in the next section. The Robot Operating System 2 (ROS 2), which is the de facto standard operating system for robotics and automation and as previously mentioned, also uses DDS as the default middleware. We also noted that, according to a confidential document leaked online, NVIDIA has listed DDS as a strategic tool for system-virtualization and cloud-computing applications, mainly for exchanging data within and across virtual machines.

7.3DDS and Real-Time Publish-Subscribe (RTPS) Packets

There are many software-based controlled systems in the world that connect sensors, actuators, and controlling and monitoring applications. DDS was invented for such systems, with a strong focus on interoperability and fault tolerance. It is optimized for publish-subscribe and peer-to-peer applications as most applications can’t afford a single point of failure. The middleware relies on multicast (group communication or data transmission to multiple recipients) for discovery, allowing everything to run without needing initial configurations.

From a software developer standpoint, DDS is a communication middleware that facilitates interoperability of processes across machines in all main programming languages. From another viewpoint, DDS is a data-centric, publish-subscribe communication protocol that allows developers to build a flexible shared data “space” or “bus” for virtually any application that requires two or more nodes to exchange typed data.

The DDS layer is encapsulated into real-time publish-subscribe (RTPS) packets, which for now can be considered as a collection of sub-messages (such as timestamp, discovery, data, and security metadata), as shown in Figure 5. Given the strong dependency between DDS and RTPS, we focused our research on RTPS for increased generality. Because of its flexibility, DDS and its underlying layers do not come as a ready-to-use, off-the-shelf product like other middleware tools (such as Message Queuing Telemetry Transport or MQTT). Rather, DDS is a programming library that developers use to build custom middleware protocols with advanced features such as custom data types, quality of service (QoS) policies, network partitioning, authentication, and encryption.

Research Scope: RTPS, DDS, and Robot Operating System 2 (ROS 2)

In addition to the DDS standard specifications, we focused our investigation on the six DDS implementations listed in Table 7.1. These implementations are used globally and have customers and users in the critical sectors identified in the same table. Because DDS depends on RTPS as a lower-layer standard protocol, each DDS implementation ships with its own RTPS implementation. In other words, DDS data is contained as a sub-message within RTPS, thereby focusing on both protocols. To make sure we highlight the criticality of this for robotics, again, the Robot Operating System 2 (ROS 2), which is the default standard meta-operating system for robotics and automation, has DDS as its default middleware. For this reason, the impact of each vulnerability extends beyond DDS alone, and includes all ROS 2 instances.

7.4Research Methodology and Technical Details

Refer to for details on the final findings derived from this research. This subsection instead discusses the technical details of how those findings were encountered. The offensive methodology explained here applies to other (software) robotic endeavours. Figure 7.5 provides an overview of the research methodology we followed. We analyzed each DDS implementation from two main angles: network and source — or binary — code. We wanted to have a deep understanding of the low-level details of the RTPS network layer to craft arbitrary test packets, and prove that it is possible for an attacker to mass scan a network and map the attack surface. In a parallel investigation, we manually studied the original or decompiled source code, with the main goal to find good fuzz targets (i.e., functions that receive and process untrustworthy data like network packets).

A New Scapy Layer to Dissect and Forge RTPS and DDS Data

Although Wireshark already includes an RTPS dissection plugin, we needed something more scriptable. Since we spent some time manually crafting RTPS packets at the beginning, we decided to develop a RTPS Scapy-based dissector. We released the resulting Scapy layer as open-source code under the GNU General Public License v2.0.74 which is available ever since https://github.com/secdev/scapy/pull/3403.

Crafting RTPS probes with Scapy

Without going into the details of our Scapy RTPS implementation, note that it can be used to programmatically create RTPS packets by writing Python code, like any other Scapy layer. In practice, that’s seldom what a researcher would do, especially for “thick” protocols with lots of options.

Instead, the developer’s typical workflow can be:

• Intercept traffic. Use Tcpdump or Wireshark to collect the traffic generated by the “hello world” example typically provided with a DDS distribution.

• Extract UDP payload. Use Scapy (or manually via Wireshark) to select the packet of interest and extract the UDP payload (which contains the RTPS layer, as shown in Figure 7.5).

• Dissect with the RTPS class. Pass the extracted payload to the Scapy RTPS class, which will automatically dissect it.

• Generate Python code automatically. Use Scapy’s built-in .command() method to output the Python code that will declaratively generate the packet that has just been dissected. If necessary, modify the packet so obtained (as exemplified in Figure 7.11).

• Test modified packet against the endpoint. Either use Scapy’s built-in send()/sendp() functions or Python’s socket module to send the packet to a target DDS endpoint and check if it triggers the desired behavior.

The following section briefly describes how, almost by accident, we discovered the amplification vulnerability while dissecting and modifying packets during the early stages of the development of our Scapy RTPS layer.

Finding the Amplification Vulnerability

Although network fuzzing via Scapy was not directly effective in our research, creating a Scapy layer helped and motivated us to investigate all the RTPS packets’ fields in depth. The activity led us to find the amplification vulnerability (CVE-2021-38425, CVE-2021-38429, CVE-2021-38487, CVE-2021-43547). In the long run, we recommend that developers and users leverage our Scapy RTPS layer — or similar libraries — as a building block for building continuous network fuzzers for RTPS and DDS.

The goal of the RTPS discovery phase is to send “probe” packets (e.g., to multicast addresses) and wait for responses from new locators. Locators could be IP-port pairs (see the PID_DEFAULT_UNICAST_ LOCATOR in Figure 7.6, right side of the screenshot) or memory offsets in a shared-memory transport. Before reading the specifications in depth, we assumed that an RTPS discovery packet would allow us to restrict the locator to the IP addresses within the network the machine is connected to, and would not blindly send RTPS data to any IP-port found in the locator field. On a second read, however, this is exactly how discovery works by design.

We discovered this by setting the PID_DEFAULT_UNICAST_LOCATOR to the first IP address that came to mind (in this case, the Google DNS, because it’s easy to type at “8.8.8.8”). Almost immediately, a flow of outbound packets came from the DDS node, as shown in Figure 7.7 and 7.9.

Source-code and Binary Fuzzing

Of all the implementation vulnerabilities that we disclosed for this research, all but three have been found through source-code or binary fuzzing, and three through scripting a file-format input mutator (RADAMSA). There are many fuzzing tools freely available to researchers, and we chose one based on what has been used successfully for years by the largest public fuzzing platform (Google OSS-Fuss), which uses a combination of AFL++, libFuzzer, and Honggfuzz. Although the choice of the specific tool can influence the efficiency of a fuzzing campaign, we focused our attention on the most important piece: finding good fuzz targets and writing good fuzzing harnesses.

Source-code Fuzzing with AFL++ and libFuzzer

We used AFL++ for fuzzing with multiple sanitizers in LLVM. AFL++ requires the project compile with the latest version of LLVM and the build system of some DDS implementations required some work. Aside from this, most of the effort in this phase went into finding the right fuzz target and implementing a harness while keeping the code deterministic (for example, no threading).

From the high-level viewpoint depicted in Figure 7.11 and 7.14, we were interested in finding the most self-contained function in charge of processing data coming from the network. We found a repeating pattern in all the DDS implementations: upon receiving network data (i.e., recv() or some abstraction on top of it), there are one or more deserialization functions in which we likely find a switch-case control structure, used to dispatch the RTPS sub-message IDs to the right routine.

Given the importance of finding the right fuzz target, we dedicate the remainder of this section to showcase some examples of fuzz targets.

We began with the supplied Docker images or make files to compile example programs, run them under GDB, and inspect debug traces, like exemplified for OpenDDS in Figure 7.11.

By following the function calls with the aid of the code analyzer part of Visual Studio Code, we were able to see that all three DDS implementations were using very similar procedures for deserializing network payloads. In particular, we found that they all had a switch-case to handle the RTPS sub-message types, as exemplified for OpenDDS in Figure 7.15.

In some cases, we adjusted the source code right before the beginning of the de-serialization to dump the binary data being passed to the first function. This was useful to confirm that it was the network payload that we expected the function to receive, as exemplified for Cyclone DDS in Figure 7.16.

The following list showcases the groundbreaking vulnerabilities discovered by the author of this thesis, in collaboration with an international cohort of security researchers . Over an intensive 18-month study dedicated to Data Distribution Service (DDS) protocols, the author played a pivotal role in identifying these novel security flaws. These contributions, which are original and previously undocumented in the literature, highlight the author’s research results and innovative approach to cybersecurity.

Our methodical and responsible disclosure process further underscores our commitment to ethical research practices. Each vulnerability was first reported to the respective manufacturers, allowing for necessary remediation, and subsequently to relevant authorities after a stipulated grace period, ensuring an orderly public disclosure . This diligent process not only demonstrates our dedication to advancing the field but also our leadership in setting new standards for responsible vulnerability reporting. These discoveries represent significant advancements in DDS security and mark a notable contribution to the broader cybersecurity community in robotics.

CVE-2021-38445 (OpenDDS): Failed Assertion Check in RTPS Handshake

Using the harness (shown in Figure 7.17) we found out that, in OpenDDS less or equal v3.17, while receiving a RTPS packet with valid headers, with DATA sub-message, any attached serialized sub-data segment with a parameterLength of 0 will cause an assertion to fail in Serializer::doread, which subsequently called Serializer::smemcpy with a const char* from of zero. This vulnerability can be exploited via the network even without authorization and can cause the DDS node to crash. It cannot be developed into a buffer overflow so it does not grant any code-execution primitives.

More specifically, Serializer::doread does not check for segments of 0 length and continues to handle the messages. This is passed by RtpsCoreTypeSupportImpl.cpp near:

bool operator >>(Serializer\& outer_strm,
    ::OpenDDS::RTPS::Parameter\& uni) 

which extracts size information from parameterLength but does not check if it is a valid value. It only makes sure extracting values from the serializer is successful.

CVE-2021-38445 (OpenDDS): Memory exhaustion

The opposite occurs with CVE-2021-38445 explained in the previous section, wherein the serializer is tricked into allocating very large chunks of memory. AFL++ found a crash in OpenDDS’s serializer: While deserializing data with parameter ID type 0x55, it does not sanitize the value in its length field. This allows attackers to exhaust a server’s memory by crafting a packet with a very large number in that field. In

bool operator>>(Serializer\& strm,
    ::OpenDDS::RTPS::FilterResult_t\& seq)
    (RtpsCoreTypeSupportImpl. cpp:1977)

a check should be made to make sure it never allocates more memory than it’s allowed, or a hard limit should be implemented.

CVE-2021-38441 and CVE-2021-38443 (Cyclone DDS): XML Parsing to Heap-write

Some DDS implementations had networking functionalities plugged deep into the application code, which required some mock functions in the harness, as exemplified in Figure 7.18 for Cyclone DDS.

The harness in Figure 7.18 found several crashes, which led to two vulnerabilities. One is exemplified in Figure 7.19, a multi-byte heap-write primitive. Upon checking the source code, we noticed that there were multiple inputs that can lead to a heap overflow in the XML parsing routines. This causes at least a crash and can be exploited to write in the heap, potentially overflowing into the stack. Without heap protections, this vulnerability is exploitable as it is a write primitive of at least 8 bytes, and certainly causes the program to crash in the best case.

Binary Fuzzing with UnicornAFL

The trial licenses for RTI Connext DDS, CoreDX DDS, and Gurum DDS grant access only to binary distributions of the libraries. After compiling the example programs that ship with the original software distribution, we used GDB to inspect run traces. This turned out to be quite verbose given the presence of several debug symbols. We filled the missing information by inspecting the listing via Ghidra and IDA Pro. This allowed us to find interesting fuzz targets, as seen in Figure 7.20. For coverage-guided fuzzing we used UnicornAFL, which is a fork of AFL++ that uses the Unicorn emulation engine to “execute” the target and employs block-edge instrumentation in a similar fashion to AFL’s QEMU mode.

In practice, we dumped the context of a running process with GDB and prepared a harness (see Figure 7.21) that lets UnicornAFL restore that context, set registers and memory state, the RIP register, and start emulation. Like AFL, UnicornAFL will take care of mutating the input, passing it to the fuzz target, and keep track of the coverage. The main shortcoming is that we had to re-implement some memory management functions (e.g., malloc, memset). This approach is inherently slow due to emulation, but was good enough for initial vulnerability research. It costed us a few hours of AWS EC2 computation (c5a.8xlarge), and we found that AMD EPYC machines were three times faster than Intel Xeons while fuzzing using UnicornAFL.

CVE-2021-38435 (RTI Connext DDS): Segmentation Fault on Malformed RTPS Packet

The UnicornAFL instrumentation that we prepared found a segmentation-fault in the RTPS deserializer in RTI Connext DDS when receiving a malformed packet. This would cause runtimes to exit immediately and a DoS. In particular, the RTICdrStream_skipStringAndGetLength() function does not properly check inputs, using the result straight from RTICdrStream_align(), thus triggering a segmentation fault. Both publisher and subscriber are affected.

CVE-2021-38439 and CVE-2021-38423 (Gurum DDS): Heap Overflow and Segmentation Fault

While using UnicornAFL on Gurum DDS fuzz targets, we discovered that there is a heap overflow in the RTPS routine that handles payload parsing. This causes a segmentation fault leading to DoS. More specifically, the crash is triggered in rtps_read_AckNackMessage() function when called in read_ Submessage(), which creates a multi-byte heap overflow condition. We found this crash by using a harness that passes RTPS payload directly to the rtps_read_Data(…, buf, len, …) function through the buf argument.

Another case we 25 found is in the rtps_Parameter_load2() function, which does a type conversion from a buffer and does a check to exclude specific IDs. During this conversion, we found some inputs causing a segmentation fault. We verified that this is exploitable via network by crafting a packet based on the crash dump provided by the fuzzing engine.

Scripting RADAMSA to Mutate XML Files

At the beginning of this research and before using AFL++ and UnicornAFL, we used RADAMSA directly, with some simple shell scripting (see Figure 7.23). Without any prior knowledge on the target software, this simple technique can be surprisingly effective at finding crashes, which can also lead to the discovery of vulnerabilities.

CVE-2021-38427 and CVE-2021-38433 (RTI Connext DDS): Stack-based Buffer Overflows Python Bindings

The simple “harness” shown in Figure 7.23 allowed us to find two vulnerabilities; one could be exploited beyond just a crash to control a pointer using a malformed XML file. When the length of an attribute value in a configuration XML file is longer than a certain limit, RTIXMLObject_lookUpRef() would trigger a buffer overflow. If the length is exactly 894 characters, we could overwrite RIP register (see Figure 7.23). We have not investigated further, but we see the possibility of preparing a ROP chain for this target. However, the XML parser does not accept arbitrary hexadecimal characters, so we’re limited within the Unicode range.

A variant of this vulnerability is in RTIXMLDtdParser_getElement(), which does not properly validate the length of an element prior to copying it to a fixed-length stack buffer.

Unmaintained XML Parsing Libraries

Unfortunately, we discovered that Gurum DDS uses ezXML, an open-source XML library that has been in beta status since 2006 that has never been updated. The mailing list of the project has been silent since 2010, showing that no users are actively discussing it. The latest version was 0.8.6, but an inspection of the binary code revealed that the developer changed the version number to 1.0.0, which was the only change. EzXML currently has 16 known vulnerabilities (eight in 2021), all with medium to high severity ratings and have never been fixed. Probably because of its small footprint, we discovered that ezXML is also used in many embedded applications like router firmware, and has hundreds of downloads per week. We have reached out to Gurum DDS several times — about this and other vulnerabilities — since the summer of 2021 and have received no response.

Internet-wide Scanning for RTPS Endpoints

We wanted to demonstrate how an attacker could leverage the RTPS built-in discovery protocol for automated, large-scale reconnaissance of RTPS/DDS endpoints. We found hundreds of exposed services as a byproduct, which was unexpected. Understanding that RTPS/DDS was designed for local-network applications, we did not expect to find more than a couple of endpoints exposed by mistake. Not only did we find several hundreds, but 35 of them have never stopped sending responses to our scanner despite the fact that we only sent them one single RTPS packet.

After trying to use readily available internet scanning services (such as Shodan, Censys, and LeakIX), we ended up implementing our own scanning prototype because of the intricacies of the RTPS discovery phase. This makes it a bit convoluted to correctly fingerprint a service.

Challenges of RTPS/DDS Reconnaissance

In summary, the main challenges of RTPS/DDS reconnaissance are:

• Dynamic and arbitrarily large port range. Depending on the number of participants in a RTPS/DDS network, there can be tens of thousands of ports to check. The formula to calculate the port is defined in the specifications , and each implementation has distinct defaults, as shown in Table 7.2.

• Latency and connectionless nature. Although RTPS/DDS are transport-agnostic, the de facto standard is to use UDP, which makes efficient scanning techniques useless. To verify if there is a valid RTPS/DDS endpoint bound to a given address (IP and UDP port), we need to wait for an answer, which may or may come within a few seconds. Given the size of the public IPv4 space, it’s impractical to wait for answers upon each request.

• Addressing information at application layer. Addressing information is exchanged at the application layer. Sending a valid RTPS packet to the correct UDP port (for example, the default 7400 discovery port) does not guarantee a response, even if there is an RTPS service running. To trigger a response, the RTPS discovery packet must include correct locator information (for instance, IP and UDP port), which will receive a response.

Scanning Approach

Given the challenges mentioned in the previous section, we implemented a distributed scanning system (see Figure 7.9) that we first validated in a private network against all six DDS implementations, under the simplifying assumption that the developer would not go too far from the “default” set of ports listed in Table 7.2.

As shown in Figure 7.11, we created a template RTPS packet (using our Scapy RTPS layer, as shown in Figure 7.11) with a parametric locator IP and port number. We then generated an actual RTPS discovery packet by fixing the locator IP and port numbers according to the collector that we set up to receive the (reflected) packet. The collector will know what packet to expect given its IP and port number. Since there are several honeypots that simply reply to every request by echoing traffic they receive, the collector filters these “echoed” packets and keeps only valid responses. The collector checks if a received RTPS packet is valid by using the Scapy layer to dissect it and checks whether the globally unique identifier field (GUID) is new.

7.5Conclusion

This chapter has provided an in-depth analysis of robot software with particular focus on the the Data Distribution Service (DDS) standard and its implementation across various critical sectors, revealing significant security vulnerabilities and widespread exposure of DDS services. Through rigorous research methodologies, including network and binary fuzzing, our investigation uncovered numerous vulnerabilities, resulting in the issuance of new CVE IDs. The findings highlight the urgent need for continuous security testing and robust supply chain management in DDS and similar technologies.

The extensive exposure of DDS services on public-facing networks and the discovery of vulnerabilities in standard specifications call for immediate and long-term mitigation strategies. As we move forward, the focus should be on enhancing the security frameworks of DDS implementations and advocating for more secure practices in the development and deployment of critical middleware technologies. This research serves as a foundation for future work in securing robot software and other similar systems.

8Attacking Robot Hardware

8.1Introduction

Robotics is the art of system integration, as noted by . Building a robot requires one to carefully select components that exchange information across networks while meeting timing deadlines. In a way, a robot is a network of networks. One that comprises sensors to “read” the world, actuators to produce a physical change, and one for dedicated computational resources to process it all and respond coherently, in time, and according to its application. Roboticists often conceive the robot not as one of its parts, but as the complete system including all of its components, whether they are assembled under the same structure or physically distributed. In the case of a robotic manipulator, these robots are often presented physically distributed and include the robot arm mechanics (which generally include actuators and sensors), the HMI or teach pendant, the controller (the main compute substrate for reasoning), and any additional safety mechanism related to the robot operation. The robotic system is thereby the composition of all these sub-systems and networks.

Under such system integration complexity, it is not uncommon for one of the robot sub-components to fail over time, often leading to the complete system malfunction. Given the high price point of robots, it is reasonable to consider the need for repairing these machines, often replacing individual faulty components for continued operation, or simply for re-purposing them. The European Commission (EC) showed early interest on this topic in a report by evaluating different scoring systems for repairing and upgrading different consumer-oriented products, including robots. More recently, and as part of the Circular Economy Action Plan , the EC has shown commitment towards establishing a new Right to Repair in the context of reviewing directive 2019/771. summarizes major events in the U.S. with regard the Right to Repair and highlights that it wasn’t until 2012 that the Automotive Right to Repair passed in Massachussets, empowering customers with tools to fight planned obsolescence. Hatta summarizes how material obsolescence works:

• Making items difficult to repair (by raising the cost of repair, requiring special tools, etc.)

• Failing to provide information (for instance, manuals are not provided)

• Systematic obsolescence (making parts among models incompatible or making it impossible to fix newer models with parts from the older models)

• Numbering (frequently changing the model numbers to make it psychologically less attractive to use old models)

• Legal approaches (prohibiting access and modification to the internal structure of products by means of copyrights and patents)

noticed that, similar to Ford in the 1920s, most robot manufacturers follow several of these practices nowadays and organize dealers (often called distributors) or approved system integrators into private networks, providing repair parts only to certified companies in an attempt to discourage repairs and evade competition. Amongst the most recent examples we observe an interesting development from Teradyne, where two of its owned robotics companies (Universal Robots and Mobile Industrial Robots), follow this practice. The case of Teradyne is of special interest because its robots are advertised as collaborative, that is: designed to augment human capabilities by closely (physically) cooperating without causing any harm. Past research however hints that the lack of security measures in these robots leads to safety hazards, as concluded by .

Cybersecurity in robotics is still on its early stages, as demonstrated by . Therefore, as in many other fields, it remains addressed mostly in disconnected silos. With most efforts concentrated in IT, hardware security in robotics has received very limited attention. Building secure robots, however, demands consideration throughout domains (hardware, firmware, OS, application, network, cloud, etc.), , and across the robot lifecycle, .

The present chapter introduces and promotes robot teardown as a systematic process to repair robots, improve robot hardware and research its security. We advocate against the business priorities set in industry, which avoid repairs and planned obsolescence. Instead, we advocate for a Right to Repair in robotics as a means to reduce robot e-waste and recycle components, both across robots and throughout use-cases. Ultimately, we argue that, in the long run, the more researchers and practitioners will learn to systematically teardown robots, the more this practice will impact the quality assurance of hardware in robotics, putting pressure on manufacturers to produce robots with better (hardware) security measures, thereby safer. Our contributions are fourfold: first, we discuss the empirical results of three robot teardowns performed on popular industrial collaborative robots and uncover various quality, security, and safety flaws in the process. Second, we demonstrate how, as a result of a teardown, we gain repairing capabilities in the robots. This leads us to acquire means to mitigate security flaws early, by simply extending the robotic system with additional, off-the-shelf hardware elements that increase the overall cybersecurity posture with a minimal cost impact. Third, we show how teardown helps pinpoint security vulnerabilities across internal and external robot networks while discussing some of them. Fourth, we show evidence of planned obsolescence practices in robotics on leading industrial collaborative robots and demonstrate how by applying minor fixes, we managed to bypass the obsolescence limitations obtaining full control of the hardware across subsequent releases.

The content below is organized as follows: Section 8.2 describes the robot teardown process in three different robots and Section 8.3 the posterior reversing exercise to gain repairing capabilities. Section 8.4 argues about the obsolescence indicators encountered and demonstrates how to bypass them as a result of the results in previous sections. Finally, Section 8.5 summarizes our work and draws some conclusions.

8.2Robot teardown

A teardown is the process of taking apart a product to understand how it is made and works. More formally, it is the approach to modeling the functional behavior and physical components of a product, as described by . Robot teardown is thereby the process to study robot hardware architectures through systematic disassembly to understand how the robot works and what physical sub-systems compose it.

The motivation behind teardowns was previously researched by other groups such as . In robotics, we identify three key purposes: a) dissection and analysis to evaluate the status of a product, b) competitive benchmarking against similar products, and c) gain engineering experience and knowledge. This chapter focuses on a) and c). Particularly, we show three case studies on the robots from Universal Robots (UR) and Mobile Industrial Robots (MiR). Our motivation for selecting these targets is twofold: first, these robots are arguably widely used across use cases in the professional and industrial environments, with tenths of thousands of units sold and operating in close contact with humans (as collaborative robots). Second, past research by has shown a lack of security concern and readiness from these two manufacturers making them attractive targets for adversaries aiming to disrupt industrial processes or causing injuries as reported by . Disruption-based attacks, unfortunately, continue to be the most effective leverage used by financially driven threat actors such as DarkSide 26 , just to name the most damaging and recent one (as of June 2021).

Based on common teardown practices , we present in Figure 8.1 our teardown methodology for robots. The process involves 5 steps: 1. Identifies the purpose and scope of the teardown exercise. 2. Prepares for the teardown gathering required tools for documentation and dissasembly. 3. Examines the supply chain identifying how to acquire parts, what’s the installation process and who’s entitled for repairs, including costs and liabilities. 4. Takes apart the robot, documenting each step and avoiding the damage of any component. 5. Extracts relevant data (e.g. firmware version) from each robot component, constructs a BOMs and gathers additional information by researching public resources. To the best of our knowledge, we are the first to propose and document a teardown approach for industrial robotic products. The following subsections provide a walk-through on three case studies and discuss the most interesting findings on each one of them.

Case Study 1: Teardown of an industrial collaborative robot

Figure. UR3 collaborative robot teardown. Adapted from .

Figure 8.2 shows a selection of images obtained from the complete teardown of the UR3 CB3.1 industrial collaborative robot. Our goal is to show how a systematic teardown can lead to understanding how to obtain repairing capabilities of the complete robot, including the controller (i.e., the “brain” of the robot), teach pendant, and robot arm mechanics. We put particular emphasis in the CB3.1 controller since most safety-related electronics reside in there. The total time spent in the teardown is of five hours and 30 minutes.

An interesting observation is depicted in Figure 8.2f, which displays that the compute substrate in charge of implementing the safety logic is the NXP LPC4437JET256 microcontroller. While doing hardware reconnaissance we found the following excerpt within the part datasheet of the corresponding microntroller:

Suitability for use — NXP Semiconductors products are not designed, authorized or warranted to be suitable for use in life support, life-critical or safety-critical systems or equipment, nor in applications where failure or malfunction of an NXP Semiconductors product can reasonably be expected to result in personal injury, death or severe property or environmental damage. NXP Semiconductors and its suppliers accept no liability for inclusion and/or use of NXP Semiconductors products in such equipment or applications and therefore such inclusion and/or use is at the customer’s own risk.

In other words, this observation leads us to question the quality and reliability of the safety implementation within robots of the CB-Series by Universal Robots. indicated that vendors have historically opposed to teardowns under the argument that closed networks of dealers guarantee quality. However, our first observation indicates the exact opposite. Third parties with the required technical expertise might be able to identify and pinpoint hardware components that don’t meet the quality standards for the safety situations the robot may have to face, leading to an overall improved scenario for end-users.

Case Study 2: Teardown of a next-gen industrial collaborative robot

Following the CB-Series, we proceeded and disassembled one of the latest releases from Universal Robots, the UR3e, an e-Series. Figure 8.3 depicts the complete process through selected images. The complete teardown of the robot took us a total time of three hours 27 .

Figure. UR3e collaborative robot teardown. Adapted from .

We observe how, while the overall outer look remains similar, the internals have suffered a significant change:

• The e-Series controller integrates a single PSU, as presented in Figure 8.3e, while the CB-series had two (see Figure 8.2h).

• While the CB-Series presented two boards containing compute, power, and safety logic (Figures 8.2c and 8.2f, respectively), the e-Series presents only one single PCB named as “SAFETYCONTROLBOARD” and depicted in Figures 8.3f, 8.3g, 8.3h and 8.3i.

• Figure 8.3g shows that the new PCB includes a Xilinx Artix-7 series FPGA, widely used for implementing safety logic in a variety of automotive and control domains, and a much more reliable compute substrate for safety-related tasks than a MCU.

• Figure 8.3j shows that the base filter PCB—which helps interface power and RS485 communications from the controller (e-Series) to the robot arm mechanics—is similar to the one present in the CB-series. We also note that, while the arm mechanics connector changed in the e-Series (see Figure 8.3k), power and communications lines remain coherent (through the base filter board).

• For the most part, the electronics contained in the arm mechanics (Figure 8.3l) do not present relevant changes from an interoperability perspective. This facilitates re-purposing and reusing them (see Section 8.3).

Looking at the results of our teardown, we highlight the following observations.

Figure. MiR-100 mobile industrial robot teardown. Adapted from .

Case Study 3: Teardown of a mobile industrial robot

Figure 8.4 depicts the teardown process of a MiR-100, a popular mobile robot manufactured by the Danish Mobile Industrial Robots (MiR), also owned by the US Teradyne. The teardown took four hours and 20 minutes approximately. The first impression is that various components of the robot could be improved from a safety perspective, as highlighted in Figure 8.4g or 8.4i). Moreover, the teardown helped understand how this robot presents multiple (internal and external) networks and how each one of the sensors and actuators are connected across these networks, forming the data layer graph. One interesting finding resulting from the teardown is obtaining a better understanding of the robot’s computational graph (the behavior itself). The robot itself is powered by ROS () and gaining further understanding of the ROS computational graph requires understanding also its underlying hardware mapping (from which one derives the data layer graph). The teardown exercise yields exactly this, and allows to produce a data layer graph represented in the form of a hardware schematic which can then be used in combination with the computational graph to gain further understanding of the robot.

8.3Teardown-enabled security research

The previous section highlighted how teardown helped identify quality and safety issues in robots, as well as obtaining a better understanding of their architectures by matching each robot’s data layer graph with their corresponding computational graph. Beyond this, we argue that robot teardown is also key for security research in robotics. Recall that safety and cybersecurity are very related and influence one another, as noted by .

Teardown, as a process, is an essential part of a hardware reverse engineering task, and brings useful lessons and insights for the design of current and future robot systems. Generally, teardown supports Kerckhoffs’ principle in revealing all the details and weaknesses of a security system, excluding volatile secrets such as keys or credentials that are stored in memory and most likely disappear naturally once the power supply is taken away (with the exception of keys stored in permanent memory, which is generally discouraged, and would be discovered along the teardown process). Overall, the history of proprietary systems violating Kerckhoffs’ principle by pursuing “security through obscurity” is rich of failure cases (with the military domain as the sole exception), as a vast amount of related work demonstrates.

Reverse engineering has always been invaluable to discover vulnerabilities and develop remedies in many domains: network security (), access control (), embedded systems (), software engineering (), or the internet of things (). By promoting systematic teardown we want to extend this successful concept to the analysis of abandoned robots.

Particularly, and as part of this research, our group identified more than 100 security flaws across the three robots described above over a period of two years. Most of the flaws were cataloged as vulnerabilities and 17 obtained new Common Vulnerabilities and Exposures (CVE) IDs all of which are publicly disclosed at the RVD, proposed by . Table A.7 introduces some of the selected security vulnerabilities found. The information obtained through teardown helps pinpoint flaws across the multiple (internal and external) robot networks. In most cases, these robots present few or no security measures, allowing adversaries to easily exploit the flaws of internal components (e.g., , so compromising the robot behavior or taking full control of it.

We advocate for robot teardowns as a means to improve security in robotics and encourage manufacturers, integrators and end-users to carefully consider the underlying hardware architecture to protect their robotic systems. Similarly, we encourage teardowns as a tool to mitigate outstanding security flaws. Proper knowledge of the hardware helps determine which additional elements can help mitigate security issues when the manufacturer does not react. As an example, our group introduced an additional commercial off-the-shelf hardware firewall within MiR’s internal network, between the main controller and the safety PLC, by SICK AG, mitigating without having to modify any parts of the firmware. This modification could enable users and system integrators frustrated with MiR’s security policies to secure their robots directly.

8.4Lessons learned

Through this work we learned about the underlying hardware architectures that popular collaborative robots deploy today, researching security in the process. As a result of the teardown case studies described in Section 8.2, our group also identified several of the planned obsolescence indicators previously introduced in Section 8.1. Planned obsolescence was particularly evident in the robots from Universal Robots. To further illustrate this, Figures 8.5a and 8.5b depict the simplified electrical diagrams of the UR3 and UR3e robots. From an electrical point of view, these two robots present a similar layout for interfacing with the robot arm.

While we appreciate certain changes in the electronics, given the teardown results, we find no real reason why backwards or forward compatibility between controllers and robotic arms should not be possible. This would mean that existing customers with UR3 robots could repair and replace parts in either the controller or the robotic arm, without being forced to pay the premium price of buying a complete new set including both.

Unsurprisingly, we observe that the manufacturer introduced subtle changes meant to make this particular intent harder. One of such actions is depicted in Figure 8.3k, which shows the replacement of the controller-to-arm connector, which we can only justify with attempts to exercise obsolescence practices. Another of such actions includes the obscurity around the changes introduced in the UR3e robot arm itself. These changes can be summarized with the addition of an extra 6-axis force-torque sensor at the end of the robot. The exact same result can be achieved in UR3 robot arms by adding commercial off-the-shelf robot components, gaining such capabilities.

8.5Conclusions

In this chapter we presented robot teardowns as an approach to study robot hardware architectures, obtain repairing capabilities and research its security. We discuss the empirical results of three robot teardowns and the findings affecting quality and safety throughout the process. We then discuss how teardown is a relevant tool for security research in robotics which helps pinpoint security flaws early across the multiple internal and external networks in a robot. Moreover, we introduce our security findings and propose mitigations powered by the hardware know-how and repairing capabilities acquired. Ultimately, we research planned obsolescence practices in the robots from Teradyne and propose actions that could be taken to bypass obsolescence.

Our results show evidence that robot teardowns can help the robotics industry and supply chain by improving significantly quality, safety and security. Our findings extrapolate to most of the robots manufactured by Teradyne and its subsidiaries. We show concern for the currently growing trend in robotics to create private networks of certified groups, a common practice shown by manufacturers like MiR or UR, both owned by Teradyne. This difficulties system integration, repairability and ultimately security. We advocate for a ‘Right to Repair’ in robotics and encourage end-users to reflect their needs into their supply chains and into the original upstream robot manufacturers.

Figure. Simplified electrical diagrams of Universal Robots UR3 CB-Series (8.5a) and UR3e e-Series (8.5b) collaborative robots. Reproduced from .

9Attacking Robots in Industry

9.1Introduction

In the context of computer security, ransomware is malicious software (malware) that either locks a computer, prevents from accessing the data using encryption, or both, until the subject has paid a ransom. First ransomware Proof of Concept (PoC) appeared in 1989 when Joseph Popp, an evolutionary biologist and AIDS researcher, carried out an experimental attack by distributing 20,000 floppy disks in a Conference by the World Health Organization focused in AIDS research. The conference spanned to researchers from more than 90 countries, and malware was distributed claiming that the disks contained a program that analyzed risk of acquiring AIDS through the use of a questionnaire. Thereafter, ransomware called "AIDS Trojan" got distributed. Since then, it has evolved leading into two big subgroups:

• Crypto ransomware: which encrypts data and asks the user for a ransom in exchange for the decryption key.

• Locker ransomware: locks the system by some means, prevents its use and asks for a ransom to re-enable it.

9.2Background

According to literature , from 1989 to 2007 ransomware instances mostly focused on crypto-ransomware. In 2007, locker-ransomware began to appear and went mainstream. These ransomware instances locked systems and intimidated by displaying certain content (mostly pornographic images) while demanding a ransom to remove such content and unlock the systems. In 2013, Richardson et al. observed that attackers pivot back to crypto-ransomware. According to the authors, the most famous piece of ransomware was released in August 2013. Named as CryptoLocker, it was originally distributed via a botnet and later through e-mail. CryptoLocker used public/private cryptographic key pairs to encrypt the target’s file. Decrypting the files required to pay a ransom of 2 bitcoins (100 $ at the time) within the first three days. To the extent of our literature review and to date, variations of CryptoLocker remain being the most wide spread instances of ransomware .
In robotics, no targeted malware has yet been observed out of the PoC phase. Cesar Cerrudo and Lucas Apa earlier published a ransomware attack over Nao, a social robot by Softbank Robotics, which got the media attention. According to research being conducted on the security concerns of the robotic market 28 , only 9 % of robotics users have witnessed a cyber attack. This preliminary figure indicates that there is still very little activity yet known to the general public however, according to the same source, 51 % of the users inquired confirmed having identified security flaws in robotic systems which leads to consider that there exists a relevant landscape of insecurities. Confirming this hypothesis, users inquired assigned a rating of 8 out 10 to the security relevance in robotics yet only 26 % of the inquired acknowledged to have invested in robot cybersecurity, which includes evaluating security and protecting existing robot setups. Therefore, it seems that robot users do not fully apprehend the insecurity by design governing robots in the market.
In this chapter we aim to illustrate the existing insecurity status in some robotics vendors . We have selected one of the most popular industrial collaborative robots and present Akerbeltz, an instance of ransomware targeting industrial robotic systems. We present the PoC ransomware attack, describe the rationale behind our target and discuss the general flow of the attack including the initial cyber-intrusion, lateral movement and later control phase. We then briefly discuss the resulting consequences from the installation of Akerbeltz and wrap up by sharing some major conclusions.

9.3Akerbeltz

In basque mythology, Akerbeltz is an antique deity impersonated in a male goat which is the animal-kind protector. Ethimologically coming from the Basque works Aker (male goat) and Beltz (black), its origin is attributed to a meadow in the surroundings of Zugarramurdi caves (Navarra), a pligrimage place for Basque Mythology. Akerbeltz is the demon that is chairing "Akelarres" or Basque witch (sorginak) meetings. Some authors note that the mythological figure Akerbeltz represents was adored in many European countries, some of which remain up to present day. Akerbeltz often shows two different faces. On the one hand, it is the protector of animals and is even able to heal their illnesses if needed. Indeed, belief on Akerbeltz is thought to be the origin of hosting a black goat as the protector of all animals within a "Baserri" (Basque cottage or farm). On the other hand, when Akerbeltz participated Akelarres, it showed his darkest face, where witches obeyed and adored him as the genius representing revolution against established status-quo, amidst banquets of human flesh.
We advocate for a change in (most) robot manufacturer’s attitude towards security and to do so, we take Akerbeltz as inspiration and present below the first instance of industrial robotic ransomware. Due to our concerns about malicious applications of the software, authors will not be releasing the source code nor the low level method of operation of this industrial robot ransomware.

Target selection and rationale

Our target was selected based on the results obtained from prior research on the vulnerability landscape across several industrial robot manufacturers. Our critical attitude was previously introduced by Alzola-Kirschgens et al. and essentially builds on the fact that several robot manufacturers, particularly collaborative robot (cobot) vendors, are profiting the popularity of these devices, via third parties (distributors and integrators) totally disregarding the ethical consequences of not caring about security, to the point that security vulnerabilities are left unadressed, or "up to the end user". To further understand our viewpoint, we ask the reader and potential robot users to consider the following questions: Thousands of insecure robots are being deployed all around the world, some of them thought to be networked and to increasingly collaborate with humans, who will be held responsible when these security holes get exploited and cause human damages? How would robot vendors respond when these reported, non-patched and public vulnerabilities cause safety hazards? Would they continue claiming that "their robots are designed to be open" and thereby never meant to be used beyond research? Should authorities regulate the use of these devices and their corresponding incomplete standards?
The authors discussed on all the above and coherently, decided to select what arguably is the most popular collaborative industrial robot manufacturer: Universal Robots. We select one of their best-selling robots, the UR3 and implement our PoC on it. Figure 9.1 shows a picture of our target together with the control box and teach pendant.

Before our work, Universal Robots had other groups assessing their insecurity. In 2017, Cerrudo and Apa reported five 0-day vulnerabilities. Several months later, representatives from the vendor acknowledged that security patches had been applied yet there seems to be no public information available. Instead, it appears, the vendor disregarded the previously reported issues under the claim that attackers required very specific conditions. Further to that, Jacob Bom Madsen, Software Product Manager at Universal Robots publicly claimed that Universal Robots is "proud to have a fairly open architecture, that allows system integrators and UR+ Partners to easily develop and integrate the solutions they need." . This attitude conflicts directly with the very principle of Universal Robots safety claims, previously highlighted by Cerrudo and Apa : "Do not change anything in the safety configuration of the software (e.g. the force limit). If any safety parameter is changed the complete robot system shall be considered new, meaning that the overall safety approval process, including risk assessment, shall be updated accordingly". In other words, any modifications of the safety setup in the UR3 will lead to the complete invalidation of the robots’ compliance with ISO 10218-1 incurring in potential relevant losses and conflicts for the end user. Beyond the human and economical damages caused by modifying the safety setup of the UR3, the claim by Madsen leads to a troubling an arguably unethical statement: "The lack of security facilitates system integration". We have seen this statement repeatedly and yet, once again, this time coming from a leading cobot vendor, we see how openness and feature inclusion is used to justify the lack of security. Wielding the interoperability pitch, vendors push security up to their partners, collaborators or ultimately, to "the community" avoiding security actions, critical for the use of these products in human environments.
At the time of writing, our team knows yet of no security patch mitigating these vulnerabilities. Moreover, we performed a penetration testing assessment in the UR3 CB series robot confirming the still unpatched existence of several of the previously reported flaws. Furthermore, we found more than 300 new vulnerabilities of relevant severity according to robot-specific scoring mechanisms . The overall picture depicts a vendor which shows the little care not only for security, but also for quality of software.
The following subsection elaborates on how Akerbeltz acts on our UR3 CB series.

Ransomware’s flow

Cyber intrusion

Initial infection gets realised by exploiting unpatched vulnerabilities in the robot. Akerbeltz’s initial cyber intrusion is implemented exploiting these well known vulnerabilities and deployed via one of the following two attack vectors:

• physical USB ports in the teach pendant: Exploiting CVE-2019-19626 29 an attacker can execute malicious code with root privileges by simply connecting an external USB stick. This attack vector is likely the easiest to implement given the exposure of the teach pendant in most UR3 robots. Moreover, the same attack vector applies not only in the teach pendant but also in the control box which exposes internal USB ports that can be accessed with standard mechanical fixations.

• remote access via adjacent networks: RVD#672 (CVE-2018-10633) permits an adjacent network attacker to easily ssh into the control box. Alternatively, RVD#6 allows an adjacent attacker to exploit a stack-based buffer overflow in the UR3 ModBus TCP service, and execute commands as root equally fulfilling the cyber intrusion. Akerbeltz can be configured to exploit these flaws and take action over industrial LANs.
  

Lateral movement

Since the cyber-intrusion allowed to obtain root privileges in the control box, no privilege escalation phase is necessary within Akerbeltz. The "open" architecture of the robot facilitates lateral movement to other sub-devices within the robot. We briefly explored such setup and found that it is trivial to access the robot user interface, the PolyScope. Figure 9.2 shows a preview of the ransomware’s message in the teach pendant:

The UR3 lack of security allows for additional lateral movement. Both the BIOS of the robot controller and the safety PLC are easily accessible and exposed. While it remains beyond the scope of our study, we argue that targeting any of these systems (or both together) will likely lead to malware that could damage the robot to a point of no return and would likely be matter of future security research.

Control

• Locking: In our PoC, our team was able to lock the whole system while displaying a message (refer to Figure 9.1) after booting, using previously mentioned CVEs, we managed to access the control box, acquire root privileges, disable the default safety configuration and change the default user and password. We also identified and disabled several open ports used to control the robot from the outside, blocking other mechanisms to reestablish normal operation. Our work with Akerbeltz finalized by performing some minor and non-exhaustive hardening, meant to avoid users to unlock the system.

• Encrypting: After identifying most of the critical files used for the control of the robot, we located previous programs and IP deployed within the control box. These files are encrypted using the local pgp binary in combination with a series of robot-specific identifiers.

Discussion

In an attempt to responsibly disclose and mitigate the existing flaws, we generated a series of reports for selected vulnerabilities and approached the manufacturer by e-mail. To this date, no formal answer with intention to establish discussions on security has been received. In a second attempt, making use of well established and de facto approaches in security, we filed for a CVE identifier in one of the new discovered vulnerabilities. At the time of writing, no formal communication has been established via this channel either.
In a third attempt, the PoC attack was disclosed to Universal Robots directly, in a public robotics industrial conference, the ROS-Industrial Conference in Europe (Stuttgart, December 2019). While maintaining the vulnerabilities undisclosed, we publicly presented 30 to the vendor and the rest of the audience the consequences of their insecurity. We briefly presented Akerbeltz and followed with a possible solution to mitigate existing flaws in their robots.
The deployment of Akerbeltz in a UR3 via the physical attack vector has been recorded and made available at https://youtu.be/VF7fcV5j1t0 arguing that by making this public and available we finally call to a reaction of this particular vendor or its associated value chain. Last but not least, our final discussion point is that these kind of targeted attacks have the potential to spread rapidly and across the use cases in which the Universal Robots CB-series are deployed. Given the low complexity of the attack, the easiness of programming on top of Universal Robots programming environment and the relative high cost of the assets involved, a series of attacks on this taxonomy are foreseen.

9.4Conclusions

The current insecurity status in robotics allows for malware to be created easily and rapidly by simply evaluating known vulnerabilities. These pieces of malware could be used and exploited by malicious actors to gain economic profits via extorsion. In this chapter we presented Akerbeltz, a piece of ransomware that locks and encrypts industrial collaborative robots from Universal Robots. After the targeted attack, the result is rendering the industrial robot totally useless, losing IP within the robot and potentially leading to economical losses and human or environmental damages. We described and prototyped Akerbeltz for the UR3 CB series robot which could be introduced following physical or network based attacks on zero-day vulnerabilities.
At the time of writing, Universal Robots has yet to answer our vulnerability reports, which in some cases provide the manufacturer relevant insights on how to mitigate the most relevant flaws exploited by the attack. As a particular example, for the physical attack vector we exploited in Akerbeltz, a possible mitigation involves the validation and sanitization of specific files as well as the implementation of Discretionary Access Control (DAC) model by providing Mandatory Access Control (MAC) within the control box. This will limit the programs’ capabilities with per-program profiles (e.g. via AppArmor).
When asked directly, face to face, representatives of the vendor, they indicated once again that their robot "is designed to be open" or that we are "disabling features" and therefore, "it’s just normal". We would like to express our strong concern for a reiterated lack of security actions. While our team did not explore malicious attacks or their profitability on the robot any further, we argue that a more aggressive individual or group, with bad intentions, might easily come up with some sort of "self-destructing" malware (by disabling safety and repeatedly crossing the boundaries of the kinematics model and hitting itself) or worse, a "human or environment damaging" one, or any other kind of creative exploits on top of the vulnerabilities exploited by Akerbeltz.
The lack of communication from the vendor’s perspective makes us guess that, at best, there is security by obscurity around Universal Robots. Once again as we did in the past, we argue against this and advocate for a proactive interaction of robot manufacturers with security researchers.
Other industry case studies have also been conducted as part of this research and have been made available at . These studies cover a broad spectrum of penetration testing scenarios, including the analysis of widely-used industrial robot arms from manufacturers like ABB and KUKA, the security testing of Mobile Industrial Robots (MiR) in healthcare environments, and the exploration of cybersecurity vulnerabilities in commercial drones.

Automation

10Determining the Attack Target with Game Theory

As we venture into the intricate landscape of offensive robot cybersecurity, particularly within the realm of automation (Part V), the act of selecting an attack vector transcends brute force or mere choice—it becomes a strategic decision of profound consequence for both defense and offense approaches. This chapter ventures into the analytical domain where the discipline of cybersecurity intersects with the predictive prowess of game theory. Herein, we dissect the art of anticipating an adversary’s move, transforming what is traditionally an intuitive decision (heuristics) into a calculable prediction.
The quintessence of this approach lies not just in its offensive application but equally in its defensive potential. By leveraging game-theoretic principles, we argue an automation engine can forecast an attacker’s most probable vector of exploitation (the attack vector). Such predictive capacity is paramount; it informs how we can judiciously deploy our finite security defensive resources where they are most needed—effectively outmaneuvering potential breaches before they materialize. This is ultimately the objective pursued in this thesis with the offensive robot cybersecurity approach.

This chapter is not only a discourse on methodology and theoretical concepts but also a tactical guide. It reveals how game theory serves as the toolbox in the creation of cybersecurity cognitive engines that adapt dynamically to the ever-shifting tides of digital conflict. These engines, armed with the ability to discern the most vulnerable vectors, provide an indispensable tool for entities tasked with safeguarding automated systems. Through a meticulous blend of theory and practical application, this chapter embarks on a quest to ascertain the optimal point of action—be it for a defender fortifying their bastion or an attacker seeking a breach—in the digital arena where automation’s role is ever-expanding and critical.

10.1Introduction

The daily business of a CISO concerns seeking a proactive defense against an invisible intruder and more elaborated attacks. Such threats are often well represented by contemporary APTs, a type of cyber attack executed by well-resourced and capable attackers, such as nation-states or organized criminal groups, aiming to infiltrate a network to extract valuable information or disrupt operations. These threats are "advanced" in the sophistication of their tactics, "persistent" in their long-term objectives, and stealthy in nature, often remaining undetected for extended periods. APTs typically involve a high degree of covertness over a prolonged duration, with the intention to steal data rather than cause immediate harm. This work is concerned with the daily business of defense, under the assumption that the infection has already happened, but there has not been any damage so far, thus we are in the “incubation” phase in the APT life-cycle. A refined view on the evolution of an APT is the kill chain , depicted initially in Figure 2.6.

The contributions presented here focus on the study of a game theoretic model intended to support CISOs and/or to serve as a tactical guide that helps autonomous cybersecurity agents to make decisions and reason about the threat landscape from an attacker’s perspective. Thus, coherent with APTs, we assume that the adversary is already in the system 31 , and the CISO duty is preventing damage. The battlefield on which the CISO, hereafter called the defender, matches the stealthy adversary, is an attack graph. This is a graph-theoretic model of a system, relating individual system components to threats and exploits on them, and thereby visualizing possible attack paths towards defined target nodes. Such attack graphs can be compiled from threat modeling activities and further refined with penetration tests and help of topological vulnerability scans (see, e.g., ). Alternatively, attack graphs (also referred to as the attack tree or the exploit tree in this thesis) can be built by tools like ExploitFlow . Figure 10.1 depicts how this work fits into our architecture to develop fully automated offensive robot cybersecurity tools, a cognitive cybersecurity engine we denominate Malism.

On the attack graph, we play a stealthy intrusion game with infinite time horizon and repetitions, in which the defender takes action to keep the attacker away from the critical assets in the enterprise, while the adversary is on its way towards some target asset, along one or more attack paths. The game instantly terminates if the attacker has reached the critical asset, in which case the defender (permanently) lost. The interaction between the two players is constrained as follows:

Assumption 1: The attack graph \(G=(V,E)\) consists of \(V\) nodes and \(E\) are the edges (e.g., exploits). We assume a single starting node (for all attacks), and a single target node, representing some critical asset to be captured. The graph is assumed as acyclic (thus, the attacker will never enter infinite loops along accomplished exploits), and all paths, more precisely the attack paths/vectors, lead to the target asset. We denote this target as \(v_0\in V\) hereafter. The nodes in \(V\) represent system threats and vulnerabilities, or system states, while edges represent threats or exploits to get from one component/state into the next component/state. For example, the adversary may jump from a desktop computer \(v_{pc}\) to a server \(v_{server}\), or may gain root privileges, denoted as \(v_{pc,root}\) from user-level privileges \(v_{pc,user}\) on the same computer. In either case, we would have a directed edge \(v_{pc}\to v_{server}\), or \(v_{pc,user}\to v_{pc,root}\) to express this attack path in the graph model.

Assumption 2: The adversary is already somewhere in the system at an unknown location when the defender enters the gameplay (i.e., we are past the event of infection/reconnaissance). We also assume that the defender has no indication of adversarial activity (for example, from an Intrusion Detection System (IDS)); the adversary is stealthy. In absence of adversarial signals, the defender may assume all possible locations of the adversary as uniformly distributed (the inclusion of signals is discussed in Section 10.4.5).

Assumption 3: the adversary may run parallel or concurrent attacks, thereby exploiting several, up to all, attack paths simultaneously to maximize its chances to conquer \(v_0\).

This general setting was converted into a game theoretic model in , named Cut-The-Rope: Like on a chess-board, the adversary in Cut-The-Rope runs parallel attacks, one on each attack path available. Since the attacker’s location on each attack paths is unknown to the defender, this player imagines a whole “cohort” of avatars starting from all possible locations in the network and moving towards \(v_0\). The strategic choice of the adversary, from the defender’s perspective, is about existing attack paths, but the defender does not know where the attacker is, equivalently, how far the adversary has already come down an attack path \(\pi\). To tackle this uncertainty, the defender plays the game as if the adversary would first (strategically) choose the path \(\pi\), and move all avatars on \(\pi\) simultaneously towards \(v_0\). In other words, it moves avatars at all possible, not necessarily also probable, locations. The target asset (and security game) is lost (to the attacker), if at least one of the adversary’s avatars reaches \(v_0\).

For the adversarial movement, Cut-The-Rope lets the defender assume a random distribution on how many steps an avatar can take when it is on the move. Figure 10.2 provides a high-level overview of the games involved in the method. This random movement pattern is a model design choice, and the original work about Cut-The-Rope assumed one specific movement regime, in which the defender acts periodically, and the attacker has some “aggressiveness level” \(\lambda\), interpretable as an “expected number exploits per day” or within a defined unit of time. This amounts to a Poisson-distributed number of steps taken along the attack path.

Research Questions and Contribution

Some companies have CISOs with a fixed working schedule, which corresponds to a periodically active defender (working days, day/night-shifts, etc.). What if there is a 24/7 continuous response team available, like in bigger (globally distributed) companies that run their own security operations center? These (multiple) defenders may become active at random time intervals and at any time. For the security game, it means that the defender will not be active periodically, but rather at random times and possibly at any time. This is the first new movement contributed and studied in this chapter.

Both, the original Poisson movement and the just described continuous security response policy are agnostic of the particular details of exploits, like their difficulty or severity. However, many attack graphs do carry additional information about exploits, and if so, it is desirable to use it in the security model. These additional details can range from a security threat research and risk assessment or scoring like CVSS, up to proof of concept implementations for each exploit. The second new movement pattern proposed in this chapter makes the attacker’s traversal dependent on exploit complexities (threats assigned a higher complexity would thereby be probabilistically less feasible), as far as they are known, and studies the defense performance against a defender that is again periodically active.

The third new movement pattern is a combination of a defender that can take action at any time in a 24/7 continuous security provisioning, against an attacker that has to deal with threats and exploits of different complexity.

We stress that none of these cases assumes a purely reactive defense, i.e., we do not study security response patterns when the incident has been noticed. This is due to the assumption of stealthiness of the intrusion; once the attacker becomes visible, it is because the target asset \(v_0\) was lost (permanently).

The other contributions of this chapter are two case studies: First, we provide an experimental study and illustration of how to use Cut-The-Rope in different settings, based on two documented attack graphs for industrial robots. Specifically, we look at the Modular Articulated Robotic Arm (MARA) and Mobile industrial Robotics MiR100 robots , for which attack graphs have been compiled by security experts. On these, we instantiate Cut-The-Rope and compute results in the aforementioned settings of a periodically/randomly active defender versus an attacker that traverses an attack path with uniform speed at an average number of exploits per time unit, or mounts attacks with individually distinct exploit complexities, thus being slower or faster, depending on the chosen path. Experiments are conducted on the MARA robot, for which the threats and exploits are known, but without a CVSS rating or further details. In that case, the original Poisson model from and the first of the new patterns announced above are usable. The other case study is on the MiR100 robot, whose attack graph carries additional vulnerability and exploit details.

10.2Case Studies

We dedicate the next couple of subsections to numeric results, starting with a brief correction to past calculations in the literature, and then moving onward to the new case studies and the comparison of defense policies optimized with Cut-The-Rope, versus a heuristic common-sense defense policy.

To assess the game w.r.t. a real-life application, we conducted two case studies on the industrial robots in Section 10.2.2. The game is similar to capture-the-flag competitions known in ethical hacking, since there and also here, the goal is to “capture” a target asset \(v_0\). Our analysis, different from ethical hacking, is purely game-theoretic and optimization-based here.

Implementation Remarks

We adapted the implementation from and thereby discovered a few bugs in this older code that we corrected in our version. The original code used fictitious play on the full distribution \(U=(u_1,u_2,\ldots,u_n=\Pr\)(adversary reaches \(v_0))\) obtained from equation (B.5).

We compute an optimum w.r.t. a lexicographic order from right to left, first minimizing the last coordinate \(u_n\), and breaking ties by continuing to minimize \(u_{n-1}\) while keeping \(u_n\) at minimum. The next tie is broken using \(u_{n-2}\), while keeping the so-far optimized coordinates at their minima and so on. This introduces a dependence on the ordering of the coordinates, corresponding to a likewise ordering of locations in the attack graph. Therefore, the solution returned by the implementation from is ambiguous in the sense of depending on the node ordering.

The optimization, however, independently of the node order, always minimizes the chances to reach the target asset, and hence provides a valid defense policy w.r.t. the targets of the defender. Our implementation inherits this dependence on the node order, but since our sole interest is reaching or avoiding to reach \(v_0\) anyway as (B.7) and (B.8) define, this ambiguity is not a limitation. In light of this, we chose the graph-topological sorting to order the probabilities in \(U'\) other than for \(v_0\), which is the last element in this vector.

The attack graphs for our robot case studies have several entry points for the attacker and also several targets to reach. To handle them all in a single run of the analysis, we added an artificial (virtual) entry node from which all (real) entry nodes are trivially reachable (with probability 1). Since Cut-The-Rope in the original version, analyzed here, assumes only one target, we contracted the multitude of target nodes into a single “compound” target node. This corresponds to the target being to reach any of the possible target nodes, not distinguishing which in particular. A target node is, by default in our implementation, any node that does not have descendants (zero out-degree in the attack graph). Consequently, all inner nodes, except the virtual start, are possible defense spot-check locations. The technical simplification towards having one target (only) is to avoid multi-criteria optimization, which is theoretically possible (even supported by the packages to run the optimization practically), but is more involved to interpret for a defense policy.

Robot Case Studies

For both of the robot cases to follow, we give computational results and a discussion of their practicality. To avoid confusion between the attack graphs appearing here and those found in the cited literature, we use the original versions thereof to visualize the battlefield and results. The actual simulation was done on an attack graph with added virtual starting and a single compound target node (if more than one exists).

It is perhaps practically interesting to remark that both attack graphs have inner nodes that classify as attack targets, but have descendant nodes as subsequent attack targets. With the convention of taking nodes with zero out-degree in the graph as targets (see Section 10.2.1), the simulation will include all “inner” nodes as defense locations even though they may be attack targets too. This is not precluded by the game design, and may be interpreted as considering inner nodes as “intermediate targets” whose prevention may avoid subsequent final, perhaps more dangerous, attack targets. The game’s defense policy would then advise to prevent a certain attack sub-target in the attack graph, with a certain level of effort (expressed as likelihood). From a simulation perspective, including or excluding any node from the defense policy is a simple matter of defining the action set for the defender accordingly.

Case #1: Modular Articulated Robotic Arm (MARA)

MARA is a collaborative robotic arm with ROS 2.0 in each actuator, sensor or any other representative module. Each module has native ROS 2.0 support, can be physically extended in a seamless manner and delivers industrial-grade features including synchronization, deterministic communication latencies, a ROS 2.0 software and hardware component life-cycle and more. Altogether, MARA empowers new possibilities and applications in the professional landscape of robotics. The use case considered contemplates the MARA modular robot operating in an industrial environment while performing a pick & place activity. Details about MARA for this case study can be found in .

Case #2: MiR100 - Mobile industrial Robotics

The MiR100 autonomous mobile robot is advertised as a safe and cost-effective mobile robot that quickly automates your internal transportation and logistics. The robot claims to optimize workflows, freeing staff resources so you can increase productivity and reduce costs. A case study analyzing the cyber-resilience of MiR100 robots was conducted and documented at and , which considered a single robot operating in a structured environment while connected to a local area network that gets compromised. Through the local area network, prior work demonstrated how an attacker could exploit vulnerabilities, pivoting across subsystems in the robot all the way into its safety system, disabling it fully in a remote manner.

For both robots, we took attack graphs out of industrial security assessments, which, in the particular case of MiR100, were also annotated with CVE and CVSS information, which allows an assessment of the “hardness” of vulnerabilities along the attack path. Such annotations were not available for the MARA use case, which, in lack of such details, suggests an application of the Poissonian movement pattern of Section B.3. The more detailed attack graph for the MiR100 robot enables the consideration of probabilistic success on exploits as Section B.3.2 described.

10.3Results and Comparison

To evaluate how much a game-theoretic defense may add to the security, we do not only give the absolute results from the simulations, but also compare them to a heuristic best-effort defense policy, described in Section 10.3.1. Its simulation is run likewise with each of the four movement patterns from Sections B.3, with the probability to reach \(v_0\) given for each case as (i) optimized by Cut-The-Rope versus (ii) according to a best-effort defense.

Baseline Comparison: A Best-Effort defense Policy

For an assessment of the quality of the game theoretic defense, let us use the following heuristic defense policy to compare:

• We assume that an adversarial avatar will always follow the shortest, or “easiest” attack path towards \(v_0\). The distinction between shortest and easiest is made in dependence of how much is known about exploit complexities. In the MARA use case, the path choice will be for shortest, in terms of the number of exploits, since there is no further detail given about the exploit complexities. In the more detailed MiR100 use case, we have attack complexities and can likewise apply a shortest path algorithm to guide the attacker to the path whose success probability (as the product of all exploit success probabilities) is maximal 32 .

• The defender, unbeknownst of where the attacker is, and unable to actively detect it, applies a uniformly random defense strategy. That is, if the attacker is equally likely to be anywhere in the system, the defense policy would likewise be a uniformly random spot checking.

Under these hypotheses, we apply the same mechanism as in Cut-The-Rope, i.e., we let the attacker follow its chosen (shortest/easiest) path, and be occasionally sent back by the defender upon a coincidental cut of the path equivalently, closure of any backdoor. If so, then the avatar will keep retrying, until it hits the final target \(v_0\). Note that this regime also includes lateral movement, since we still have a multitude of avatars attacking in parallel, each on its individually optimal route from its starting location \(\theta\) towards \(v_0\).

We implemented this defense policy simulation by adapting the code from the implementations of Cut-The-Rope accordingly, to implement the heuristic defense and attack policy of above. Like for the game optimization, the heuristic defense implementation outputs the probability to reach \(v_0\) by simulating this defender-attacker interaction. We remark that this heuristic defense may still be overly optimistic relative to real life situations, in which defense teams may have only an incomplete view on the attack graph \(G=(V,E)\). The defender would thus only be active on a subset \(D\subset V\), so that all nodes in \(V\setminus D\) would be zero-day exploits.

Overview of Experiments

In total, comparing the periodic/exponential defense strategy against a randomly moving adversary in two use cases, gives a total of 4 evaluation scenarios, each accompanied with its own comparison to the baseline heuristic of Section 10.3.1. Table 10.1 relates the sections and figures in the following to these four configurations.

MARA: Results

The attack graph for the MARA robot is taken from and shown in Figures 10.3 and 10.4. This graph has 11 nodes and 10 edges in total, among them one entry point (node 1) for the attacker, and two targets (nodes 6, and 9).

We played Cut-The-Rope on this graph with a periodic defender versus an attacker that takes an average of 2 moves per time unit (i.e., in-between two appearances of the defender, e.g., per day). Figure 10.3 shows a table with the probabilities to spot-check each node on the attack graph. For the visualization, we have put bubbles on the attack graph, whose size corresponds to the probability of spot-checking there. That is, the larger the bubble, the more effort should be out on defending at this point.

Optimal defense policy:

Efficacy of the optimal vs. heuristic defense:

Turning to the case of the defender coming back in random time intervals, we let the game run in three configurations, with the defender moving slower (\(\lambda_D=1<\lambda\)), at equal speed (\(\lambda_D=\lambda=2\)) and faster than the attacker \((\lambda_D=3>\lambda)\) in (B.12). The resulting spot checking probabilities are again displayed as bubbles located at the respective nodes in the attack graph, and put over one another in Figure 10.4.

Optimal defense policy:

Efficacy of the optimal vs. heuristic defense:

The numbers and bubbles are almost of the same size, showing that for the defense locations, the speed of spot checking has only a negligible impact, while the performance of the defense accordingly becomes better if the defender is “more active”. The performances of the defense policy as displayed in the bottom table of Figure 10.4 show that the optimized defense pays over the heuristic “blind” spot checking policy.

The takeaways from these findings is not that a more intense defense activity will reduce the chances of the attacker (this would be obviously the case), but rather giving the defender an indication of where to allocate its (limited) resources to gain the best possible effect. Without signaling and without additional information in the attack graphs, the results are necessarily a crude approximation of reality, and Cut-The-Rope has been designed to be workable in such a situation of limited information, as well as with cases when more details are available, such as for the MiR100 robot following next. The results in the rather little detailed MARA use case are quite evident but therefore also plausible (“guard the closest graph cut between the asset and the defender”). The nontrivial indication here is the advice to let the attacker come “close” to the asset, while a defender would perhaps otherwise try to guard the outer perimeter of the system to keep the intruder out in first place. The optimum to be at the closest graph-cut towards the asset is here explainable by our assumption that the attacker is stealthy and can start from anywhere, and in a practical situation, the defender may indeed have no reliable information about infected parts (otherwise, it would be trivial to disconnect and repair/replace the malfunctioning component). The defense policy that Cut-The-Rope computes is for practitioners operating blue teams that need to protect a large attack surface with no monitoring or signaling. A game-theoretic defense can help prioritize resources.

MiR100: Results

Similarly as for MARA, we used an attack graph for the MiR100 robot as shown in Figures 10.5 and 10.6. The attack graph has 16 nodes and 24 edges. The attacker can enter at four points (nodes 1…4), and four targets (12, 13, 14 and 16).

We conducted the likewise experiments under the same configurations as for the MARA use case, but this time making use of the CVE annotations to give information on how hard it is for the attacker to mount an exploit. For the defender, we again assume this one to be periodically active (as in Section B.3.1) and to randomly spot check (as in Section B.3.4). Note that in this case we do not have an attack rate \(\lambda\) as for the MARA use case before, since the movement of the attacker is solely governed by the difficulty to mount exploits.

Optimal defense policy:

Efficacy of the optimal vs. heuristic defense:

It is interesting to note that the optimal defense policy does not advise to guard node 7 or 9, which is a way towards reaching goal node 14. This may be assumption of the game, of the attacker already being somewhere in the network. The defense policy accounts for this and hence does not put more weight on lower nodes with higher incidence index. This way, the model accounts for defense in depth rather than entry prevention.

Finally, let us turn to the case of the adversary working towards \(v_0\) only in random time intervals between two appearances of the defender. This time, the defender’s parameter \(\lambda_D\) is the average “window size” \(W\) (see Appendix B section B.6), measured in units of time, e.g., days. It is the time that we give the attacker to mount activities in the game. The results are shown in Figure 10.6.

Optimal defense policy:

Efficacy of the optimal vs. heuristic defense:

Similar as for the MARA use case, the defense locations are the same in all cases, with the defense efforts only slightly differing according to how large the window is for the attacker, respectively, how frequently the defender comes back. The performance of the defense is shown in the bottom table of Figure 10.6. Consistent with the intuition, the attacker’s chances to reach \(v_0\) become larger if the defense window is made larger. In both, the experiments with the heuristic defense and optimized under Cut-The-Rope, the value \(\lambda_D\) gives the average number of time units before the defender comes back. That is, larger \(\lambda_D\) give the attacker more time to exploit (conversely to the interpretation of \(\lambda_D\) in the other experiments, where it was the frequency of the defender’s return). Again, the experiments show that Cut-The-Rope outperforms the heuristic defense considerably.

10.4Discussion

The experimental findings suggest that the apparent optimal defense strategy delivered by Cut-The-Rope is to guard the immediate neighborhood of the target asset, so as to cover cases where the attacker has already deeply penetrated the system when the game begins. Indeed, an analytic characterization of the optimal defense under Cut-The-Rope is obtained in Appendix B section B.4 as Proposition B.4.3. It confirms a certain graph cut to be optimal under certain assumptions, but not in all of our test cases. For this reason, we leave the discussion of analytic results as an appended remark here, and continue the discussion with more practical aspects.

Incomplete Attack Graphs and Zero-Day Exploits

The heuristic defense of Section 10.3.1 may in reality be still over-optimistic, in its assumption of complete knowledge about the attack graph. Practical defense teams may only have a limited knowledge or possibility to construct the entire attack graph, and it is generally unaccomplishable for the defender to get exactly the same attack graph as the adversary has. The simulations implemented in this chapter have been made with the possibility to include only a randomly chosen subset of nodes in the defender’s possibilities to spot check, so include such incomplete knowledge in the analysis. Concretely, the code was made to randomly reduce the defender’s spot check locations to, for example, only 75% of the nodes in the attack graph. Under such reduced possibilities, the game runs against an attacker with more, i.e., full, knowledge about the attack graph. We confine ourselves here to reporting that the defense policies performed worse than under full knowledge (not surprisingly), but both policies (Cut-The-Rope and the heuristic) lost performance at approximately equal magnitudes, leaving their relative quality over one another without substantial changes.

We emphasize that a simulation under such reduced knowledge for the defender, whereas giving the attacker full knowledge, can be viewed as a study of the impact of zero-day exploits used by the attacker. That is, any node excluded from the defense, but used by the attacker is nothing else than a zero-day vulnerability. Since a systematic account for this would be beyond the scope (and space limits) of this work, we will explore this route along future work.

Cutting the rope vs. Changing the Attack Graph

In our experimental instance of the game, we let the attack graph remain unchanged over time. In particular, we assume that none of the defender’s actions causes a permanent removal of a certain backdoor. This is practically motivated by the fact that spot checking may remove some, but not all vulnerabilities, so that, for example, one buffer overflow vulnerability in a secure shell implementation might get fixed, but other exploits of the same kind remain open, making the respective nodes remain unchanged in the graph after an inspection. Likewise, remote shell access may be required for the business workflow and hence cannot be deactivated, but only the access credentials might be updated. In that case, the remote shell access exists before and after an inspection. Even though the game model itself uses a static attack graph (see Figure 10.2), this one may itself require an update from time to time upon changes in the infrastructure. This is part of the business continuity management related to security, and accordingly changes the action sets for the defender and attacker. The implementation of the game, however, remains unaffected, except for the specification/input of the attack graph. We close the discussion at this point, referring to Appendix B section B.5 for a continuation of this discussion.

Further Generalizations

The movement patterns as studied admit further modifications and generalizations, yet to be explored, such as:

Probabilistic Success on Spot Checks

First, to the advantage of the attacker, suppose that the defender is not necessarily successful on wiping out the adversary inside a node \(c\), which may the more “probable” case in an enterprise or embedded network. It is not difficult to generalize the model towards this: If we write \(p_c\) for the likelihood to actually cut the rope at \(c\in V\) upon trying so, (B.5) becomes a mix of cut and uncut paths, \[\begin{aligned} \Pr(&\text{adversary's location}=v) = \\ &p_c \cdot \Pr(\text{adversary's location}=v|V(\pi|_c))\\ &\quad+ (1-p_c)\cdot\Pr(\text{adversary's location}=v|V(\pi)),\end{aligned}\] and (B.5) is defined alike by the entirety of all these values for all \(v\in V\). Thus, the computation as such does not change, only the code needs to use the above formulas to compute the payoffs. If the probabilities are made conditional on the system state, the analysis can be made to account for changing system conditions too.

Multiple Adversarial Targets

Cut-The-Rope may be modified towards a multi-criteria game, treating all target nodes as individual targets in the game. The concept of a security strategy has a multi-goal counterpart, which the software used for the experimental implementation already supports. The experiments reported here could, possibly, be re-conducted without the merge of targets, i.e., the graph-theoretic contraction.

Complexity and Scalability

The complexity of the analysis is governed by the time to solve a sequence of linear optimizations. The dimensions of these problems depend on the number of strategies for both players. Using interior point methods, the computational complexity is \(O(\left\lvert AS_1\right\rvert\cdot p(\left\lvert AS_1\right\rvert\cdot\left\lvert AS_2\right\rvert))\) for a polynomial \(p\) that depends on the chosen optimization algorithm. The need for an exhaustive enumeration of attack paths can raise scalability issues, since the number of attack paths is worst-case exponential. However, the number of paths in an attack graph may become large only because many paths overlap in large portions, and the defender may consider using only a subset of paths that cover all edges in the attack graph, so as to cover all known exploits (which is a polynomial number), rather than all possible paths (whose number is exponential). We did not implement such a dimensionality reduction, this modification of the model is a possible aisle of future studies. For the use cases in our research, the number of paths was sufficiently small to admit an exhaustive enumeration. Likewise is the number of paths feasibly small if the battlefield is an attack tree, rather than an attack graph. In any case, Cut-The-Rope itself does not conceptually change if the restriction is imposed only on the cardinality of the strategy sets to be polynomial in the number of nodes in the attack graph.

Including Signals about Adversarial Activities

The model assumes zero information for the defender about where the adversary is located. Many real-life systems use intrusion detection, and other signaling means (here explicitly not to be understood in the game-theoretic sense of signaling games). If we can compile the entirety of indications about the adversary’s activity into a weight \(\Pr(\theta)\neq 1/\left\lvert AS_2\right\rvert\) for location \(\theta\) to possibly start from. Higher values may be assigned where we have stronger indication of recent adversarial activity at location \(\theta\).

10.5Conclusions and Outlook

Cut-The-Rope has been designed for ease of use in applications with little information (such as exemplified with the MARA robot use case), but also situations where there is detailed information encoded in the attack graph (such as for the MiR100 use case), or even when adversarial indications are available from auxiliary security systems, such as intrusion detection or others (including is possible as outlined in Section 10.4.5). The accuracy hence depends on how much information we can bring into the game, being a rather crude approximation for MARA, but much more fine-grained for the MiR100 use case. In both cases, however, the defender gets nontrivial advice on where to allocate its typically scarce resources for a best defense, beyond just guarding a graph-cut or choke point towards the critical asset \(v_0\). When there are several such (evident) critical regions in the network to defend, different choke points may be of different criticality, depending on how many attack scenarios (each executed by another avatar in Cut-The-Rope) actually make use of this area in the attack graph.

The results obtained show some limitations for the practical use. First, and most substantially, the results depend on the ordering of the nodes, and – in addition to the general non-uniqueness of equilibria – hence may be ambiguous for the defender, leaving a residual chance of there being other defense possibilities. The algorithms applied in this work give only one solution, among perhaps many others. Second, the setting of probabilities from CVE, CVSS or likewise annotations is a nontrivial matter on its own, with only few first steps towards a systematic and sound derivation available in the literature . The assumption of invisibility of the intruder can perhaps be weakened by including signals from intrusion detection or other side-information in the defender’s policy. The model simulated here does not include this possibility. Finally, the condensation of several attack goals into a single target node comes with the price of losing accuracy and some information about which attack goal may be more likely to be reached, thus making multi criteria optimization an interesting generalization to study.

Generally, Cut-The-Rope opens up an interesting class of games of mixed timing of moves between the actors, unlike as in extensive or normal form games, where players usually take actions in a fixed order. Likewise, and also different to many other game models, Cut-The-Rope has no defined start or finish for the defender (“security is never done”), while only one of the players knows when the game starts and ends, and the attacker can send its avatars from all possible locations in the network. The model is thus complementary to FlipIt, while it allows the attacker to spend any amount of time in the system, as long as the vital asset remains out of reach. This is actually to reflect the reality of security management: we cannot keep the adversary out, we can only try keeping as far away as possible.

In conclusion, Cut-The-Rope work presented in this chapter epitomizes the dynamic interplay between attack and defense in the realm of robotic cybersecurity, embodying a nuanced approach to resource allocation and threat anticipation. Its adaptability across various information landscapes—from sparse to rich data scenarios—provides a robust framework for decision-making in cybersecurity. While acknowledging its inherent limitations and the non-uniqueness of solutions, Cut-The-Rope underscores the perpetual nature of cyber defense. It represents a significant stride in the evolution of cybersecurity models, transitioning from static defense strategies to a more proactive, predictive stance. This model not only facilitates effective defense posturing but also enriches our understanding of the complex cyber battlefields, filled with automation, and where the security landscape is constantly reshaped by emerging technologies and sophisticated adversaries.

11Guiding Attacks with Machine Learning

In the rapidly evolving field of offensive robot cybersecurity, particularly in automated systems (referenced in Part V), the challenge lies in effectively comprehending and navigating the complex threat landscape. Following the previous Chapter 10, which focused on automating attack vector selection through game theory, we now turn to the task of guiding an autonomous agent. This involves choosing the correct sequence of actions (like reconnaissance, exploitation, and exfiltration) and appropriate payloads to effectively utilize the chosen attack vector. This necessitates a fusion of in-depth cybersecurity knowledge with specialized domain expertise, such as robotics, each bringing unique challenges and technological nuances.

In this chapter, we delve into the sophisticated realm of guiding autonomous cybersecurity agents, whether for offensive or defensive purposes. The challenge lies in the enormity and obscurity of the cybersecurity state and action spaces, coupled with the unpredictability of the environment. While traditional machine learning approaches, including Reinforcement Learning, have demonstrated proficiency in learning basic policies, their effectiveness is often limited to the data sets they are trained on, lacking the capacity to generalize across diverse cybersecurity scenarios.

This chapter introduces a novel method for autonomously steering cybersecurity agents towards specific objectives. Our approach harnesses the power of foundational models and Large Language Models (LLMs), which encompass a broad spectrum of general-purpose knowledge. These models are particularly adept at parsing, reasoning, and generating security-related artifacts, thereby playing a pivotal role in guiding cybersecurity exercises. We specifically focus on the application of Generative Pre-trained Transformer (GPT) models , with our prototype, PentestGPT, demonstrating remarkable capabilities. PentestGPT leverages the extensive knowledge base and analytical prowess of LLMs to offer innovative solutions and insights in the field of robotic cybersecurity.

Through this exploration, we aim to showcase how leveraging advanced machine learning techniques can profoundly impact the field of cybersecurity, particularly in automating and refining attack strategies. This chapter not only highlights the potential of these technologies but also sets the stage for future developments in the automated orchestration of cybersecurity tactics.

11.1Introduction

Guaranteeing a system’s immunity to potential attacks is a formidable challenge. Offensive security methods, such as penetration testing (pen-testing) or red teaming, have become essential in the security lifecycle. As detailed by Applebaum , these methods require security teams to attempt breaches of an organization’s defenses to uncover vulnerabilities. They offer marked advantages over traditional defensive mechanisms, reliant on incomplete system knowledge and modeling. Guided by the principle “the best defense is a good offense”, this study focuses on offensive strategies, particularly penetration testing.

Penetration testing is a proactive offensive technique aiming at identifying, assessing, and mitigating as many security vulnerabilities as possible. This involves executing targeted attacks to confirm diverse flaws (e.g., erratic behaviors) and is efficacious in creating a comprehensive inventory of vulnerabilities complemented by actionable enhancement recommendations. As a widely-employed practice for security appraisal, penetration testing empowers organizations to discern and neutralize potential vulnerabilities in their networks and systems before exploitation by malicious entities. Despite its significance, the industry often leans on manual techniques and specialized knowledge , making it labor-intensive. This has generated a gap in responding to the escalating demand for adept and efficient security evaluations.

Recently Large Language Models (LLMs) are making striking progress, exhibiting an increasingly nuanced understanding of human-like text and effectively executing various tasks across diverse domains. One intriguing aspect of LLMs is their emergent abilities , which are not explicitly programmed but arise during the training process. These abilities enable LLMs to perform complex tasks such as reasoning, summarization, question-answering, and domain-specific problem-solving without requiring specialized training. Such capabilities indicate the transformative potential of LLMs across various sectors, including cybersecurity. A critical question thus emerges: can LLMs be leveraged in cybersecurity, particularly for performing automated penetration testing?

Motivated by this question, we set out to evaluate the capabilities of LLMs on real-world penetration testing tasks. Unfortunately, the current benchmarks for penetration testing are not comprehensive and fail to assess progressive accomplishments fairly during the process. To address this limitation, we construct a robust benchmark that includes test machines from HackTheBox and VulnHub —two leading platforms for penetration testing challenges. Comprising 13 targets with 182 sub-tasks, our benchmark encompasses all vulnerabilities appearing in OWASP’s top 10 vulnerability list . Also, it offers a more detailed evaluation of the tester’s performance by monitoring the completion status for each sub-task.

Armed with this benchmark, we conduct an exploratory study using GPT-3.5 , GPT-4 , and Bard as representative LLMs. We interactively test these models by guiding them to complete the penetration tasks against our benchmark targets. This interaction involves setting a penetration testing goal for the LLM, soliciting it for the appropriate operation to execute, implementing it in the testing environment, and feeding the test outputs back to the LLM for next-step reasoning (Figure 11.2). By repeating this cycle, we derive the final penetration testing results. To evaluate the performance of the LLMs, we compare their results against baseline solutions provided by official walkthroughs and solutions from certified penetration testers. By analyzing similarities and differences in their problem-solving approaches, we aim to better understand LLMs’ penetration testing capabilities and discern how their problem-solving strategies diverge from those of human experts.

Our investigation yields intriguing insights into the capabilities and limitations of LLMs in penetration testing. We discover that LLMs demonstrate proficiency in managing specific sub-tasks within the testing process, such as utilizing testing tools, interpreting their outputs, and suggesting subsequent actions. Compared to human experts, LLMs are especially adept at executing complex commands and options with testing tools, while models like GPT-4 excel in comprehending source code and pinpointing vulnerabilities. Furthermore, LLMs can craft appropriate test commands and accurately describe graphical user-interface operations needed for specific tasks. Leveraging their vast knowledge base, they can design inventive testing procedures to unveil potential vulnerabilities in real-world systems and CTF challenges. However, we also note that LLMs have difficulty in maintaining a coherent grasp of the overarching testing scenario, a vital aspect for attaining the testing goal. As the dialogue advances, they may lose sight of earlier discoveries and struggle to apply their reasoning consistently toward the final objective. Additionally, LLMs might overemphasize recent tasks in the conversation history, regardless of their vulnerability status. As a result, they tend to neglect other potential attack surfaces exposed in prior tests and fail to complete the penetration testing task.

The outcomes of our empirical study are promising, revealing that LLMs possess the necessary domain knowledge to perform penetration testing tasks. In particular, they are great at providing an intuition of what to do in a given networking scenario. However, what they lack is effective guidance to carry out these tasks independently and maintain a cohesive grasp of the testing scenario. On the other hand, as investigated in a prior research publication focused on capturing the exploitation route (or flow) for automation. Given the complexity of the (network) state space, the state itself is not enough to reason about what are the best actions to pentest. It rapidly becomes evident that a heuristic is needed to support autonomous pentesting which helps pick actions to achieve given goals. With this understanding, we aim to contribute unlocking the potential of modern machine learning approaches and develop a fully automated penetration testing framework that helps produce cybersecurity cognitive engines. Our overall architecture is depicted in Figure 11.1, which shows our current work so far and near future planned contributions. Our proposed framework, Malism, is designed to enable a user without in-depth security domain knowledge to produce its own cybersecurity cognitive engine that helps conduct penetration testing over an extensive range of targets. This framework comprises three primary components:

1. ExploitFlow : A modular library to produce cyber security exploitation routes (exploit flows). ExploitFlow aims to combine and compose exploits from different sources and frameworks, capturing the state of the system being tested in a flow after every discrete action which allows learning attack trees that affect a given system. ExploitFlow’s main motivation is to facilitate and empower Game Theory and Artificial Intelligence (AI) research in cyber security. It provides a unique representation of the exploitation process that encodes every facet within it. Its representation can be effectively integrated with various penetration testing tools and scripts, such as Metasploit to perform end-to-end penetration testing. Such representation can be further visualized to guide the human experts for the reproduction of the testing process.

2. PentestGPT (this chapter): An automated penetration testing system that leverages the power of LLMs to produce testing guidance and intuition at every given discrete state. It functions as the core component of the Malism framework, guiding the LLMs to efficiently utilize their domain knowledge in real-world testing scenarios.

3. PentestPerf: A comprehensive penetration testing benchmark developed to evaluate the performances of penetration testers and automated tools across a wide array of testing targets. It offers a fair and robust platform for performance comparison.

The harmonious integration of these three components forms an automated, self-evolving penetration testing framework capable of executing penetration tests over various targets, Malism. This framework to develop fully automated penetration testing tools, which we named back in Chapter 11 cybersecurity cognitive engines, aims to revolutionize the field of penetration testing by significantly reducing the need for domain expertise and enabling more comprehensive and reliable testing.

Building on our insights into LLMs’ capabilities in penetration testing, we present PentestGPT, an interactive system designed to enhance the application of LLMs in this domain. Drawing inspiration from the collaborative dynamics commonly observed in real-world human penetration testing teams, PentestGPT is particularly tailored to manage large and intricate projects. It features a tripartite architecture comprising Reasoning, Generation, and Parsing Modules, each reflecting specific roles within penetration testing teams. The Reasoning Module emulates the function of a lead tester, focusing on maintaining a high-level overview of the penetration testing status. We introduce a novel representation, the Pentesting Task Tree (PTT), based on the cybersecurity attack tree . This structure encodes the testing process’s ongoing status and steers subsequent actions. Uniquely, this representation can be translated into natural language and interpreted by the LLM, thereby comprehended by the Generation Module and directing the testing procedure. The Generation Module, mirroring a junior tester’s role, is responsible for constructing detailed procedures for specific sub-tasks. Translating these into exact testing operations augments the generation process’s accuracy. Meanwhile, the Parsing Module deals with diverse text data encountered during penetration testing, such as tool outputs, source codes, and HTTP web pages. It condenses and emphasizes these texts, extracting essential information. Collectively, these modules function as an integrated system. PentestGPT completes a complex penetration testing task by bridging high-level strategies with precise execution and intelligent data interpretation, thereby maintaining a coherent and effective testing process.

We evaluate PentestGPT using our benchmark to showcase its efficacy. Specifically, our system achieves remarkable performance gains, with 228.6% and 58.6% increases in sub-task completion compared to the direct usage of GPT-3.5 and GPT-4, respectively. We also apply PentestGPT to the HackTheBox active penetration testing machines challenge , completing 4 out of the 10 selected targets at a total OpenAI API cost of 131.5 US Dollars, ranking among the top 1% players in a community of over 670,000 members. This evaluation underscores PentestGPT’s practical value in enhancing penetration testing tasks’ efficiency and precision. The solution has been made publicly available on GitHub 33 , receiving widespread acclaim with over 4,700 stars to the date of writing, active community engagement, and ongoing collaboration with multiple industrial partners. In summary, we make the following contributions:

• Development of a Comprehensive Penetration Testing Benchmark. We craft a robust and representative penetration testing benchmark, encompassing a multitude of test machines from leading platforms such as HackTheBox and VulnHub. This benchmark includes 182 sub-tasks covering OWASP’s top 10 vulnerabilities, offering fair and comprehensive evaluation of penetration testing.

• Empirical Evaluation of LLMs for Penetration Testing Tasks. By employing models like GPT-3.5, GPT-4, and Bard, our exploratory study rigorously investigates the strengths and limitations of LLMs in penetration testing. The insights gleaned from this analysis shed valuable light on the capabilities and challenges faced by LLMs, enriching our understanding of their applicability in this specialized domain.

• Development of an Innovative LLM-powered Penetration Testing System. We engineer PentestGPT, a novel interactive system that leverages the strengths of LLMs to carry out penetration testing tasks automatically. Drawing inspiration from real-world human penetration testing teams, PentestGPT integrates a tripartite design that mirrors the collaborative dynamics between senior and junior testers. This architecture optimizes LLMs’ usage, significantly enhancing the efficiency and effectiveness of automated penetration testing.

11.2Background

Penetration Testing

Penetration testing, or “pentesting”, is a critical practice to enhance organizational systems’ security. In a typical penetration test, security professionals, known as penetration testers, analyze the target system, often leveraging automated tools. The standard process is divided into seven phases : Reconnaissance, Scanning, Vulnerability Assessment, Exploitation, and Post Exploitation (including reporting). These phases enable testers to understand the target system, identify vulnerabilities, and exploit them to gain access.

Despite substantial efforts in the field, a fully automated penetration testing pipeline remains elusive. The challenges in automating the process arise from the comprehensive knowledge needed to understand and manipulate various vulnerabilities and the demand for a strategic plan to guide subsequent actions. In practice, penetration testers often use a combined approach integrating depth-first and breadth-first search techniques . They begin by obtaining an overarching understanding of the target environment (utilizing a breadth-first approach) before focusing on specific services and vulnerabilities (employing a depth-first approach). This strategy ensures a thorough system analysis while prioritizing promising attack vectors, relying heavily on individual experience and domain expertise. Additionally, penetration testing requires many specialized tools with unique features and functions. This diversity adds complexity to the automation process. Therefore, even with the support of artificial intelligence, creating a fully unified solution for automated penetration testing remains a formidable challenge.

Large Language Models

Large Language Models (LLMs), including OpenAI’s GPT-3.5 and GPT-4, are prominent tools with applications extending to various cybersecurity-related fields, such as code analysis and vulnerability repairment . These models are equipped with wide-ranging general knowledge and the capacity for elementary reasoning. They can comprehend, infer, and produce text resembling human communication, aided by a training corpus encompassing diverse domains like computer science and cybersecurity. Their ability to interpret context and recognize patterns enables them to adapt knowledge to new scenarios. This adaptability, coupled with their proficiency in interacting with systems in a human-like way, positions them as valuable assets in enhancing penetration testing processes. Despite inherent limitations, LLMs offer distinct attributes that can substantially aid in the automation and improvement of penetration testing tasks. The realization of this potential, however, requires the creation and application of a specialized and rigorous benchmark.

11.3Penetration Testing Benchmark

Motivation

The fair evaluation of Large Language Models (LLMs) in penetration testing necessitates a robust and representative benchmark. Existing benchmarks in this domain have several limitations. First, they are often restricted in scope, focusing on a narrow range of potential vulnerabilities, and thus fail to capture the complexity and diversity of real-world cyber threats. For instance, the OWASP benchmark juiceshop is commonly adopted for evaluating web vulnerability testing. However, it does not touch the concept of privilege escalation, which is an essential aspect of penetration testing. Second, existing benchmarks may not recognize the cumulative value of progress through the different stages of penetration testing, as they tend to evaluate only the final exploitation success. This approach overlooks the nuanced value each step contributes to the overall process, resulting in metrics that might not accurately represent actual performance in real-world scenarios.

To address these concerns, we propose the construction of a comprehensive penetration testing benchmark that meets the following criteria:

Task Variety. The benchmark must encompass diverse tasks, reflecting various operating systems and emulating the diversity of scenarios encountered in real-world penetration testing.

Challenge Levels. To ensure broad applicability, the benchmark must include tasks of varying difficulty levels suitable for challenging novice and expert testers.

Progress Tracking. Beyond mere success or failure metrics, the benchmark must facilitate tracking of incremental progress, thereby recognizing and scoring the value added at each stage of the penetration testing process.

Benchmark Design

Following the criteria outlined previously, we develop a comprehensive benchmark that closely reflects real-world penetration testing tasks. The design process progresses through several stages.

Task Selection. Our first step is to meticulously select tasks from HackTheBox (HTB) and VulnHub . These platforms are widely recognized and frequently utilized for penetration testing practice. Our selection process is guided by a desire to incorporate a diverse and challenging set of tasks. Capture The Flag (CTF) exercises and real-world testing scenarios have been included. The targets are drawn from various operating systems and encompass a broad spectrum of vulnerabilities. This approach ensures a wide representation of real-world penetration testing tasks. To account for different skill levels, the selected tasks cover a broad range of difficulty. While HTB and VulnHub offer reference difficulty levels, we further validate these with input from three certified penetration testers 34 , including the authors of this work. This collaborative process yields a consensus on the final difficulty rating for each target, aligning with the conventional categorization of penetration testing machines into easy, medium, and hard levels. It is worth noting that our benchmark does not explicitly include benign targets for evaluating false positives. This is because the iterative and exploratory nature of penetration testing inherently involves investigating services within the target that may ultimately be deemed benign. In this process, our primary focus is successfully identifying genuine vulnerabilities.

Task Decomposition. We further parse the testing process of each target into a series of sub-tasks, following the standard solution commonly referred to as the “walkthrough” in penetration testing. Each sub-task corresponds to a unique step in the overall process. Specifically, a sub-task may represent a micro-step involving the use of a particular penetration testing tool (e.g., performing port scanning with nmap ) or exploiting a unique vulnerability identified in the Common Weakness Enumeration (CWE) (e.g., exploiting SQL injection). To standardize decomposition, we arrange the sub-tasks into a two-layer structure. Initially, we categorize each sub-task according to the five phases of penetration testing, as described in Section 11.2. Then, we label the sub-task with either the corresponding CWE item it targets or the specific tools employed. These two steps enable us to formulate an exhaustive list of sub-tasks for every benchmark target. We include this list in Appendix C, and the complete sub-task information is accessible on our anonymous open-source project .

Benchmark Validation. The final stage of our benchmark development involves rigorous validation. This step ensures that our benchmark accurately reflects real-world penetration testing scenarios and offers reproducibility. During validation, three certified penetration testers independently attempt the penetration testing targets, refining the sub-tasks as needed. We adjust our task decomposition accordingly because some targets may have multiple valid solutions.

By the end, we compile a benchmark of 13 penetration testing targets with 182 sub-tasks in 25 categories. The benchmark includes all types of vulnerabilities as listed in the OWASP Top 10 Project. Detailed information on the included categories is listed in the Appendix C. To contribute to community development, we have made this benchmark publicly available online at our anonymous project website .

11.4Exploratory Study

We conduct an exploratory study to assess the capabilities of LLMs in penetration testing. Our primary objective is determining how well LLMs can adapt to the real-world complexities and challenges associated with penetration testing tasks. Specifically, we aim to address the following two research questions:

RQ1 (Capability): To what extent can LLMs perform penetration testing tasks?

RQ2 (Comparative Analysis): How do the problem-solving strategies of human penetration testers and LLMs differ?

We utilize the benchmark described in Section 11.3 to evaluate the performance of LLMs on penetration testing tasks. In the following, we first delineate our testing strategy for this study. Subsequently, we present the testing results and an analytical discussion to address the above research questions.

Testing Strategy

LLMs cannot perform penetration tests directly. Their capabilities are primarily text-based, responding to queries and providing suggestions. However, penetration testing often involves operations with user interfaces (UI) and understanding graphical information, such as website images. This necessitates a bridge between the test machine and the LLM to facilitate task completion.

We introduce an interactive loop structure to evaluate the LLM’s abilities in penetration testing within our benchmark. This process, depicted in Figure 11.2, consists of the following stages: We present the target information to the LLM and request recommendations for penetration testing actions. This initiates a looped testing procedure. We implement the actions suggested by the LLM, which encompass both terminal commands and graphical interactions. We gather the results of the actions. Text-based output, such as terminal responses or source code, is recorded directly. Human penetration testers provide concise summaries and descriptions for non-textual results (e.g., images). The summarized information is returned to the LLM to inform subsequent actions. This cycle continues until we identify a solution or reach a standstill. We compile a record of the testing procedures, encompassing successful tasks, ineffective actions, and any reasons for failure, if applicable.

Evaluation Settings

We proceed to assess the performances of various LLMs in penetration testing tasks using the strategy mentioned above.

Model Selection. Our study focuses on three cutting-edge LLMs that are currently accessible: GPT-3.5 and GPT-4 from OpenAI and LaMDA from Google. These models are selected based on their prominence in the research community and consistent availability. To interact with the LLMs mentioned above, we utilize chatbot services provided by OpenAI and Google, namely ChatGPT and Bard . For this chapter, the terms GPT-3.5, GPT-4, and Bard represent these three LLMs.

Experimental Setup. We conduct our experiments in a local environment where the target and testing machines are part of the same private network. The testing machine operates on Kali Linux , version 2023.1. Several measures are implemented to validate the effectiveness of our testing procedures. First, we repeat the tests to account for inherent variability in the LLM outputs. In particular, we test each target with each LLM five times. We performed 195 tests in total, i.e., 5 repetitions * 3 models * 13 targets, wherein the 13 targets correspond to the Easy, Medium and Hard target scenarios 35 . In this process, a sub-task is considered successful if it succeeds in at least one trial, and a penetration task is considered successful as long as one trial succeeds. Second, we make the best efforts to translate UI operations and graphical information into natural languages accurately. In addition, we ensure the precise execution of the instructions provided by the LLMs. Third, we maintain the integrity of the testing process by strictly limiting the tester’s role to executing actions and reporting results without adding expert knowledge or guidance. Finally, the testing and target machines are rebooted after each test to reset their states, ensuring a consistent starting point for each test.

Tool Usage. Our study aims to assess the innate capabilities of LLMs without reliance on automated vulnerability scanners such as Nexus and OpenVAS . Consequently, we explicitly instruct the LLMs to refrain from using these tools. However, we follow the LLMs’ recommendations for utilizing other tools designed to validate specific vulnerability types (e.g., sqlmap for SQL injections). Occasionally, versioning discrepancies may lead the LLMs to provide incorrect instructions for tool usage. In such instances, our penetration testing experts evaluate whether the instructions would have been valid for a previous version of the tool. They then make any necessary adjustments to ensure the tool’s correct operation.

Capability Evaluation (RQ1)

To study RQ1, we begin by assessing the overall performance of three prominent LLMs: GPT-4, Bard, and GPT-3.5. The results of these evaluations are compiled in Table 11.1. The experimental results show that the three LLMs completed at least one end-to-end penetration testing task. This achievement underscores their ability to conduct a broad spectrum of testing operations, particularly within environments of less complexity. Among the models, GPT-4 stands out with superior performance, achieving success with 4 targets of easy difficulty and 1 of medium difficulty. Bard and GPT-3.5 also demonstrate commendable capabilities, completing 2 and 1 targets of easy difficulty, respectively. When examining sub-tasks, GPT-4 accomplishes 52 of 77 on easy difficulty targets and 27 out of 71 on medium ones, underlining its potential for significant contributions to more complex penetration testing scenarios. Though not as proficient as GPT-4, GPT-3.5 and Bard still show promise, completing 13 (18.31%) and 16 (22.54%) of sub-tasks on medium difficulty targets, respectively. However, the performance of all three models noticeably diminishes when challenged with hard difficulty targets. While each model can complete the initial reconnaissance phase on these targets, they fall short in exploiting the identified vulnerability. This outcome is not entirely unexpected, as the hard difficulty machines are deliberately crafted to be exceedingly difficult. They often include services that appear vulnerable but are, in fact, non-exploitable—a trait commonly referred to as rabbit holes . Additionally, the routes to successfully exploiting these machines are typically inventive and unforeseeable, making them resistant to straightforward replication by automated tools. For instance, the benchmark target Falafel involves deliberately crafted SQL injection vulnerabilities, which are resistant to sqlmap and can only be exploited through manually designed payloads. Existing LLMs do not exhibit the capability to solve them solely without the guidance of human experts.

We further examine the detailed sub-task completion performances of the three LLMs, as presented in Table 11.2. Analyzing the completion status, we identify several areas where LLMs excel. First, they adeptly utilize common penetration testing tools to interpret the corresponding outputs, especially in enumeration tasks correctly. For example, all three evaluated LLMs successfully perform all nine Port Scanning sub-tasks. They can configure the widely-used port scanning tool, nmap , comprehend the scan results, and formulate subsequent actions. Second, the LLMs reveal a deep understanding of prevalent vulnerability types, connecting them to the services on the target system. This understanding is evidenced by the successful completion of sub-tasks related to various vulnerability types. Finally, LLMs demonstrate their effectiveness in code analysis and generation, particularly in the tasks of Code Analysis and Shell Construction. These tasks require the models to read and generate codes in different programming languages, essential in penetration testing. This often culminates in identifying potential vulnerabilities from code snippets and crafting the corresponding exploits. Notably, GPT-4 outperforms the other two models regarding code interpretation and generation, marking it the most suitable candidate for penetration testing tasks.

Comparative Analysis (RQ2)

To address RQ2, we examine the problem-solving strategies that LLMs employ, contrasting them with human penetration testers. In each penetration testing trial, we concentrate on two main aspects: (1) Identifying the unnecessary operations that LLMs prompt, which are not conducive to successful penetration testing, as compared to a standard walkthrough; and (2) Understanding the specific factors that prevent LLMs from successfully executing penetration tests.

We analyze the unnecessary operations prompted by LLMs by breaking down the recorded testing procedures into sub-tasks. We employ the same method to formulate benchmark sub-tasks, as Section 11.3 outlines. By comparing this to a standard walkthrough, we identify the primary sub-task trials that fall outside the standard walkthrough and are thus irrelevant to the penetration testing process. The results are summarized in Table 11.3. We find that the most prevalent unnecessary operation prompted by LLMs is brute force. For all services requiring password authentication, LLMs typically advise brute-forcing it. This is an ineffective strategy in penetration testing. We surmise that many hacking incidents in enterprises involve password cracking and brute force. LLMs learn these reports from accident reports and are consequently considered viable solutions. Besides brute force, LLMs suggest that testers engage in CVE studies, SQL injections, and command injections. These recommendations are common, as real-world penetration testers often prioritize these techniques, even though they may not always provide the exact solution.

We further investigate the reasons behind the failure of penetration testing trials. We manually categorize the causes of failure for the 195 penetration testing trials, with the results documented in Table 11.4. This table reveals that the predominant cause of failure is the loss of session context. The three examined models face difficulties in maintaining long-term conversational memory uniformly, frequently forgetting previous test results as the dialogue progresses. This lack of retention may be attributable to the limited token size within the LLM conversation context. Given the intricate nature of penetration testing—where a tester must skillfully link minor vulnerabilities across different services to develop a coherent exploitation strategy—this loss of context substantially undermines the models’ effectiveness.

Secondly, LLMs strongly prefer the most recent tasks, adhering rigorously to a depth-first search approach. They concentrate on exploiting the immediate service, rarely deviating to a new target until all potential paths for the current one have been pursued. This can be attributed to the attention of LLMs focusing more on the beginning and end of the prompt, as revealed in . Experienced penetration testers generally assess the system from a broader standpoint, strategizing the subsequent steps likely to provide the most substantial results. When combined with the aforementioned memory loss issue, this tendency causes LLMs to become overly fixated on a specific service. As the test progresses, the models completely forget previous findings and reach a deadlock.

Lastly, LLMs have inaccurate result generation and hallucination issues, as noted in . This phenomenon ranks as the second most frequent cause of failures and is characterized by the generation of false commands. In our study, we observe that LLMs frequently identify the appropriate tool for the task but stumble in configuring the tools with the correct settings. In some cases, they even concoct non-existent testing tools or tool modules.

Our exploratory study of three LLMs within penetration testing reveals their potential for executing end-to-end tasks. Nevertheless, challenges arise in maintaining long-term memory, devising a testing strategy beyond a depth-first approach, and generating accurate operations. In the following section, we elucidate how we address these challenges and outline our strategy for designing our LLM-powered penetration testing tool.

11.5Methodology

Overview

In light of the challenges identified in the preceding section, we present our proposed solution, PentestGPT, which leverages the synergistic interplay of three LLM-powered modules. As illustrated in Figure 11.3, PentestGPT incorporates three core modules: the Reasoning Module, the Generation Module, and the Parsing Module. Each module reserves one LLM session with its conversation and context. The user interacts seamlessly with PentestGPT, where distinct modules process different types of messages. This interaction culminates in a final decision, suggesting the subsequent step of the penetration testing process that the user should undertake. In the following sections, we elucidate our design reasoning and provide a detailed breakdown of the engineering processes behind PentestGPT.

Design Rationale

Our central design considerations emerged from the three challenges observed in the previous Exploratory Study (Section 11.4): The first challenge (Finding 3) pertains to the issue of penetration testing context loss due to memory retention. LLMs in their original form struggle to maintain such long-term memory due to token size limits. The second obstacle (Finding 4) arises from the LLM chatbots’ tendency to emphasize recent conversation content. In penetration testing tasks, this focuses on optimizing the immediate task. This approach falls short in the complex, interconnected task environment of penetration testing. The third obstacle (Finding 5) is tied to the inaccurate results generation by LLMs. When tasked to produce specific operations for a step in penetration testing directly, the outputs are often imprecise, sometimes even leading to

PentestGPT has been engineered to address these challenges, rendering it more apt for penetration testing tasks. We drew inspiration from the methodologies employed by real-world penetration testing teams, where a director plans overarching procedures, subdividing them into subtasks for individual testers. Each tester independently performs their task, reporting results without an exhaustive understanding of the broader context. The director then determines the following steps, possibly redefining tasks, and triggers the subsequent round of testing. Essentially, the director manages the overall strategy without becoming entrenched in the minutiae of the tests. This approach is mirrored in PentestGPT’s functionality, enhancing its efficiency and adaptability in conducting penetration tests. Our strategy divides penetration testing into two processes: identifying the next task and generating the concrete operation to complete the task. Each process is powered by one LLM session. In this setup, the LLM session responsible for task identification retains the complete context of the ongoing penetration testing status. At the same time, the generation of detailed operations and parsing of information is managed by other sessions. This division of responsibilities fosters effective task execution while preserving the overarching context.

To assist LLMs in effectively carrying out penetration testing tasks, we design a series of prompts that align with user inputs. We utilize the Chain-of-Thought (CoT) methodology during this process. As CoT reveals, LLMs’ performance and reasoning capabilities can be significantly enhanced using the input, chain-of-thought, output prompting format. Here, the chain-of-thought represents a series of intermediate natural language reasoning steps leading to the outcome. We dissect the penetration testing tasks into micro-steps and design prompts with examples to guide LLMs through processing penetration testing information step-by-step, ultimately leading to the desired outcomes. The complete prompts are available at our anonymized open-source project .

Reasoning Module

The Reasoning Module plays a pivotal role in our system, analogous to a team lead overseeing the penetration testing task from a macro perspective. It obtains testing results or intentions from the user and prepares the testing strategy for the next step. This testing strategy is passed to the generation module for further planning.

To effectively supervise the penetration testing process and provide precise guidance, it is crucial to translate the testing procedures and outcomes into a natural language format. Drawing inspiration from the concept of an attack tree , which is often used to outline penetration testing procedures, we introduce the notion of a pentesting task tree (PTT). This novel approach to testing status representation is rooted in the concept of an attributed tree :

Given the definition of attributed tree, PTT is defined as follows:

As outlined in Figure 11.3, the Reasoning Module’s operation unfolds over four key steps operating over the PTT. Initially, the module absorbs the user’s intentions to construct an initial PTT in the form of natural language. This is achieved by carefully instructing the LLM with examples and definitions of PPT using meticulously crafted prompts. The LLM outputs are parsed to confirm that the tree structure is accurately formatted. Note that due to the nature of the tree structure, it can be represented in the natural language format through layered bullets, as illustrated in Figure 11.4. The Reasoning Module effectively overcomes the memory-loss issue by maintaining a task tree that encompasses the entire penetration testing process. After updating the tree information, a verification step is conducted on the newly updated PTT to ascertain its correctness. This process checks explicitly that only the leaf nodes of the PTT have been modified, aligning with the principle that atomic operations in the penetration testing process should only influence the status of the lowest-level sub-tasks. This step confirms the correctness of the reasoning process, safeguarding against any potential alterations to the overall tree structure due to hallucination by the LLM. If discrepancies arise, the information is reverted to the LLM for correction and regeneration. With the updated PTT, the Reasoning Module evaluates the current tree state and pinpoints viable sub-tasks that can serve as candidate steps for further testing. Finally, the module evaluates the likelihood of these sub-tasks leading to successful penetration testing outcomes. It then recommends the top task as the output. The expected results of this task are subsequently forwarded to the Generation Module for an in-depth analysis. This is feasible, as demonstrated in the exploratory study, since LLMs, particularly GPT-4, can identify potential vulnerabilities when provided with system status information. This procedural approach enables the Reasoning Module to address one of the inherent limitations of LLMs, precisely their tendency to concentrate solely on the most recent task. Note that in cases where the tester identifies that the correct task is incorrect or not completed in a preferred way, he could also manually revise the PTT through the interactive handle further discussed in Section 11.5.6.

We devise four sets of prompts to sequentially guide the Reasoning Module through the completion of each stage. To bolster the reproducibility of our results, we optimize these prompts further with a technique known as hint generation . From our practical experience, we observe that LLMs are adept at interpreting the tree-structured information pertinent to penetration testing and can update it accurately in response to test outputs.

Generation Module

The Generation Module translates specific sub-tasks from the Reasoning Module into concrete commands or instructions. Each time a new sub-task is received, a fresh session is initiated in the Generation Module. This strategy effectively isolates the context of the overarching penetration task from the immediate task under execution, enabling the LLM to focus entirely on generating specific commands.

Instead of directly transforming the received sub-task into specific operations, our design employs the CoT strategy to partition this process into two sequential steps. This design decision directly addresses the challenges associated with model inaccuracy and hallucination by enhancing the model’s reasoning capability. In particular, upon the receipt of a concise sub-task from the Reasoning Module, the Generation Module begins by expanding it into a sequence of detailed steps. Notably, the prompt associated with this sub-task requires the LLM to consider the possible tools and operations available within the testing environment. Subsequently, the Generation Module transforms each of these expanded steps into precise terminal commands ready for execution or into detailed descriptions of specific Graphical User Interface (GUI) operations to be carried out. This stage-by-stage translation eliminates potential ambiguities, enabling testers to follow the instructions directly and seamlessly. Implementing this two-step process effectively precludes the LLM from generating operations that may not be feasible in real-world scenarios, thereby improving the success rate of the penetration testing procedure.

By acting as a bridge between the strategic insights provided by the Reasoning Module and the actionable steps required for conducting a penetration test, the Generation Module ensures that high-level plans are converted into precise and actionable steps. This transformation process significantly bolsters the overall efficiency of the penetration testing procedure.

An Illustrative Example. We utilize a real-world running example to illuminate how the Reasoning Module and the Generation Module collaboratively operate to complete penetration testing tasks. Figure 11.5 illustrates a single iteration of PentestGPT working on the HackTheBox machine Carrier , a medium-difficulty target. As depicted in a-1), the PTT, in natural language format, encodes the testing status, revealing the open ports (21, 22,80) on the target machine. The Reasoning Module is subsequently instructed to identify the available tasks. As highlighted in red, service scanning is the only available task on the leaf node of the PTT. This task is therefore chosen and forwarded to the Generation Module for command generation. The generated command is executed in the testing environment, and the execution result is conveyed to the Reasoning Module to update the PTT. In a-2), the Reasoning Module integrates the previous scanning result into the PTT, cross-referencing it with the earlier PTT to update only the leaf nodes. It then looks for the available tasks to execute. In this case, two tasks emerge: scanning the web service on port 80 and checking the SSH service for known vulnerabilities. The LLM evaluates which task is more promising and chooses to investigate the web service, often seen as more vulnerable. This task is passed to the Generation Module. The Generation Module turns this general task into a detailed process, employing nikto , a commonly used web scanning script. The iterative process continues until the tester completes the penetration testing task.

Parsing Module

The Parsing Module operates as a supportive interface, enabling effective processing of the natural language information exchanged between the user and the other two core modules. Two needs can primarily justify the existence of this module. First, security testing tool outputs are typically verbose, laden with extraneous details, making it computationally expensive and unnecessarily redundant to feed these extended outputs directly into the LLMs. Second, users without specialized knowledge in the security domain may struggle to extract key insights from security testing outputs, presenting challenges in summarizing crucial testing information. Consequently, the Parsing Module is essential in streamlining and condensing this information.

In PentestGPT, the Parsing Module is devised to handle four distinct types of information: (1) user intentions, which are directives provided by the user to dictate the next course of action, (2) security testing tool outputs, which represent the raw outputs generated by an array of security testing tools, (3) raw HTTP web information, which encompasses all raw information derived from HTTP web interfaces, and (4) source codes extracted during the penetration testing process. Users must specify the category of the information they provide, and each category is paired with a set of carefully designed prompts. For source code analysis, we integrate the GPT-4 code interpreter to execute the task.

Active Feedback

While LLMs can produce insightful outputs, their outcomes may sometimes require revisions. To facilitate this, we introduce an interactive handle in PentestGPT, known as active feedback, which allows the user to interact directly with the Reasoning Module. A vital feature of this process is that it does not alter the context within the Reasoning Module unless the user explicitly desires to update some information. The reasoning context, including the PTT, is stored as a fixed chunk of tokens. This chunk of tokens is provided to a new LLM session during an active feedback interaction, and users can pose questions regarding them. This ensures that the original session remains unaffected, and users can always query the reasoning context without making unnecessary changes. If the user believes it necessary to update the PTT, they can explicitly instruct the model to update the reasoning context history accordingly. This provides a robust and flexible framework for the user to participate in the decision-making process actively.

Discussion

We explore various design alternatives for PentestGPT to tackle the challenges identified in Exploratory Study. We have experimented with different designs, and here we discuss some key decisions.

Addressing Context Loss with Token Size: a straightforward solution to alleviate context loss is the employment of LLM models with an extended token size. For instance, GPT-4 provides versions with 8k and 32k token size limits. This approach, however, confronts two substantial challenges. First, even a 32k token size might be inadequate for penetration testing scenarios, as the output of a single testing tool like dirbuster may comprise thousands of tokens. Consequently, GPT-4 with a 32k limit cannot retain the entire testing context. Second, even when the entire conversation history fits within the 32k token boundary, the API may still skew towards recent content, focusing on local tasks and overlooking broader context. These issues guided us in formulating the design for the Reasoning Module and the Parsing Module.

Vector Database to Improve Context Length: Another technique to enhance the context length of LLMs involves a vector database . By transmuting data into vector embeddings, LLMs can efficiently store and retrieve information, practically creating long-term memory. Theoretically, penetration testing tool outputs could be archived in the vector database. In practice, though, we observe that many results closely resemble and vary in only nuanced ways. This similarity often leads to confused information retrieval. Solely relying on a vector database fails to overcome context loss in penetration testing tasks. Integrating the vector database into the design of PentestGPT is an avenue for future research.

Precision in Information Extraction: Precise information extraction is crucial for conserving token usage and avoiding verbosity in LLMs. Rule-based methods are commonly employed to extract diverse information. However, rule-based techniques are engineeringly expensive given natural language’s inherent complexity and the variety of information types in penetration testing. We devise the Parsing Module to manage several general input information types, a strategy found to be both feasible and efficient.

Limitations of LLMs: LLMs are not an all-encompassing solution. Present LLMs exhibit flaws, including hallucination and outdated knowledge. Our mitigation efforts, such as implementing task tree verification to ward off hallucination, might not completely prevent the Reasoning Module from producing erroneous outcomes. Thus, a human-in-the-loop strategy becomes vital, facilitating the input of necessary expertise and guidance to steer LLMs effectively. Moreover, it must be higlighted that reproducibility of results is an issue here, as LLMs are in continuous evolution. In our case, given the novelty of the technology and lack of availability at the time of starting this work, we did not host the LLM deployments ourselves, but instead used third party providers. This setup is sub-optimal from a scientific experiment point of view. For future work, we foresee building our own LLMs by performing post-training phases on top of existing open source models and freezing them after a certain stage of learning. This will lead to results being reproducible given the right input context, and fully reproducible on statistical average. This topic is further discussed in Section 11.7.

11.6Evaluation

In this section, we assess the performance of PentestGPT, focusing on the following four research questions:

RQ3 (Performance): How does the performance of PentestGPT compare with that of native LLM models and human experts?

RQ4 (Strategy): Does PentestGPT employ different problem-solving strategies compared to those utilized by LLMs or human experts?

RQ5 (Ablation): How does each module within PentestGPT contribute to the overall penetration testing performance?

RQ6 (Practicality): Is PentestGPT practical and effective in real-world penetration testing tasks?

Evaluation Settings

We implement PentestGPT with 1,700 lines of Python3 code and 740 prompts, available at our anonymized project website . We evaluate its performance over the benchmark constructed in Section 11.3. In this evaluation, we integrate PentestGPT with GPT-3.5 and GPT-4 to form two working versions: PentestGPT-GPT-3.5 and PentestGPT-GPT-4. Due to the lack of API access, we do not select other LLM models, such as Bard. In line with our previous experiments, we use the same experiment environment setting and instruct PentestGPT to only use the non-automated penetration testing tools.

Performance Evaluation (RQ3)

The overall task completion status of PentestGPT-GPT-3.5, PentestGPT-GPT-4, and the naive usage of LLMs is illustrated in Figure 11.6a. As the Figure shows, our solutions powered by LLMs demonstrate superior penetration testing capabilities compared to the naive application of LLMs. Specifically, PentestGPT-GPT-4 surpasses the other three solutions, successfully solving 6 out of 7 easy difficulty targets and 2 out of 4 medium difficulty targets. This performance indicates that PentestGPT-GPT-4 can handle penetration testing targets ranging from easy to medium difficulty levels. Meanwhile, PentestGPT-GPT-3.5 manages to solve only two challenges of easy difficulty, a discrepancy that can be attributed to GPT-3.5 lacking the knowledge related to penetration testing found in GPT-4.

The sub-task completion status of PentestGPT-GPT-3.5, PentestGPT-GPT-4, and the naive usage of LLM is shown in Figure 11.6b. As the Figure illustrates, both PentestGPT-GPT-3.5 and PentestGPT-GPT-4 perform better than the standard utilization of LLMs. It is noteworthy that PentestGPT-GPT-4 not only solves one more medium difficulty target compared to naive GPT-4 but also accomplishes 111% more sub-tasks (57 vs. 27). This highlights that our design effectively addresses context loss challenges and leads to more promising testing results. Nevertheless, all the solutions struggle with hard difficulty testing targets. As elaborated in Section 11.4, hard difficulty targets typically demand a deep understanding from the penetration tester. To reach testing objectives, they may require modifications to existing penetration testing tools or scripts. Our design does not expand the LLMs’ knowledge of vulnerabilities, so it does not notably enhance performance on these more complex targets.

Figure. The performance of GPT-3.5, GPT-4, PentestGPT-GPT-3.5, and PentestGPT-GPT-4 on overall target completion and sub-task completion. Reproduced from .

Strategy Evaluation (RQ4)

We then investigate the problem-solving strategies employed by PentestGPT, contrasting them with those of LLMs and human experts. By manually analyzing the penetration testing process of PentestGPT, we synthesize its underlying approaches to problem-solving. We surprisingly find that PentestGPT decomposes the penetration testing task in a manner akin to human experts, successfully achieving the overall goal. Instead of focusing solely on the most recently discovered task, PentestGPT can pinpoint potential sub-tasks likely to lead to successful outcomes.

Figure 11.7 provides an illustrative example, demonstrating the strategic differences between GPT-4 and PentestGPT while handling the VulnHub machine, Hackable II . This target comprises two vulnerable services: an FTP service allowing arbitrary file uploads and a web service enabling file viewing through FTP. A successful exploit necessitates exploiting both services by uploading a malicious PHP shell via the FTP service and triggering it through the web service. As depicted in the figure, GPT-4 begins by enumerating the FTP service and successfully identifies the file upload vulnerability (-). However, it fails to correlate this with the web service, resulting in an incomplete exploit in the following steps. Conversely, PentestGPT follows a more holistic approach, toggling between enumerating the FTP service and browsing the web service. In particular, PentestGPT firstly enumerates the FTP service and web service to understand the general situation. It then prioritizes the FTP service, and eventually discovers the file upload vulnerability. More importantly, in this process, PentestGPT identifies that files available on FTP are the same as those on the web service. By connecting these findings, PentestGPT guides the tester to perform a shell upload, leading to a successful reverse shell. This strategy aligns with the walkthrough solution and highlights PentestGPT’s comprehensive understanding of the penetration testing process and its ability to make effective decisions on the optimal sub-task to pursue next. This reveals PentestGPT’s strategic thinking and ability to integrate different aspects of the testing process.

Our second observation is that although PentestGPT behaves more similarly to human experts, it still exhibits some strategies that humans will not apply. For instance, PentestGPT still prioritizes brute-force attacks before vulnerability scanning. This is obvious in cases where PentestGPT always tries to brute-force the SSH service on target machines.

We then analyze the failed penetration testing cases to understand the limitations of PentestGPT. Beyond the absence of some advanced penetration testing techniques, two primary issues emerge. First, PentestGPT struggles to interpret images. LLMs are limited to text comprehension, so they cannot accurately process images. This issue might be addressed by developing large multimodal models to understand text and visual data. Second, it cannot grasp certain social engineering tricks and subtle cues. For instance, real-world penetration testers often create brute-force wordlists using information gathered from the target service. Though PentestGPT can retrieve a list of names from a web service, it fails to instruct the use of tools to create a wordlist from those names. These limitations underline the necessity for improvement in areas where human insight and intricate reasoning are still more proficient than automated solutions.

Ablation Study (RQ5)

We perform an ablation study on how the three modules: Reasoning Module, Generation Module, and Parsing Module, contribute to the performance of PentestGPT. We implement three variants:

1. PentestGPT-no-Parsing: the Parsing Module is deactivated, causing all data to be directly fed into the system.

2. PentestGPT-no-Generation: the Generation Module is deactivated, leading to the completion of task generation within the Reasoning Module itself. The prompts for task generation remain consistent.

3. PentestGPT-no-Reasoning: the Reasoning Module is desabled. Instead of PTT, this variant adopts the same methodology utilized with LLMs for penetration testing, as delineated in the Exploratory Study.

All the variants are integrated with GPT-4 API for testing.

Figure. The performance of PentestGPT, PentestGPT-No-Annotation, PentestGPT-Operation-Only, and PentestGPT-Parameter-Only on both normalized average code coverage ($μ LOC$) and bug detection. Reproduced from .

The results of the three variants tested on our penetration testing benchmarks are depicted in Figure 11.8. In general, PentestGPT demonstrates superiority over the three ablation baselines regarding overall target and sub-task completion. Our key findings are as follows: (1) In the absence of the Parsing Module, PentestGPT-no-Parsing attains marginally lower performance in overall task and sub-task completion relative to the full configuration. While parsing information is advantageous in penetration testing, the 32k token size limit often suffices for various outputs. Given the Reasoning Module’s inherent design to maintain the entire testing context, the lack of the Parsing Module does not substantially impair the tool’s performance. (2) PentestGPT-no-Reasoning fares the worst, completing only 53.6% of the sub-tasks achieved by the full solution, an outcome even inferior to the naive application of GPT-4 in testing. We attribute this to the Generation Module adding supplementary sub-tasks to the LLM context. Since the prompts are not tailored for scenarios without the Reasoning Module, the resulting outputs are irrelevant for the naive LLM without the Generation Module. Furthermore, the extended generation output obscures the original context, hindering the LLM’s ability to concentrate on the task, thus failing the test. (3) PentestGPT-no-Generation realizes performance slightly above that of GPT-4 employed naively. This occurs because, without the Generation Module, the testing procedure closely resembles the usage of LLMs. Notably, the Generation Module is principally intended to guide the tester in executing precise penetration testing operations. Without this module, the tester may depend on supplementary information to operate the tools or scripts essential for completing the test.

Practicality Study (RQ6)

We demonstrate that PentestGPT exhibits practicality for real-world penetration testing beyond the crafted benchmark. For this purpose, we engage PentestGPT in the HackTheBox active machine challenges, a series of penetration testing objectives open to global testers. Each challenge consists of two components: a user flag, retrievable upon initial user access, and a root flag, obtainable after gaining root access. Our evaluation encompasses five targets of easy difficulty and five of medium difficulty. During this exercise, PentestGPT, utilizing GPT-4’s 32k token API, conducts up to five tests on each target. Success is defined solely by the capture of the root flag. Table 11.5 details the performance of PentestGPT in these challenges 36 . Ultimately, PentestGPT completes three easy and five medium challenges. The total expenditure for this exercise amounts to 131.5 USD, averaging 21.92 USD per target. This cost is markedly lower than employing human penetration testers and falls within an acceptable range. Our evaluation, therefore, underscores PentestGPT’s capability to yield viable penetration testing results in real-world settings at an efficient cost, thereby highlighting its potential as a practical tool in the cybersecurity domain.

11.7Discussion

We recognize that the penetration testing walkthrough might have been part of the training material for the tested LLMs, potentially biasing the results. To mitigate this, we take two measures. First, we manually verify that the LLM does not have prior knowledge of the target machine. We do this by prompting the LLMs if the tested machine is within their knowledge base. Second, we include penetration testing target machines released after 2021 in our benchmark, which falls outside the training data of OpenAI models. The practicality study on the most recent HackTheBox challenges also demonstrates that PentestGPT can solve challenges without prior knowledge of the target.

The rapidly evolving nature of LLMs and inconsistencies in available APIs could invalidate PentestGPT’s designed prompts. We strive to make prompts general and suitable for various LLMs. However, due to their hacking nature, some LLMs resist generating specific penetration testing content, such as concrete reverse shell scripts. Our prompts include jailbreak techniques to guide the LLM to generate penetration-testing-related information. How to generate reproducible outcomes is an important direction we are working towards.

We identify hallucination in Large Language Models as a significant challenge where the model’s outputs diverge from its training data. This affects the reliability of our automatic penetration testing tool. We are actively exploring various techniques to reduce hallucination and enhance our tool’s performance. As an ongoing work, we believe such an attempt will lead to a more robust and effective automatic penetration testing tool.

11.8Conclusion

This chapter paved the way towards integrating Large Language Models (LLMs) with the operational processes of cybersecurity, particularly in guiding cybersecurity software agents to perform autonomously and more specifically, automating and refining penetration testing strategies. The implementation of PentestGPT, our innovative prototype, showcases the potential of LLMs to emulate human-like decision-making in cyber-attacks. While PentestGPT demonstrates adeptness in fundamental cybersecurity tasks, it also reveals the limitations of current models, particularly in maintaining context and focusing attention over prolonged interactions.

PentestGPT’s architecture, inspired by the dynamics of professional penetration testing teams, divides complex tasks into manageable segments, enabling a collaborative approach to threat analysis and response. This modular structure of Reasoning, Generation, and Parsing replicates the cognitive synergy found in human teams, ensuring a comprehensive and nuanced approach to cybersecurity challenges.

The insights gleaned from our evaluation of PentestGPT suggest that while LLMs offer significant advantages, they are not a panacea. There remains a clear need for human intuition and expertise, particularly in complex and novel scenarios that exceed the training parameters of existing models. As we look to the future, the integration of LLMs in cybersecurity presents an exciting frontier with the potential to revolutionize the field. The continuous development of PentestGPT and similar tools promises to enhance our defensive and offensive capabilities, making our digital infrastructures more resilient against the evolving threats they face. Our work lays a foundation for future research, where the symbiosis of human and machine intelligence will lead to unprecedented advancements in maintaining cybersecurity.

12Conclusion and future work

This thesis, titled "Offensive Robot Cybersecurity," presents a multifaceted exploration of safeguarding robots by adopting a preemptive hacking stance, using Game Theory and Machine Learning as primary tools for automation. The journey began with a motivation (Part I), which provided a general introduction to the field and a description of the thesis structure and framework (Chapter 1). This is followed by some general background (Chapter 2) which provided more context to the intersection of cybersecurity, robotics and artificial intelligence involved in this work.

The content then continues with an in-depth ethical investigation (Part II), setting the landscape (Chapter 3) for responsible and ethically-aligned cybersecurity practices in robotics.

The development of fundamental blueprints (Part III) introduced a suite of core know-how, tools and frameworks, fostering a secure robotics research environment and empowering both defensive and offensive research. This includes DevSecOps for Robotics and ROS 2 security mechanisms (Chapter 4), benchmarking robots and their security (Chapter 5) and reducing the timing overhead and computational bottlenecks introduced by security layers in robotics through hardware acceleration (Chapter 6).

The heart of the thesis lies in the meticulous cyber-attacking of robots (Part IV), a necessary evil that unveiled a plethora of robot vulnerabilities. This research was conducted attacking robot software (Chapter 7), hardware (Chapter 8) and industry deployments (Chapter 9), providing invaluable insights into the current state of robot cybersecurity.

The finale (Part V) of the thesis ventured into the realm of autonomous offensive cybersecurity strategies for robots, harnessing the analytical prowess of Game Theory (Chapter 10) to determine the most vulnerable attack targets and vectors, followed by the use of modern Machine Learning approaches to guide the offensive behavior. In particular, Large Language Models helped bring up adaptive intelligence to our cybersecurity cognitive engines (Chapter 11).

The resulting framework and architecture for our cybersecurity cognitive engines (introduced in Chapter 10) is again presented above in Figure 12.1 and captures the results obtained in this thesis. In relation to the content of this thesis, Parts I and II helped understand better the User and Target entities. Part III helped developed tools such as ExploitFlow, which sits at the core of our architecture providing capabilities for composing exploitation flows to interact with the robotic security environment and achieve desired goals. Part IV helped populate tools like ExploitFlow with actions that corresponded with both exploitation and mitigation of robot security flaws. Finally, Part V completes and connects the architecture altogether with three final contributions: a game theoretic extension of ExploitFlow (through implementing Cut-The-Rope) that allows to reason about attack trees and identify the best attack vectors to achieve desired goals, b) a security reasoning engine (PentestGPT) that digests such annotated attack trees, the attack vectors and the ultimate goal and guides the process and c) a comprehensive penetration testing benchmark (PentestPerf) that helps compare various implementations for the penetration testing task.

In conclusion, this thesis successfully demonstrates the critical role of offensive measures in enhancing robot cybersecurity, charting a path towards a future where robots are not only resilient to cyber threats but also equipped to autonomously defend themselves by hacking-themselves-first with Game Theory and Machine Learning. In a soon-to-come autonomous ever-evolving digital battleground, Malism aims to provide a reference guideline for future implementations. In summary, the work presented in this thesis lays a solid foundation for the future of offensive cybersecurity in robotics, paving the way for innovative solutions that can autonomously protect and secure robotic systems in an increasingly interconnected world. The advancements achieved through this research not only contribute to the academic field by demonstrating how robots can be made more secure with an offensive stance, but also have far-reaching implications for the safety and security of next-generation robotic applications.

Future work will focus on implementing Malism in a scalable form factor, demonstrating fully autonomous offensive and defensive cybersecurity behaviors, and extending its concepts to industrial setups and OT-machinery beyond robotics. Achieving embedded form factors for Malism-like cybersecurity cognitive engines is critical for real-world applicability, necessitating the integration of efficient Machine Learning (ML) and Game Theory (GT) algorithms within limited computational resources. Leveraging hardware accelerators like FPGAs can empower these deployments, ensuring real-time threat detection and response without compromising primary robotic functions. Embedded systems will enable localized, on-device processing, reducing dependency on vulnerable external cloud services and enhancing security response times. Moreover, distributed security frameworks facilitated by embedded Malism-like systems will enhance resilience and coordination in networked environments. This approach is crucial for dynamic, time-sensitive applications in autonomous vehicles, smart manufacturing, and critical infrastructure, where robust, real-time security solutions are essential.

In reflecting on this research journey, we find ourselves at the exhilarating intersection of robotics and cybersecurity, where offensive tactics become the ultimate defense. This thesis has danced through the realms of ethical hacking, intricate game theory strategies, and the boundless potential of machine learning, culminating in the creation of autonomous guardians named cybersecurity cognitive engines for our robotic systems. As we look to the future, Malism stands as a reference concept that deserves further development and time. Such a concept may eventually lead to a world where robots not only outsmart cyber adversaries but evolve with every challenge they face.

Appendices

Appendix AResearch Results

A.1Thesis framework

The topics of interest for the purpose of this thesis are presented as building blocks. These blocks represent a framework for the development of this dissertation. Results are categorized accordingly, with each research product aligning to one of these Blocks as hinted in the tables below. From bottom to top: (0) Ethics and law, (1) Offensive security blueprints for robots and their components, (2) Cyber-attacking robots and (3) Cyber-attacking robots with Machine Learning and Game Theory. See Figure 1.2.

A.2Past work

Tables A.1 and A.2 present a summary of the work performed prior to engaging formally with the PhD. Several articles were produced focusing mostly in blocks 0 (Ethics and law) and 1 (Blueprints). Since building a solid base was instrumental for the PhD research, these grounding publications capture the robotics know-how acquired after multiple years in industry and represent the baseline from wherein the research started.

A.3Peer-reviewed articles

Tables A.3 and A.4 show the peer-reviewed articles published within the PhD period:

A.4Preprint articles (including under review)

Table A.5 summarizes other non peer-reviewed results while classifying them according to the four levels of Figure 1.2.

A.5Tutorials and talks

Table A.6 summarizes participation in peer-reviewed tutorials and talks across workshops and conferences while classifying them according to the four thesis building blocks of Figure 1.2.

A.6Vulnerabilities found

Table A.7 summarizes vulnerabilities found during the research period that have been awarded with a CVE ID:

Appendix BGame Theoretic Appendix

B.1Cut-The-Rope Background

APTs, like most targeted attacks conducted by cybercriminals, due to their diverse combination of attacks, hardly admit a single model to capture them; rather, they call for a combination of models designed for different aspects or characteristics of the attack. Game theoretic defense models may be distinguished according to the nature of APT that they cover: there is the parasitic type, in which the attacker tries to steal resources for as long and much as possible, but does not aim to kill its victim. Related models are FlipIt and its descendants. Minimizing the total time that the attacker spends in the system may not necessarily minimize damage too, since the attacker may entirely destroy the asset \(v_0\) even within a very short period of time. The defender may nonetheless suffer a permanent defeat (upon loss of \(v_0\)). For example, if the attacker can gain access to the security controls of a nuclear power plant even for a very short time, this may be sufficient to cause an unstoppable meltdown. Conversely, the attacker may spend a considerably larger amount of time in other areas of the nuclear power plant’s system; as long as there is no vital subsystem to fiddle with, the damage to the infrastructure may be bearable. This motivates the consideration of the second type of APT, for which the game model Cut-The-Rope is tailored to: there, the attacker aims to kill the victim and silently prepare the final blow. A documented case of this is Stuxnet , and Cut-The-Rope is a game model designed for this latter type.

Many other game models are aligned with the phases in the kill chain, and most related work is specific for at least one of them. We note that the ADAPT project covers a wide spectrum of aspects and phases here. Specific defense models include the detection of spying activities , tracing information flows , detection of malware , deception also via honeypots , attack path prediction , path selection to support malware detection in distributed networks , and general network defense to name only a few. Our game is in a way similar to that of the seminal work , yet differs from this previous model in not being stochastic, and in using payoffs that are not real-valued. The stochastic element is included in a much simpler way in our model, yet preserving information about uncertainty in a full distribution, to avoid losing information by averaging out randomness (for example, replacing a random payoff by a real-valued expected payoff).

Since the methods applied here come from the risk management field, this relates our work to that of , who presents a framework to optimally respond to a detected APT. Their work is thus an a posteriori treatment after the APT succeeded, while ours complements the risk management here by an a priori treatment to prevent the APT from success. Likewise notable is also the work of , who consider inter-dependency graphs in relation to attack graphs in a game-theoretic analysis of targeted attacks. Their work adds constraints to budgets or desirable risk levels, and is specifically about investments in defenses of nodes and edges, but also works with crisp payoff measures (such as, e.g., paths of maximal attack probability or similar).

A different classification of related work is based on the protection targets. defenses can be optimized for confidentiality , the monetary value of some asset upon theft or damage , or the time that an adversary has parts of the system under control . This distinction can be important depending on the context, as industrial production typically puts priority on availability and integrity, with confidentiality as a secondary or tertiary interest. Conversely, whenever personal data is processed, confidentiality becomes the top priority, putting availability further down on the list.

The techniques applied to capture and defend against APTs are manifold, but in most of these (like in our work), the network graph is in the center of attention: it may define how an attack evolves as a dynamical system inside the graph topology, with the challenge of optimized orchestrated defense. A good defense design that needs to account for new vulnerabilities potentially being opened up when closing known security holes. The work of , in this regard, utilizes a game model for graph coloring for a systematic and optimized defense, applying these results to industrial bus systems. Another dynamic yet queuing-based model is that of , which like our model computes optimal resource allocations by the defender and attacker, as an aid for decision making. Tailoring the attack model more closely to the application domain for the sake of a more accurate description, the work of provides insightful connections of graph topological properties of a power grid, and how areas in danger of becoming attacked are identifiable from analyzing the graph.

The work of takes a more birds eye perspective on the domain of the IoT, and applies it directly to or varies the FlipIt game (see the references above and ) to model individual parts of a cloud-based IoT infrastructure, combining these submodels into a larger hybrid game model that allows certain equilibria to play optimally against the adversary. Another cloud-related and -specific APT defense model is . Like us, they adopt a leader-follower model, but different to our work, they use a Stackelberg equilibrium concept.

Taking the APT as a long term yet one-shot event, an attack graph can be treated as a (big) game in extensive form. From this point of view, it is possible to think of the APT as an instance of the induced gameplay, to which Bayesian or subgame perfect equilibria can be sought . More similar to this work, we can treat the APT as a game of inspections, to discover optimal strategies of inspection in different depths of a shell-structured defense . An aspect of strong relevance concerns the use of probabilities: the work of are most interesting in its account for subjective probability and prospect theory, since this includes the way of how humans bring in their individual risk attitudes in decision making under uncertainty (especially about defenses). We avoid this conceptual and practical difficulty in the modeling by designing our game with as few probabilistic parameters as possible.

Cut-The-Rope is, in two ways, different from most other game theoretic models: first, it can let the players act in different time axes, meaning that the defender can be active in discrete or continuous time, while the attacker is (here always) acting in continuous time. This is in contrast to most other models in which both players act in fixed schedules (such as in extensive form games), or both can take actions continuously (such as in differential games). The second aspect is the added suggestion of tie-breaking if there are several equilibria. Cut-The-Rope implicitly addresses the equilibrium selection problem by refining the set of possibly many defense actions based on the probabilities to reach not only \(v_0\), but also to get nearby it. Formally, the optimization, after having minimized the chance to conquer \(v_0\), continues by minimizing the chances to reach a node close to \(v_0\). As mentioned in Section 10.2.1 this induces a dependency on the ordering of nodes, but this ordering is up to the choice of the defender setting up the model. In any case, the defender is not left with a choice among possibly many equilibria, but can have the calculation automatically refine it in an interpretable sense. This equilibrium selection problem is not usually intrinsically addressed in other security game models.

B.2The Model

In the following, we let sets appear as upper case letters, and vectors and matrices in boldface font. Given a finite set \(X\), the symbol \(\Delta(X)\) denotes all (categorical) probability distributions supported on \(X\), i.e., an element \(\mathbf x\in\Delta(X)\) has the elements \((p_1,\ldots,p_{\left\lvert X\right\rvert})\) with \(p_i=\Pr(x_i\in X\) is chosen\()\). The symbol \(\left\lvert X\right\rvert\) is the cardinality of the set \(X\).

We refrain from replicating the full formal description of Cut-The-Rope, and instead summarize its concept in Fig. B.1. The game is played entirely from the defender’s viewpoint: the defender knows the attack graph \(G=(V,E)\) and can enumerate the attack paths on which the adversary can be. For simplicity, we assume that the number of these routes is tractably small. Generally, the number of routes can be exponential in the cardinality \(\left\lvert V\right\rvert\) of nodes, but by strategic domination and other heuristics, some routes may be safely excluded from consideration. We will revisit this point later in Section 10.4.4.

To express the uncertainty about where exactly the adversary is, the defender acts as if the attacker would move a whole cohort of avatars towards \(v_0\), each avatar starting from another possible location in the attack graph with uniform probability. The game is round-based, where the exact meaning of a round depends on the moving patterns of the defender and the attacker:

• If the defender acts periodically in fixed intervals (e.g., daily), a round of the game is one period of activity for the defender (e.g., one day). During this period of time, the attacker can take a random (unlimited) number of steps along the attack path towards the goal.

• If the defender is taking action at random, e.g., taking exponentially distributed pause times, then a round of the game is, in each instant, the random idle time of the defender. Again, during these periods, the attacker can take any number of actions, depending on its “configuration” and/or the attack path. In the terminology of the FlipIt game , this is called an exponential defense strategy.

In both cases, we do not explicitly model the time to complete a spot-check and merely assume this completion to be possible within one unit of time. Including the defender’s costs for spot-checking as a separate goal (to minimize) makes the game multi-criteria and calls for Pareto-optimization, which we leave out of our scope in this work (and up to future considerations). We will come back to the exact meaning of a “round” or “unit of time” in Section B.3.1. Let us first complete the description of the gameplay: Figure B.1 displays two attack paths, with the lower path showing the step-by-step traversal of an avatar towards the goal \(v_0\). Every possible action of the defender is here called a spot-check at any node in \(V\setminus\set{v_0}\), where the target node is excluded to avoid trivialities 37 . A spot-check can mean any action that, for example, (i) cleans a component from malware, (ii) disables certain services that an exploit would rely on, (iii) changes in the security policy or implementation that invalidates the adversary’s knowledge (e.g., access control mechanisms), or similar. Common to all actions of the defender is their transient efficacy, which means that the effect of such an action is not permanent (the opposite case is discussed later in Section 10.4.2). After the action, and not necessarily known to the defender, the attacker is sent back on the attack path to an earlier position (upper part of Figure B.1). For example, if the so-far accomplished route has at some point used access credentials for a computer, and the defender has just changed them, the route is essentially closed at this point, and the attacker has to re-try just before this point 38 . The avatars can go unaffected by the defender’s action in two cases: (i) if it travels on a different route that the defender did not inspect in this moment (e.g., lower attack path in Figure B.1), or (ii) the attacker started from a location below the cut point (e.g., if the attacker is left to the cut point \(\checkmark\) in Figure B.1). This assumption implicitly accounts for “out of attack graph” ways of the attacker having reached this location In either case, the avatar’s journey is not intercepted.

The attacker may at any point decide to try a different route instead. This is called lateral movement. It is naturally included in this modeling by having avatars on all attack routes, which makes lateral movement nothing else than moving other avatars on another route. Cut-The-Rope is played under the assumption that an avatar can be thrown back to an earlier point by the defender, but will in any case re-try its current attack path, until it (or any of its clones) has reached the goal.

The payoffs in the game are zero-sum, and come to the probability of reaching \(v_0\) in a single round of the game. This is the payoff to the attacker, and likewise the loss of the defender, who seeks to minimize this probability. Its computation depends on the probability distribution law that governs how many steps \(N\) can be taken during the defender’s idle periods. This is the main ingredient whose influence is studied in this work, relative to a heuristic best-practice defense.

The payoff to the adversary is the chances for any of its avatars to reach, from its current position, the target \(v_0\) within \(N\) steps and within the time-limit \(W\), during which the defender is idle 39 . This can be fixed (for a periodic defender) or random (for an exponential defense strategy). We collect all avatars in a set \(\Theta\subseteq V\setminus\set{v_0}\), and denote individual avatars as \(\theta\in\Theta\). The exclusion of \(v_0\) from this set is to avoid the trivial case where the attacker has already reached \(v_0\) before the defense game starts. The payoffs to both players are: \[\begin{aligned} u_{\text{attacker}} = -u_{\text{defender}} &= \Pr(\text{adversary reaches~}v_0)\end{aligned}\] We will formally determine this quantity in Section B.2.1 in expressions (B.7) and (B.8), which make the dependency on the strategic choices of the defender and attacker visible and explicit.

Strategies

We now turn to the description of how the defender’s and attacker’s action determine the probability to reach \(v_0\). The strategic choices of both players towards maximizing or minimizing \(\Pr(\text{adversary reaches~}v_0)\) are the following:

• The defender has a choice from the set \(AS_1:=V\setminus\set{v_0}\) to spot-check, giving a total of \(n=\left\lvert AS_1\right\rvert\) actions. We will write \(\mathbf x\in\Delta(AS_1)\) for a randomized such spot-checking rule.

• The attacker can likewise use a total of \(m=\left\lvert AS_2\right\rvert\) attack paths in \(G\), collected in the set \(AS_2\). Each avatar starts from a different location \(\theta\in\Theta=V\setminus\set{v_0}\) and traverses one of the (perhaps many) routes from \(\theta\) towards \(v_0\). The adversary solution in the game is the best choice of attack paths from \(AS_2\). Likewise, we will write \(\mathbf y\in\Delta(AS_2)\) for a random choice from the set of attack paths.

Every avatar takes action by being moved forward along the attack path that it is on, and draws/samples a random number \(N\) from a fixed step-distribution \(f_N\). This is not a strategic choice, but rather a part of the game’s payoff mechanism. Low-level procedures of how the avatar technically mounts exploits are not expressed nor modeled in the game itself (due to the heterogeneity and sheer number of possibilities of exploits in a real-life attack graph).

Definition of Payoffs

For the sake of rigor, let us concretize (B.1) by showing how it is practically obtained. This will also display the role of the movement patterns (periodic, exponential) in the experimental analysis. Working out the adversary’s utility is a matter of conditioning the attack step distribution \(F_N\) on the current situation in the network, i.e, the position of the avatar and where the defender took action.

Let \(\pi_{1}, \pi_{2},\ldots,\pi_{m}\) be an (exhaustive) enumeration of all attack paths, each starting from another location \(\theta\in\Theta\subseteq V\setminus\set{v_0}\). Each starting location is thus identified with one avatar, and the adversary moves all of them towards \(v_0\). Let \(m\) be the total number of all attack paths.

Each such path is again a sequence of nodes, written as \(\pi=(\theta,w_1,w_2,\ldots, v_0)\) with all \(w_i\in\set{v_1,v_2,\ldots}=V\) and \(\theta\in\Theta\) being the starting point of the route, one-to-one corresponding to an adversarial avatar. The set of nodes constituting \(\pi\) is \(V(\pi)\). Furthermore, let \(d_\pi(u,v)\in\mathbb{N}\) count the edges on the path \(\pi\) from \(u\) to \(v\). It is a graph-theoretic distance.

Then, the location distribution for the attacker assigns to each node \(v\in V\) the mass \[ \Pr(\text{avatar location}=v|V(\pi))=\frac{f_N(d_\pi(\theta,v))}{\Pr_{N}(V(\pi))},\] in which \(f_{N}(n)=\Pr(N=n)\), where \(N\in\set{0,1,2,3,\ldots}\) is the random number of steps undertaken by the avatar, and \[ \Pr_{N}(V(\pi))=\sum_{x\in V(\pi)}\Pr_{N}(d_\pi(\theta,x))=\sum_{x\in V(\pi)}f_{N}(d_\pi(\theta,x)).\] The probability density \(f_N\) will be the main element to vary when describing different attacker-defender scenarios (such as announced in the introduction under the contributions). We will give various options to define \(f_N\) in equations (B.10), (B.11), (B.12) and (B.13).

Now, the defender attempts to break the attacker’s chain of exploitation (“cut the rope” in the wording of ). Let \(c\in V\) be the checked node, then the possibly truncated path is \[ \pi|_c = \left\{ \begin{array}{ll} (\theta,w_1,w_2,\ldots,w_{i-1}), & \hbox{if }c=w_i~\text{for some~}w_i\,\,\text{on~}\pi \\ (\theta,w_1,\ldots,v_0), & \hbox{otherwise.} \end{array} \right.\] The closing of a backdoor here becomes a conditioning of the distribution of the avatar’s location on the shorter (cut) path \(\pi|_c\). The formula is the same as (B.2), only with \(\pi\) replaced by \(\pi|_c\) now. Since \(c\sim \mathbf x\) follows the defender’s mixed spot checking strategy (possibly degenerate), and the set of paths \(\pi\) along which avatars proceed, the defender can determine the possible locations of the attacker, based on the imagined avatars, as the vector of probabilities \[ U=(\Pr(\text{adversary's location}=v|V(\pi|_c)))_{v\in V},\] which depends on the random choices of the defender (“where to cut?”) and the attacker (“which route to take?”). This is what the implementation of Cut-The-Rope computes.

The actual quantity of interest for the game, coming back to (B.1), is the mass that \(U\) assigns to \(v_0\). This is the utility for the adversary and conversely the loss of the defender. Since the game is, from the attacker’s perspective, a strategic choice \(\mathbf y\in\Delta(AS_2)\) of an attack path, the payoffs in the game are obtained from the following consideration:

\[\begin{aligned} \Pr&(\text{attacker reaches~}v_0) = \Pr(\text{at least one avatar reaches~}v_0)\nonumber\\ &= \sum_{\theta\in V\setminus\set{v_0}} \Pr(\text{avatar reaches~}v_0\text{~starting from~}\theta)\cdot\Pr(\theta)\nonumber\\ & = \sum_{c,\pi}\sum_{\theta\in V\setminus\set{v_0}}\underbrace{\Pr[\text{avatar~}\theta\text{~has location~}v_0~|~V(\pi|_c)]}_{\text{from eq. (B.2) and (B.4)}}\\ &\quad\cdot\underbrace{\Pr(\text{path~}\pi\text{~is chosen and defender cuts at~}c)}_{\text{strategic choices to optimize}}\cdot\underbrace{\Pr(\theta)}_{=1/\left\lvert AS_2\right\rvert}\nonumber\\ & = \sum_{c,\pi}\sum_{\theta\in V\setminus\set{v_0}}\Pr\Big[\text{avatar~}\theta\text{~has location~}\nonumber\\ &\quad v_0~|~V(\pi|_c)\Big]\cdot\Pr_{\mathbf x}(c)\Pr_{\mathbf y}(\pi)\cdot\Pr(\theta) =: u_{\text{attacker}}(\mathbf x,\mathbf y)\\ &= -u_{\text{defender}}(\mathbf x,\mathbf y)\end{aligned}\]

The equality in the second line herein follows from the fact that the attacker will move one avatar at a time, so that no two avatars will simultaneously reach \(v_0\). The first avatar to reach \(v_0\) will make all others stop, so that the respective events become disjoint.

Solution Concept

An instance of Cut-The-Rope is a quintuple \((G,v_0,AS_1,AS_2,f_N)\), containing: the attack graph \(G=(V,E)\), the target node \(v_0\in V\), the defender’s possible spot check locations \(AS_1\subseteq V\setminus\set{v_0}\), the possible locations \(AS_2\subseteq V\setminus\set{v_0}\), for the attacker’s avatars. These avatars will move towards \(v_0\) along the attack paths encoded in \(G\), taking a random number \(N\) of steps distributed according to the probability density \(f_N\). This density determines the particular behavior of the attacker, relative to the defender’s actions, and will be generally given in Section B.3, and instantiated for the two real-life use-cases in Section 10.3.

A solution for a given instance is obtained with standard techniques to compute Nash equilibria: With both players having a finite set of choices, and the utility \(\Pr(\text{adversary's location}=v~|~V(\pi|_c))\) derived from the location distribution (B.5) that depends on the attack path \(\pi\), movement pattern \(f_N\) and spot-check location \(c\), we end up with a (normal-form) matrix game that we can analyze for an equilibrium using known techniques. The solution concept used in this work is a security strategy for the defender, having the following (informal) semantics: it is the best randomized choice rule \(\mathbf x^*\in\Delta(AS_1)\) such that \[ u_{\text{defender}}(\mathbf x^*, \mathbf y^*)\leq u_{\text{defender}}(\mathbf x^*,\mathbf y)\quad \text{for all~}\mathbf y\in\Delta(AS_2)\] That is, the defender can, upon playing the optimal spot checking strategy \(\mathbf x^*\), enforce the worst-case minimal likelihood for the attacker to reach \(v_0\), for all choice rules \(\mathbf y\in\Delta(AS_2)\), i.e., irrespectively of what the attacker actually does.

The security strategy is computable by solving a conventional matrix game, which is finite since there are only finitely many spot check locations, and likewise finitely many attack paths. The game matrix is thus computable by evaluating formula (B.7), for all locations \(c\in AS_1\) and all paths \(\pi\in AS_2\). The Nash equilibrium of this game is \((\mathbf x^*,\mathbf y^*\)), in which \(\mathbf x^*\) is the sought security strategy, and \(\mathbf y^*\) is the optimal choice rule for the attack paths towards \(v_0\).

The latter information is, however, of limited use for the defender, since equilibria are generally not unique. Therefore, taking \(\mathbf y^*\) as a guidance on where to find for the invisible intruder with highest probability can be misleading, since there may be (plenty of) other equilibria giving entirely different advice.

On the contrary, since the saddle point value giving the lower bound value in (B.9) is invariant w.r.t. different equilibria \((\mathbf x^*,\mathbf y^*)\) any alternative defense advice cannot accomplish any better lower bound for the defender. Hence, \(\mathbf x^*\) is in fact useful as optimal advice.

B.3Movement Patterns

In lack of any particular knowledge about the difficulty of the attack path, a simple heuristic is to just use the shortest path, in a graph theoretic sense. This will later also be the intuitive benchmark (see Section 10.3.1) to compare the defense obtained from Cut-The-Rope to a defense based on the (plausible) assumption that the attacker takes the shortest/easiest route towards \(v_0\).

Periodically Active Defender

In the simplest case, originally proposed in , we assume that the defender becomes active in fixed time intervals that are known to the attacker. The unit of time (see the previous section) is herein the period in which the defender becomes active (each day, each week, or similar). Furthermore, we assume no particular cost for the attacker to penetrate (this case is covered in Section B.3.2). This corresponds to the situation of having a “just conceptual” attack graph, displaying general strategies to penetrate, but without reference to concrete exploits, CVE numbers or similar.

During the defender’s idle times, we assume an average number \(N\sim \mathcal{P}ois(\lambda)\) of steps towards its target at “average speed \(\lambda\)”. This analytical choice is common in related literature (see, e.g., the FlipIt Game to describe APTs, calling this strategy “exponential”). Empirically estimating the rate parameter from data, for example, taken from intrusion detection or other monitoring systems is an interesting challenge of independent research.

The function \(f_N\) for a periodic defender and attacker with average speed \(\lambda\) is the Poisson distribution density \[ f_N(n) = f_{\mathcal{P}ois(\lambda)}(n)=\frac{\lambda^n}{n!}e^{-\lambda},\] which would be substituted into (B.2) and (B.3) to set up the game.

The value \(\lambda\) must be set relative to the frequency at which the defender takes actions. For example, if the attacker makes two attempts per day, and the defender does one spot check per week, then we have \(\lambda=2\times 7=14\). If the defender checks twice per day, then the attack rate is \(\lambda=2\times \frac 1 2=1\). The actual choice of \(\lambda\) was, experimentally, found to mostly impact the likelihood to hit \(v_0\). The defense advice, however, did not significantly change (see Appendix B, section B.4), meaning that an inaccurate choice of \(\lambda\) in practice will deliver a respectively inaccurate estimate on how likely \(v_0\) will fall, but can nonetheless deliver a valid defense recommendations.

We assume that the defender has knowledge (or a reasonable assumption) about \(\lambda\), so that he is able to adapt the defense to it accordingly, as the security resources permit. The choice of \(\lambda\) itself can be considered as a strategic decision for the attacker too, knowing the defender’s behavior. However, we do not explore this variation here any further, as it leads to a different game, but point out this investigation as a separate research question. We refer to the work of as being a game about computing the optimal check-intervals explicitly with help from prospect theory, and under some assumptions on the attacker’s attitude (risk appetite, etc.), but not considering attack graphs.

Probabilistic Success on Exploits

The attacker may not necessarily succeed in every penetration that it attempts. As before, if we assign probabilities 40 \(q(e)\) to express the chance of a successful exploit \(e\) on the respective attack path. Formally, \(q(e)\) could be equated to the likelihood of meeting some precondition to penetrate a node. Let us slightly change the view to think of an attack path \(\pi\) as a sequence of exploits \(\pi=(e_1,e_2,\ldots)\) (instead of nodes). Then, the chances to progress forward by a lot of \(n=0,1,2,\ldots\) steps is no longer Poisson distributed; rather, assuming stochastic independence of exploits, the chances to take \(n=0,1,2,\ldots\) steps are \[ f_N(n)=(1-q(e_{n+1}))\cdot\prod_{k=1}^{n}q(e_k),\] i.e., the probability to succeed with exactly \(n\) exploits, and to fail on the \((n+1)\)st step on the attack path. This function then goes into (B.2) and (B.3) to instantiate the game under the setting described here. A unit of time is, again, the period between two appearances of the defender in the system, again taken as fixed and constant over time (e.g., one day, one week, etc.).

Checks with Random Intervals (“Exponential Strategy”)

If the defender becomes active at its own random (Poisson) rate \(\lambda_D\), the attacker will be able to take a \(\mathcal{P}ois(\lambda)\)-distributed number of steps in an exponentially distributed pause time controlled by the defense intensity \(\lambda_D\). This defense regime defines a random unit of time, whose long run average is exactly \(\lambda_D\).

This change of the setting amounts to a humble change of the Poisson distribution into a geometric distribution, because: we now have two types of events to consider, which are activity of the attacker at rate \(\lambda\) and activity of the defender, at rate \(\lambda_D\). Within a unit of time, we will thus have a number \(k_A\) of attack events, vs. a number \(k_D\) of defense actions. So, the likelihood of the defender to become active is (frequentistically) approximated as \(p=k_D/(k_A+k_D)=\frac{1/n\cdot k_D}{1/n\cdot (k_A+k_D)}\) for all \(n>0\). The last term, however is the average number of events per \(n\) time units, which upon \(n\to\infty\) converges to \(\lambda\) for \(k_A/n\) and to \(\lambda_D\) for \(k_D/n\). Thus, the probability for an action to be taken by the defender is \(p=\lambda_D / (\lambda_D+\lambda)\), and the number of trials that the attacker can take until the defender becomes active again is a geometric distribution with that parameter \(p\). Conceptually, the model thus remains unchanged, except that the attacker’s step number is now computed using the geometric distribution density with the given rate parameter. Consequently, we have \[ f_N(n) = p\cdot(1-p)^n\quad \text{with} \quad p=\frac{\lambda_D}{\lambda_D+\lambda}\] in (B.2) and (B.3).

Spot Checks with Random Intervals and Probabilistic Success on Exploit

Unlike before, we now consider a fixed unit of time, in which an exploit for a given vulnerability can be tried. The defender comes back in random intervals, measured in the this (fixed) unit of time, and has an average return time of \(\lambda_D\). Consequently, the time window for the attacker to run exploits is an exponentially distributed random variable \(W\sim \mathcal{E}xp(\lambda_D)\). Within this time window \(W\), the attacker ought to accomplish \(n\) exploits, along an attack path \(\pi=\theta\to w_1\to w_2\to\ldots v_0\), in the notation of Section B.2.1. Like in Section B.3.2, let us call \(e_k\) the edge into node \(v_k\), which carries a known exploit complexity as the quantity \(q(e_i)=\Pr(\)exploit on \(e_i\) is successful within a (fixed) unit of time). Then, an exploit on edge \(e_i\) takes an exponentially distributed time \(T_i\sim \mathcal{E}xp(1/q(e_i))\). The total time for \(n\) exploits is thus \(T_1+T_2+\ldots+T_n\), which, unfortunately, does not admit a closed analytical expression for its distribution, since the values can be assumed independent, but not identically distributed. To escape the issue, we simplify matters by assuming the avatar to move at a uniform velocity along the attack path, instead of being faster and slower depending on the attack complexities. We believe this assumption to be mild, since our main concern is the time it takes to reach the end \(v_0\) anyway, and we are not as much interested in determining the avatar’s location anywhere in the middle of the attack path.

This simplification comes to a geometric mean of the probabilities \[\overline{q}=\text{geomean}\set{q(e_i)~|~e_i\text{~is on the chosen attack path}}.\] The point is that the product of the actual probabilities, i.e., the chance to hit \(v_0\), remains unchanged hereby, since \(\prod_i q(e_i)=\overline{q}^{\left\lvert V(\pi)\right\rvert}\) where \(\left\lvert V(\pi)\right\rvert\) is the length of the attack path. Let us put \(\lambda_{\pi} := 1/\overline{q}\) to bring the notation closer to that of Section B.3.3, since the result (to come later) will also be close to this previous finding. The subscript \(\pi\) to \(\lambda_\pi\) herein reminds about the attack rate now to depend on the chosen path.

Under this simplification, the time for \(n\) exploits is the sum of all identically \(\mathcal{E}xp(1/\overline{q})\)-distributed random variables \(E_n := T_1+T_2+\ldots+T_n\sim \mathcal{E}rl(\lambda_{\pi},n)\) that is Erlang distributed. We are interested in the probability of \(T_1+T_2+\ldots+T_n\leq W\), which is a matter of computing a convolution integral. We shift the algebraic details to the appendix B, and directly give the result here: \[ f_N(n) = \begin{cases} \Pr(E_N \leq W) = \left(\frac{\lambda_{\pi}}{\lambda_{\pi}+\lambda_D}\right)^n & n\geq 1;\\ \Pr(E_1 > W) = 1-\Pr(E_1\leq W) & n=0. \end{cases}\] Observe that this is movement pattern is like in Section B.3.3, which is yet another geometric distribution, only with the different parameterization.

The approach of geometric averaging over the entire attack path deserves a bit of discussion: we could equally well average only across the segment of length \(n\) that the attacker targets to overcome, and/or exclude all exploits with \(q(e_k)=1\) from the averaging. We refrained from both these options for two reasons: first, removing the 1es from the averaging would unrealistically shorten the attack path to less than its physical reality. Even if an exploit has a 100% chance to be used within short time, there is nonetheless a time step necessary to do it, so including it in the geometric mean seems plausible. An attack path that is longer will, despite the same product probability of accomplishing it, take a proportionally longer time to traverse. Second, concerning the focus on only a segment, this may miss the actual intention of the adversary, since it targets the end of the attack path, and not only a specific segment. In other words, geometrically averaging only over the first \(k\) exploits would be the assumption that the adversary would stop at the \(k\)-th step, even if there is time left before the defender comes back. Since the target is getting to the end of \(\pi\), it appears plausible to include all exploits towards this end.

B.4Analytic Results

For a plausibility assessment of the game’s results, we analytically study the results on attack graphs with no information at all, so that we can intuitively (and without any model) determine a best defense. The analytic results to follow are consistent with the numeric findings for MARA, and, more importantly, are independent of the attack rate \(\lambda\) (see Proposition B.4.3.

Given a set \(AS_2\) of attack paths, the defender’s best strategy in Cut-The-Rope is to find and guard a minimal (in a sense to be defined later) graph-theoretic cut \(C\subseteq V\) between the starting node (set) of all attack paths, and the target asset \(v_0\). Towards proving this claim, suppose that the defender would focus on a set \(S\subset V\) that is not a cut. Then, there is a path \(\pi\) that bypasses \(S\), i.e., \(S\cap V(\pi)=\emptyset\), but this makes \(\pi\) a winning strategy for the attacker (since the defender can never catch the attacker on that route). Suppose that the defender’s guarded set \(C\) were not minimal, i.e., the defender spot-checks on a superset \(S'\supset C\). Then, we can distinguish two cases:

1. either no attack path passes through the nodes \(S'\setminus C\), in which case defending them is useless, and hence defending \(S'\) is a sub-optimal strategy (as it consumes too many resources), or

2. there is at least one attack path through a node in \(S'\setminus C\) and another node in \(C\). In that case, we can safely remove either of the two, since both would cut the rope in the sense we desire. This strictly shrinks \(S'\), and we can repeat this reduction until the resulting set has become minimal (in terms of cardinality).

Compiling the thoughts above concludes the proof of the next result:

Lemma B.4.1 makes no assertion about what cut to choose if there are several. For example, if we have only one attack path overall, then every node on it would be a valid cut. Intuitively, the best option is cutting the (single) rope as close as possible near \(v_0\), in order to get the most likely locations covered from which an attacker’s avatar could start. The proof of Lemma B.4.2 makes this rigorous:

Now, we can compile the findings so far into a generic characterization of the defender’s best choice:

Consistency of Numeric and Analytic Results

The numeric findings for the MARA use case agree with the analytic predictions to defend the graph cut that is closest to the target nodes. The formal arguments in Section B.4 assume the same distribution for all possible paths, which does not hold for the MiR100 use case. Thus, the optimal defense no longer needs to be a graph cut, and the numeric results about the MiR100 use case confirm this possibility. Since in the MiR100 case, the attack paths have different efficacies, strategic dominance among the attack paths may affect the results accordingly. Since the results, in this more general case, depend on the distribution conditional on the attack path, it appears unlikely that comparable analytic predictions can be made for the movement pattern of Section B.3.4, and we leave this as an open problem.

Regarding the heuristic defense, its bad performance in comparison to Cut-The-Rope can be attributed to the defender blindly checking everywhere on the attack graph, while the intuition (also behind the formal arguments of Section B.4) would rather advise to defend closer to the goal. This suggests that the optimization that Cut-The-Rope may be reasonably replaced by a heuristic defense, only focused on a graph cut subset of nodes, and indeed, the numbers for the MARA use case show an approximately uniform defense of nodes on such a cut to be optimal. Overall, however, it is advisable to run an optimization, since just adding the analytic prediction of where to defend to the heuristic is incorrect in the case where the traversal of an attack path depends on the path’s properties, such as distinct difficulties to exploit, as in the MiR100 use case. Here, the performance of the defense is substantially better than for the heuristic, but the apparent focus on a graph cut is not found in the results.

B.5Application for Risk Control

Actions with a permanent effect change the attack surface by blocking certain paths, increasing the attack detection capabilities, or similar. Examples include the installation of a firewall, malware scanners, deactivation of services or accounts, and many more.

If the defender’s action space includes at least one with potentially permanent effect, the attack graph, and hence the overall game, changes with the defender’s activity, and the game must be re-instantiated before the next round after pruning the attack graph. This turns Cut-The-Rope into a dynamic game, but it is still repeated with infinite time horizon. It is fair to remark that the tree may not only become pruned, but introduce new attack paths upon inserting new components, installing new software or similar.

In both cases, the setup of the game may (but does not need to) start from the results of a topological vulnerability analysis, with repetitions being either from the existing defense equilibrium strategy (static instance) or including the re-instantiation and equilibrium computation (dynamic instance); see Figure B.2 for a flowchart-like presentation.

B.6Derivation of the probability (B.13)

The density of the \(\mathcal{E}rl(n,\lambda)\) distribution family is for \(x\geq 0\) given by \(f_{\mathcal{E}rl(n,\lambda)}(x)=\frac{\lambda^n x^{n-1}}{(n-1)!}e^{-\lambda x}\) and \(f(x)=0\) for \(x<0\). The density of the exponential distribution is a special case thereof, \(f_{\mathcal{E}xp(\lambda)}(x) =f_{\mathcal{E}rl(1,\lambda)}(x)\). Abbreviating the total time as \(T=T_1+T_2+\ldots +T_n\), with all i.i.d. summands \(T_i\sim\mathcal{E}xp(\lambda_\pi)\), we are interested in whether \(T\leq W\iff Z:=W-T\geq 0\). The case \(Z=z\) for \(z\in\mathbb{R}\) occurs if and only if \(W=t+z\) and \(T=t\) for any \(t\in\mathbb{R}\), and we get the convolution-like integral for the density of \(T-W\) as \[f_{T-W}(z)=\int_{-\infty}^{\infty} f_{\mathcal{E}rl(\lambda_\pi,n)}(t)f_{\mathcal{E}rl(1,\lambda_D)}(t+z)dt.\] We are, however, only interested in the probability \(p=\Pr(T-W\geq 0)\), which adds a second integral to get the quantity of interest \[\begin{aligned} p&=\int_0^\infty f_{T-W}(z)dz\\ &= \int_0^\infty \int_{-\infty}^{\infty} f_{\mathcal{E}rl(\lambda_\pi,n)}(t)f_{\mathcal{E}rl(\lambda_D,1)}(t+z)dtdz.\end{aligned}\] A bit unexpectedly, the double integral makes things easier to evaluate here, since we can swap the order of integration (by the Fubini-Tonello theorem), to get \[\begin{aligned} p&=\int_{-\infty}^{\infty} \int_0^\infty \underbrace{f_{\mathcal{E}rl(\lambda_\pi,n)}(t)}_{\text{const. w.r.t. $z$}}f_{\mathcal{E}rl(\lambda_D,1)}(t+z)dzdt\\ &=\int_{-\infty}^\infty f_{\mathcal{E}rl(\lambda_\pi,n)}(t)\underbrace{\int_0^\infty f_{\mathcal{E}rl(\lambda_D,1)}(t+z)dz}_{=e^{-\lambda t}}\,dt\\ &=\int_{-\infty}^\infty \underbrace{f_{\mathcal{E}rl(\lambda_\pi,n)}(t)}_{=0\text{~for~}t<0}e^{-\lambda t}\,dt =\int_0^\infty f_{\mathcal{E}rl(\lambda_\pi,n)}(t)e^{-\lambda t}\,dt\\ &= \left(\frac{\lambda_\pi}{\lambda_\pi+\lambda_D}\right)^n\end{aligned}\]

Appendix CMachine Learning Appendix

C.1Illustrating Example

Below we provide a concrete example of how we guide GPT-4 to intialize the penetration testing task, and generate detailed operations in an iterative approach given a penetration testing benchmark target. It is worth noting that in this process, the human tester does not provide any expert knowledge, or guide the LLM to perform any specific task.

Read the original paper ↗