Overview

Risk Mapping and threat modelling, when combined, is the main approach used to assess risk in testing, and is performed as part of testing analysis, design and planning. The combined approaches are most famously used in Governance, Risk and Compliance (GRC) when it comes to information and cybersecurity. There are many standards available that cover risk mapping and threat modelling for security, however risk mapping and threat modelling can be used to assess and test for risk for any value category. (IEC, 2019), (ISO/IEC, 2022), (NIST, 2012), (van Daele, 2016).

 

Approach

Risk mapping and threat modelling is a structured approach to assessing risk and threats guided by a set of steps. An initial, formal risk mapping and threat modelling session can be done early in the software project lifecycle, typically during design and architecture. Revisiting risks and threats are done informally throughout the software project lifecycle through to deployment and operation. This means that risks are managed inline with agile principles and are continuously performed throughout all phases of testing in the DevOps lifecycle.

While testers can be coaches and facilitators of the risk mapping and threat modelling process, risk mapping and threat modelling is performed as cross-functional teams where everyone shares ownership and responsibility for the process and managing the risks that come out of the process.

Value, Assets & Scope Quality Categories Risks Threats Vulnerabilities Management

Risk Map and Threat Model

Value

Analyse and identify the business value and stakeholders this value is delivered to. This is typically done during test analysis activities early on in the project during business ideation and communicated as user stories. The company or team’s risk appetite and risk tolerance is also considered at this stage as a criteria for evaluating risks later on.

Scope

Analyse, decompose and decide upon the technical scope. Create a visual representative of the software system: it’s major components, integrations and processes including data flow and trust boundaries for internal and external components and services. The key question to ask here is “what are we building or working on?”

Assets

Identify the important functions, data, interfaces or other system components that hold or deliver value. Mark these on the technical scope model above.

Categories

Decide upon the risk categories that are most important and prioritise the top few depending on project size and complexity.

Risks

Identify, discuss and brainstorm the major risks. Ask the question “what could go wrong?” including what potential loss or harm and who that applies to. Use the categories identified above to guide ideas but also think about other risks that may require revisiting the above.

Threats

For each of the risks, identify the threats that could bring these risks into reality. These are specific events or scenarios where users or other stakeholders can potentially interact with the system to exploit a vulnerability leading to a bug. In the case of security, people (threat actors) and natural events also represent threats. Threat intelligence (such as previous bugs reported or databases of common threats) and threat frameworks can be used to help identify threats.

Vulnerabilities

For each of the threats, identify vulnerabilities or weaknesses in the system, be it specific components, processes or even some types of data that, if exploited by a threat, could lead to a risk of lost value or harm.

Manage

The team decides what to do with each of the risks (key question: “what are we going to do about it?”), whether to accept (retain), mitigate (control), avoid or transfer. For mitigate (control), this can involve putting measures in place to reduce the risk with quality control or assurance processes, software changes (including bug fixes for any uncovered risks during testing) and other actions. At this stage, testers design and implement tests that can attempt to prove the risk (bug) or to uncover new risks that weren’t considered at the time.

Individual owners may also want to be assigned to each risk in the same way that other project work items are, with regular status reports on the risk.

Priorities

Analyse and determine the likelihood and impact of risks using a qualitative or quantitative approach, such as scoring low, medium, high on a risk matrix or using projected costs. From this, prioritise each of the risks accordingly using the same approach. It’s important at this stage to consider risks as a whole and how they impact each other, for example, two risks on their own may not have a high likelihood or impact, but if combined together, may represent a much bigger risk.

Follow-Up Activities

Document

Ensure all risks are documented and communicated for the whole team and management. One approach is to create a risk register which includes the likelihood and impact assessment, and priority, but could contain any of the information listed above plus any recommendations. Another approach is to create work tasks in the project management software that sit alongside user stories and bugs. Key Risk Indicators (KRI) metrics may also be defined at this stage.

Monitor

Keep on top of the risks and threats throughout development and operation. Test and retest for regressed risks and threats in the software system especially due to code changes. Also monitor for issues in production where risks have manifested into bugs that have impacted customers and end users.

Retrospect

Iterate and improve the process with key questions “Have we done enough and have we done well enough?”. Feed all new information back into existing risk maps and threat models, and update them accordingly. Revisit and update the whole process for the next project as part of retrospectives and root cause analyses. Along with agile principles for all project activities, the goal is always continuous learning and improvement.

 

Example

Value: As a passenger, I want to set my pickup location, so I can get picked up easily and without having to walk to far.

Scope: A technical diagram showing the self-driving taxi, the phone application used by the passenger, satellite navigation to pinpoint the pick up position and navigation for the taxi to make their way to the pick-up position.

Assets: The functionalities to specify a pick-up location, to find a route there via road network, to navigate there via LIDAR, RADAR and AI systems.

Categories:

• Capability – to get to the destination using all asset functionalities
• Performance – to get there in good time without undue waiting
• Usability – for the passenger to set the pick-up location easily
• Security – to prevent unauthorised use of the passenger’s phone app and to prevent third-parties from preventing the taxi getting to the pick up point

Risks:

• Passenger unable to set up a pickup location near them despite being near a road
• Taxi loses satellite navigation meaning it can’t get to the pick up location keeping the passenger waiting indefinitely or cancelling the journey
• Navigation or AI systems fails causing the taxi to crash into someone or something causing physical harm or property damage
• |Taxi’s pickup location is leaked or manipulated

Threats:

• Other road others accidentally or maliciously trying to block the taxi
• Snow or other adverse weather affecting navigation
• AI’s training data not taking into account all road types
• Passenger clicking on the wrong menu items or getting lost on the phone app
• A malicious hacker rerouting the taxi for themselves or uncovering the passenger’s current location

Vulnerabilities:

• The AI system’s “fail safe” stopping at any location
• LIDAR system’s inability to penetrate heavy weather
• Gaps in the AI training data
• Overlapping or misplaced GUI elements
• API for pickup locations open to SQL injection attack

Priorities:

• The passenger pick-up failure near a road is low likelihood as this is reliable proven software but medium impact as it could cause the passenger to walk a great distance or abandon the ride altogether.
• The taxi losing satellite navigation is possible intermittently, but to lose it altogether is unlikely, but if it did, then the risk is medium-high as it could leave the passenger waiting indefinitely or cancel the ride losing a sale.
• Taxi’s pickup location getting hijacked is low likelihood due to extensive cryptography and planned penetration testing, however the impact would be high
• The taxi navigation and AI systems failing to recognise roads or road users is medium likeliness as it does happen regularly however the impact of this could be serious loss or life or property.

Document: All the risks above with their priority assessment are created in a risk register and also as a project work item alongside user stories to track per sprint

Manage:

• For the passenger pick-up location will be mitigated to only be able to select locations near a road only, and will be tested by searching for potentially problematic areas (inaccessible roads, open plazas far away from a road, highways closed to motor traffic or other such locations unsuitable for pick-up including tunnels and bridges.
• For the taxi losing satellite navigation, we’ll mitigate store location information and also use the passenger’s phone for navigation as a backup. We’ll test this with varying lengths of lost satellite navigation to see what points it gives up and how the taxi handles the situation.
• For the taxi’s location getting hijacked, we’ll conduct extensive penetration testing with third-party companies and ensure we have robust compliance and security procedures
• For the navigation and AI systems failures, we’ll test this extensively though phases of internal and beta on physical and simulated environments trying anything we can to cause any type of driving mistake by the taxi. This will be a highly open and exploratory approach.

Monitor:

• Extensive sensors on the taxi and logging on both the taxi and phone application will collect data about potential problems feeding them back to a database for investigation. We’ll also invite users and beta testers to feed back their experiences with a special focus on what went wrong.

Retrospect

• The risk assessment will be revisited every sprint and at the start of each new feature project to update and revise risks as required, adding new risks from feedback.

 

Citations and Further Reading

• • van Daele, 2016. Consult the TestSphere. Isle of IT. Link
  • IEC 31010:2019. Risk management — Risk assessment techniques. Link (Alt)
  • ISO/IEC 27005:2022. Information security, cybersecurity and privacy protection — Guidance on managing information security risks. Link
  • NIST SP 800-30 Rev. 1 2012. Guide for Conducting Risk Assessments. Link