Resources: Red Team

Web3 Devops Penetration Testing

Threat actors have leveraged DevOps access to perform what are known as “software supply chain attacks.” Such attacks are especially effective when performed from within popular software packages.

Web3 Devops Penetration Testing

In recent years, software development infrastructure (DevOps) has come under increasing scrutiny by threat actors because of its large technological footprint, its traditionally low sophistication of security controls, and its privileged position which allows modification of source code as well as deployment of code to production. In the past, threat actors have leveraged DevOps access to perform what are known as “software supply chain attacks.”  Such attacks are especially effective when performed from within popular software packages. As an example, a breach at the software vendor SolarWinds[1] was used by threat actors to insert software into their Orion Network Management System. Orion customers installed this SolarWinds product on-premises within their enterprise network. When the update containing the malicious software insert was distributed, the threat actor was able to gain access to and compromise these customers, which included the United States Treasury department.

Components Tested

For organizations developing blockchain code and/or code which leverages the blockchain, performing a penetration test against DevOps which targets the following components is recommended:

  • Developer access to production credentials
  • Code repository access / contents
  • Authentication / authorization to sensitive development components
  • Access to production deployment
  • Secrets management
  • CI/CD infrastructure (e.g., Jenkins, Teamcity, GitLab CI/CD)

Attack Surface

The security posture should be assessed from the simulated perspective of a compromised or malicious developer account with the most common types of developer access rights. This imitates a successful spearphishing or exploitation campaign under realistic conditions. In a DevOps environment with a mature security posture, a compromised developer account with properly configured security controls present within the organization should not severely impact the security of production code, and offensive testing will aim to confirm this. The following is a non-exhaustive list of common attack vectors against DevOps:

Projects which are built from systems of systems tend to have more complex deployment processes, pulling code or artifacts from many different locations within the DevOps environment. Testing should confirm that these locations, scripts, or documented build procedures are free from exploitable defects which would allow an attacker to modify code as it is being prepared or built for deployment.

Development environments frequently employ artifact storage services to store portions of compiled code which are not frequently re-built. Build systems take this code from artifact storage and integrate it into final packages. If these artifact storage systems are misconfigured or vulnerable to exploitation, an attacker may be able to insert infected packages into the build process.

Software, especially software running on trusted servers, often requires access to third-party services which require secret tokens or API keys. If production secrets are present within the code repository, developer documentation, build log, or deployment log, then they are being improperly distributed to a wider audience (the development team) than is strictly necessary, putting these keys and tokens at greater risk of compromise. 

Infrastructure such as Gitlab, Perforce, Jenkins, and Teamcity are frequently treated as “fire and forget” despite having long histories of severe exploitable vulnerabilities. These applications, as well as supporting infrastructure such as Active Directory and cloud service providers, must be included in any offensive testing strategy to gain a complete picture of the DevOps security posture.        

Code management systems, unless modified from their default configurations, do not usually support commit validation “out of the box.” This allows an attacker with developer access to the code repository to spoof the authorship of code commits and merge unapproved code more easily. In addition to authorship identity, attackers are also able to modify commit timestamps and other metadata. These techniques should be included in DevOps testing to audit adherence to the team’s merge and deployment policies, which should include at least two reviewers prior to merging code into production.