Securing your CI/CD pipelines in the cloud

As thousands of organizations make the move to the cloud, they are becoming increasingly reliant on CI/CD pipelines to facilitate their cloud workloads. Securing these pipelines has therefore become a primary objective, with many companies looking to adopt a shift-left security approach in order to keep up with the constant barrage of cyber threats. The shift-left security paradigm plays a crucial role, as it ensures the implementation of security comes first and, in turn, helps to avoid security debt.

Scanning images and implementing the role-based access control (RBAC) mechanism at the beginning of the software development lifecycle are important steps in adopting a shift-left approach, especially when using in-house CI/CD pipelines and CI/CD platforms such as CircleCI. CI/CD platforms are generally more secure and better configured than in-house CI/CD pipelines, since the platforms are built by developers who truly understand CI/CD pipelines and how they work. Platforms also allow for fast updates when new CI/CD pipeline vulnerabilities are discovered.

In many cases, misconfigurations are the cause of CI/CD pipeline failure. However, cloud CI/CD platforms such as CircleCI are configured to provide reliable security features and measures such as RBAC. Following best practices for CI/CD security can also help prevent misconfigurations.

In this article, we take a look at methods for securing your CI/CD pipelines in the cloud.

What are CI/CD platforms?

CI/CD platforms and tools offer a set of features to facilitate CI/CD pipeline processes such as building, packaging, and testing software components. They maintain CI/CD scripts, autoverify, and simplify the deployment process. This allows you to identify problems with deployment and builds.

A cloud CI/CD platform is preferred over an in-house CI/CD platform. First, you won’t have to spend thousands of dollars on infrastructure. Second, and perhaps the greatest benefit, you won’t have to maintain the infrastructure. Some examples of CI/CD tools and platforms include:

• GitLab CI
• GitHub Actions
• TravisCI
• CircleCI 
• AWS CodePipeline

Each of these platforms are run by developers explicitly dedicated to fine-tuning the security of the platform and who stay up to date on CI/CD platform cybersecurity. 

There are many benefits to hosting your codebase in the cloud while using CI/CD platforms:

• RBAC: Usually out-of-the-box role-based access control mechanisms over access to certain subsystems and operations
• Rollbacks: Baked-in ability to rollback to an older version if the new one is performing below defined standards
• Easy integration: Straightforward connection to your toolchain and code repositories
• Logs & metrics: Only metrics can indicate the death or health of your application. CI/CD platforms provide logs and metrics indicating rollback frequency and deployment time.

Implementing security measures in CI/CD cloud tools

Insecure coding, insufficient access controls, security misconfigurations, exposure of secrets, use of flawed third-party libraries, and supply-chain attacks are just a few of the cloud CI/CD pipeline threats you should be on the lookout for. In the next section, we’ll review methods to combat these CI/CD pipeline threats.

Continuously monitor and analyze your CI/CD logs

Combating the growing number of cyber attacks requires dynamic application security. Log analysis and customer feedback are crucial when it comes to detecting vulnerabilities. Continuously analyzing CI/CD logs will give you insight into how your build processes are performing. Setting up a logging system using tools such as Mezmo will enable you to identify builds and deployments that are taking longer than expected. 

When logging CI/CD pipelines, there are a number of critical metrics you should be collecting and analyzing:

• Code commit time
• Deployment times
• Build times
• Important application info such as application metadata

These metrics will help spot inconsistencies. If a deployment or a build process takes too long, this indicates an issue that needs to be addressed. Code commit time helps identify when specific code that might have caused issues was committed.

Protecting access to your cluster through RBAC

Both Kubernetes and Docker are major components of DevOps and CI/CD cloud platforms. Protecting access to your Kubernetes clusters is therefore crucial. 

Implementing the role-based access control mechanism—which allows you to select which objects and cluster users can access specific cluster resources—is one way to enhance CI/CD platform security. It also enables you to block certain users from accessing cluster secrets. RBAC contains two important definitions: Roles, which define which resources can be accessed; and RoleBindings, which specify the users and service account with access permissions to the resource as well as what actions can be performed on the resource.

Following is an example of a Role that gives the user access to services with limited actions:

apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata:   namespace: organization   name: service-access rules: - apiGroups: [""] # "" indicates the core API group   resources: ["services"]   verbs: ["get", "watch", "list"]

 

Scanning Docker images

Continuous security auditing and image scanning are key when it comes to knowing what’s happening in your pipelines and codebase. Containers, clusters, source code, and infrastructure must be scanned regularly. There are many tools available to scan your clusters, such as Kubescape and the docker scan command. If you have Docker, use the following command to scan your images:

$ docker scan [enter the name of the image here]

 

You will receive output showing you any vulnerabilities that have been discovered in your image:

✗ Medium severity vulnerability found in openssl/libcrypto1.1   Description: Inadequate Encryption Strengthsername and password. Note this will not work if   Info: https://security.snyk.io/vuln/SNYK-ALPINE315-OPENSSL-2941810   Introduced through: openssl/libcrypto1.1@1.1.1n-r0, openssl/libssl1.1@1.1.1n-r0, apk-tools/apk-tools@2.12.7-r3, libretls/libretls@3.3.4-r3, python2/python2@2.7.18-r4   From: openssl/libcrypto1.1@1.1.1n-r0   From: openssl/libssl1.1@1.1.1n-r0 > openssl/libcrypto1.1@1.1.1n-r0   From: apk-tools/apk-tools@2.12.7-r3 > openssl/libcrypto1.1@1.1.1n-r0   and 6 more... docker scan  getting-started   Image layer: 'apk add --no-cache python2 g++ make're to provide both SNYK_INTEGRATION_NAME and SNYK_INTEGRATION_VERSION together!   Fixed in: 1.1.1q-r0 dependencies for getting-started ✗ Medium severity vulnerability found in node   Description: HTTP Request Smuggling   Info: https://security.snyk.io/vuln/SNYK-UPSTREAM-NODE-2946723   Introduced through: node@12.22.12   From: node@12.22.12   Fixed in: 14.20.0, 16.16.0, 18.5.0   Image layer: Introduced by your base image (node:12.22.12-alpine3.15)   Fixed in: 14.20.0, 16.16.0, 18.5.0 ✗ Medium severity vulnerability found in node   Description: DLL Hijacking   Info: https://security.snyk.io/vuln/SNYK-UPSTREAM-NODE-2946727   Introduced through: node@12.22.12   From: node@12.22.12   Fixed in: 14.20.0, 16.16.0, 18.5.0   Image layer: Introduced by your base image (node:12.22.12-alpine3.15)   Fixed in: 14.20.0, 16.16.0, 18.5.0

 

Encrypting secrets

Secrets are widely used in DevOps because there are many tools that communicate using APIs, which utilize endpoints. Secrets give access to endpoints, the path to data storage, or any software resources. Thus, you need to ensure your secrets are secured using the best encryption algorithm before storing them in the ETCD. 

Sealed Secrets (formerly known as Kubeseal) is one helpful tool for encrypting and sealing secrets. This makes them safe to be stored in a public repository because the owner alone is the one who can decrypt them.

Implementing multi-factor authentication

Repositories and Git accounts must be secured with multi-factor authentication. This ensures you will be alerted if hackers attempt to enter your system but fail to gain access, since the access verification code will be sent to your phone or email address.

Scanning workload deployment manifests

Software vulnerability exploitation is a daily occurrence, and Kubernetes security exploitations make an even greater impact since K8s orchestrates multiple containers carrying various applications. Kubernetes security is thus at the core of secure application delivery.

Monitoring and implementing security measures is made easy with the use of automated tools that scan and analyze cluster security performance, identifying vulnerabilities you were unaware of. Even better, they offer suggestions for how to fix the detected vulnerability. 

Tools that offer Kubernetes security solutions include:

• Kubescape
• Datree
• Kubeval
• Trivy
• Kubeaudit

Next steps for securing CI/CD pipelines in the cloud

Following best practices is key to keeping your cloud CI/CD pipelines secure. Even with the pressure to meet release dates,  CI/CD pipeline security, combined with cybersecurity training must be a priority. Moreover, implementing security measures at the beginning of the CI/CD pipeline will help you avoid security debt in any of your software layers; while consistent software testing and checking code test efficiency will help you keep up with the ever-evolving threat landscape.

Looking to implement a DevSecOps approach with your risk remediation? The Vulcan Cyber® risk management platform correlates, prioritizes, and manages risk across all your attack surfaces—and at scale. Get your free trial and start owning your risk.