Demonstrating end-to-end traceability with pull requests

Understanding what was changed and why is vital to understanding the state of any system. This is especially true for software teams in regulated industries, as they are required to demonstrate compliance with various standards and regulations to internal and external auditors. While each standard varies a bit depending on the issuer and/or industry, most share several common requirements, including a need to demonstrate end-to-end traceability of deployed changes as part of a standard change control process.

In many companies, “compliance” and “traceability” can be synonymous with phrases like “high friction,” “onerous processes,” and “additional tooling.” GitHub provides companies with the necessary capabilities to design standards-compliant workflows and demonstrate their compliance to auditors while keeping the developer experience centered around something developers use every day—the pull request.

Even if your organization isn’t required to meet a particular compliance standard, following these practices can help to build confidence in your process and provide greater transparency into all deployed changes by:

• Making it easier to troubleshoot production issues. The ability to explicitly understand what was changed, when it was deployed, and the reasons for the change help engineers and response teams efficiently triage issues, identify problems, and work toward resolution.
• Tracking the business impact of changes. Mapping business requirements to production changes greatly improves the ability to analyze whether or not the change resulted in the desired impact.
• Minimizing the effort and cost associated with internal investigation and audits. A recent Forrester report notes GitHub customers building compliance into their workflow saved significant time and expense preparing for audits. When workflows are automated, standardized, and easily traceable the time required to prepare and perform an audit or investigation goes down significantly.
• Improving the developer experience. A single, agreed-upon source of truth for developer workflows helps keep business teams, ops, security, and development all on the same page and empowers them to have more effective, informed discussions.

A note on compliance standards

When creating compliant workflows, it is important to remember compliance is assessed against your overall process and is not a simple pass-through from a vendor. Standards achieved by vendors are indicative of the vendor’s internal processes. Non-compliant workflows can be built with tools from a “compliant” vendor and compliant workflows can often be built from tools whose vendors may not meet the standard for their internal processes. It all depends on the context of the service provided.

It’s equally important to consider the applicability of the standard. For instance, standards that apply to the storage and maintenance of protected data, such as Personally Identifiable Information (PII) and Personal Health Information (PHI), require certain guarantees of your deployment targets and data stores that don’t apply to systems managing source code and work tracking. Neither of these systems should ever contain such information.

Looking to the standards

When discussing software changes, standards often focus on the ability to link deployed changes to:

• An initial request (which may include detailed requirements)
• Authorship of the code
• Review and approval by the appropriate person(s)
• Test and build artifacts

Let’s take a look at few examples:

Requirement 6.5.1 of the PCI/DSS standard requires every change be made through an established procedure that includes:

• The reason for, and description of, the change
• Documentation of security impact of the change
• Documented approval by authorized parties
• Compliance with secure development processes
• Security testing

ISO 27001 A.12.1.2 calls for formal change management process that includes:

• The request for change
• An evaluation of the change
• Approval of the change
• Implementation of the change

It is expanded upon in A.14.2 to further include:

• Formal change control procedures for all systems in the development lifecycle (A.14.2.2)
• Review and testing to ensure no adverse impact on security (A.14.2.3)

The COBIT framework, commonly used by companies to define and audit their internal processes, can be mapped to many common compliance standards. COBIT’s practice description for BAI06.01 specifically calls for organizations to:

“Evaluate all requests for change to determine the impact on business processes and I&T services, and to assess whether change will adversely affect the operational environment and introduce unacceptable risk. Ensure that changes are logged, prioritized, categorized, assessed, authorized, planned and scheduled.”

Starting from the pull request

Each of the above common compliance standards and frameworks focus on ensuring proper controls exist for all deployed changes. GitHub has a construct specifically designed around evaluating “requests for change”—the pull request. Committed code is discussed, reviewed, tested, analyzed for security issues, approved, and deployed within the context of a pull request, making them the logical place to center your audit and traceability efforts.

Tracing back from the pull request to planning

GitHub Issues are where the pre-work of development happens. They capture feature requests, bugs, requirements, and discussions about possible implementation patterns. Pull requests can be linked to one or more GitHub Issues by referencing them in the description or in a commit message.

Issues and pull requests are managed in GitHub Projects. Using GitHub Projects, teams can efficiently plan, prioritize, assign, schedule, and track their work. The flexible nature of projects enables teams to create multiple, tailored views into their activity.

Unplanned work happens. Typos, minor UI corrections, and small bug fixes may not have associated issues when they are addressed. Since pull requests can be tracked and planned in GitHub Projects and can include rich context, just like issues, you may find requiring every change to have an associated issue to be redundant. Looking back to the ISO 27001 A.12.1 requirement, a pull request includes:

• The request for change (after all, this is what a pull request is)
• Evaluation of the change (via human review and automated testing)
• Approval of the change (via human review of the change and/or deployment after required tests have passed)
• Implementation of the change (via a deployment)

For those cases when you desire a required link back to at least one issue, you can use GitHub Actions to check for, and require, linked issues. The community-maintained nearform-actions/github-action-check-linked-issues can check for issues linked via the pull request description.

Tracing within the pull request to approvals

Approvals from both human reviewers and automated systems are surfaced directly within the pull request, greatly aiding traceability.

Human review and approval

GitHub provides repository owners with the ability to enforce code reviews before a change can be merged, and use CODEOWNERS to explicitly state from which teams or individuals review is required based on the content modified. Reviews received, and whether or not the reviewer approved the change, are logged to the pull request. Repository owners can also specify that approving reviewers must not have contributed to the pull request, which can help satisfy separation of duties requirements between authors and reviewers.

Automated review

Automated testing is crucial to ensuring proposed changes will not “adversely affect the operational environment and introduce unacceptable risk” (COBIT BAI06.01) and doesn’t “adversely impact system security.” (PCI DSS 4.0 – 6.5.1)

With GitHub Actions users can run their automated test suites and require successful completion before a change can proceed toward production. The status of all automated workflow runs is reported directly in the pull request. This same approach can be applied to security testing through the use of GitHub code scanning. Repository owners can require that tests pass prior to pull requests being mergeable.

Tracing forward from the pull request to deployments

Pull requests deployed by GitHub Actions log all corresponding events directly to the pull request timeline.

Deployments can require review from specified individuals and/or teams before code is deployed into an environment. This is also logged to the associated workflow run.

Batched deployments

When a single deployment must be linked to the many pull requests constituting the entire deployment, data retrieval can be a challenge. Getting the list of commits in the deployment and linking them to their associated pull requests is possible via the GitHub API, but takes some effort and needs to be scripted into your deployment process. Thankfully, there is an easier way: GitHub releases.

GitHub releases allow you to associate a tag in your git repository with a release of your software, store and/or link off to any assets, and provide detailed release notes. GitHub can even auto-generate your release notes, including:

• All merged pull requests that are part of the release
• All committers to the release
• A complete changelog

A complete API is available to integrate the creation of a GitHub release, complete with auto-generated release notes, into your deployment process.

Data retention

One of the best things about focusing audit and traceability around the pull request is that everything has a URL. An auditor, when granted access to the repository, can follow along in the pull request, viewing every step of the process and linking out to other assets and systems when necessary.

GitHub retains your pull requests for as long as your repository is stored with us, including all timeline events. Checks data, including which checks passed and failed, are retained for 400 days. Retention of logs generated by GitHub Actions workflows can be configured for a period up to 400 days.

If you need to retain data in a separate location, or for a longer period of time, much of it can be obtained directly from your audit log, which can be streamed to a location of your choice. Additional information, if desired, can be captured from GitHub using webhooks to notify your internal system of events and the GitHub API to retrieve detailed information.

Conclusion

GitHub makes it easy to demonstrate, and document, end-to-end traceability across your software projects. Building audit and traceability around pull requests keeps compliance work within existing developer workflows, reducing friction for both development teams and auditors.

Written by

Philip Holleran

@pholleran