code-scanning

CodeQL now officially supports customizing the build configuration for Go analysis in the Actions workflow file. This aligns the Go configuration experience with the C/C++, C#, and Java analysis. The new customization options allow for more flexibility, for example when the build fails, or if analysis is desired on different source files.

All your existing CodeQL workflows for Go analysis will continue to work and continue to be supported. You don’t need to take any action to keep Go analysis running.

Example Actions workflow steps using Go build customization

steps:
  - name: Checkout repository
    uses: actions/checkout@v3

  - name: Initialize CodeQL
    uses: github/codeql-action/init@v2
    with:
      languages: go

  - name: Build code
    run:
      # You can modify these commands or add new commands to customize the build process
      make bootstrap
      make release

  - name: Perform CodeQL Analysis
    uses: github/codeql-action/analyze@v2

Learn more about CodeQL and code scanning.

We have started creating and storing CodeQL databases for the most popular open-source projects on GitHub.com. If you use CodeQL for security research, you can now obtain these databases easily and directly through the CodeQL extension for Visual Studio Code, which makes it much easier to write and run your own custom CodeQL queries.

Using CodeQL for security research

The CodeQL engine powers GitHub code scanning: it analyses source code and flags up potential security problems (for example, in pull requests). By default, code scanning runs a large set of open source queries that are able to identify the most important and common security problems.

CodeQL is also a powerful tool for variant analysis and other types of security research. CodeQL treats source code as data, and anyone can write custom CodeQL queries to explore a codebase and identify vulnerabilities. Like code search on steroids!

The first step of any CodeQL analysis is extracting the source code into a CodeQL database. This database contains a relational representation of the source code — including elements like the abstract syntax tree, the data flow graph, and the control flow graph. You can create CodeQL databases yourself using the CodeQL CLI, but with the feature we shipped today, it's much quicker to get started: you can download a ready-built CodeQL database from GitHub.com.

Downloading CodeQL databases from GitHub.com in VS Code

To download a CodeQL database for use in the CodeQL extension in VS Code:

1. Make sure you have set up the CodeQL extension for VS Code. For more information, see Setting up CodeQL in Visual Studio Code.
2. Open the CodeQL databases view in the extension.
3. Hover over the sidebar, click the GitHub icon, and specify the owner/repo identifier of the public repository you'd like to analyze.

   

Once you've downloaded a CodeQL database, you're ready to start your research. Find more information in the CodeQL documentation.

FAQs

How many CodeQL databases are available?

We currently store databases for over 200,000 repositories on GitHub.com. That list is constantly growing and evolving to make sure that it includes the most interesting codebases for security research.

What languages are can you download CodeQL databases for?

We create and store databases for all of the languages that we support in CodeQL code scanning. For more information, see About code scanning with CodeQL.

Can I download CodeQL databases outside VS Code?

Yes, you can also download CodeQL databases using the GitHub REST API. For more information, see Downloading databases from GitHub.com in the CodeQL CLI documentation.

Why is there no CodeQL codebase available for my favourite open source repository?

If there is a repository that you'd like to analyze, but a CodeQL database is not available yet, then you can trigger the creation (and storing) of a database by enabling GitHub code scanning with the CodeQL engine. Alternatively, you could fork the repository and enable code scanning on the fork. For more information, see the code scanning documentation.

It's now easier to debug CodeQL analysis problems in code scanning: click Re-run jobs from the GitHub Actions workflow run page, check the Enable debug logging box, and hit the Re-run jobs button.

The data will be uploaded as an Actions artifact named debug-artifacts, attached to the workflow run. Such artifacts contain CodeQL logs, CodeQL databases, and the SARIF files that were produced.

These artifacts will help you when you're debugging problems with CodeQL code scanning. When contacting GitHub support, you might be asked for this data.

As part of the analysis, CodeQL extracts your source code into a relational database format. The debug artifacts include more detailed information about CodeQL extraction errors and warnings that occurred during database creation. If you want to permanently enable debug logging for the CodeQL analysis, or would like more information about troubleshooting CodeQL, please follow these instructions.

This feature is now available to all users on GitHub.com and will also be available in GitHub Enterprise Server 3.7.

On March 30, 2022, we released CodeQL Action v2, which runs on the Node.js 16 runtime. The CodeQL Action v1 will be deprecated at the same time as GHES 3.3, which is currently scheduled for January 2023.

How does this affect me?

Users of GitHub.com, GitHub AE, and GitHub Enterprise Server 3.5 (and later)

All users of GitHub code scanning (which by default uses the CodeQL analysis engine) on GitHub Actions on the following platforms should update their workflow files:

• GitHub.com (including open source repositories, users of GitHub Teams and GitHub Enterprise Cloud)
• GitHub AE
• GitHub Enterprise Server (GHES) 3.5 and later

Users of the above-mentioned platforms should update their CodeQL workflow file(s) to refer to the new v2 version of the CodeQL Action.

Users of GitHub Enterprise Server 3.4 (and older)

Users of GitHub Enterprise Server 3.4 (and older) are not recommended to update their configuration to use the v2 version of the CodeQL Action:

• GHES 3.3 (and older) does not support running Actions using the Node 16 runtime and is therefore unable to run the v2 version of the CodeQL Action. Please upgrade to a newer version of GitHub Enterprise Server prior to changing your CodeQL Action workflow files.
• While GHES 3.4 does support Node 16 Actions, it does not ship with v2 of the CodeQL Action. Users who want to migrate to v2 on GHES 3.4 should request that their system administrator enables GitHub Connect to download v2 onto GHES before updating their workflow files.

The upcoming release of GitHub Enterprise Server 3.5 will ship with v2 of the CodeQL Action included.

Exactly what do I need to change?

To upgrade to the CodeQL Action v2, open your CodeQL workflow file(s) in the .github directory of your repository and look for references to:

• github/codeql-action/init@v1
• github/codeql-action/autobuild@v1
• github/codeql-action/analyze@v1
• github/codeql-action/upload-sarif@v1

These entries need to be replaced with their v2 equivalents:

• github/codeql-action/init@v2
• github/codeql-action/autobuild@v2
• github/codeql-action/analyze@v2
• github/codeql-action/upload-sarif@v2

Can I use Dependabot to help me with this upgrade?

Yes, you can! For more details on how to configure Dependabot to automatically upgrade your Actions dependencies, please see this page.

What happens in January 2023?

In January 2023, the CodeQL Action v1 will be officially deprecated (at the same time as the GHES 3.3 deprecation). At that point, no new updates will be made to v1, which means that new CodeQL analysis capabilities will only be available to users of v2. We will keep a close eye on the migration progress across GitHub. If many workflow files still refer to v1 of the CodeQL Action we might consider scheduling one or more brownout moments later in the year to increase awareness.

GitHub code scanning helps open source maintainers and organizations find potential vulnerabilities in their code, before these can make their way into deployments. CodeQL, our very own analysis engine, powers the majority of those checks. Over the past few months, we have been working hard to improve the depth and breadth of our analysis to cover more CWEs, add support for a host of new language versions, and improve our platform compatibility.

Before we dive into the details: If you haven’t tried GitHub code scanning with CodeQL yet, you can enable it now on your repositories by following this guide! It’s free for open-source projects and available as part of GitHub Advanced Security for our enterprise customers.

All improvements below are available to users of GitHub code scanning on GitHub.com today, and will be part of the next GitHub Enterprise Server release (GHES version 3.5). Users of other GHES versions can also update their CodeQL version to benefit from these analysis improvements straight away.

Language Support

Today, CodeQL already supports JavaScript/TypeScript, Python, Ruby, Java, C#, Go, and C/C++. These languages are themselves under constant development, and we now support the following language versions:

• C# 10 / .NET 6,
• Python 3.10,
• Java 17, and
• TypeScript 4.5

The standard language features in those language releases are now fully supported by CodeQL.

Performance and Compatibility

For our Linux users, we have fixed an issue that caused the CodeQL CLI to be incompatible with systems running glibc version 2.34 and older.

For users of the CodeQL Apple Silicon support (beta), we are now bundling a native Java runtime for improved performance. Rosetta 2 and macOS Developer Tools are still required for other CodeQL components.

Security Coverage

The Common Weakness Enumeration (CWE) system is an industry-standard way of cataloging insecure software development patterns. CodeQL runs hundreds of queries out of the box that are able to detect an even greater number of CWEs. We went back through our existing queries, and aligned dozens of them with updated CWE IDs to give users better insight into the potential impact of a security issue when an alert is flagged up by code scanning.

We’ve added and improved detection for a large number of CWEs. These are the most significant changes:

• CWE-190 – Integer Overflow: The cpp/uncontrolled-arithmetic query for C/C++ detects potential user-controlled inputs to calculations that could produce an overflow condition
• CWE-319 – Cleartext Transmission of Sensitive Data: The cpp/cleartext-transmission query for C/C++ detects network transmissions of sensitive data without encryption
• CWE-120 – Buffer Overflow: The cpp/very-likely-overrunning-write query for C/C++ now detects cases of out-of-bounds writes based on advanced range analysis
• CWE-732 – Incorrect Permission Assignment for Critical Resource: The cpp/open-call-with-mode-argument (and optional cpp/world-writable-file-creation) query for C/C++ detect issues that could lead to stack memory disclosure or attacker-writable files
• CWE-295 – Improper Certificate Validation: The java/insecure-trustmanager query for Java now detects missing or lax certificate handling that could lead to man-in-the-middle attacks
• CWE-829 – Inclusion of Functionality from Untrusted Control Sphere: The js/insecure-dependency query for JavaScript/TypeScript detects dependency downloads over unencrypted communication channels
• CWE-347 – Improper Verification of Cryptographic Signature: The js/jwt-missing-verification query for JavaScript/TypeScript detects scenarios in which a JWT payload is not verified with a cryptographic secret or public key
• CWE-918 – Server-Side Request Forgery: SSRF detection queries for Python have been improved, and now differentiate between partially and fully (py/full-ssrf) user-controlled URLs

Behind the scenes, we’re also working on support for mobile application security, with additional support for Kotlin and Swift on our roadmap. In the meantime, we’ve also added more coverage for mobile security issues for our existing Java support:

• CWE-266 – Incorrect Privilege Assignment: The java/android/intent-uri-permission-manipulation query detects Intent URI permission manipulation in Android applications
• CWE-470 – Use of Externally-Controlled Input to Select Classes or Code: The java/android/fragment-injection and java/android/fragment-injection-preference-activity queries detect issues that could enable malicious applications to bypass access controls
• CWE-927 – Use of Implicit Intent for Sensitive Communication: The java/android/implicit-pendingintents query detects potential information disclosure through the use of PendingIntents