Dev ToolingDev Tooling
What Code Quality Tools Actually Do
Dev Tooling

What Code Quality Tools Actually Do

Lewis SmithArticle by Lewis Smith

Write the rule configuration into a file, pin the analyser to one exact version, and commit both next to the source they judge, because everything downstream depends on that file being identical on every machine. Without it, two clones produce two finding sets, the dashboard ranks noise as debt, and no reviewer can say whether a fix closed a defect or moved it.

That file is step one of eight. The rest of this page runs the work in the order it happens, and each step ends with a result you can check before starting the next.

A service delivered to a customer normally includes three analysers: a formatter, a language linter, and a static analyser carrying a compliance rule pack. They perform four jobs and no others. They parse source into a syntax tree, match that tree or the raw text against patterns, compute aggregates across the whole repository, and order what they found. A vendor sells one of those jobs behind a different interface. A team that cannot name which job a tool performs will buy the same job twice and then argue about the duplicate findings.

1. Pin the analyser and commit the rule file

Select the rule set before the tool, because the set determines which findings exist. BrowserStack's guide to coding standards sets out the governance split: MISRA C and C++ came out of the automotive industry, is closed and paid for, is revised by working groups to a fixed blueprint, and enforces uniformity across vendors; CERT is free, web-based, and takes public comment into its revisions.

Maintainer, access model and typical use for each rule set named on this page, from the BrowserStack coding standards guide and Parasoft's static code analysis page:

Rule set Maintainer Access model Where it fits
MISRA C and C++ Motor Industry Software Reliability Association working groups Paid and closed, revised on a fixed blueprint Automotive and embedded safety code
CERT C Community wiki with public comment Free Security across industries
AUTOSAR C++14 Automotive working group Licensed with the standard C++ in vehicles
OWASP and CWE Open community projects Free Application security
PCI DSS Payment card industry body Published and audited Cardholder data systems
DISA ASD STIG United States Defense Information Systems Agency Published Government and defence systems

Parasoft's scanners map findings to OWASP, CWE, CERT, PCI DSS and DISA ASD STIG rule identifiers, so the mapping between a rule set and a rule identifier is a property of the tool, not of the standard.

Write the choice into the repository, pin the version, and print it during the build.

# .quality/rules.conf
analyser      = static
version       = 1.14.2
ruleset       = cert-c
severity_min  = major
baseline      = .quality/baseline.json

Check: two clean clones built on two machines produce the same finding set, and the version string appears in the build log.

2. Run one full pass and freeze the output as a baseline

Run the analyser across the whole tree, export the findings, and store the export as an artefact. The sentence to be rewritten is required. The baseline is the only device that lets a later gate separate a new finding from an old one; without it, the first pull request after you enable the gate fails on a violation that arrived two years earlier. The xcubelabs review of static analysis practice reports that these tools detect up to 70 per cent of potential defects before the code runs, which justifies the wait, but that figure describes the tree as a whole and provides no information about the file you are editing.

Check: two consecutive runs on unchanged source produce the same finding count per rule.

3. Tag every rule with the job it performs

Open the rule file and write one of four labels against each rule: parsing, pattern matching, aggregation, ordering. Write the team that owns the fix next to it. A rule with a job and no owner produces a finding that remains open, and a rule with an owner and no job cannot be defended in review.

The job What it reads What it writes What it cannot decide
Parsing Source text against the language grammar A syntax tree per file Anything about intent
Pattern matching The tree, or the raw text Findings with a rule identifier and a location Whether the pattern matters in this file
Aggregation Every tree in the repository Complexity, duplication and coupling figures per module Which module you are about to change
Ordering Findings plus history A ranked list Whether the ranking matches your release

Check: Every rule identifier in the file has one job label and one owning team, and no rule is left untagged.

4. Join the findings to change history before ranking anything

Extract per-file change counts from version control, then intersect that signal with the findings. A module that both contains a major finding and changes every week is a choice; a module that contains the same finding and has not changed in a year is a note.

git log --since=12.months --name-only --pretty=format: \
  | sort | uniq -c | sort -rn > churn.txt

CodeScene's code health gallery shows the Apache Zookeeper codebase carrying modules in its worst code health band while the hotspot view shows most of them stable, which makes them a low priority. The same evaluation of code quality at scale calculates Kubernetes at 4 million lines of code and finds test code frequently in worse health than application code, with no difference in maintenance cost between the two.

Check: Your prioritised list contains files that have both a finding and recent change activity, and it is shorter than the raw finding list.

5. Put the gate on the merge, not in the editor

The editor plugin is a convenience and never the enforcement point. Parasoft's Jtest triggers a live static analysis scan whenever a source file is opened or saved, and the same product publishes continuous integration results to its dashboard for triage. Those are two different jobs: The editor scan informs the author and the pipeline gate determines the branch. Configure the gate to fail on findings the baseline does not contain, and to pass a file that still has its old violations. Graphite's code quality guidelines recommend tracking the average number of review cycles until a merge, which is the closest available measure of whether the guidelines are being followed at all.

Check: a commit that introduces a violation fails the pipeline, and a commit that edits a file with existing violations passes.

6. Triage with history, and expiry on every suppression

Record each triage decision with a date, a reason and a name. Parasoft's dashboard applies machine learning to past triage actions and predicts which violations to prioritise for remediation and which to suppress, and its build-to-build analysis groups findings by root cause, both of which fail when the triage record is empty. The same discipline appears in plainer form in the best-practice list from xcubelabs: customise the rules to the standards the project actually answers to, order the results by severity, and fix the high-risk items first.

Check: Every suppressed finding has an expiry date and a named owner.

7. Fix only what a test can verify

For each finding you accept, identify a test that fails on the current commit and passes after the change. Graphite's guidelines tell teams to refactor as part of regular development rather than let technical debt accumulate, and AWS's explanation of code quality says that code must be tested before and after refactoring so that no defects are introduced, and mentions testing frameworks, performance testing and security verification along with static analysis because they answer different questions about the same commit. That fixes the order: the failing test exists first, the change is second, and the passing test is what closes the finding. A change with no test is a refactor you cannot verify, and it belongs in its own commit with its own review.

Check: the diff that closes a finding contains a test that failed before it and passed after it, and the gate reports no new findings on that commit.

8. Read the numbers at ninety days

Run the pipeline for a quarter before changing the rule set again. The attributes worth reading are the ones AWS groups under code quality: reliable, extendible, testable, portable and reusable. Attribute names come from AWS's explanation of code quality; the thresholds below are the ones this procedure sets, and one decision is recorded per row.

Attribute What it reads Threshold Action when crossed
Reliable Unhandled errors and failed runs recorded against a release tag More than one unhandled error in a release Hold the next tag until the finding is closed or suppressed with an expiry
Extendible Findings per module joined to the change count from step four A top-quartile churn module carrying a major finding for two consecutive months Refactor it in the next iteration, or suppress it with a named owner
Testable Findings located in test code as a share of all findings Test code in worse health than the application code it covers Fix the test code first, because it decides whether every later fix is verified
Portable Gate results that differ between a developer machine and the pipeline Two mismatches in a month Move the rule into the pipeline configuration and re-baseline
Reusable Duplicated blocks above the limit the team agreed One block above the limit, found in two modules Extract it behind an interface and delete the copies

Check: the ninety-day review produces one recorded decision per attribute, either the threshold was crossed and an action was written down, or it was not and the attribute keeps its current setting. A review that produces a list and no decision has not run.

The file from step one is what makes the other seven steps repeatable. Every later question about a finding reduces to which rule produced it, at which version, against which baseline, and that question is answerable only if the file is in the repository next to the code it judged.

Sources

The figures behind this

Potential defect detection
70% xcubelabs
Kubernetes codebase size
4 million lines
Typical analyser count
3 tools

Related guides

Static Code Analysis Tools: Where to Start
Static Code Analysis Tools: Where to Start
How to Evaluate Code Review Platforms
How to Evaluate Code Review Platforms
Debugging Tools For Developers: What to Look For
Debugging Tools For Developers: What to Look For

← All Guides