Guides / Computing costs and local AI

How we audited GitHub Actions cost and run time

How we audited repeated CI work, separated waiting time from runner minutes and chose simpler checks with measurable trade-offs, with a worked example.

Engineering note

By Steven Michael Barber · Published October 25, 2026 · Updated November 3, 2026 · 6 min read

Our two-model development process grew quickly: Claude and Codex built and reviewed changes, checks accumulated, and every extra safeguard had a reason. Then GitHub Actions became slower and more expensive. We asked both models to audit the process and agree on a simpler plan. The useful question was what each execution proved, and whether we were paying to prove the same thing twice.

Our October 25, 2026 audit counted 300 continuous integration (CI) runs over roughly six days. 77 were canceled. That was a reason to investigate, not evidence that a quarter of the bill could disappear. This guide explains the measurements, the first changes and the work still to measure.

This is the record of that audit and its first pilots. Our October 31 follow-up explains why the reductions did not meet the monthly target and what moving jobs to a desktop changed.

Measure waiting time and resource use separately

A workflow is a sequence or collection of jobs; a runner is the machine executing a job. Several runners can work at once. One of our full CI runs finished in about 17 minutes but consumed 67.35 combined runner minutes. The developer's wait and the resources used were different quantities.

Record three measures: elapsed time from the run's reported start to its last update; the sum of completed job durations, including failed and canceled work; and the account's usage report. The first describes waiting, the second helps locate work, and the third determines charges. Queue delays and rerun attempts need separate attention when comparing timestamps.

Split the sample into pull requests, pushes to the main branch and manual runs. A quick documentation check and a complete release check should not share one average. Compare similar change types over equal periods, and keep slow examples alongside ordinary ones.

Follow one change through the whole process

Draw the path from an author's push through review, merge, verification and deployment. Beside each execution, write the code revision tested, the checks completed and the decision it supports.

We found a successful main-branch run and a later manual run testing the same commit. They consumed 67.35 and 70.98 runner minutes. Our release verifier already accepted either trigger; the separate manual run existed to produce a verification document. We changed record generation so a qualifying main-branch run can supply that evidence, retaining a manual fallback when no eligible full run exists.

Reuse needs more than a green workflow badge. Confirm the trusted source, exact revision (the commit), required jobs and results. A run that skipped required tests cannot stand in for one that completed them. Keep production approval and the deployment result even when the evidence comes from an earlier eligible run.

Worked example with fifteen nearly empty jobs

One product deployment started a 16-job matrix, meaning sixteen jobs created from a list of products. The selected product deployed; the other fifteen checked the selection and exited. Each ran for four to seven seconds. These were executed jobs, not jobs skipped before obtaining a runner.

The raw runner time was 317 ÷ 60 = 5.28 minutes. GitHub rounds each job's duration up individually: the 243-second job becomes five minutes, and each of the fifteen short jobs becomes one. Rounding the combined 317 seconds once would give six minutes, which is a different calculation.

If we allocated only the selected job and its duration stayed unchanged, the comparison would be twenty rounded minutes versus five. That is fifteen rounded minutes avoided, while the raw work avoided is only 74 seconds. We implemented the allocation change during the audit. Its planning job must also be counted when measuring a deployment; the calculation above is not its final observed total. The account's runner rates, allowances and storage determine the money owed.

Remove repeated setup before reducing coverage

Our browser checks repeatedly installed Chromium and its operating-system libraries. In one slow job, the package manager fetched 22.4 MB in eleven minutes and twenty-nine seconds. Other jobs completed the installation much faster. That variability deserved attention alongside genuine test runtime.

A browser cache does not restore the system libraries installed on a fresh runner. Playwright's documentation makes that distinction. Its official container image includes browsers and their system dependencies; the project still installs its Playwright package. Our pilot used matching image and package versions. Check fonts, native dependencies and image-pull time when trying it in your own pipeline.

Grouping short jobs can reduce repeated setup too, but may lengthen the wait. Keep heavy suites separate, preserve each application's results, and let every selected application finish even if another fails. Update release verifiers and report collectors when job names change. A cheaper layout must still report missing or failed evidence.

Preserve work left unverified by cancellation

Running checks only for affected applications sounds straightforward until runs overlap. Suppose push A changes application one. Push B changes application two and cancels A's checks. If B examines only its own changes, application one's merged state may never finish validation.

Selection must account for unverified changes since a suitable successful validation baseline, with a full-check fallback when that baseline cannot be established. A deployment filter similarly compares against the last successful deployment, so a later documentation edit cannot hide an earlier failed site update.

This adds state to maintain. Our first phase therefore keeps full main-branch coverage while changing allocation and setup. Narrower checks come later if measurements justify their complexity. Reducing repetition and accepting less coverage are separate decisions.

Have the second reviewer challenge the assumptions

Claude proposed changes; Codex independently recounted runs and checked the code. Review caught the browser-cache mistake, the cancellation gap and an estimate that charged a reused release run twice.

For a small team, ask a colleague or second model to challenge the arithmetic, identify what would go untested, and name the evidence that would disprove the proposal. Give them source data, not just the conclusion you want approved. Record where both agree and leave unresolved differences visible.

Two reviewers can still miss the same problem. Their agreement is a reason to test a bounded change, not a measured reliability result. We retained revision-linked test evidence, authorized production approval, deployment checks and a rollback reference.

Run one experiment and compare the result

For each change, write its purpose, expected saving, preserved checks and reversal step. Start with a pilot small enough to explain in one review. Measure:

• Raw and individually rounded job minutes for comparable changes.
• The usual wait and a slow-end percentile, with the calculation defined.
• Failures, cancellations, missing reports and necessary reruns.
• Problems found after merge or deployment.

Check the account usage report against those measurements. Avoid adding savings from proposals that remove the same setup twice. Charge each execution once across the pull request, merge and release. Keep storage and model-provider usage separate from runner time.

We ran two pilots of the first implementation that day. They consumed 62.92 and 59.70 runner minutes, or 71 and 66 when each job was rounded up. Earlier full runs in our sample used 66 to 74 runner minutes; the slow-mirror example used 97.65. The thirteen smoke jobs became four. These are small samples of different revisions, not a controlled benchmark.

The first pilot failed its report check; the second passed. We counted the failed run because it still consumed resources. The unit, integration and regression job remained substantial at 16.5 and 15.9 minutes. Bounded parallel typechecking took 6.1 and 4.6 minutes, against 6.2 in the earlier example: an uneven result, rather than a reason to promise a fixed saving. Deployment cost and a comparable week's usage remain separate measurements.

A useful first audit ends with a short list: allocate only jobs needed, avoid repeated setup, reuse complete evidence, and measure again before narrowing checks. Choose one workflow, record its baseline and the proposed change in a decision log, then compare the next equivalent runs. That gives a small team a way to improve the process without turning optimization into another system to operate.