I took the continuous-integration workflow in Foundry's documentation, pointed it at a small staking vault, and ran every step twice: once with the Foundry release that stable resolves to today, and once with the release it resolved to last December.
Same commit. Same source. Same pinned compiler.
Every gas number in the committed snapshot changed — the smallest by 1.43x, the largest by 2.61x. One release said the suite had nine tests; the other said eight. And the forge snapshot --check step, which exists precisely to catch unintended gas changes, failed on every line in the file without anybody touching the contract.
Nothing in the repository had changed. The toolchain had.
That is the argument for pinning, and it is the part of a Solidity CI file that is easiest to get wrong, because the default is convenient and looks like it works. The rest of this article is the pipeline I would commit instead, with each step measured: what it gates, what it costs, what it silently ignores, and what it caught on a contract that passes its own tests.
The subject
Everything below runs against RewardVault, a 93-line staking contract that pays a fixed reward per second, split across deposited shares. It is wrong in five ways on purpose: ERC-20 transfers whose return value is discarded, state written after an external call, no zero-address check, an access-control write with no event, and reward debt accumulated instead of assigned.
Its own test suite — six unit tests, one fuzz test and two invariants — passes. That is the useful part. A CI pipeline that only asks "do the tests pass" reports this contract as healthy.
Foundry 1.8.3, released on the morning I measured, and Foundry 1.5.1 are the two releases under test; solc is pinned to 0.8.36; Slither is 0.11.6. Everything ran on an Apple M-series laptop, locally, with no network and no chain. The contract, the harness that produced every number below, and the raw results live in the repository behind this site under examples/articles/how-do-i-set-up-ci-for-a-solidity-repository/.
Start from the workflow the documentation gives you
Foundry's CI page ends with a "full workflow example":
- name: Check formatting
run: forge fmt --check
- name: Run linter
run: forge lint --deny warnings
- name: Build
run: forge build
- name: Run tests
run: forge test -vvv
- name: Check gas snapshots
run: forge snapshot --check
Those five steps are the right five steps. The job around them is where it comes apart: actions/checkout@v4, foundry-rs/foundry-toolchain@v1 with no version, and a job-level FOUNDRY_PROFILE: ci that also applies to the gas check.
Here is what those five commands did against one unchanged commit, with a .gas-snapshot recorded by 1.8.3. Every step ran even after an earlier one failed, so you can see which ones a real job would never have reached:
| Step | Foundry 1.8.3 | Foundry 1.5.1 |
|---|---|---|
forge fmt --check |
pass, 0.01 s | pass, 0.12 s |
forge lint --deny warnings |
fail, 0.09 s | fail, 0.43 s |
forge build |
pass, 0.10 s | pass, 0.81 s |
forge test -vvv |
pass, 1.70 s | pass, 1.69 s |
forge snapshot --check |
pass, 1.69 s | fail, 1.68 s |
The linter failing is the pipeline working: there really are eleven findings in that contract, and I will come back to them. The snapshot step failing on only one of the two releases is the pipeline lying to you.
Pin the toolchain, not just the compiler
foundry-rs/foundry-toolchain@v1 has two moving parts, and neither of them is obvious from reading the workflow.
The first is the action. v1 is a branch-like tag that the maintainers move; when I resolved it, it pointed at 908c540300062bd5a7e473851cdb4282204cee09, which is release v1.9.1 from 28 July 2026. Tomorrow it can point somewhere else. GitHub's own guidance is blunt about this: "Pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release." (GitHub Docs, secure use reference)
The second is Foundry itself. The action's version input defaults to stable, which is whatever was released most recently. On the day I measured, that was v1.8.3, published a few hours earlier. A repository whose last CI run was in December 2025 got 1.5.1 from the same unchanged workflow file.
Here is what that difference does to the gas numbers of the same six unit tests:
The cause is a default that flipped. Foundry's isolate setting executes "each top-level external call made by a test ... as a separate transaction in a separate EVM context", so cold-access costs are paid the way they would be on chain. It defaulted to false on 1.5.1 and is documented as true on 1.8.x. Running 1.5.1 with --isolate reproduces 1.8.3's number exactly on six of six tests — not approximately, to the gas unit.
That is a good change. The new numbers are the more honest ones. It is still a change that arrives in your repository without a commit, on whatever morning it happens to ship.
The same drift shows up in places that have nothing to do with gas.
The test count moved. forge test --list discovers nine tests on both releases. But 1.5.1 runs each invariant_ function as its own campaign and prints 9 tests passed; 1.8.3 runs both invariants inside one campaign and prints 8 tests passed. If anything downstream — a coverage gate, a badge, a script grepping the summary line — asserts on that number, it breaks on upgrade with no code change.
A lint suppression written for the new linter fails the old one. The fixed contract uses Foundry's inline suppression for a block.timestamp comparison that is intentional:
// forge-lint: disable-next-line(block-timestamp)
if (block.timestamp <= lastAccrual) {
return;
}
1.8.3 honours it. 1.5.1 has no block-timestamp lint, so it reports warning: unknown id: 'block-timestamp' — and under --deny warnings, the suppression comment is itself the thing that fails the build.
So pin both, and pin the actions by SHA:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
submodules: recursive
- name: Install Foundry
uses: foundry-rs/foundry-toolchain@908c540300062bd5a7e473851cdb4282204cee09 # v1.9.1
with:
version: v1.8.3
While you are there: Foundry's documented example still uses actions/checkout@v4. The current release is v7.0.1.
What the test step is actually gating
Three things are worth knowing before you trust forge test as a merge gate.
A pinned fuzz seed does not pin the counterexample any more. The fixture's foundry.toml sets fuzz = { runs = 256, seed = "0x1a7e" }. I ran a deliberately failing fuzz test twenty times on each release, clearing Forge's persisted failure cache before every run.
1.5.1 reported the same counterexample twenty times out of twenty. 1.8.3 reported three different ones, split 8 / 8 / 4. Foundry's 1.7.0 release notes list "parallelized stateless fuzzing across worker threads with shared corpus", and the input that gets reported is now whichever worker got there first — which depends on how loaded the machine is. With no seed at all, both releases produced a fresh counterexample nearly every run: 20 distinct values out of 20 on 1.5.1, 18 out of 20 on 1.8.3.
The practical consequence is not that fuzzing is broken. It is that "re-run the job and post the failing input" is no longer a reproduction step on a current Foundry. Copy the counterexample out of the failed run and write it into a regression test while you still have it.
The default invariant configuration hides reverting calls. invariant.fail_on_revert defaults to false, and a passing campaign prints the reverts it swallowed right underneath the green result:
[PASS] invariant_BalanceBacksShares
[PASS] invariant_TotalSharesEqualsSumOfHolders
RewardVaultInvariantTest invariants (runs: 64, calls: 4096, reverts: 281)
╭--------------+----------+-------+---------+----------╮
| Contract | Selector | Calls | Reverts | Discards |
+======================================================+
| VaultHandler | deposit | 1385 | 0 | 0 |
|--------------+----------+-------+---------+----------|
| VaultHandler | passTime | 1378 | 0 | 0 |
|--------------+----------+-------+---------+----------|
| VaultHandler | withdraw | 1333 | 281 | 0 |
╰--------------+----------+-------+---------+----------╯
281 of 4,096 calls reverted and the suite passed. Read the table rather than the total: every one of those reverts is a withdraw, and withdraw reverted on 21.1% of its calls. That is not the fuzzer wandering out of bounds — the handler bounds every withdrawal by the shares the actor actually holds. It is an arithmetic underflow in the vault's reward accounting.
Setting fail_on_revert = true in the CI profile found it in three calls:
[FAIL: panic: arithmetic underflow or overflow (0x11)]
[Sequence] (original: 9, shrunk: 3)
… calldata=deposit(uint256,uint256) args=[4348, 1000000000000000000 [1e18]]
… calldata=passTime(uint256) args=[1]
… calldata=withdraw(uint256,uint256) args=[598, 115792089237316195423570985008687907853269984665640564039457584007913129639935 [1.157e77]]
(The sender= and addr= fields on each line are elided.) Deposit, wait one second, withdraw. That is the whole bug. It sat behind a passing suite the entire time, and the only thing between it and the repository was one configuration key.
fail_on_revert = true is not free — a handler that is not carefully bounded will fail on its own bad inputs rather than on a contract defect, which is why the default is what it is. But it belongs in the CI profile, where the run is allowed to take longer than a local one.
The heavier profile costs about what you would guess. With fuzz.runs at 10,000 and invariant.runs at 256 by depth 128, the test step took 1.70 s against 0.28 s under the default profile. Six times the wall clock for thirty-nine times the fuzz budget. On a real protocol the ratio is worse, which is the argument for running the heavy profile on pull requests and the light one on pushes.
Static analysis: two tools, and what each does to your exit code
Foundry ships a linter. It is fast, it needs no installation, and on 1.8.3 it does not even compile the project: deleting out/ and running forge lint leaves out/ absent. That is the 0.09 s step in the table above.
On the original vault it reported 11 findings across 6 rules: erc20-unchecked-transfer, reentrancy-no-eth, reentrancy-events, block-timestamp, missing-zero-check, and missing-events-access-control.
Slither 0.11.6, run over the same source, reported 12 findings from 102 detectors on 2 contracts: 2 high impact, 2 medium, 8 low.
The overlap is larger than I expected. Five of the six forge lint rules have a Slither equivalent that also fired, and every Slither check that fired has a forge lint counterpart:
| Finding | forge lint 1.8.3 |
Slither 0.11.6 |
|---|---|---|
| ERC-20 transfer return value discarded | erc20-unchecked-transfer |
unchecked-transfer |
| State written after an external call | reentrancy-no-eth |
reentrancy-no-eth, reentrancy-benign |
| Event emitted after an external call | reentrancy-events |
reentrancy-events |
block.timestamp in a comparison |
block-timestamp |
timestamp |
| Address stored with no zero check | missing-zero-check |
missing-zero-check |
| Access-control write with no event | missing-events-access-control |
— |
On this contract, Slither found nothing forge lint had missed, and forge lint found one thing Slither did not. That is not an argument against Slither — its value is in the ninety-odd detectors this small contract never triggers, and in the cross-function reentrancy analysis it prints alongside each finding. It is an argument for putting forge lint first, because it is already installed and it costs 0.09 s.
Two things will bite you in a pipeline.
Slither's exit code is not what you would guess. With 12 findings it exited 255 — and it exited 255 at every --fail-* level except --fail-none, which returned 0. That includes --fail-high, because two of the findings are high impact. The default behaviour is --fail-pedantic: any finding at all fails the job, including informational ones. Decide the level deliberately. The crytic/slither-action exposes it as fail-on, and that action has no floating major tag at all — the latest release is v0.4.2 from January 2026 — so you are pinning a SHA whether you want to or not.
forge lint --json changed streams between releases. On 1.8.3 the diagnostics go to stdout. On 1.5.1 they go to stderr, and a pipeline that reads stdout gets an empty result and a zero exit code. If you parse lint output, assert that you found something.
One setting is worth knowing about before you turn the linter into a gate:
[lint]
ignore = ["test/**"]
The documented step, forge lint --deny warnings with no path, lints your test helpers too. On this project that cost nothing — 1.8.3 put all eleven warnings in src/ and none in test/ — but 1.5.1 did raise a naming note against a test file, and a mock written to be deliberately minimal is the kind of file that earns findings which say nothing about the contract you are shipping.
After fixing all six rules — checks-effects-interactions ordering, checked transfers, the zero-address check, an event on the operator write, and reward debt assigned rather than accumulated — the linter reports zero warnings and the whole pipeline runs green, including fail_on_revert = true.
Gas changes: what forge snapshot --check really compares
The documentation says forge snapshot --check "exits with an error if any test uses more gas than recorded". It is stricter and narrower than that, and it is worth knowing exactly how.
I took a snapshot each release had produced itself, changed exactly one number in a copy, and re-ran the check. Both releases behaved identically:
| Field changed by 1 | --check result |
|---|---|
| A unit test's gas | fails |
A fuzz test's median (~) |
fails |
A fuzz test's mean (μ) |
passes |
| A fuzz test's run count | passes |
| An invariant campaign's revert count | passes |
Two consequences. Gas that goes down fails the check too — a 1-gas reduction produced Diff in "RewardVaultTest::test_AccrualSplitsRewardAcrossShares()": consumed "(gas: 226112)" gas, expected "(gas: 226113)" gas. That is the correct behaviour for a gate whose job is to make gas changes explicit, but "more gas than recorded" undersells it.
And the invariant lines in the file are decoration. They are written on every forge snapshot, they are never compared, and they do not hold still: across five identical runs on 1.8.3, seven of the nine lines were byte-identical and each invariant line took four different values. Commit that file as-is and every unrelated pull request carries a diff in it.
So record and check the snapshot without them, and pin the profile for that one step so the fuzz budget is the same on both sides:
- name: Check gas snapshots
run: forge snapshot --check --no-match-contract Invariant
env:
FOUNDRY_PROFILE: default
Reproducible artifacts
The last thing a Solidity CI job should do is say what it built, in a form somebody can compare.
I compiled the same single source file ten ways — two Foundry releases, two directory depths, four individual settings changed one at a time, and one run with an empty HOME — and hashed the deployed bytecode of each. One source file produced six distinct artifacts:
| Variant | solc | EVM version | Deployed | sha256 |
|---|---|---|---|---|
| Pinned, Foundry 1.5.1 | 0.8.36 | prague | 1,945 B | 560cd1e8911f… |
| Pinned, Foundry 1.8.3 | 0.8.36 | prague | 1,945 B | 560cd1e8911f… |
| Pinned, 1.8.3, different directory | 0.8.36 | prague | 1,945 B | 560cd1e8911f… |
| Nothing pinned, 1.5.1 | 0.8.36 | prague | 1,999 B | 22cbd2f0b462… |
| Nothing pinned, 1.8.3 | 0.8.36 | osaka | 1,999 B | 1e53e8e30d05… |
| Nothing pinned, 1.8.3, empty compiler cache | 0.8.37 | osaka | 1,999 B | c46353513b97… |
solc_version = "0.8.25" |
0.8.25 | cancun | 1,945 B | c3888f2cedad… |
optimizer_runs = 1000 |
0.8.36 | prague | 2,084 B | 56d0ea3a69c7… |
Pinned but evm_version = "osaka" |
0.8.36 | osaka | 1,945 B | 560cd1e8911f… |
| Pinned but metadata left on | 0.8.36 | prague | 1,999 B | 22cbd2f0b462… |
The pinned configuration is the boring row, and boring is the goal: identical bytes across both releases and across two different checkout paths.
The last two rows explain the interesting failure. Changing the EVM version alone, with metadata off, produced identical bytecode — this contract emits no opcode that Osaka changed. Leaving metadata on, with everything else pinned to prague, reproduced the unpinned 1.5.1 artifact exactly.
Put those together and the two unpinned releases make sense: they produced the same 1,956 leading bytes and differed only in the last 43. That is the CBOR metadata blob. Foundry's default bytecode_hash = "ipfs" appends a digest of the metadata JSON, the metadata records the EVM version, and the EVM version default moved from prague to osaka between the releases. The code is identical; the fingerprint is not. Set bytecode_hash = "none" and cbor_metadata = false if you want two machines to agree on a hash.
The last row is the one that would have surprised me. With auto_detect_solc = true, Foundry resolves the pragma against the compilers the machine already has. On my laptop that meant 0.8.36. With HOME pointed at an empty directory — which is what a fresh CI runner looks like — the same command downloaded and used 0.8.37, released five days earlier. Your laptop and your runner compile different bytecode from the same commit, and nothing in the repository records which.
So the compiler settings go in foundry.toml, not in a pragma range:
[profile.default]
solc_version = "0.8.36"
auto_detect_solc = false
optimizer = true
optimizer_runs = 200
evm_version = "prague"
bytecode_hash = "none"
cbor_metadata = false
And the job prints what it built:
560cd1e8911f3fcfea8106d8ae6e844ebbd72694ce3fc7227770be219d61c333 1945 RewardVault.sol:RewardVault solc 0.8.36+commit.8a079791 prague
That line comes out of a 40-line script over out/, uploaded as a build artifact. It is the cheapest possible supply-chain control: when somebody asks whether the address on chain matches the audited commit, the answer is a file, not an afternoon.
Cache the thing that actually costs time
The toolchain action's cache input is on by default, and Foundry's CI page presents it as "speed up CI by caching dependencies". It caches exactly one directory: ~/.foundry/cache, which holds fork RPC and Etherscan responses. The action's README says so directly, and its source confirms it — the cache key is even prefixed foundry-chain-fork-.
If your suite has no fork tests, that setting does nothing for you. Here is where the time actually goes on this project, median of three runs each:
| Runner state | forge build |
|---|---|
| No compiler, no build output | 6.85 s |
| Compiler present, no build output | 0.70 s |
| Everything warm | 0.10 s |
Downloading solc is 90% of a cold build on a project this small. It is a fixed cost, so on a large repository it matters proportionally less — but it is also trivial to remove, and unlike out/, a compiler cache cannot go stale in a way that produces a wrong answer:
- name: Cache the Solidity compiler
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.svm
~/.local/share/svm
key: solc-${{ runner.os }}-${{ hashFiles('foundry.toml') }}
Both paths, because svm-rs uses ~/.svm when it already exists and the platform data directory otherwise — ~/.local/share/svm on Linux, ~/Library/Application Support/svm on macOS. Keying on foundry.toml means the cache is invalidated by exactly the file that decides which compiler you need.
I would not cache out/. It was 1.7 MB here and saved 0.6 s, and a stale artifact directory is a class of CI failure that is genuinely unpleasant to debug.
The workflow I would commit
name: CI
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
env:
FOUNDRY_PROFILE: ci
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
submodules: recursive
- name: Install Foundry
uses: foundry-rs/foundry-toolchain@908c540300062bd5a7e473851cdb4282204cee09 # v1.9.1
with:
version: v1.8.3
cache: false
- name: Cache the Solidity compiler
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.svm
~/.local/share/svm
key: solc-${{ runner.os }}-${{ hashFiles('foundry.toml') }}
- name: Check formatting
run: forge fmt --check
- name: Lint
run: forge lint --deny warnings
- name: Build
run: forge build
- name: Test
run: forge test
- name: Check gas snapshots
run: forge snapshot --check --no-match-contract Invariant
env:
FOUNDRY_PROFILE: default
- name: Fingerprint the build
run: node scripts/fingerprint.mjs
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: bytecode-fingerprint
path: results/fingerprint.json
analyze:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
submodules: recursive
- name: Slither
uses: crytic/slither-action@b52cc1cbfee9ca3e8722dd5224299d16c9a6b80f # v0.4.2
with:
fail-on: medium
Every SHA in that file was resolved on 15 September 2026 by a script, not typed. Re-resolve them when you adopt it, and let Dependabot move them afterwards.
The matching [profile.ci]:
[profile.ci]
verbosity = 3
fuzz = { runs = 10000, seed = "0x1a7e" }
invariant = { runs = 256, depth = 128, fail_on_revert = true }
What this actually buys
Pinning is the part with the clearest return. It costs a version: line and a handful of SHAs and it is the difference between a gas gate that means something and one that goes red on a Tuesday for reasons nobody in the repository can explain. Pin the Foundry release, pin the compiler, pin the actions by SHA, and let a bot propose the upgrades as pull requests you can read.
Gating is where judgment comes in, and the measurements point in a consistent direction: gate on the things that are deterministic and cheap. Formatting, linting, the build, unit gas. Record the things that are not — fuzz counterexamples, invariant revert counts, timings — but do not let them fail a merge, because you will train the team to re-run the job until it is green, and then the gate is worse than nothing.
The last piece is the one that is easy to skip because nothing fails without it. A build fingerprint uploaded on every commit is thirty lines of script and it answers, months later, the only question that really matters about a deployment: is this the code we reviewed?
Two of the findings above — the fuzz counterexample that will not reproduce, and the invariant campaign passing with 281 swallowed reverts — are about the limits of what an automated suite can promise. If you want the fuller version of that argument, my benchmark of the same Solidity tests under Foundry and Hardhat 3 is where I worked through how much of a measured difference is the tool and how much is the harness around it. And if your repository is a Hardhat 3 project rather than a Foundry one, most of this transfers — the pinning, the fingerprint, the caching — but the commands and the gas gate do not.