Formal Verification of Warp-Specialized GPU Kernels.
SentryWard is a formal verifier for warp-specialized GPU kernels. Given a kernel’s PTX and a launch configuration, it proves correct synchronization and freedom from data races in every execution schedule within the configured input scope and hardware model. With a PyTorch reference, it also proves that the kernel computes the same function in real arithmetic.
From manual validation to an automated loop
Coding agents can automatically generate and optimize high-performance GPU programs at low cost. As generation becomes cheaper, the bottleneck shifts from programming kernels to verifying their correctness. SentryWard automates that verification step, giving agents proofs or diagnostics to guide the next revision.
BeforeManual validation
Each revision waits for a person- Coding agentGenerate or optimize
- Candidate kernelCompile to PTX
- Human validationInspect · test · debug
- Tested candidateBenchmark performance
- Time-consumingA person must review each revision.
- Error-proneRare interleavings are easy to miss.
- Hard to scaleMore candidates need more review.
AfterSentryWard
Agents generate, verify, and revise- Coding agentGenerate or optimize
- Candidate kernelCompile to PTX
- SentryWardProve across schedules
- Verified candidateBenchmark performance
- Fast verification10.3 seconds median to complete verification
- Strong guaranteesProofs cover every execution schedule.
- Fully automatedAgents call the CLI or Python API.
What goes in. What comes out.
Input
- PTX kernel
- Your compiled NVIDIA GPU kernel.
- Input configuration
- A JSON file describing input shapes, launch dimensions, kernel parameters, and any TMA descriptors.
- Optional PyTorch reference
- A
reference(...)function and a binding that maps its arguments and outputs to the kernel. No separate reference input file is needed.
Output
- A clear verdict
race_freemeans synchronization and memory checks passed without a reference.verifiedmeans reference equivalence was proved too.- Check results and witnesses
- JSON from the CLI, or a Python dictionary, with each check’s status and evidence for findings such as
potential_race. - Explicit limits
unsupportedortimeoutexplains why analysis stopped. An undecided check is never reported as a pass.
What SentryWard verifies
- Synchronization & deadlocks
- Barriers and mbarriers, including TMA byte counts, behave consistently across schedules, ruling out deadlocks.
- Data races
- Conflicting accesses to global, shared, and tensor memory are ordered, including asynchronous copies and tensor-core operations.
- Resource lifetimes
- Tensor memory and barriers are never used after they are freed or invalidated.
- Mathematical correctness
- The kernel’s outputs equal those of your PyTorch reference in real arithmetic.
- Unnecessary waits
- Use verification to remove redundant synchronization: remove an operation, then verify again. Safe removals depend on the input shape.
Covers Blackwell TMA, tcgen05 MMA and tensor memory, mbarriers, thread-block clusters, and asynchronous copies. Proofs apply to fixed shapes and launches. By default, control flow cannot depend on tensor data. Floating-point rounding, fences, and the full PTX memory model are not modeled. Scope and limitations
How SentryWard works
Verifying a CUTLASS GEMM from SentryWard’s examples against its PyTorch reference.
Installation
With Python 3.10 or newer, install and run synchronization and memory checks:
pip install sentryward
sentryward kernel.ptx \
--input input.json > output.json
The CLI prints JSON; the command above saves it to output.json.
Also prove mathematical correctness
pip install "sentryward[reference]"
sentryward kernel.ptx \
--input input.json \
--reference reference.py \
--reference-binding reference_binding.json \
> output.json
Use the Python API
from sentryward import verify
result = verify(
"kernel.ptx",
input="input.json",
reference="reference.py", # optional
reference_binding="reference_binding.json", # optional
)
print(result["result"], result["checks"])
Install sentryward[reference] when supplying a PyTorch reference.
Full usage and options, including time and memory limits.
Read the technical post for the algorithms, examples, and results