SentryWard

Formal Verification of Warp-Specialized GPU Kernels.

SentryWard is a formal verifier for warp-specialized GPU kernels. Given a kernel’s PTX and a launch configuration, it proves correct synchronization and freedom from data races in every execution schedule within the configured input scope and hardware model. With a PyTorch reference, it also proves that the kernel computes the same function in real arithmetic.

From manual validation to an automated loop

Coding agents can automatically generate and optimize high-performance GPU programs at low cost. As generation becomes cheaper, the bottleneck shifts from programming kernels to verifying their correctness. SentryWard automates that verification step, giving agents proofs or diagnostics to guide the next revision.

BeforeManual validation

Each revision waits for a person
  1. Coding agentGenerate or optimize
  2. Candidate kernelCompile to PTX
  3. Human validationInspect · test · debug
  4. Tested candidateBenchmark performance
  • Time-consumingA person must review each revision.
  • Error-proneRare interleavings are easy to miss.
  • Hard to scaleMore candidates need more review.

AfterSentryWard

Agents generate, verify, and revise
  1. Coding agentGenerate or optimize
  2. Candidate kernelCompile to PTX
  3. SentryWardProve across schedules
  4. Verified candidateBenchmark performance
  • Fast verification10.3 seconds median to complete verification
  • Strong guaranteesProofs cover every execution schedule.
  • Fully automatedAgents call the CLI or Python API.

What goes in. What comes out.

Input

PTX kernel
Your compiled NVIDIA GPU kernel.
Input configuration
A JSON file describing input shapes, launch dimensions, kernel parameters, and any TMA descriptors.
Optional PyTorch reference
A reference(...) function and a binding that maps its arguments and outputs to the kernel. No separate reference input file is needed.

Output

A clear verdict
race_free means synchronization and memory checks passed without a reference. verified means reference equivalence was proved too.
Check results and witnesses
JSON from the CLI, or a Python dictionary, with each check’s status and evidence for findings such as potential_race.
Explicit limits
unsupported or timeout explains why analysis stopped. An undecided check is never reported as a pass.

What SentryWard verifies

Synchronization & deadlocks
Barriers and mbarriers, including TMA byte counts, behave consistently across schedules, ruling out deadlocks.
Data races
Conflicting accesses to global, shared, and tensor memory are ordered, including asynchronous copies and tensor-core operations.
Resource lifetimes
Tensor memory and barriers are never used after they are freed or invalidated.
Mathematical correctness
The kernel’s outputs equal those of your PyTorch reference in real arithmetic.
Unnecessary waits
Use verification to remove redundant synchronization: remove an operation, then verify again. Safe removals depend on the input shape.

Covers Blackwell TMA, tcgen05 MMA and tensor memory, mbarriers, thread-block clusters, and asynchronous copies. Proofs apply to fixed shapes and launches. By default, control flow cannot depend on tensor data. Floating-point rounding, fences, and the full PTX memory model are not modeled. Scope and limitations

How SentryWard works

Verifying a CUTLASS GEMM from SentryWard’s examples against its PyTorch reference.

$sentryward--input--reference--reference-binding>

    Installation

    With Python 3.10 or newer, install and run synchronization and memory checks:

    pip install sentryward
    sentryward kernel.ptx \
      --input input.json > output.json

    The CLI prints JSON; the command above saves it to output.json.

    Also prove mathematical correctness
    pip install "sentryward[reference]"
    sentryward kernel.ptx \
      --input input.json \
      --reference reference.py \
      --reference-binding reference_binding.json \
      > output.json
    Use the Python API
    from sentryward import verify
    
    result = verify(
        "kernel.ptx",
        input="input.json",
        reference="reference.py",                    # optional
        reference_binding="reference_binding.json",  # optional
    )
    print(result["result"], result["checks"])

    Install sentryward[reference] when supplying a PyTorch reference.

    Full usage and options, including time and memory limits.