Kani for Rust AI: Verify Your Code in Minutes

Rust's ownership model catches a class of bugs that would eat a C++ codebase alive. But it doesn't catch everything. A tokenizer that silently produces wrong offsets on multi-byte Unicode, a pointer cast in an ONNX binding that's valid today and undefined behavior tomorrow, a bounds check that works on every test input you thought to try — these slip through. And in an AI pipeline, one silent error doesn't just break one function. It poisons everything downstream.

Kani is a model checker for Rust, developed at Amazon Web Services and described in the research paper published on arXiv. It doesn't run your code. It mathematically explores every possible execution path and tells you whether your assertions can ever be violated. The gap between "I tested this" and "I know this" is exactly what Kani closes.

This guide shows you how to drop Kani into an existing Rust AI project — tokenizers, unsafe FFI blocks, model loading code. No formal methods background required.

What Kani Actually Checks (and Why AI Code Needs It)

Standard testing is a sampling problem. You pick inputs, run the function, check the output. If the bug lives in an input combination you didn't think of, the test passes and the bug ships.

Kani approaches this differently. It uses a technique called bounded model checking: it systematically explores all possible executions of your Rust code within a specified bound and asks a solver whether any assignment of inputs can violate your assertions. If the solver finds one, Kani shows you the counterexample. If it can't find one within the specified bounds, you get a proof that no such input exists.

That distinction matters in AI code for two specific reasons.

First, AI pipelines process arbitrary input. A tokenizer might see any string a user types. A model loader might read any byte sequence from a file. The space of "inputs you tested" is always a tiny fraction of "inputs that can arrive in production."

Second, Rust AI code often reaches for unsafe blocks. ONNX runtime bindings, hand-rolled SIMD kernels, zero-copy tensor views built on raw pointer arithmetic — these all bypass Rust's normal safety guarantees. Rust trusts you to get them right. Kani checks whether you did.

Compare these two approaches to the same function:

Before (standard assertion — tests one input):

assert!(my_function(input) == expected_output);

After (Kani harness — explores all valid inputs):

#[kani::proof]
fn verify_my_function() {
    let input: u32 = kani::any();
    let result = my_function(input);
    assert!(result == expected_output_for(input));
}

The first version tells you the function worked on that one input. The second tells you whether the assertion can be broken by any u32. That's not a style difference — it's a different class of guarantee.

Setup: Adding Kani to an Existing Rust AI Crate

Kani ships as a standalone tool installed separately from the standard Rust toolchain. Check the Kani installation guide for the up-to-date command, since the exact invocation has changed across versions. As of recent releases, the recommended approach is:

cargo install --locked kani-verifier
cargo kani setup

The setup step downloads the supporting components Kani needs to analyze your code. Once it completes, you're ready to add harnesses to your crate.

Adding verification to an existing crate is three steps.

Step 1: Create a proof harness

A proof harness is a function annotated with #[kani::proof]. Inside it, you use kani::any() to declare symbolic inputs — values Kani treats as "could be anything of this type" — then assert properties that should always hold.

#[cfg(kani)]
mod verification {
    use super::*;

    #[kani::proof]
    fn verify_add_no_overflow() {
        let x: u32 = kani::any();
        let y: u32 = kani::any();
        // Constrain inputs to prevent overflow for this proof
        kani::assume(x <= u32::MAX / 2);
        kani::assume(y <= u32::MAX / 2);
        let result = add(x, y);
        assert!(result == x + y);
    }
}

Wrapping harnesses in #[cfg(kani)] keeps them out of your normal build, so they're only compiled when you're running verification.

Step 2: Run Kani on a specific harness

cargo kani --harness verify_add_no_overflow

Kani will report either a proof success or a counterexample with the exact input values that caused the failure.

Step 3: Scope the verification

Kani works best when you start narrow: one function, one property, one harness. Trying to verify a 500-line module in one shot will either time out or produce results too broad to act on. Start with the functions that touch unsafe code or process unbounded input.

Real Workflow Examples: Tokenizers, Model Loading, and Unsafe FFI

These are the three areas in a typical Rust AI codebase where Kani earns its keep.

Tokenizers

Tokenizers fail on edge cases: empty strings, strings consisting entirely of whitespace, strings that mix ASCII and multi-byte characters in sequences the training data never included. Manual testing catches the obvious cases. Kani catches the rest.

#[cfg(kani)]
mod verification {
    use super::*;

    #[kani::proof]
    #[kani::unwind(32)]
    fn verify_tokenizer_length() {
        // Symbolic input: a byte array of up to 32 bytes
        let len: usize = kani::any();
        kani::assume(len > 0 && len <= 32);
        let input: Vec = (0..len).map(|_| kani::any()).collect();

        if let Ok(s) = std::str::from_utf8(&input) {
            let tokens = tokenize(s);
            // Property: token count must be at least 1 for non-empty input
            assert!(tokens.len() >= 1);
        }
    }
}

The #[kani::unwind(32)] attribute controls how many loop iterations Kani will explore. For tokenizers, start low and increase if you need to cover longer inputs. Tighter bounds run faster. Wider bounds give stronger guarantees.

Unsafe pointer reads in FFI bindings

This is the highest-value use case. An unsafe block that reads from a raw pointer has no automatic bounds checking. Kani can verify that the access is always within the allocated region:

#[cfg(kani)]
mod verification {
    use super::*;

    #[kani::proof]
    fn verify_buffer_read_in_bounds() {
        let buffer = [0u8; 10];
        let index: usize = kani::any();
        // Prove the access is only attempted within bounds
        kani::assume(index < buffer.len());
        let value = unsafe { *buffer.as_ptr().add(index) };
        assert!(value == buffer[index]);
    }
}

Without kani::assume(index < buffer.len()), Kani will find a counterexample where index >= 10 and flag the out-of-bounds access. That's the point — you want to know whether your actual calling code always satisfies that precondition before the unsafe block executes.

Model loading and numeric conversions

Casting between numeric types during model loading (index widths, dimension sizes, dtype conversions) is a quiet source of truncation bugs. Kani verifies that a cast is lossless for all inputs in a range:

#[cfg(kani)]
mod verification {
    use super::*;

    #[kani::proof]
    fn verify_dimension_cast() {
        let dim: u64 = kani::any();
        // Verify the cast to usize is lossless on this platform
        kani::assume(dim <= usize::MAX as u64);
        let converted = dim as usize;
        assert!(converted as u64 == dim);
    }
}

CI/CD Integration: A GitHub Actions Template

Verification you run once and forget doesn't protect you. The goal is to run Kani on every pull request so that a new unsafe block or a refactored tokenizer path can't slip through without being checked.

Here's a minimal GitHub Actions workflow structure. Kani's supported runner environments can change between releases, so consult the Kani documentation for the current recommended runner and any updates to the install steps. As of recent releases, Ubuntu 20.04 is a confirmed supported environment. Pin your actions/checkout version to whatever your project currently uses, and verify it against the latest GitHub Actions documentation:

name: Kani Verification

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  kani:
    runs-on: ubuntu-20.04
    steps:
      # 1. Check out your repository (pin to your project's preferred version)
      - uses: actions/checkout@v3

      # 2. Install a stable Rust toolchain (use your preferred action here)
      - name: Install Rust toolchain
        run: rustup toolchain install stable --profile minimal

      # 3. Install Kani
      - name: Install Kani
        run: |
          cargo install --locked kani-verifier
          cargo kani setup

      # 4. Run all Kani proofs
      - name: Run Kani proofs
        run: cargo kani

A few practical notes on this setup:

• Kani installation adds time to your CI run. Cache the ~/.cargo directory between runs using an appropriate caching action to keep it manageable.
• If you have many harnesses, use --harness flags to run only the proofs relevant to changed files, rather than the full suite on every commit.
• Check the Kani documentation for how proof failures surface in CI output so you can configure your pipeline's failure conditions appropriately.

For teams integrating AI agents into their development workflows, this kind of automated verification step pairs well with the broader question of how AI agents can augment rather than replace engineering judgment — Kani handles the exhaustive checking; you decide what properties matter.

Kani vs. Miri vs. Prusti: A Decision Matrix for AI Projects

These three tools are often mentioned together but they solve different problems. Picking the wrong one wastes setup time and gives you false confidence.

Miri is the easiest entry point. Run Miri against your existing test suite and it catches undefined behavior without requiring new harnesses. The limitation: it only checks the inputs your tests already use.

Prusti takes a deductive verification approach. With the right annotations, it can prove functional correctness properties that go beyond what model checking covers. The annotation overhead is real, though manageable — writing Prusti specs for a non-trivial function is still closer to writing a formal proof than writing a unit test, which is why it's best reserved for components where certified correctness is a hard requirement.

Kani sits in the practical middle. You write short harnesses, use kani::any() to represent the input space, and get exhaustive coverage within whatever bounds you set. The tradeoff is worth understanding: Miri requires no new harnesses but only covers what your tests exercise; Prusti covers more ground but demands annotation work; Kani asks for harnesses and gives you coverage over the full input space within your stated bounds.

A reasonable workflow: use Miri during development to catch obvious undefined behavior quickly, add Kani harnesses for the functions that matter most (tokenizers, unsafe FFI, numeric conversions), and revisit Prusti only if a specific component needs certified correctness.

Frequently Asked Questions

How do I run Kani on my Rust AI project without learning formal methods?

You don't need a formal methods background to start. Pick one function — ideally one that handles arbitrary input or contains an unsafe block. Add a #[kani::proof] harness, declare inputs with kani::any(), and write one assert! about a property that should always hold. Run cargo kani --harness your_harness_name. That's the whole workflow. Expand from there once you've seen it work once.

Can Kani verify unsafe code used in Rust ML libraries?

Yes, and that's where it adds the most value. Wrap the unsafe block in a proof harness, constrain the inputs with kani::assume() to match the preconditions your calling code is supposed to guarantee, and assert the postconditions that need to hold. If Kani finds a counterexample, it means either the preconditions aren't being enforced upstream or the unsafe block has a bug. Either finding is worth knowing.

What's the difference between Kani and Miri for AI developers?

Miri is a dynamic interpreter. It runs your code on specific inputs and flags undefined behavior when it encounters it. Kani is a static model checker. It doesn't run your code — it reasons about all possible inputs simultaneously. Miri is faster to set up and requires no new harnesses, but it only covers what your existing tests exercise. Kani requires harnesses but gives you coverage over the full input space within your stated bounds. For production AI code where a rare edge case can corrupt model state, Kani's exhaustive coverage is the stronger guarantee.

How do I add Kani to a GitHub Actions workflow?

Add two steps: one that installs Kani (cargo install --locked kani-verifier && cargo kani setup) and one that runs your proofs (cargo kani). Cache your Cargo directory between runs to avoid reinstalling Kani from scratch on every CI run. Check the Kani documentation for the current recommended runner environment and details on how proof failures surface in CI output.

Is Kani suitable for verifying Rust code that calls into Python AI models?

Kani verifies Rust code specifically. It can verify the Rust side of a Python FFI boundary — pointer handling, buffer sizes, type conversions — but it has no visibility into Python code or the AI model's behavior. Use it to prove that your Rust bindings never produce undefined behavior on the inputs they receive, and handle validation of the AI model's outputs separately.

The Practical Case for Starting Today

Most Rust AI codebases already have the highest-risk code identified: it's the functions with unsafe in the signature, the tokenizer that's been patched three times, the numeric conversion that someone commented "TODO: check this for large inputs." Those are your first Kani harnesses. Write three of them this week and run cargo kani. Either everything passes — and you've got your first proof — or Kani finds something, and you fix a real bug before it reaches production.

AI being a useful partner here is worth naming. You can use an AI assistant to generate the initial harness structure from a function signature, then you review what property is actually being asserted and whether the kani::assume() constraints match your real calling conventions. AI handles the scaffolding. You handle the judgment about what correctness actually means for your code. That division of labor is where you get the most out of both. For more on that pattern, the piece on keeping AI in a partnership role rather than a dependency role is worth reading.

Verification isn't a research activity anymore. It's a straightforward setup with a tool that runs in your existing CI pipeline. The question isn't whether your AI code is worth verifying. It's whether you want to find the edge case in development or in production.

If you want pre-built prompt templates for generating Kani harnesses and CI configurations, Ultra Prompt's formal verification template collection has structured prompts for exactly this workflow.