Rust → MLIR → PTX · Apache 2.0
Ship a vision model as one binary.
teenygrad compiles ordinary Rust functions into GPU kernels. There is no Python on the device, no ONNX export step, and no vendor runtime to install alongside your code — one language from the model definition to the thing you flash.
teenygrad
edge path
Every dashed segment above is a format handoff — and a version pair you have to keep matched on the device.
01 / Install
Pick your board. Copy one line.
Cross-compilation is set up already — you build on your workstation and get an artifact that runs on the target.
cargo install cargo-teeny
cargo teeny build --target orin-nano --releaseJetPack 6.2.2 · CUDA 12.6 · SM_87
// A kernel is a function. That is the whole API. #[kernel] pub fn conv1d_forward< T: Triton, D: Num, const KL: i32, const STRIDE: i32, const PAD: i32, const BLOCK_OL: i32, >( x_ptr: T::Pointer<D>, w_ptr: T::Pointer<D>, y_ptr: T::Pointer<D>, _B: i32, C_IN: i32, C_OUT: i32, L: i32, OL: i32, ) { // Block-tiled loads, accumulate, store — plain Rust. } // Compile ahead of time, for the board on your desk. // $ cargo teeny build --target orin-nano
Nothing to install on the device
The interpreter, the packages and the version drift all stay on your workstation. What lands on the board is compiled code.
The backend is a trait
Lowering targets sit behind one interface. CUDA today, SPIR-V and RISC-V in progress — the model code does not change.
Debug it like Rust, because it is
Types, cargo, the borrow checker and your existing CI. No second build system bolted on for the model.
02 / Speed
It is slower than TensorRT. Here is by how much.
On a Jetson Orin Nano running YOLO26 inference, teenygrad is roughly 7× behind TensorRT and 2.5× behind ONNX Runtime. Vendor toolchains have years of hand-tuned fused convolutions in them, and this is a young compiler.
The primary cause is operator fusion — vendor toolchains fuse chains of ops into single kernels, and teenygrad mostly does not yet. Closing that gap is the current priority.
If your frame budget is already tight, use TensorRT. If you are spending more engineering time on the export-and-deploy path than on the model itself, that is the trade this is built for.
Relative inference latency, lower is better. Jetson Orin Nano Super, MAXN SUPER, JetPack 6.2.2. Reproduction steps in the repo.
03 / Status
What works, and what doesn't yet.
teenygrad is early. Here is the honest state of it, so you can tell in thirty seconds whether it is any use to you.
| Capability | State |
|---|---|
| Rust → PTX pipeline, end to end MIR through to loadable GPU code | Works |
| Object detection inference on Jetson Orin Nano YOLO26 · verified bounding boxes · parking demo | Works |
| Training MNIST / LeNet to 97% | Works |
| Cross-compilation from x86 to aarch64 CUDA 12.6 | Works |
| Detection model training compiles and runs, not yet validated | Rough |
| One-command build and deploy to a board currently a manual copy step | Rough |
| Transformer detection (RF-DETR) compiles, not yet run | Not yet |
| Language models kernel work in progress | Not yet |
| Non-NVIDIA targets SPIR-V and ARM GPUs planned | Not yet |
04 / About
Who is building this.
teenygrad started as a compiler experiment: if a kernel is just a Rust function, how much of the edge-deployment toolchain disappears? Enough of it did that we kept going.
We build for teams shipping vision models onto real hardware — inspection rigs on a factory line, autonomy stacks on a drone, cameras in places with no Python runtime and no room for one.
The compiler and the vision SDK on top of it are both Apache 2.0 and developed in the open. We offer support, training and consultancy for teams who want it.
- Started
- 2025
- Built in
- Rust
- Licence
- Apache 2.0
05 / Contact
Tell us what you are trying to ship.
Hardware you are targeting, model you are running, where the current toolchain hurts. We read everything.