pawan
Cintu07 · rust / c++ · ml compilers & inference
founding engineer at cortex, voice agents that pick up the phone for restaurants and clinics. before that co-founder at onepurplepen building onedb.net. cto at dhanamcollections.com, which i built on my own and still run.
i am good at ml compilers and inference, and i write rust and c++.
right now i am mostly in gpu kernels and inference internals. rust and c++, and whatever is underneath them. my laptop is aarch64, which turns out to be the reason i see half the things i see.
what i want to work on next is the memory and context side of ai, the infra side of ml, and anything low level enough that i have to care what the machine is actually doing.
projects
see all →heresy // finds the code that disagrees with the rest of your go codebase.
reads a go codebase, works out the rules the code already follows, and flags the few places that break them. nothing to configure. across 16 big go repos and 15,290 files it flagged one site, a real lock leak in minio.
ciot // ternary nets on cpu. no multiplies in the matmul, just adds and subtracts.
cpu inference for ternary neural nets. weights are -1, 0 or 1, so the matmul is adds and subtracts and no multiplies at all. c++ and simd intrinsics, no dependencies.
strata // 200b mixture of experts on a 16gb laptop, with nvme on the critical path of every token.
an inference engine for a 200b mixture of experts model on a 16gb laptop, where nvme sits on the critical path of every token and not just the cold start. two mechanisms, predicting which experts are needed several layers ahead, and scheduling prefill around experts rather than around layers. everyone else assumes the hot set eventually fits in ram.
talos // layer 4 firewall in ebpf. drops the packet before it reaches the stack.
layer 4 firewall in ebpf and xdp. filtering happens in the kernel before the packet reaches the stack. dynamic blacklist, rate limits, tui on top.
