This talk is about using LLMs to build and modernize software where correctness matters. It follows the work at two scales: modernizing a similarity-digest algorithm that must match its reference implementation bit-for-bit, and building malbench, a local-first triage and clustering workbench for malware analysts.
malbench is a local-first triage and clustering workbench for malware analysts. It pulls samples and threat intelligence from various sources, computes similarity digests, runs YARA rules, and then turns a flat pile of hashes into structure: it clusters and graphs files by features they actually share and weaves several of those lenses into a single view.
I needed a wider variety of hashing algorithms, so in addition to ssdeep and TLSH, I took a dormant 2021 Go port of sdhash, a similarity-digest algorithm used in malware triage and code-reuse detection, and modernized it into a clean, dependency-free, fully-tested library. A deterministic verification corpus and parallel C++/Go harnesses turn the original reference implementation into an oracle: millions of pair comparisons, zero unexplained divergences. Around that sit total test coverage as a drift alarm, strict session and context hygiene, profile-before-you-optimize with one change measured at a time. The result is a repeatable process for adopting and modernizing old code and for building new tools on top of it.
How do you use AI models on code that has to be correct, without their confident mistakes silently landing in your output? This talk answers that with a concrete, repeatable process, demonstrated end to end on a real library, and on the larger tool that library is being built for. That tool is malbench, a local-first triage and clustering workbench for malware analysts. It pulls samples and threat intelligence from various sources, computes similarity digests, runs YARA rules, and then turns a flat pile of hashes into structure: it clusters and graphs files by what they actually share and weaves several of those lenses into one view.
The subject of the overall process is sdhash, a similarity-digest (fuzzy hashing) algorithm that fingerprints binary data into bloom filters and scores two fingerprints for similarity. It has a C++ reference by Vassil Roussev and Candice Quates and a 2021 Go port that had gone dormant for over three years. This is exactly the kind of valuable-but-abandoned code worth adopting rather than rewriting. The goal was to bring it forward into a clean, modern, dependency-free Go library with an idiomatic sealed API, full documentation, and total statement coverage.
The spine of the effort, and the core defense against hallucination, is that ground truth comes first. Before trusting a single line the model produced, I built a deterministic, seed-reproducible corpus and parallel C++/Go harnesses, ran it through both, and diffed the outputs with an independent tool. That turns the reference into an oracle: any divergence, a real bug or something a model invented, surfaces immediately as a mismatch. Generation parity held across more than a hundred thousand files and scoring parity across nearly three million pair comparisons in both modes, at zero unexplained divergences.
The process surfaced first three correctness problems in the C++ implementation scoring as well as a problem with the hashing algorithm itself. Each change to the algorithm was carefully isolated and the effects on scoring measured. This talk goes into detail all of the tooling and process used to get sdhash up to snuff and then plug it in to malbench. You will also learn about the graphing and clustering algorithms used to visualize the results of the different malware hashes.