Kmemo

A semantic cache for LLM calls. It recognises requests it has already answered even when they are worded differently, and rejects the ones where similarity is high but the right answer is not the same. The core compiles for the JVM, iOS, macOS, Linux, Windows, JS and WasmJS.

An exact-match cache cannot tell that two wordings are the same question. A semantic cache embeds the request, finds the nearest one it has already handled and reuses that answer, which cuts model calls and latency. The known weakness of this approach is the false positive. «Convert 100 USD to EUR» and «Convert 250 USD to EUR» sit around 0.99 similarity on every major embedding model. No threshold separates them, because on that axis they are closer than plenty of legitimate rephrasings. A cache that trusts the threshold alone returns the wrong answer and raises nothing. Kmemo treats that case as the primary requirement. Similarity is only the first filter: candidates that clear it are then read as text by a chain of eleven lexical checks, which look for real differences in numbers, units, entities, time references, negations, antonyms, reversed comparisons and a clause one wording adds to narrow the question. An optional twelfth check reads the cached answer instead of the two questions alone, which is the only way to catch two honest rephrasings whose difference sits in the answer. The two errors do not cost the same. A wrong rejection costs one API call; a wrong acceptance costs a wrong answer. The defaults follow from that, and a check abstains when it is unsure. The similarity threshold has to be measured against the embedding model you actually use, so the library ships a calibrator for it. The numbers are measured on blind corpora, written before the checks and never used to tune them. A cache that trusts the threshold alone serves every false positive it is shown; Kmemo's checks cut that to roughly a third while keeping 88% of the legitimate rephrasings. The comparison against GPTCache is published together with its cost: that cross-encoder serves fewer of them, and refuses more than half the correct rephrasings to do it. Those three corpora share one weakness no process removes: the same person wrote the pairs and the checks, so they cover the edge cases somebody thought of rather than the ones that exist. Since version 2.1 there is a fourth corpus this project did not write. PAWS was built by Google Research in 2019 to measure whether a model can tell a rephrasing from a near-rephrasing when almost every word matches. On it the checks refuse 14% of the false positives and keep 79% of the rephrasings, far worse than the internal numbers. Two things pull in opposite directions and neither improves it: a corpus built to defeat lexical overlap is harder than real traffic, and its pairs are encyclopedia sentences where the checks read questions. It is still the only figure on this page nobody here could have tuned, which is why it is published beside the others. Since version 2.0 the core is no longer tied to the JVM. It compiles for iOS, macOS, Linux, Windows, JS and WasmJS without adding a single dependency. A mobile app pays for every call over a cellular network and a browser tool has no server to cache on, which are the deployments a JVM-only library never reaches. The Redis and PostgreSQL stores and the framework integrations stay on the JVM, where the drivers they wrap live.