Building a Persistent Key Value Store from Scratch in Go

While building an automated CVE scanning system, I ran into a bottleneck that wasn’t a security problem. It was a systems problem.

Repeatedly parsing large NVD feeds and matching CPE records was slow and redundant. PostgreSQL was great for storage, but not for ultra fast repeated lookups. What I really needed was a fast, persistent, concurrent key value store to act as a cache and index layer. So instead of reaching for Redis lite, I built one from scratch to understand how these systems actually work.

Interactive Terminal

Every command you type is sent to a live instance of vulnkv. The server logs each change to disk using a write ahead log, updates an in memory store for speed, and responds in real time.

The Bottleneck I Hit

The CVE scanner frequently needs to answer: “Given this software version, what vulnerabilities match it?” That requires mapping CPE records to CVEs, which means parsing and searching large datasets over and over again. I needed something that could keep this index in memory for speed, persist across crashes, and handle concurrent lookups safely. That’s the problem vulnkv solves.

Why This Design Works

• Clients send commands over TCP.
• Mutations are written to a write ahead log before memory is updated.
• Reads come directly from memory for speed.
• On restart, the log is replayed to restore the exact state.

What Makes This Nontrivial

Building a key value store is easy. Building one that survives crashes, handles many concurrent clients, persists without a database, and enforces TTL without background sweeps is where real systems engineering starts.

Quick Demo Snapshot