Writing / Engineering tradeoffs

Zerodha's Blobasaur, and why they turned down S3.

On 19 November 2026 Zerodha explained why they built a sharded SQLite blob store and rejected S3. Their workload is write-heavy. The image pipeline I worked on is a read-through delivery path for more than 50 million images a day, and that difference is the point.

On 19 November 2026, Zerodha published an account of Blobasaur, a sharded blob store they built on SQLite after deciding not to put this workload on S3 or MinIO. The post, by Sarat Chandra on the Zerodha Tech Blog, is the primary source for what follows. The figures and reasons are theirs.

The contrast with my work is the access pattern. At a US consumer app I worked on a read-through image pipeline that delivers more than 50 million images a day. That system puts repeated reads on a Cloudflare edge cache, with durable copies in R2. Zerodha's store, as they describe it, is write-heavy, overwrite-heavy and full of JSON blobs. Those are different jobs, and they lead to different storage choices. I have not benchmarked SQLite for this, and I have not rerun their tests.

What they needed to store

Zerodha's trading platform, Kite, has about 20 million customers, according to the post. Users save chart layouts, indicator templates, vector drawings, workspace widgets and other UI preferences. The backend stores those as opaque blobs, mostly JSON, ranging from a few kilobytes to several megabytes.

They say they sustain thousands to tens of thousands of requests per second reading and writing these blobs, that the traffic is bursty because of market volatility, and that writes are 50% of the traffic. For several years this lived in SSDB, a disk-backed store that speaks the Redis protocol. The post says two problems forced a replacement: SSDB became unmaintained, and it runs on a single node. At one point they capped blob sizes at 1 MB because latency degraded on larger payloads.

They wanted a key-value store that could handle multi-megabyte blobs, scale horizontally, compress well and stay economical to run, with Redis protocol compatibility so existing clients could stay as they were.

Why they turned down S3

They also looked at ScyllaDB, PostgreSQL BYTEA columns, YugabyteDB, RocksDB, Ceph, SeaweedFS and a raw filesystem, and in their tests or assessment each failed a requirement they cared about.

Their reasons for rejecting S3 and MinIO are specific. They cite high per-request latency for small blobs, high request costs under write-heavy workloads, and coarse lifecycle TTLs instead of exact key expiry. They also say S3 would incur significant costs given thousands of operations per second and tens of millions of operations per day. They do not publish a dollar figure, and I am not going to invent one.

Their three reasons map to the workload: writes are half the traffic, per-request latency hurts small blobs, and they need exact key expiry rather than coarse lifecycle TTLs. The post is also clear that this store holds non-critical UI and workspace preferences, not critical financial data. They accepted relaxed durability as a tradeoff: a sudden power loss might drop a recent chart-layout tweak, which they call annoying, but it does not affect trading.

What they built

Blobasaur is a sharded blob server written in Rust. It speaks the Redis protocol and uses SQLite as its storage engine. They have released it under the AGPLv3 license.

A single SQLite database in WAL mode allows concurrent readers but only one writer at a time. Because writes are such a large share of traffic, they split data across multiple SQLite files, or shards, on each machine. Each shard has a dedicated writer that commits a batch of queued writes in one transaction. Reads use read-only connections and can run at the same time as the writer. Across machines they implement the Redis Cluster protocol. They note that live slot rebalancing and automatic failover are not implemented yet.

They report a benchmark of 1 shard against 16 shards, using 64 concurrent connections and 64 KiB JSON payloads, with the data directory on tmpfs to isolate locking and CPU from disk variance. In the HSET-only run, throughput rose from 7,740 to 25,324 operations per second, which they describe as a 3.3x increase, and they say median write latency dropped from 8.2 ms to 2.1 ms. Read-only throughput stayed nearly the same. These are their numbers, from a single 16-core laptop running both the server and the load generator, with 100,000 keys, synchronous writes and a batch size of 50. They are not an independent measurement.

In production they run a three-node cluster, each node with 4 vCPU and 8 GB of RAM. Over a full trading session, 09:00 to 15:30, they say p99 latency across all commands stays around 2 ms on every node, with p50 between 250 and 550 microseconds. They compress with zstd, and they report that an hour of production HSET traffic shrank by about 11x on average, from 63.6 KiB sent by the client to 5.5 KiB stored in SQLite.

They also record a few operational lessons: deleted rows did not shrink SQLite files until they enabled incremental auto-vacuum and added a vacuum command that reclaims freelist pages and then checkpoints the WAL, and a single large TTL DELETE produced 30 to 50 ms p99 write spikes until cleanup shared the shard's writer and ran in chunks.

How they moved off SSDB

They did not cut over in one shot. New Terminal-mode workspace storage went to Blobasaur first. For existing workloads they dual-wrote to SSDB and Blobasaur, backfilled historical keys, and read from Blobasaur first, falling back to SSDB on a miss or error. After running dual-writes for a month with zero discrepancies, they turned off SSDB. On the client side, the change was switching to a Redis Cluster client pointed at the three Blobasaur nodes.

That is a migration designed around comparison on real traffic. It is their procedure, not a method I am claiming as my own.

A different workload: repeated image delivery

The image pipeline I worked on, at a US consumer app, had a different shape. The published case study is 50 million images. A much lighter origin.. The longer write-up is Designing a read-through image pipeline at 50M+ images per day.

The problem on those pages is repeated image delivery. The app served more than 50 million images a day. Requests kept reaching the origin even when the underlying image had not changed, and image-path CPU was approximately 90%. The origin had to remain the source of truth and generate missing variants.

I gave each layer a specific job, as the case study describes. A Cloudflare Worker normalized requests into canonical variant keys. The edge cache served hot responses. R2 held durable copies for cold locations. Requests reached the origin only when the required variant was missing. D1 tracked the object-key inventory used for lifecycle changes.

The documented result is 50 million-plus images a day, with image-path origin CPU down from approximately 90% to below 15%.

Object storage is in this design because the common operation is a read of an image that already exists. R2 is the durable copy that a cold location can use without waking the origin. D1 is not the image store; it is the inventory of derived keys. The published problem being solved is the volume of repeated reads.

The useful comparison is the access pattern

It is tempting to treat "just use object storage" or "just put blobs in SQLite" as general advice. Neither published account supports that. Zerodha turned down S3 for a workload they describe as half writes, often small, frequently overwritten, and sensitive to per-request cost and latency, with expiry that had to be exact. They also chose that design for data they describe as non-critical UI and workspace preferences, not critical financial data, where relaxed durability was acceptable.

The image pipeline went toward object storage and an edge cache because the documented pain was repeated delivery of images that had not changed. The origin still generates a missing variant. The layers in front of it exist to stop that from being the default path for every request.

Zerodha's account ties its design to write share, object size, overwrite rate, exact expiry and durability needs. The image-pipeline account ties its design to repeated reads of images that had not changed. That is analysis of the two published accounts, not a claim that I tested Zerodha's traffic, and not a claim that their production numbers would transfer to another app.

The primary source for Blobasaur is Distributed, high-performance blob storage on SQLite. The image-pipeline facts above are the ones already on the case study and the engineering write-up.

If a storage or delivery problem like this is sitting in your business, I take that work through Developer Labs.

Bhavin Kamani

A small command line for a curious visitor.