Zerodha has migrated its trading platform Kite’s user preference storage from SSDB to Blobasaur, an open-source distributed blob storage system built on SQLite. The new system is designed to handle high volumes of read and write requests with low latency, addressing limitations in the company’s previous storage infrastructure.
Blobasaur is designed to support tens of thousands of read and write operations per second, with single-digit millisecond latency for data ranging from a few kilobytes to several megabytes. Zerodha uses the system to store JSON-based preferences for millions of users on Kite.
The platform allows users to customise their trading interface, save multi-chart layouts, create indicator templates, store drawings and organise workspaces using widgets. These customisations generate data blobs that need to be stored and retrieved efficiently.
The migration was driven by limitations in SSDB, which Zerodha had been using after initially relying on CouchDB. While SSDB worked reliably for several years, it was no longer maintained and could operate only on a single machine. The company also had to restrict blob sizes to 1 MB because latency increased with larger payloads.
Zerodha’s workload presented additional performance challenges, with requests fluctuating significantly during market hours due to changing trading activity.
According to Sarat Chandra, a software engineer at Zerodha, the company’s preference-storage workload is unpredictable and bursty, driven by stock market volatility and requests from users and internal services.
Why Zerodha chose SQLite
Zerodha evaluated several alternatives, including ScyllaDB, PostgreSQL, object storage and distributed file storage systems, before selecting SQLite as the foundation for Blobasaur.
Although SQLite is generally associated with lightweight, embedded database applications, Zerodha adapted it for distributed blob storage. The system is written in Rust and supports the Redis protocol, allowing existing backend services to connect using familiar client libraries without requiring significant changes to their architecture.
Chandra noted that SQLite can be up to 35% faster than raw filesystem input/output (I/O), which contributed to the company’s decision.
Performance and production deployment
In benchmark testing, a 16-shard configuration of Blobasaur recorded 29,538 combined read and write operations per second, more than twice the throughput of a single-shard configuration. The test was conducted on a 16-core laptop, with the server and load generator sharing the machine.
In production, Zerodha operates Blobasaur across three nodes, each equipped with four virtual CPUs and 8 GB of memory. The company reported that 99th-percentile latency remains around 2 milliseconds across commands during a full trading session.
The migration was completed in stages, with new workloads moved first before existing services were transitioned. Zerodha also ran dual writes for a month to verify data consistency before shutting down SSDB.
Open-source release
Blobasaur has been released under the AGPLv3 open-source licence and is available on GitHub. However, certain scaling capabilities, including automatic failover and live redistribution of data between nodes, have not yet been implemented.
The development reflects Zerodha’s efforts to build infrastructure capable of supporting the scale and performance requirements of its trading platform while using open-source technologies to address specialised storage challenges.
-By: Aakriti Jaiswal




