Grab slashes counter service latency in half with Aerospike migration

Grab has successfully transitioned its Counter Service—a critical backend powering its fraud prevention—from a wide-column database to Aerospike, achieving dramatic improvements in speed and efficiency. The effort produced a 50% reduction in production P99 read latency, compressed on-disk storage from 3 TB to 1 TB, and cut per-node costs by 45–50%.

The challenge: time-windowed counting at massive scale#

The Counter Service handles time-sensitive queries for Grab's anti-fraud mechanisms, answering questions like "how many ride requests came from this account in the last hour?" It processes tens of thousands of queries per second and about one billion requests daily.

The original architecture stored 15-minute, hourly, and daily aggregations as separate rows. Each new event triggered three parallel reads, an in-memory update, and a batch write—requiring four network round trips and creating excess operational overhead.

Rearchitecting for Aerospike#

Rather than cut over immediately, Grab built abstraction layers to decouple storage logic from its Rust service. A storage facade supported legacy, Aerospike, and mock implementations, switchable via configuration into single-backend, shadow-read, and split-traffic modes. The team gradually ramped shadow reads from 5% up to 50% and then 100%, validating data consistency before migrating live traffic.

The bigger shift was the data model itself. Storing one row per time bucket meant that Aerospike's ~64 bytes of index overhead per record became prohibitive. Instead, Grab consolidated all time buckets for a counter into a single record holding a timestamp-ordered map. Atomic map-increment operations replaced per-bucket reads and writes, slashing record cardinality by more than an order of magnitude and dramatically cutting index and disk overhead.

Read paths also evolved. The old system returned paginated, server-filtered rows; Aerospike now returns full records for client-side filtering. Grab leveraged Aerospike's batch API to group multiple record operations per request, reducing network round trips for multi-key access patterns.

Rollout and client improvements#

During deployment, Grab encountered friction with the Aerospike Rust client—initially only synchronous and troubled by DNS resolution during cluster transitions. The team adopted the async client and collaborated with upstream maintainers to resolve DNS issues before completing the migration.