From Polling to Pub/Sub
The Notification Queue Handler polled Redis to find out when a bank's payment form became available. Pub/Sub should have been a clean swap. It wasn't, because of how Spring's RedisTemplate serializes channel names underneath you.
- System
- Notification Queue Handler
- Company
- Paytm Payments Services
- Impact
- Eliminated redundant Redis reads in the payment callback path, cut latency, and resolved production incidents tied to webhook signature-verification failures with banking partners.
Context
The Notification Queue Handler needs to know the moment a bank-hosted payment form becomes available so it can route the next step of the transaction. The existing implementation polled a Redis key on an interval — simple, but wasteful: most polls found nothing, and worst-case latency was bounded by the poll interval rather than by how fast the form actually became ready.
Redis Pub/Sub replaces that with a push: subscribe to a channel keyed by the transaction's cache request ID, and the publisher notifies you the instant the form is ready.
The part that wasn't simple
Spring's `RedisTemplate` runs configured serializers over both keys and channel names by default. That's usually invisible, because most code only ever reads back through the same `RedisTemplate` it wrote through — the serialization round-trips cleanly and nobody looks twice.
Pub/Sub broke that assumption: the channel name needed to match, byte-for-byte, between the side publishing the bank-form-ready event and the side subscribing to it — and those two sides didn't agree on serialization, because the subscriber needed to construct and listen on a raw channel name derived from the request ID, not a value round-tripped through the same template configuration.
The result was subscriptions that looked correct in code and simply never fired, because the actual bytes on the wire for the channel name didn't match what was published.
The fix
The fix was a deliberate low-level bypass: constructing and matching the Redis Pub/Sub channel name outside `RedisTemplate`'s default serialization path, so publisher and subscriber agreed on the exact channel bytes regardless of what the surrounding template was configured to do with regular key/value operations. Once that was in place, I moved the relevant behaviour behind an FF4J flag so the pub/sub path could be rolled out and rolled back independently of a deploy.
The latency win in the payment callback path was real, but the bigger effect was on debuggability: some banking-partner webhook signature-verification failures that had been intermittently showing up in production traced back to timing sensitivity in the old polling path, and cleared up once the event delivery became push-based instead of poll-based.
The lesson
The failure mode of 'my Pub/Sub subscriber never fires' is almost never the Pub/Sub model — it's a serialization mismatch between whoever names the channel and whoever listens on it. Frameworks that quietly serialize things for you are only safe as long as every caller goes through the same framework.
Next case study
Seventy Secrets, One Vault →