Two Codebases, One Fallback Loop
Merging two parallel notification stacks meant one of them had to be able to fall back into the other on failure — without the two systems retrying into each other forever.
- System
- Payment Gateway Notification Service
- Company
- Paytm Payments Services
- Impact
- Removed dual-maintenance cost across the OCL and PPSL stacks and shipped a fallback path with explicit loop prevention built in from the start, not bolted on after an incident.
Context
Paytm's notification stack existed as two separately deployed codebases — OCL and PPSL — with their own FF4J feature flags, MongoDB template stores, and Kafka bootstrap configuration, covering overlapping merchant traffic. Every change had to be made twice, tested twice, and could drift out of sync between the two at any point.
The problem with a naive fallback
The unification plan required OCL's Communication Gateway to fall back to PPSL's when its own outbound notification retries were exhausted, so that a merchant's webhook or SMS still had a path to delivery even if one side of the fallback pair was degraded.
The obvious way to build that is dangerous: if PPSL's attempt also exhausts its retries, and it falls back to OCL, and OCL falls back to PPSL again, a transient partial outage turns into two systems bouncing the same notification back and forth indefinitely — consuming resources and never actually failing closed.
What shipped
The fallback path I built into the Communication Gateway carries explicit loop-prevention state through the S2S notify layer — the affected surface touched constants and service implementations across every merchant-specific notify service in the codebase (AOA, Chargeback, CoFT, BPCL, DMRC, MTNL, Manipur GRAS, and more), each of which needed to respect the same "have we already tried the other side" signal before attempting a cross-stack fallback, plus dedicated PPSL Kafka producers so the fallback path didn't share failure domains with the primary one.
This shipped alongside environment configuration to control the fallback behavior per deployment, so the two stacks could be merged incrementally rather than as a single high-risk cutover.
The lesson
Fallback logic is only as safe as its exit condition. The interesting engineering problem in "make system A fall back to system B" is never the happy path — it's guaranteeing the bounce terminates when both sides are unhealthy at once.
Next case study
From Polling to Pub/Sub →