hamsa software
← All articles
Integrations7 min read

A paid booking should survive a delayed payment event

Three deliveries can turn a paid booking back into a pending one and send two welcome messages. A worked projection shows where the interpretation goes wrong.

Two intersecting orbital rings surround a central circle and two satellites, suggesting repeated event arrivals around a stable booking state.

The second message describes an earlier moment

A booking is paid, then pending, then paid again. The customer has paid once; the integration sends two welcomes. This can happen when a delayed pending event arrives between two deliveries of the same paid event and each handler copies its snapshot into the booking. The handler’s rule sounds reasonable in isolation: update the status, then welcome a customer whose booking has just become paid. The three deliveries expose what that rule forgot.

The application has allowed the order of messages to rewrite the order of events. It has also mistaken a second change in its own status field for another welcome being due. Checking event IDs helps with repeated notifications, but there are still two questions to answer: what is true of the payment now, and has this booking’s welcome already been handled? I would give those questions their own records rather than make one duplicate flag answer both.

The example uses three invented deliveries: evt-B says paid, evt-A carries an earlier pending snapshot, and evt-B arrives again. They are teaching records rather than Stripe payloads. Throughout this run, the provider’s current object stays paid, so a fresh lookup has a definite answer. Keeping that assumption visible matters: later we will let the provider and local workers change state, which exposes a second ordering problem.

There are three identities in the room

The provider event identifies a notification. Store its identity so a repeated delivery can be recognised. The provider payment or invoice object identifies the thing whose state is changing. Your booking identifies the business relationship. A link between those objects is more dependable than matching an email address that a customer can change, share or mistype.

If the link to the booking is missing, keep the event unresolved rather than creating a new booking from whichever customer detail happens to match. A delayed event can legitimately arrive before the local relationship has been recorded. Retain the provider object identity, the missing relationship and a retry or review path. Guessing a match makes the handler appear complete while transferring the ambiguity into a customer record that may be much harder to repair.

A welcome can have an identity such as booking-17:welcome:v1. That names the action the business intends to carry out once under its agreed policy. Several provider events may establish that the same welcome is due, and a single event may justify more than one different action. An event receipt therefore cannot double as the welcome’s history. Even in a small implementation, preserving that distinction lets a colleague explain a repeated notification without promising a repeated message.

Stripe’s documentation explicitly says event delivery order is not guaranteed. It also warns against using snapshot created timestamps to establish event order or duplicate processing, because distinct events can share a timestamp; it recommends tracking event IDs and retrieving missing objects through the API. [1] A timestamp is useful context, but it is not a universal sequence number supplied by the network.

Walk the deliveries through both interpretations

The naive handler starts with a pending booking. On evt-B it writes paid and sends a welcome message. On evt-A it writes pending, because that is the latest message it has received. On the repeated evt-B it sees another local pending-to-paid transition and sends again. No second payment occurred in the fixture. The delayed status write created a second local transition for the same confirmation obligation.

The alternative fixture checks the event identity, retrieves the current object for a previously unseen event and projects that state. The first lookup says paid, so it records the paid booking and the welcome-action identity. The delayed evt-A causes another lookup, still paid, so the booking remains paid and the action identity already exists. The repeated evt-B is recognised as a duplicate delivery and does no new work.

The downloadable program follows both interpretations through those three deliveries. The naive path ends with two simulated sends; the guarded path retains one welcome-action identity. Look at the intermediate pending write as well as the final paid status. A check of the final status alone would approve both paths and miss the duplicate welcome. Keeping the action history beside the booking exposes the difference without mistaking a simulated send for a message delivered through a provider.

The same three deliveries produce two different stories

A synthetic pending/paid example contrasts overwriting from each arriving snapshot with projecting the current object and retaining one welcome-action identity.

Scroll sideways to see all columns.

ArrivalSnapshotNaive booking / sendsCurrent-state projection / actions
1 · evt-Bpaidpaid · 1 welcomepaid · 1 action identity
2 · evt-Aearlier pendingpending · 1 welcomepaid · 1 action identity
3 · evt-B againpaid duplicatepaid · 2 welcomespaid · 1 action identity; duplicate ignored
Deterministic synthetic fixture output, not a real provider incident. The fixture models decisions in memory; durable receipt, serialisation and side-effect handling still require implementation.

A current lookup can still become an old write

A fresh lookup removes one source of stale information, but the result still has to survive the trip into the database. Worker A can fetch a state and pause while B reads a newer one and commits. A then resumes and overwrites B. Both workers have diligently consulted the provider; their writes have recreated the original problem locally. The observation was current when A read it, which is a narrower promise than being current when A finally uses it.

One design is to process changes for the same business object serially, holding an appropriate per-object processing claim while fetching and updating its projection. Another can use a trustworthy provider revision together with a conditional local update, if the provider actually supplies the required revision semantics. Do not invent such a revision from event time. The right mechanism depends on the provider contract and the database; PostgreSQL’s isolation documentation describes the concurrent phenomena local transactions must address. [2]

Even serial local processing does not freeze the provider. Its object can change after a lookup, and another notification may be needed to learn that. State the freshness the application can offer and add an independent reconciliation route for persistent mismatches. A remote read supplies evidence for the projection at a particular time; subsequent observations still need to update it under the same concurrency rules.

A fresh lookup still needs a guarded write

Two local workers can reverse the order of observations even when both retrieve provider state.

Follow the labelled arrows. On a narrow screen, scroll sideways if needed.

A fresh lookup still needs a guarded writeA reads, then pauses. A retains an older provider observation while another local worker can make progress. B reads newer state. B can retrieve newer state and commit its projection before A resumes. A resumes with the old observation. An unconditional write can overwrite B’s newer projection even though both workers fetched provider state. Coordinate per-object work. Hold the appropriate serial-processing claim while fetching and projecting the object. Use a trustworthy provider revision. Where the provider documents suitable revision semantics, make the update conditional and reject an older projection. Remote state can still change. Local serialisation does not freeze the provider. Reconcile subsequent observations. From A reads, then pauses to A resumes with the old observation: Retained observation. From B reads newer state to A resumes with the old observation: Newer projection committed first. From A resumes with the old observation to Coordinate per-object work: Coordinate local work. From A resumes with the old observation to Use a trustworthy provider revision: If supported by the provider. From Coordinate per-object work to Remote state can still change: Does not freeze remote state. From Use a trustworthy provider revision to Remote state can still change: A read is still an observation.Retained observationNewer projectioncommitted firstCoordinate local workIf supported by theproviderDoes not freeze remotestateA read is still anobservationA reads, then pausesA retains an older providerobservation while anotherlocal worker can makeprogress.B reads newer stateB can retrieve newer stateand commit its projectionbefore A resumes.A resumes with the oldobservationAn unconditional write canoverwrite B’s newerprojection even though bothworkers fetched providerstate.Coordinate per-objectworkHold the appropriateserial-processing claim whilefetching and projecting theobject.Use a trustworthyprovider revisionWhere the provider documentssuitable revision semantics,make the update conditionaland reject an olderprojection.Remote state can stillchangeLocal serialisation does notfreeze the provider.Reconcile subsequentobservations.
This concerns projection concurrency, separately from event deduplication and welcome-action identity.

Paid is not the whole membership policy

A refund or cancellation complicates the apparently simple instruction to read the current state. A partial refund may leave access intact. A full refund may have a separate cancellation or grace policy. A welcome already sent does not become unsent when a financial state changes. Model those consequences explicitly rather than allowing every new status to repeat or reverse every action attached to an older status.

For this booking, the welcome is a one-time confirmation action under a particular policy version. A later refund receipt or access notice is a different action, rather than another chance to run the welcome code. Its record should name what made it due, which object supplied that information and whether the send finished or remains uncertain. That gives the operator a customer story they can follow, instead of asking them to infer one from a succession of paid and pending labels.

AWS’s idempotent-API discussion explains why a request identity and its mutating effects need careful handling together. [3] A durable welcome-action record is useful, but recording intent before a send and completion after it still leaves an uncertain interval if the process stops. Use the actual mail provider’s idempotency or lookup behaviour where available. Otherwise, define reconciliation or manual handling; a database uniqueness constraint alone cannot make an arbitrary remote send exactly once.

Test the explanation you will give the customer

Start with the three-delivery fixture, then vary one condition at a time. Deliver the same event twice concurrently. Reverse two different events with equal timestamps. Pause a worker after lookup but before its local write. Stop after the external message is accepted but before completion is recorded. Each case tests a different relationship, and each should leave an interpretable state rather than merely a green handler response.

The operator view needs to keep a received notification separate from completed customer work. A durable inbox entry may still await interpretation, and a locally confirmed booking may still have an uncertain welcome send. Those distinctions let a colleague answer the customer without reading accepted as finished. They also leave room for the reception path’s own duties: follow the provider’s rules for signature checks, durable recording and acknowledgement before treating a notification as safely received.

There is a useful check that starts outside the event stream: compare the paid bookings with the confirmations or access the policy says they should have. That can find work for which no notification reached the handler at all. Review the discrepancies rather than replaying everything. In the three-delivery example, the paid booking needs to stay paid, while an uncertain welcome needs investigation of its own. A single status cannot describe both outcomes without hiding something the next person still has to do.

Working on a workflow like this? Hamsa helps finance and operations teams connect their tools and handle the exceptions, starting with one workflow.

See the Payment Operations Pilot

References & further reading

Primary sources behind this article. Living documentation may change.

  1. Stripe: webhook ordering, duplicate delivery and event handling
  2. PostgreSQL: transaction isolation and concurrent updates
  3. AWS Builders’ Library: idempotent requests and side effects
Hamsa Jama

Practical engineering. A human perspective.

More about Hamsa Software and the person behind it. Editorial notes.

A few more things
worth a read.

Back to the reading room
Integrations7 min read

A valid signature still needs a trust decision

An Oprindo sample had a matching asset hash and a valid signature, alongside an untrusted credential result. The useful story is in those separate answers.

Integrations6 min read

Should this old instruction still run?

Repairing a consumer makes replay possible. Current state, dependencies and recovery admission decide whether it is still appropriate.

Integrations7 min read

The backfill arrived after the live update

Historical work needs a population boundary, revision policy and crash-safe checkpoint strategy before its final page can mean done.