Duplicate protection has two jobs:

  1. Stop the same webhook delivery from being processed twice.
  2. Stop one business action from creating two downstream results.

A delivery ID addresses the first job. Idempotent writes and action records address the second. Small businesses need both when repeated events could create duplicate contacts, send two emails, assign duplicate tasks, or trigger payment-related work twice.

Score Your Readiness

Give the workflow one point for every statement that is true.

Readiness check Score 1 point when this is true
Delivery identity Each accepted webhook has a stable delivery ID, event ID, or composite key.
Durable storage Deduplication records are kept in persistent storage rather than browser, session, or temporary memory storage.
Early recording The event key is claimed before the workflow performs an irreversible action.
Duplicate response A repeated delivery is recognized, logged, and stopped without another result.
Destination protection CRM and database writes update or upsert records using a stable external identifier.
Action-level protection Messages, invoices, payments, documents, and fulfillment actions use their own idempotency or action key.
Retry coverage Records remain stored longer than all retry, queue-delay, and approved replay windows.
Ordering protection The workflow can reject or safely handle stale updates when record order matters.
Replay procedure A failed event can be replayed without bypassing deduplication records.
Ownership and visibility One person owns exceptions, and duplicate decisions appear in logs or workflow records.
Score Readiness level Use the workflow for
0–3 High duplicate risk Manual review or internal testing only. Add durable event storage and destination-side uniqueness before customer-facing actions.
4–6 Basic delivery protection Low-impact internal alerts and task updates, with duplicate events visibly logged.
7–8 Strong operational foundation Many CRM updates, lead-routing flows, and internal notifications. Review replay handling and ordering before expanding it.
9–10 Well-controlled workflow Workflows protected across retries, destination writes, and downstream side effects. Revisit the design after CRM, integration, or retry-policy changes.

Three gaps should stop a workflow from handling customer messages, invoices, payments, or fulfillment:

  • No stable event or action key
  • No durable record written before the downstream action
  • A destination action that creates a new record every time it runs

A high score elsewhere does not offset any of those failures.

Use Three Layers of Protection

1. Delivery-level deduplication

Store the webhook provider’s event or delivery ID before processing it. When that ID arrives again, stop it before it creates another action.

This blocks exact redelivery of the same webhook. It does not block two separate event IDs that represent the same lead submission or CRM change.

2. Business-level idempotency

Use a stable key for the destination action. Useful keys include a form submission ID, external customer ID, order ID, appointment ID, or CRM object ID paired with a version number.

This handles separate deliveries that represent the same logical action. It is also useful during migrations, queue reprocessing, and approved manual replays.

3. Destination-side uniqueness

Configure the destination to update an existing record when the same external identifier appears. A workflow that creates a new contact, account, or transaction every time it runs turns a delivery issue into a data-cleanup issue.

For actions outside the destination database, keep a separate action record. An upsert can prevent duplicate contacts while two welcome emails still go out unless the message action has its own protection.

Choose a Key That Matches the Action

Method Prevents Best fit
Event ID cache Exact redelivery of the same webhook Internal alerts, simple CRM updates, task creation
Object ID plus version or timestamp Repeated processing of the same record state and stale updates Deal stages, ticket updates, appointment status changes
External business ID plus upsert Duplicate contacts, accounts, and transaction records Lead intake, contact synchronization, account updates
Action ledger Repeated completion of a specific downstream action Messages, documents, payments, and fulfillment handoffs
Payload hash Repeated identical payloads during a defined period Simple form submissions without a stable source ID

A CRM contact ID is not a universal deduplication key. One contact can have many valid changes over time. Suppressing every later event for that contact would block useful work.

Pair the object ID with an event type, record version, state transition, action name, or submission ID. For example:

  • deal_1042 : stage_changed : qualified
  • lead_submission_48291 : send_welcome_email : completed

The first identifies a business transition. The second records a completed side effect.

Patterns for Common CRM Workflows

New lead capture

Use the original submission ID when the source provides one. Create or update the CRM contact through an external identifier, then record the welcome-email action separately when the workflow sends one.

This suits teams that need clean contact records and consistent follow-up. A contact upsert prevents duplicate records, but not duplicate welcome messages.

Deal-stage notifications

Use the CRM object ID, event type, and target stage or version. Send a notification for a meaningful stage transition rather than every update to the deal.

Do not key the rule only to the deal ID, because that suppresses later stage changes. A key based only on the stage can repeat a message when a deal moves backward and later returns to the same stage. Include version or transition context.

Appointment and service updates

Use the appointment ID plus a revision number or modified timestamp when the scheduling system provides one. Store the latest processed revision so an older delayed event cannot overwrite a newer status.

This approach fits confirmations, reschedules, cancellations, and staff assignments. Timestamp comparisons require consistent time zones and timestamp precision.

Payments, invoices, and fulfillment

Use a durable action ledger and a destination-side idempotency key. Keep financial and fulfillment actions outside a simple trigger-to-action flow unless retries can safely return to the same action record.

The workflow needs to distinguish between a new action, an action in progress, a completed action, and a failed action before trying again. Businesses without a documented owner and recovery procedure should keep these actions under manual control.

Claim the Record Before the Side Effect

A short-lived event ID cache can stop repeated delivery of the same webhook in low-impact workflows. Its limits are important: it cannot identify different event IDs that represent the same business action, and it cannot fully protect a workflow that fails after creating a downstream result but before recording completion.

An action ledger provides stronger control. Keep a compact record containing:

  • Deduplication key
  • Event type
  • Processing status
  • Created and updated timestamps
  • Destination record or action reference
  • Failure reason when applicable

Useful statuses include received, processing, completed, failed, and duplicate.

Keep raw webhook payload retention separate from deduplication retention. A compact record of keys, statuses, timestamps, and destination references avoids storing every raw payload when it is not needed for duplicate handling.

Concurrent deliveries need an atomic claim. If two processes both look for a key before either saves it, both can continue. Use a unique database constraint, atomic insert operation, queue-level deduplication feature, or transactional inbox pattern. Only one process should claim a new key; the other should receive a duplicate result.

Set Retry, Replay, and Ordering Rules

Keep deduplication records longer than the longest retry, queue-delay, and approved replay period across the workflow. Include the CRM, middleware, queue, destination API, and staff recovery process when setting retention.

Some webhook senders expect a quick success response. A slow workflow can trigger retries even when downstream work eventually succeeds. Where possible, durably record the incoming event, return the appropriate response, and process the work from that record.

When a destination API rate-limits a request, retry with the same destination idempotency key. Creating a new key for every retry recreates the duplicate risk.

Do not assume webhooks arrive in event order. A delayed “stage changed to qualified” event can arrive after “stage changed to closed lost.” Use version numbers, sequence values, or last-modified timestamps when the workflow must reject stale updates.

Webhook signatures and timestamp validation protect against unauthorized requests. They do not deduplicate valid events. Keep authentication, replay protection, and duplicate suppression as separate controls.

Before You Enable Automation

Use this checklist before moving a workflow beyond low-impact internal work.

  • Every accepted event has a stable delivery ID, business ID, or defined composite key.
  • Records are stored outside temporary browser, session, and in-memory storage.
  • The key is claimed before any irreversible side effect.
  • The destination updates by external ID or accepts an idempotency key.
  • Emails, texts, payments, invoices, documents, and fulfillment actions have separate action-level protection.
  • Retention exceeds the longest retry, queue-delay, and replay period.
  • The workflow handles stale updates when event order matters.
  • Duplicate events create a log entry with the event key and disposition.
  • Failed events have a replay path that respects deduplication records.
  • One named owner can pause the flow, inspect failures, and approve replays.

After a major change, send a controlled duplicate delivery in a non-production workspace. The desired outcome is one accepted business action and one recorded duplicate decision, not two destination records followed by cleanup.

Bottom Line

For straightforward lead capture and internal CRM alerts, begin with durable event ID storage, destination upserts, and a defined retry window. That combination blocks many duplicate records and repeat notifications.

Appointment changes, customer messaging, invoices, payments, and fulfillment need action-level idempotency plus a replay process that respects completed actions. Start with low-impact workflows, record duplicate decisions, and expand only after repeat deliveries create one predictable business result.

FAQ

What is the difference between webhook deduplication and duplicate contact prevention?

Webhook deduplication stops the same event from being processed more than once. Duplicate contact prevention stops the CRM from creating multiple records for the same person or business. A safe workflow uses both.

How long should deduplication records be stored?

Store them longer than the longest retry, queue delay, and approved replay window across the full workflow, including staff recovery procedures.

Is a timestamp enough to deduplicate CRM webhooks?

No. Multiple valid CRM events can share the same timestamp resolution, and delayed events can arrive after newer events. Pair timestamps with an event ID, object version, sequence value, or action key.

Should duplicate webhooks return a success response?

A duplicate webhook should normally receive a successful response after the system confirms that the event was already processed or safely recorded. Returning an error can encourage more retries.

Do small businesses need a database for webhook deduplication?

Reliable deduplication needs durable storage across retries, restarts, and concurrent deliveries. That can be a database table, managed key-value store, or an integration-platform feature that persists idempotency records. Temporary in-memory storage does not survive a restart.