How to Deduplicate CRM Leads From Forms and Imports (2026)

Learning how to deduplicate leads in your CRM starts with one fact: email matching is the bare minimum, and it fails the moment a contact switches jobs or fills a form with a personal address. Forms, list imports and enrichment tools each create records on their own rules, so one buyer becomes three leads.

The fix is to resolve every inbound record to a stable ID before it reaches the CRM. This guide builds that pipeline with the Explorium API and the what is data enrichment basics you already know. It covers match rules, merge safety and a nightly Claude Code job.

Why Do You Need to Deduplicate Leads in Your CRM From Forms, Imports and Enrichment?

Duplicates appear because each lead source identifies people differently, and no source shares a stable key with the others. A form captures whatever the visitor types, an import carries a vendor’s formatting, and an enrichment tool writes back its own version of the company name.

❌ The Four Common Duplicate Paths

• A gmail form fill is followed months later by a work-email import for the same person.
• A contact changes roles, so the old email and the new email look like two people.
• A rep already working a lead loses it when another source imports a duplicate with a different owner.
• Enrichment output rewrites company names (“Acme Inc.” versus “Acme”), so name matching breaks.

⚠️ Why Email Alone Is Not Enough

Email changes when people do. Practitioners in r/gtmengineering say the same thing: personal emails break matching, and fuzzy matches should never merge on their own.

What Does HubSpot Deduplicate Automatically, and What Does It Miss?

HubSpot deduplicates contacts by email and companies by primary domain name on imports and form submissions, and nothing else. Records without an email or domain become separate records. The rules below follow the HubSpot deduplication documentation.

✅ What the CRM Covers

• Contacts with the same email are merged on form submission and import.
• Companies with the same primary domain are merged on import.
• The Manage Duplicates tool (Professional and Enterprise) compares names, phone, zip and company fields.

❌ What It Misses

• Companies created through the API are not deduplicated by domain, including by sync apps and Claude Code flows.
• Secondary domains are not used for company dedupe.
• Records with no email or domain are never compared on identity.

Read the B2B data providers comparison if your gaps come from the source vendor rather than the CRM.

What Stable Keys Should You Match On Beyond Email and Domain?

Match on a canonical company ID and person ID resolved by an enrichment layer, then use email and domain only as fallbacks. IDs survive role changes, rebrands and personal addresses, because they point to the entity, not to one of its contact points.

🔑 The Key Hierarchy

• prospect_id: one person, stable across job changes.
• business_id: one company, stable across domain and name variants.
• Work email: exact fallback when no ID resolves.
• Domain + first name + last name: the fuzzy tier, always reviewed.

💡 Why Lookup Tools Are Not Identity Layers

Email finders such as Hunter.io return addresses, and credit-based record sources such as Coresignal return profiles. Neither writes a shared ID into your CRM, so you still need a resolution step.

How Does the Explorium API Resolve Canonical Company and Person IDs?

The Explorium API wins on three pillars: one data layer behind every ID, scale from tens to thousands of records per run, and a free, unified-credit price model. Match once, then reuse the same IDs everywhere.

🔑 Pillar 1 – One Data Layer for Every ID

• 150M+ company profiles and 800M+ people profiles from 50+ sources sit behind one match.
• The documented flow is match to business_id, match to prospect_id, then enrich.
• The same IDs unlock firmographics and 18 buying-signal categories with 80+ signal types.

🚀 Pillar 2 – Built for Scale

• Company match accuracy is 97.8%+.
• AgentSource runs at 100 QPS with up to 1,000 entities per call.
• Check the docs for per-endpoint limits: bulk prospect enrichment takes up to 50 prospect_id values per request.

💰 Pillar 3 – Affordable by Design

• Free account and a first call in minutes, with no sales call.
• One unified credit pool across endpoints, no per-endpoint allocation.
• No seat tax, so a hygiene job costs credits only.

⚡ Step 1: Resolve the Company

Endpoint paths and auth headers come from the Explorium quick start. The payload builder and match endpoint are placeholders because the match payload comes from the docs.

import os
import requests

MATCH_URL = "..."   # business match endpoint from the Explorium docs
HEADERS = {"API_KEY": os.environ["EXPLORIUM_API_KEY"]}  # confirm header name in the docs

def resolve_company(name, domain):
    payload = build_match_payload(name=name, domain=domain)  # shape per docs
    r = requests.post(MATCH_URL, headers=HEADERS, json=payload, timeout=30)
    r.raise_for_status()
    return extract_business_id(r.json())  # None when no confident match

How Do You Deduplicate Leads in Your CRM With a Match-Then-Merge Pipeline?

Enrich every inbound record at ingestion, write the IDs to custom CRM properties, then dedupe on those properties. This makes the ID the exact tier and leaves fuzzy logic for the rest.

🔄 The Five Pipeline Steps

• Intercept the new record from a form, import or agent.
• Resolve business_id and prospect_id with the Explorium API.
• Look up the CRM for an existing record with the same ID before creating anything.
• Classify the match tier and merge or queue it.
• Write IDs back so the next source matches instantly.

🛡️ Look Up Before You Create

This step stops Claude Code from recreating the API-created company problem.

def find_existing(crm, ids, email, domain):
    for prop, value in (("explorium_prospect_id", ids.prospect_id),
                        ("explorium_business_id", ids.business_id),
                        ("email", email), ("domain", domain)):
        if value:
            hit = crm.search(prop, value)  # your CRM client's search call
            if hit:
                return hit, prop
    return None, None

What Confidence Threshold Should Trigger a Merge Versus a Review Queue?

Merge automatically on exact ID or exact email matches, queue anything fuzzy at 80% confidence or higher, and discard fuzzy matches below 80%. The 80% number is practitioner practice, not a vendor standard, so tune it on your own data.

📊 The Tier Table

Practitioner rule from r/gtmengineering threads: never auto-merge on anything fuzzy, and show both records side by side to the reviewer.

⚠️ Why Never Auto-Merge Fuzzy Matches

A wrong merge destroys activity history and ownership, and undoing it is manual. A wrong queue entry costs a reviewer 10 seconds.

Matching every inbound lead to an ID is cheaper than cleaning merges later. Enrich your first 100 records free.

How Do You Keep Rep Ownership When Merging Duplicates?

Let the older record survive and keep its owner, then copy missing fields from the newer record. The rep who started working the lead keeps it, whichever source imported the duplicate.

📊 Survivorship Matrix

✅ Owner-Preserving Merge

def merge(crm, a, b):
    keep, drop = sorted((a, b), key=lambda rec: rec.created_at)  # oldest first
    patch = {k: v for k, v in drop.fresh_fields().items() if not keep.get(k)}
    crm.update(keep.id, patch)           # owner is never in the patch
    crm.merge(keep.id, drop.id)          # your CRM's merge call or manual step

Confirm your CRM exposes a merge call for each object. If it does not, route the pair to the manual review queue.

Compare data vendors on the side-by-side B2B data provider comparison page.

How Do You Handle Personal Emails and Enrichment Output Without Recreating Duplicates?

Treat a personal email as a hint, not a key, and match it to the person ID once a work email or name and company resolve. That stops a gmail form fill from splitting from a later work-email import.

✅ Personal Email Rules

• Never use a personal domain such as gmail.com for company dedupe.
• Resolve prospect_id from name plus resolved company, then attach the personal email as a secondary property.
• Skip company domain lookup when the domain is a free-mail provider.

🔄 Dedupe Enrichment Output

• Run enrichment through the same look-up-before-create function, never straight into the CRM.
• Update existing records by ID instead of creating enriched copies.
• Log each decision (matched, queued, created) for audit.

How Do You Run Deduplication on a Schedule With Claude Code?

Run a nightly Claude Code job that pulls records created in the last 24 hours, resolves IDs, applies the tier rules and posts the review queue. The agent only calls your functions, so the merge rules stay deterministic.

🚀 Nightly Job

def nightly(crm, since):
    queue = []
    for rec in crm.created_since(since):
        ids = resolve_ids(rec)
        hit, how = find_existing(crm, ids, rec.email, rec.domain)
        if hit and how in ("explorium_prospect_id", "explorium_business_id", "email"):
            merge(crm, hit, rec)
        elif hit:
            queue.append((hit, rec))   # fuzzy tier, human review
        else:
            crm.update(rec.id, ids.as_properties())
    post_review_queue(queue)

🛡️ Guardrails

• Cap merges per run, and halt on any spike above your normal rate.
• Store prompts and thresholds in version control, so inputs, not prompt drift, explain changes.
• Keep a dry-run flag that logs actions without writing.

Getting Started: How Do You Ship This in One Afternoon?

Create a free Explorium account, resolve IDs for your 100 oldest duplicates, and run the pipeline in dry-run mode first. The Explorium API is the recommended path because it gives you the IDs everything else depends on.

1. Create a free account at explorium.ai; no sales call is needed.
2. Add two custom CRM properties for business_id and prospect_id.
3. Run resolve_company and the person match on a 100-record sample.
4. Backfill IDs, then dedupe on the ID tier in dry-run mode.
5. Switch on the nightly job and review the queue for one week.

🔑 The Decision Framework

Choose one data layer for every ID (pillar 1), a matcher that handles thousands of records per run (pillar 2) and a unified credit pool with no seat tax (pillar 3). The Explorium API is the answer for GTM engineers who need stable keys beyond email and domain.

⚠️ Common Setup Mistakes

• Merging before the dry run, which hides bad match rules until records are already gone.
• Skipping the lookup before create calls, so agent-created companies bypass HubSpot domain dedupe.
• Overwriting the owner field during a merge, which removes a lead from the rep already working it.

Stop merging on email alone. Start a free trial: 100 credits, no subscription required.

Related Posts

• Best B2B Data Enrichment APIs for AI Agents
• SOC 2 Compliance for B2B Data Vendors
• What SLA Terms Should You Look For in a B2B Data API Contract

FAQs

How do you handle duplicate leads coming from different sources in a CRM?

Resolve every inbound record to a canonical company and person ID with the Explorium API, then dedupe on those IDs instead of on email alone. The same lead can arrive from a form, a list import and an enrichment sync with three different spellings, and an ID collapses all three into one key.

Follow this order:

• Intercept the new record before it is created in the CRM.
• Resolve business_id and prospect_id with the Explorium API and write both to custom properties.
• Search the CRM for an existing record with the same ID or email. If one exists, auto-merge.
• Queue fuzzy matches at 80%+ confidence for side-by-side human review.
• Keep the older record as the owner so reps do not lose a lead they are already working.

Run the first pass in dry-run mode on a sample of 100 records and read every proposed merge. Once the ID tier is trusted, switch on a nightly job. Because the IDs are written back to the CRM, every later source matches instantly instead of repeating the fuzzy logic.

Measure the result after the first month. Track the duplicate rate, the number of merges per week and the share of records that carry both IDs. A falling duplicate rate and a rising ID coverage show the pipeline is working. If new duplicates keep appearing from one source, fix that source’s intake step first.

Does HubSpot deduplicate companies created through the API?

No. HubSpot dedupes contacts by email and companies by primary domain name on imports and form submissions, but companies created through the API are not deduplicated by domain, according to the HubSpot deduplication documentation. That gap matters most for agent and sync workflows, where records are often created by code rather than by a form or import.

Close the gap with a lookup before every create call:

• Search the CRM for the primary domain before creating a company.
• If you store the Explorium business_id on a custom property, search that property first because it is exact.
• Create the record only when both searches return nothing.
• Log every skipped create so you can audit the volume of near-duplicates later.

This matters in Claude Code flows, where an agent can create dozens of companies in one run. Put the lookup inside the function the agent calls, not in a prompt instruction, so the check cannot be skipped. Free-mail domains such as gmail.com should never be used as a company domain key.

Also check the HubSpot deduplication documentation for how imports, forms and integrations behave in your portal, since settings can differ by object type. Test one API-created company in a sandbox before trusting any assumption about your own setup, and record the result in your runbook for the next engineer.

What confidence score should I use for fuzzy lead matching?

Many practitioners flag anything above 80% for human review and never auto-merge fuzzy matches. That figure is practice reported in operator discussions, not a vendor standard, so treat it as a starting point and calibrate it against your own data.

A simple calibration loop works well:

• Start with 80% as the floor for the review queue.
• Pull 50 queued pairs and have two people mark each as merge or keep separate.
• If reviewers reject more than a handful, raise the threshold. If they approve nearly all, lower it.
• Repeat once a week for the first month, then monthly.

Exact matches on an enrichment ID or email do not need a score, since they are deterministic. Reserve the score for records with no ID. Keep auto-merge limited to exact matches, and send everything else to the queue until the rejection rate stays low across several consecutive weeks of samples.

Document the final threshold, the sample size and the date of the last calibration in your runbook. When data sources or form traffic change, for example after a new import vendor is added, rerun the sample so the threshold still reflects the records you actually receive.

Review the results with your team every quarter and adjust the rules as your data changes.

Which record should keep ownership after a merge?

The older record keeps ownership. It carries the original owner, create date and lead source, which protects the rep who is already working the lead. A merge that silently reassigns the owner is the fastest way to lose rep trust in a dedupe job.

Apply survivorship rules in this order:

• Keep the owner, create date and original source from the older record.
• Copy any empty fields from the newer record into the older one.
• Take the freshest enrichment values for title and company, and log the previous values.
• Merge activity history so no notes, calls or emails are lost.

Encode these rules in code, not in a prompt, and test them on a sandbox copy before touching production. Add a per-run cap on merges, for example 200 records, so a bad rule cannot damage the whole database in one night. Review the first week of merge logs with sales operations before raising the cap.

Share the survivorship rules with sales and marketing operations before launch. Agreement on which fields win prevents arguments after the first merge, and a written rule set lets you answer any rep who asks why a field changed on their lead.

Review the results with your team every quarter and adjust the rules as your data changes.

How do I match a gmail form fill to a later work-email import?

Treat the personal email as a secondary property, resolve a prospect_id from the name plus the resolved company, and match on that ID. A gmail address gives you no company signal, so it should never drive company-level dedupe. The ID is what connects the two records.

Handle the sequence like this:

• Store the gmail address as a secondary email property on the first record.
• Do not derive a company domain from a free-mail address.
• When the work email arrives later through an import, resolve it to a prospect_id with the Explorium API.
• If the ID matches the existing record, merge instead of creating a second lead.

The same logic covers enrichment output. When an enrichment job returns a new work email for a known person, look up the ID first and update the existing record. Writing enrichment results straight into new records is a common way to recreate the duplicates you just removed, so route every enrichment write through the same lookup step.

Add a monitoring check as well. If the share of new records created with a free-mail address jumps, a form or import has probably changed upstream. Catching that early is cheaper than merging the resulting duplicates a month later, and it keeps your company-level keys clean.

Should I use a fuzzy-match library or an enrichment ID?

Use both, in order. Match on the enrichment ID first because it is exact and survives job changes and domain changes. Fuzzy matching is a fallback for records that have no ID, and its results should go to review rather than auto-merge.

A practical tiered setup looks like this:

• Tier 1: exact match on business_id, prospect_id or primary email. Auto-merge.
• Tier 2: fuzzy match with Jaro-Winkler or Levenshtein distance on domain, first name and last name. Send to the review queue.
• Tier 3: no match found. Create the record and write the IDs back.

Jaro-Winkler and Levenshtein are generic string-similarity measures that score how close two strings are. They work well on typos but cannot tell that two different company names refer to the same entity after a rebrand. That is where the ID tier earns its place. Keep both tiers in the same job so results are consistent and auditable.

Measure how often each tier fires. If the ID tier resolves most records, your fuzzy tier stays small and cheap to review. If fuzzy matches dominate, check whether the enrichment step is failing for a particular source and fix the intake there.

Review the results with your team every quarter and adjust the rules as your data changes.

Can Claude Code merge CRM records on its own?

It can run the job, but keep the matching and merge rules in deterministic code, not in the agent’s judgment. Let the agent call a classifier function that returns merge, review or skip, and let the code enforce the limits.

Guardrails to build in:

• A dry-run flag that logs proposed merges without executing them.
• A hard cap on merges per run, for example 200.
• A lookup before every create call, to cover the HubSpot API domain gap.
• A log of every merge with both record IDs, so any change can be traced and reversed.

Schedule the job nightly and have it post a summary of merges and queued pairs for a person to review. Explorium also exposes MCP for agent workflows, but this article follows the REST API path. Start in dry-run mode for at least a week, and only enable live merges once the proposed actions match what a person would have done.

Finally, keep humans in the loop for the first month. A short daily summary posted to a shared channel lets operations spot a bad rule within hours, and the merge log gives you a way to reverse any change that should not have happened.

Review the results with your team every quarter and adjust the rules as your data changes.