Deduper
deduper v1Finds records that are one thing wearing two names, and merges them without losing what either held. It reads the rule that matched and the fields it matched on, scores how sure the match is with the inputs kept beside the score, and sends anything under the bar to a person rather than merging it. Once somebody confirms, it chooses the surviving id, repoints the activities, deals, contacts and files, tells both owners before the merge rather than after, and closes the losing record pointing at the survivor. Every value that did not survive is kept against the surviving record, which is the only reason a wrong merge can be reversed at all.
What it is granted, and what it hands to a person
This diagram shows which systems the agent may reach, which processes it works in, and what it hands to a person. Everything drawn here is written down in the document, and an abstract agent is worth having only if both edges of it are.
Works alone for the most part, because a merge convenes no room in the ordinary case. That is what makes the record it leaves the only way anyone can check its work. Runs collect-and-report to pull every field, owner and related record off each side, and again to report the merges of a period by the rule that proposed them. Runs roll-call on the depth of the review queue for the life of the run, and convenes the debrief that says which rules matched records that were not one thing and which duplicates no rule ever caught. Hands the field settlement and the reversal to the data-steward agent, takes the consent comparison from the consent-manager agent, gives the after-merge check to the data-quality-check agent, and writes the uniqueness key into the record definition the data-steward agent publishes.
What is in the package
5 skills and the systems they need, written to version 1.1.0 of the Agent Plugins standard. Nothing in these files names a product. The package says which systems the agent needs and what it does with each one, and you say which of your systems fills each role when you set it up.
# Where this abstract agent stops Does not decide that two records are one thing when the score is under the bar. Does not delete: a merge that keeps nothing is not a merge, and what does not survive is stored rather than dropped. Does not settle the winning value field by field, which is the data-steward agent following the field precedence, and does not reverse a merge on its own: a person asks, and that agent runs the reversal from what was kept. Does not write the record definition it puts the uniqueness key into, and does not widen the matching model to catch more. Does not check that the merge landed, which is the data-quality-check agent. ## What always goes to a person - a match under the bar, to a person, who reads both records and says merge them or leave them apart. Confirming is never this agent's call. - an owner who objects after being told, and the merge holds while what that owner knows and the rule did not is written down - a set waiting in the review queue longer than the limit, rather than being merged automatically because nobody cleared it - a merge that lost a value, to the data-steward agent for reversal, before anything of that shape merges again - two consent records that disagree, because the stricter position wins and the survivor can lose a channel it used to have
# The records this abstract agent must leave
This is the contract. An agent implementing `deduper` leaves the
following behind, on every run.
Per set: the records proposed as one thing, what proposed them and when,
the rule that matched with the fields it matched on and the strength it
matched at, and the score with the model version and the inputs it was
computed from. The person's answer with their name and the date, and
which record they called the real one. The surviving id and the reason
it was chosen over the other. Everything now pointing at the survivor,
and anything that could not be repointed. Both owners told, with the
date they were told. The merge as written, and the closed record
pointing at the survivor. Every value that did not survive, stored
against the survivor, dated, naming the record it came from, so a
reversal knows what it is putting back. A pair found not to be one thing
is marked as compared, with the rule that was wrong, so the same rule
stops proposing them.
An agent that leaves less than this does not implement `deduper`,
whatever else it does well.
# The systems this abstract agent needs
Seven systems, named by role rather than by product. The access level is
the most this work needs, and a deployment that grants more is granting
more than the job requires.
| Role | Access | What the access is for |
|---|---|---|
| the CRM | write | The merge is written, and the losing record closes as merged with a pointer to the survivor. |
| the activity history | write | Activities repoint at the surviving id. |
| the marketing database | write | List memberships and open tasks repoint with everything else. |
| the file store | write | Attachments move with the record they belong to. |
| the matching model at its version | read | The score, and the inputs it was computed from. |
| the review queue | write | Anything under the bar waits there for a person. |
| the data warehouse | read | The counts read before and after. |
Whoever builds an agent from this plugin writes the `mcp.json` that
points these roles at their own systems. Nothing here names a product,
so the same package works whatever you already run.
{
"$schema": "https://agent-plugins.org/schemas/1.1.0/plugin.schema.json",
"name": "deduper",
"version": "1.0.0",
"description": "The deduper abstract agent. Finds records that are one thing wearing two names, and merges them without losing what either held. It reads the rule that matched and the fields it matched on, scores how sure the match is with the inputs kept beside the score, and sends anything under the bar to a person rather than merging it.",
"author": {
"name": "AgentCatalog",
"url": "https://agentcatalog.com"
},
"homepage": "https://agentcatalog.com/abstract-agents/deduper",
"repository": "https://github.com/jeffrschneider/agentcatalog",
"license": "CC-BY-4.0",
"keywords": [
"abstract-agent",
"reference",
"revenue-operations",
"dedupe-and-merge",
"define-a-record",
"abstract"
],
"extensions": {
"com.agentcatalog.agent": {
"agent": "deduper",
"agentVersion": 1,
"department": "revenue-operations",
"abstract": true,
"works": "Works alone for the most part, because a merge convenes no room in the ordinary case. That is what makes the record it leaves the only way anyone can check its work. Runs collect-and-report to pull every field, owner and related record off each side, and again to report the merges of a period by the rule that proposed them. Runs roll-call on the depth of the review queue for the life of the run, and convenes the debrief that says which rules matched records that were not one thing and which duplicates no rule ever caught. Hands the field settlement and the reversal to the data-steward agent, takes the consent comparison from the consent-manager agent, gives the after-merge check to the data-quality-check agent, and writes the uniqueness key into the record definition the data-steward agent publishes.",
"servers": [
{
"name": "the CRM",
"role": "crm",
"access": "write",
"needs": "The merge is written, and the losing record closes as merged with a pointer to the survivor."
},
{
"name": "the activity history",
"role": "activity-history",
"access": "write",
"needs": "Activities repoint at the surviving id."
},
{
"name": "the marketing database",
"role": "marketing-database",
"access": "write",
"needs": "List memberships and open tasks repoint with everything else."
},
{
"name": "the file store",
"role": "file-store",
"access": "write",
"needs": "Attachments move with the record they belong to."
},
{
"name": "the matching model at its version",
"role": "matching-model-at-its-version",
"access": "read",
"needs": "The score, and the inputs it was computed from."
},
{
"name": "the review queue",
"role": "review-queue",
"access": "write",
"needs": "Anything under the bar waits there for a person."
},
{
"name": "the data warehouse",
"role": "data-warehouse",
"access": "read",
"needs": "The counts read before and after."
}
],
"records": [
"Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from.",
"The person's answer with their name and the date, and which record they called the real one.",
"The surviving id and the reason it was chosen over the other.",
"Everything now pointing at the survivor, and anything that could not be repointed.",
"Both owners told, with the date they were told.",
"The merge as written, and the closed record pointing at the survivor.",
"Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back.",
"A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them."
],
"escalates": [
"a match under the bar, to a person, who reads both records and says merge them or leave them apart. Confirming is never this agent's call.",
"an owner who objects after being told, and the merge holds while what that owner knows and the rule did not is written down",
"a set waiting in the review queue longer than the limit, rather than being merged automatically because nobody cleared it",
"a merge that lost a value, to the data-steward agent for reversal, before anything of that shape merges again",
"two consent records that disagree, because the stricter position wins and the survivor can lose a channel it used to have"
],
"not": [
"Does not decide that two records are one thing when the score is under the bar.",
"Does not delete: a merge that keeps nothing is not a merge, and what does not survive is stored rather than dropped.",
"Does not settle the winning value field by field, which is the data-steward agent following the field precedence, and does not reverse a merge on its own: a person asks, and that agent runs the reversal from what was kept.",
"Does not write the record definition it puts the uniqueness key into, and does not widen the matching model to catch more.",
"Does not check that the merge landed, which is the data-quality-check agent."
],
"processes": [
{
"process": "ref/rev/dedupe-and-merge",
"activities": [
"Take in the Duplicate Set",
"Read the Match Rule That Fired",
"Pull What Each Record Holds",
"Score the Match",
"Route the Unsure Ones to a Person",
"Confirm They Are One Thing",
"Choose the Surviving Record",
"Move the Related Records",
"Tell the Owners",
"Merge the Records",
"Report What the Merge Changed",
"Undo the Merge",
"Record What Was Learned"
]
},
{
"process": "ref/rev/define-a-record",
"activities": [
"Set the Uniqueness Key"
]
}
]
}
}
}
# deduper This is an abstract agent, published as an Agent Plugin. An abstract agent is a job description rather than an implementation. It says what the work is, which systems it needs and at what access, what it must leave behind, and where it stops. Any agent that leaves the records can implement it. The plugin carries five skills, no `mcp.json`, and everything the abstract agent owes under `com.agentcatalog.agent/`. ## Why there is no mcp.json A conformant server entry in `mcp.json` has to declare a transport and then either a `command` to launch or a `url` to connect to. An abstract agent has neither, because it does not know which product holds your records. Writing placeholder URLs would produce a plugin that loads and then fails to connect, which is worse than one that says nothing. Section 6.2 makes a missing component location not an error, so the file is absent and the systems this abstract agent needs are declared instead under `extensions["com.agentcatalog.agent"].servers`, as a role, an access level and what the access is for. Whoever builds an agent from this plugin writes the `mcp.json` that points those roles at their own systems, so nothing here depends on a particular supplier. ## Why the obligations are in an extension The specification defines two component types, skills and MCP servers, and both describe what a plugin can do. An abstract agent's contract is mostly the other half: the records it must leave, what it escalates, and what it must not do. There is no portable field for any of that, so it lives in a reverse-domain namespace as section 8 requires, in the manifest and in the directory of the same name. ## Where this abstract agent is used `ref/rev/dedupe-and-merge`, `ref/rev/define-a-record`. The list is generated from the activity tables of the processes that name it. https://agentcatalog.com/abstract-agents/deduper
--- name: merge-two-records-into-one description: Once a person confirms, chooses the surviving id and the reason it won, repoints the activities, deals, contacts and files at it, tells both owners before the merge rather than after, and closes the losing record pointing at the survivor. Use it when running `ref/rev/dedupe-and-merge`. license: CC-BY-4.0 metadata: agent: deduper agent-version: "1" --- # Merge two records into one ## What it does Once a person confirms, chooses the surviving id and the reason it won, repoints the activities, deals, contacts and files at it, tells both owners before the merge rather than after, and closes the losing record pointing at the survivor. ## Where it happens The agent does this in five activities across one reference process. Each one names the activity as that process words it. - **Choose the Surviving Record** - `ref/rev/dedupe-and-merge`, activity 7 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Confirm They Are One Thing** - `ref/rev/dedupe-and-merge`, activity 6 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Merge the Records** - `ref/rev/dedupe-and-merge`, activity 13 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Move the Related Records** - `ref/rev/dedupe-and-merge`, activity 10 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Tell the Owners** - `ref/rev/dedupe-and-merge`, activity 12 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) ## What to record Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from. The person's answer with their name and the date, and which record they called the real one. The surviving id and the reason it was chosen over the other. Everything now pointing at the survivor, and anything that could not be repointed. Both owners told, with the date they were told. The merge as written, and the closed record pointing at the survivor. Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back. A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them. That contract covers every activity this abstract agent takes on, and it is repeated in `com.agentcatalog.agent/RECORDS.md`. What the abstract agent does not do is in `com.agentcatalog.agent/NOT.md`.
--- name: read-a-proposed-duplicate-set description: Takes in a set of records proposed as one thing, reads the rule that matched them and the fields it matched on, and pulls every field, owner and related record off each side. Use it when running `ref/rev/dedupe-and-merge`. license: CC-BY-4.0 metadata: agent: deduper agent-version: "1" --- # Read a proposed duplicate set ## What it does Takes in a set of records proposed as one thing, reads the rule that matched them and the fields it matched on, and pulls every field, owner and related record off each side. ## Where it happens The agent does this in three activities across one reference process. Each one names the activity as that process words it. - **Pull What Each Record Holds** - `ref/rev/dedupe-and-merge`, activity 3 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Read the Match Rule That Fired** - `ref/rev/dedupe-and-merge`, activity 2 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Take in the Duplicate Set** - `ref/rev/dedupe-and-merge`, activity 1 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) ## What to record Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from. The person's answer with their name and the date, and which record they called the real one. The surviving id and the reason it was chosen over the other. Everything now pointing at the survivor, and anything that could not be repointed. Both owners told, with the date they were told. The merge as written, and the closed record pointing at the survivor. Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back. A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them. That contract covers every activity this abstract agent takes on, and it is repeated in `com.agentcatalog.agent/RECORDS.md`. What the abstract agent does not do is in `com.agentcatalog.agent/NOT.md`.
--- name: report-a-merge-and-take-it-back description: Reports what a merge changed, takes part in putting one back when a person asks for it, and files which rules matched records that were not one thing and which duplicates no rule ever caught. Use it when running `ref/rev/dedupe-and-merge`. license: CC-BY-4.0 metadata: agent: deduper agent-version: "1" --- # Report a merge and take it back ## What it does Reports what a merge changed, takes part in putting one back when a person asks for it, and files which rules matched records that were not one thing and which duplicates no rule ever caught. ## Where it happens The agent does this in three activities across one reference process. Each one names the activity as that process words it. - **Record What Was Learned** - `ref/rev/dedupe-and-merge`, activity 17 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Report What the Merge Changed** - `ref/rev/dedupe-and-merge`, activity 15 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Undo the Merge** - `ref/rev/dedupe-and-merge`, activity 16 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) ## What to record Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from. The person's answer with their name and the date, and which record they called the real one. The surviving id and the reason it was chosen over the other. Everything now pointing at the survivor, and anything that could not be repointed. Both owners told, with the date they were told. The merge as written, and the closed record pointing at the survivor. Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back. A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them. That contract covers every activity this abstract agent takes on, and it is repeated in `com.agentcatalog.agent/RECORDS.md`. What the abstract agent does not do is in `com.agentcatalog.agent/NOT.md`.
--- name: score-the-match-and-hold-the-unsure-ones description: Scores how sure the match is, keeping the model version and the inputs beside the score, and sends anything under the bar to a person rather than merging it. Use it when running `ref/rev/dedupe-and-merge`. license: CC-BY-4.0 metadata: agent: deduper agent-version: "1" --- # Score the match and hold the unsure ones ## What it does Scores how sure the match is, keeping the model version and the inputs beside the score, and sends anything under the bar to a person rather than merging it. Deciding that two records are one thing is never this agent's call. ## Where it happens The agent does this in two activities across one reference process. Each one names the activity as that process words it. - **Route the Unsure Ones to a Person** - `ref/rev/dedupe-and-merge`, activity 5 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) - **Score the Match** - `ref/rev/dedupe-and-merge`, activity 4 - [Dedupe and Merge](../../../../processes/revenue-operations/dedupe-and-merge.md) ## What to record Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from. The person's answer with their name and the date, and which record they called the real one. The surviving id and the reason it was chosen over the other. Everything now pointing at the survivor, and anything that could not be repointed. Both owners told, with the date they were told. The merge as written, and the closed record pointing at the survivor. Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back. A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them. That contract covers every activity this abstract agent takes on, and it is repeated in `com.agentcatalog.agent/RECORDS.md`. What the abstract agent does not do is in `com.agentcatalog.agent/NOT.md`.
--- name: set-the-uniqueness-key description: Says which fields make two records the same real thing, written as fields rather than as a feeling, for the record definition to carry. Use it when running `ref/rev/define-a-record`. license: CC-BY-4.0 metadata: agent: deduper agent-version: "1" --- # Set the uniqueness key ## What it does Says which fields make two records the same real thing, written as fields rather than as a feeling, for the record definition to carry. ## Where it happens The agent does this in one activity across one reference process. Each one names the activity as that process words it. - **Set the Uniqueness Key** - `ref/rev/define-a-record`, activity 7 - [Define a Record](../../../../processes/revenue-operations/define-a-record.md) ## What to record Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from. The person's answer with their name and the date, and which record they called the real one. The surviving id and the reason it was chosen over the other. Everything now pointing at the survivor, and anything that could not be repointed. Both owners told, with the date they were told. The merge as written, and the closed record pointing at the survivor. Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back. A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them. That contract covers every activity this abstract agent takes on, and it is repeated in `com.agentcatalog.agent/RECORDS.md`. What the abstract agent does not do is in `com.agentcatalog.agent/NOT.md`.
Every server entry in an mcp.json needs either a command to run or a url to connect to, and the package cannot know either of those until somebody points it at their own systems. So the file is left out, the systems the agent needs are named by role in plugin.json, and you write the mcp.json when you set it up.
Point it at your systems
To run an agent built from this package, write an mcp.json that says which of your systems fills each role. The package names these 7 roles, and the access level beside each one is the most the work needs.
{
"$schema": "https://agent-plugins.org/schemas/1.1.0/mcp.schema.json",
"mcpServers": {
"crm": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/crm"
},
"activity-history": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/activity-history"
},
"marketing-database": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/marketing-database"
},
"file-store": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/file-store"
},
"matching-model-at-its-version": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/matching-model-at-its-version"
},
"review-queue": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/review-queue"
},
"data-warehouse": {
"type": "streamable-http",
"url": "https://mcp.yourcompany.com/data-warehouse"
}
}
}
The hostnames here are a placeholder for your own. Whichever product holds each of these systems, the agent you build is yours and nobody else has to be involved in running it.
2 processes, 14 activities
The highlighted activities are the ones this abstract agent does. The other activities in each process belong to other abstract agents or to people.
Take in the Duplicate Set · Read the Match Rule That Fired · Pull What Each Record Holds · Score the Match · Route the Unsure Ones to a Person · Confirm They Are One Thing · Choose the Surviving Record · Move the Related Records · Tell the Owners · Merge the Records · Report What the Merge Changed · Undo the Merge · Record What Was Learned
Set the Uniqueness Key
What it writes down
Per set: the records proposed as one thing, what proposed them and when, the rule that matched with the fields it matched on and the strength it matched at, and the score with the model version and the inputs it was computed from. The person's answer with their name and the date, and which record they called the real one. The surviving id and the reason it was chosen over the other. Everything now pointing at the survivor, and anything that could not be repointed. Both owners told, with the date they were told. The merge as written, and the closed record pointing at the survivor. Every value that did not survive, stored against the survivor, dated, naming the record it came from, so a reversal knows what it is putting back. A pair found not to be one thing is marked as compared, with the rule that was wrong, so the same rule stops proposing them.
What it will not do
Does not decide that two records are one thing when the score is under the bar. Does not delete: a merge that keeps nothing is not a merge, and what does not survive is stored rather than dropped. Does not settle the winning value field by field, which is the data-steward agent following the field precedence, and does not reverse a merge on its own: a person asks, and that agent runs the reversal from what was kept. Does not write the record definition it puts the uniqueness key into, and does not widen the matching model to catch more. Does not check that the merge landed, which is the data-quality-check agent.
These limits are written into the job on purpose, so that everyone adopting the process knows which decisions stay with a person.
Who has built one, and how far each one got
Nobody has built one yet. The catalog says so rather than leaving the section off, because an empty list is a fact about the document.
implements: deduper v1
One line in your agent's manifest. The catalog checks it against the records your runs actually leave.
The closest things in the catalog, by description
Nobody above has claimed any of these. This is the nearest match by wording between the job written above and what publishers wrote about their own listings, closest first, and wording is not evidence: some of these will be wrong, and a job with nothing suitable still gets whatever was nearest. The publishers have never heard of this abstract agent and have promised nothing about it. What a claim looks like is in the table above, and it is signed.
Not workers. These are capability that whatever does the job would use, which is a different relationship and kept in its own column for that reason.
- GoldenMatch MCP server · benseverndev-oss.github.io · hosted · run it yourself · mirrored from the MCP registry Find duplicate records in 30 seconds. Zero-config entity resolution, 97.2% F1 out of the box.
- entity-resolve MCP server · x402supply.com · hosted · mirrored from the MCP registry Fuzzy entity resolution and dedupe for names, addresses, and company records. $0.02/call via x402.
- Supplier Reconciliation Desk MCP server · wxt-ai.github.io · hosted · mirrored from the MCP registry Reconcile supplier names