Stratus Cloud Solutions Stratus Docs
Documentation / Data Cleanser User Guide / Advanced Reference

Duplicate Groups and Scans

Understanding how groups form and when they appear in Review prevents false expectations about chain duplicates, scan timing, and org sizing.

Grouping is not transitive

Records group only on exact equality of the composite duplicate key. Data Cleanser does not perform transitive closure.

If A and B share key X, and B and C share key Y, but A and C do not share a key, they remain in separate groups. Chain duplicates (A≈B, B≈C, A≉C) need a scenario that produces one shared key for all three — for example by matching on a single authoritative field.

  • Minimum group size: 2 records with the same non-blank key.
  • Review visibility: a group appears only when both roll-up counts exceed 1 (Accounts__c / Contacts__c / Leads__c and Duplicate_Records__c).

When groups appear in Review

While a scan is processing, the Review queue stays empty and shows “Processing — waiting for results…”. Groups appear after the scan completes and the scenario enters the active state — not incrementally as each batch chunk finishes.

How groups are built during a scan (per batch chunk)
  1. Compute duplicate keys for scanned records; skip records with blank keys.
  2. Attach to existing Duplicate_Group__c rows where Duplicate_Key__c matches.
  3. Query CRM for other records with the same key not in the current batch: Account/Contact up to 2000; Lead up to 1000 (IsConverted = false).
  4. Pair records within the batch that share a key but were not found in the database.
  5. Create a new group only when the member set size is greater than 1.

Junction (“shadow”) records link CRM records to groups: Duplicate_Account__c, Duplicate_Contact__c, Duplicate_Lead__c. Junction inserts are chunked at 198 rows per DML call.

Scan batch chain (same-object)
  1. BatchMassDeleteRecords — deletes existing duplicate groups for the object type.
  2. BatchUpdateDuplicateKey — scans CRM via dynamic SOQL, clears keys, recomputes keys, creates or extends groups, writes master scores. Patches only package fields to avoid touching Name / Person Account fields unnecessarily.
  3. On finish: clears Activating_Scenario__c, writes a Duplicate_Report__c snapshot.

Cross-object chain: delete cross-object groups → CrossObjectBatch (Contact pass, then Lead pass) → creates Cross_Object_Group__c and Cross_Contact__c junctions. Contact match query is capped at 5000.

Only one activation batch per object type runs at a time. Starting a new scan cancels other activating scenarios for that object.

Batch sizes and performance limits
Setting Value
Scan batch scope (fresh install default) 100 records
Scan batch scope (if setting missing) 2000 records
Mass merge batch 10 groups per batch job
Mass merge members evaluated per group 20 max
Manual merge losers per request 30 max
Junction record inserts 198 per DML chunk

Install seeds Merge_Display_Defaults__c batch size 100. Large orgs may need administrator review of batch capacity and Apex queue depth.

Scenario state flags

Stored on StratusDataShieldMaster__c.

Field Meaning
Activating_Scenario__c Batch scan in flight.
Activate_Scenario__c UI “Active” scenario.
EnabledStatus__c 'true' = scan completed; groups reviewable.
Batch_Job_Id__c AsyncApexJob Id during scan.
Duplicate_Groups__c / Duplicate_Records__c Roll-up counters on the scenario.
Default install scenarios (locked)

Seeded per object (Account, Contact, Lead) on package install.

Scenario Rules
1st Pass — Strict Name + Phone + City Exact
2nd Pass — Semi-Strict Name Fuzzy, Phone + City Exact
3rd Pass — Semi-Relaxed Name Exact
4th Pass — Relaxed Name Fuzzy

All ship with Locked__c = true and are inactive until started.

Data health formula (Home dashboard) 
cleanRecords = totalRecords - (duplicateParticipants - duplicateGroups)
health% = cleanRecords / totalRecords

Cross-object health: (totalLeads - crossGroups) / totalLeads

Color thresholds: green above 90%, blue above 82%, yellow at or below 82%.