PEP and Sanctions Screening (v.3)
Version 3 of the PEP and Sanctions screening system introduces significant improvements to accuracy and configurability. This article covers how the screening pipeline works, the available settings, and what changed from previous versions.
How Screening Works
The screening pipeline processes each entity through five steps:
- Preprocessing -- Normalize and clean entity names.
- Search -- Find candidate matches in PEP and sanctions databases.
- Filtering -- Remove low-quality matches.
- Scoring -- Calculate a final match score for each candidate.
- User Verification -- Present results for human review.
1.1 Preprocessing
Before searching, entity names are normalized to improve matching consistency. The following transformations are applied:
| Transformation | Example Input | Example Output |
|---|---|---|
| Accents removed | Rene Descartes | Rene Descartes |
| Dashes split into separate terms | Ann-Bertine | Ann Bertine |
| Non-alphanumeric characters removed | O'Brien | OBrien |
| Titles removed (people) | Dr. Jane Smith | Jane Smith |
| Legal forms removed (companies) | Acme Corp AS | Acme Corp |
| Multi-word legal forms removed | Grupo Mexico S.A.B. de C.V. | Grupo Mexico |
| Leading prefixes removed (companies) | The M/V Bella | Bella |
For companies, every combination of legal forms is tried, and the name as written is always kept, so a word that only looks like a legal form still matches. The legal forms come from the countries tied to the name. For the screened company, that is the country it is registered in. For a sanctions list entry, it is every country the list states for it, such as nationality, address, registration or flag. Both sides are stripped, so Acme Corp AS on a list registered in Norway also matches a screened Acme Corp.
A company name is also tried without the words written ahead of it that are not part of it: The, Messrs, and the vessel prefixes M/V, M/T, M/S, S/S, M/Y, S/Y, R/V, F/V, LNG/C and LPG/C, however they are punctuated (MV or M V as well). Only a run of them at the start of the name is removed, in every combination, so The M/V Bella is also screened as M/V Bella, The Bella and Bella. Like legal forms, they are removed from both sides.
For sanctions screening, both the screened name and the list entries are read every way their punctuation allows, dashes included. Within one word, each punctuation character either separates terms everywhere or joins them everywhere. So Volga-Dnepr is read as both VolgaDnepr and Volga Dnepr, and In'n'Out as both InnOut and In n Out. A name and a list entry match whichever way each of them was written.
1.2 Search
The search step uses two key matching rules to find candidates:
Per-Term Matching Rule (75% Rule)
Each term in the entity name is compared individually against each term in the candidate name. A term is considered a match if it achieves at least 75% similarity.
Short term exception: Terms with 3 or fewer characters require an exact match, since even small differences in short terms are significant (e.g., "Bo" vs "Ba").
Overall Match Rule
The overall match uses subset logic: the system checks whether the terms of the entity name form a subset of the candidate name terms (or vice versa). This means:
| Entity Name | Candidate Name | Match? | Reason |
|---|---|---|---|
| Anna Johansen | Anna Bertine Johansen | Yes | Entity terms are a subset of candidate terms |
| Anna Bertine Johansen | Anna Johansen | Yes | Candidate terms are a subset of entity terms |
| Anna Johansen | Bertine Andersen | No | No subset relationship |
An overall similarity score is then calculated based on the proportion of terms matched and their individual similarity scores.
1.3 Filtering
After the search step, candidates are filtered based on:
- Similarity score -- Candidates below the configured threshold are removed.
- PEP retention duration -- PEP hits are retained for a configurable period after the person leaves their position, then filtered out.
1.4 Scoring
The final match score is a weighted arithmetic mean of several components:
- Name similarity (configurable weight)
- Date of birth match (configurable weight)
- Country match (configurable weight)
- SSN match (configurable weight)
The formula is:
score = (w_name * name_score + w_dob * dob_score + w_country * country_score + w_ssn * ssn_score)
/ (w_name + w_dob + w_country + w_ssn)Weights are configurable per team in the screening settings.
SSN override: If the entity's social security number (SSN) matches the candidate's SSN exactly, the score is automatically set to the maximum regardless of other factors.
1.5 User Verification
After scoring, results are presented to the user for verification. Each match includes the score breakdown, the matched data points, and a link to the source record.
Settings
The following settings control the screening behavior:
Search Settings
- Match threshold (minimum similarity score)
- Per-term matching sensitivity
- Subset matching direction
Filtering Settings
- Minimum similarity score for candidates
- PEP retention duration (months after leaving position)
Scoring Settings
- Weight for name similarity
- Weight for date of birth
- Weight for country
- Weight for SSN
Alerts
When Alerts Are Sent
- A new PEP or sanctions match is found that was not present in the previous screening run.
- An existing match has changed (e.g., updated sanctions list data, new PEP role).
- A previously matched entity is no longer found in the data source (removed hit).
When Alerts Are NOT Sent
- The match already existed in the previous run and has not changed.
- The match score falls below the configured threshold.
- The PEP retention period has expired and the match has been filtered out.
- The entity has been manually verified and dismissed in a previous review.
Improvements in v3
Version 3 introduces several improvements over previous versions:
- Removed search caps -- Previous versions had a hard limit on the number of candidates returned from the search step. V3 removes this cap, ensuring all potential matches are evaluated.
- New subset logic -- The matching algorithm now uses bidirectional subset checking, so names with additional middle names or variations are correctly matched in both directions.
- Per-term matching -- The 75% per-term rule replaces the previous whole-name matching approach, providing more granular control over what constitutes a valid match and reducing false positives from partial name overlaps.
Last updated on