Data quality metrics: how to measure master data before cleansing

To measure master data quality before a cleansing project, pick the critical fields for each domain, define the numerator and denominator of every metric and calculate only over active records. Five metrics are enough for a baseline: completeness, duplication, validity, description standardization and consistency. The result becomes the scope and the priority order of the cleansing work.

"Our master data is a mess." I heard that sentence in many meetings during the years I spent implementing SAP. Procurement complains about repeated materials, the tax team complains about missing product tax codes, finance complains about suppliers whose registration does not match the tax authority's records. Everyone agrees. Then someone asks: how big a mess?

Nobody answers. Without a number, the budget request for cleansing turns into one opinion against another. And when the project ends, nobody can show what improved. That is why I insist on measuring before touching a single record. Today's measurement is the ruler that will judge the cleansing tomorrow.

What does data quality mean for master data?

I like to use the Government Data Quality Framework as a reference. It was published by the Government Data Quality Hub of the UK government and describes six quality dimensions:

• Completeness: the degree to which records are present.
• Uniqueness: the degree to which there is no duplication in records.
• Consistency: the degree to which values in a data set do not contradict other values representing the same entity.
• Timeliness: the degree to which the data is an accurate reflection of the period it represents.
• Validity: the degree to which the data is in the range and format expected.
• Accuracy: the degree to which data matches reality.

The same framework notes that perfect quality may not be achievable and that the focus is on data that is fit for purpose. For master data, that means the product tax code has to work for the tax team, the unit of measure has to work for the warehouse and the supplier's registration status has to work for whoever pays that supplier.

One warning before the math: the formulas in this article are an operational recommendation for building a baseline. They are not a regulation or an official standard. Adapt them to your company and, above all, document each rule so you can repeat the measurement later. For a broader view of the dimensions and the tools around them, we have an article on how to ensure accurate data.

A note on context: the examples come from Brazil. There, every company has a CNPJ, the national corporate taxpayer registry number managed by the Receita Federal (Brazil's federal tax authority), and individuals have a CPF. Materials carry an NCM code, the eight-digit Mercosur product classification used for tax purposes. The logic of the metrics works in any country; only the official sources change.

Which records should go into the calculation?

This is the mistake that distorts a baseline the most. Many teams calculate metrics over the whole database, including the bolt nobody has bought since 2012 and the supplier that closed years ago.

A record with no activity does not weigh the same in daily operations. If it goes into the denominator, the number looks worse than the real problem or, in some cases, better. Either way, the priorities come out wrong.

Step 1: define what an active record is

Choose a time window and a usage criterion. For example: materials with a purchase order, invoice, reservation or stock in the last 24 months; suppliers with an order or payment in the same period; customers with a sale or an open receivable. Blocked records stay out.

Step 2: measure inactive records separately

Inactive records do not disappear from the analysis. They become a separate number that usually feeds a simple decision: block instead of cleanse. Rewriting the description of an item nobody uses is a waste of a specialist's time.

Which fields are critical in each domain?

A critical field is one that, if empty or wrong, stops a process or creates risk. Choose them together with the teams that use the data and keep the list short. A list of forty critical fields becomes a wish list that nobody can act on.

• Materials: description, unit of measure, NCM code, material group or class, material type and the characteristics required by the descriptive standard.
• Suppliers: CNPJ or CPF, legal name, state tax registration where applicable, address, bank details and registration status.
• Customers: CNPJ or CPF, legal name, delivery and billing address, state tax registration where applicable and payment terms.

Which metrics should you track and how do you calculate each one?

The table below summarizes the five baseline metrics. Validity takes two rows because it is measured in two layers (format and official source), and timeliness is optional. The denominator is always explicit, because it decides whether the number can be compared later.

Metric	What it measures	Formula (numerator / denominator)	Sample rule by domain
Critical field completeness	Whether the fields that processes rely on are filled in	Active records with all critical fields filled / active records assessed	Material: description, unit, NCM and group filled. Supplier: CNPJ or CPF, address and bank details. Customer: CNPJ or CPF and delivery address.
Duplication	How much of the active base sits in groups that look like the same item or company	Active records in suspect groups / active records	Material: same manufacturer and part number, or same characteristics with a different description. Supplier and customer: same full CNPJ (not just the same root) or same CPF under more than one active code.
Format validity	Whether the value follows the expected format and allowed values	Records with a value in the expected format / records with the field filled	Material: eight-digit NCM and a unit of measure from the allowed list. Supplier and customer: CNPJ or CPF with correct check digits.
Validity against the official source	Whether the data matches the public source	Active records with an active registration status at the Receita Federal / active records with a CNPJ checked	Supplier and customer: CNPJ checked with the Receita Federal; counted as nonconforming when the status is anything other than active. CPF stays out of this calculation or is checked against another source. Material: NCM checked against the current table used by the tax team.
Description standardization	Whether the description follows the defined descriptive standard	Items that follow the descriptive standard / items assessed	Material: description built from the class standard, with characteristics in the defined order and no free abbreviations. Supplier and customer: address with no free abbreviations and legal name kept apart from the trade name.
Consistency	Whether fields or systems contradict each other	Records with no contradiction under the rule / records assessed by the rule	Material: purchase unit different from base unit has a conversion factor. Supplier: state in the address matches the state of the tax registration. Customer: same CNPJ shows the same legal name in the ERP and the CRM.
Timeliness (optional)	Whether the data was reviewed within the defined period	Records reviewed within the period / active records	Supplier: bank details and documents reviewed in the last 12 months. Customer: address confirmed in the last review cycle. Material: item reviewed after a manufacturer change.

How do you count duplicates without inflating the number?

Duplication is where I see the most confusion. Some teams count pairs. The problem is that a group of four identical records generates six pairs, so the number grows much faster than the problem itself.

I prefer to count records inside suspect groups and report two numbers:

• Share of records in suspect groups: active records that fell into any group, divided by total active records. It shows the size of the review effort.
• Excess records: records in suspect groups minus the number of groups. Each confirmed group keeps one code and retires the others. It shows how many codes could go.

Note the word suspect. Until someone reviews it, it is only a suspicion. That is why it pays to review a sample of groups and record the confirmation rate. It adjusts the estimate and shows where the matching rule is producing false positives. The terms and the grouping logic are in our deduplication glossary.

A domain note for Brazil. When the ERP validates CNPJ or CPF as a unique field, duplicate suppliers or customers tend to be less common, and the real supplier problem usually lies in validity. Check how your system is configured, because not every database has that validation turned on. In the grouping rule, also separate the CNPJ root (the first 8 digits, shared by a head office and its branches) from the full CNPJ: the same root with a different full CNPJ may be a legitimate branch, while the same full CNPJ under more than one active code is a suspected duplicate. Duplication hurts most in the material master, where the same item shows up under different descriptions.

How do you measure validity against the official source?

Split validity into two layers, because they answer different questions.

Layer 1: format

Does the CNPJ have the correct check digits? Does the NCM have eight digits? Is the unit of measure on the allowed list? This layer runs on the whole base with simple rules and catches typing errors.

Layer 2: official source

A CNPJ can be perfect in format and still belong to a company whose registration status is not active. Checking the registration status is a public service of the Receita Federal, which offers the registration and status certificate and publishes the CNPJ open data on its CNPJ page (in Portuguese).

The calculation is: active suppliers with an active status at the Receita Federal, divided by active suppliers checked. Record the date of the check along with the result, because registration status changes over time and the baseline needs to say when it was taken.

How do you measure description standardization?

You can only measure standardization if a standard exists. For materials, that means a descriptive standard per item class: item name, required characteristics and the order in which they appear. We call it a material descriptive standard (PDM).

If the company has no standard yet, that is already a baseline finding, and one of the most important. Deduplicating free-text descriptions is pointless if the next record is created the same old way.

When to use the whole base and when to sample

Completeness, format validity, CNPJ checks and the search for suspect groups can be automated. Run them on the entire active base. Description standardization and duplicate confirmation need human eyes. For those, use a random sample, ideally stratified by material group, sized to what the team can review carefully. Write down the criterion and the sample size, or the next measurement will not be comparable.

What does the baseline look like in numbers? (hypothetical example)

Hypothetical example, with made-up round numbers just to show the calculation. This is not a real case.

Materials

• A base of 50,000 registered materials. Of these, 30,000 had activity in the last 24 months. The other 20,000 are measured separately as blocking candidates.
• Completeness: 24,000 active materials have description, unit, NCM and group filled. 24,000 / 30,000 = 80%. Field by field, 4,500 active materials have no NCM, so NCM completeness is 25,500 / 30,000 = 85%.
• Duplication: the search found 1,200 suspect groups containing 3,000 active materials. Share of records in suspect groups: 3,000 / 30,000 = 10%. Excess records: 3,000 minus 1,200 = 1,800, or 6% of the active base.
• Confirmation: in a sample of 200 reviewed groups, 150 were confirmed (75%). The estimate of real excess records becomes 1,800 x 0.75 = 1,350.
• Standardization: in a sample of 600 active materials, 330 follow the descriptive standard. 330 / 600 = 55%.

Suppliers

• A base of 6,000 registered suppliers, of which 2,000 had an order or payment in the last 24 months.
• Format validity: 1,990 have a CNPJ or CPF with correct check digits. 1,990 / 2,000 = 99.5%.
• Validity against the official source: of the 1,900 active suppliers with a CNPJ, 1,824 have an active status at the Receita Federal. 1,824 / 1,900 = 96%. The other 76 need analysis.

Customers

• Consistency: of 10,000 active customers present in both the ERP and the CRM, 9,200 show the same legal name for the same CNPJ in both systems. 9,200 / 10,000 = 92%.

How do you turn the snapshot into cleansing scope and priorities?

A number alone decides nothing. What decides is the number combined with impact. In the hypothetical example above, the reading would go roughly like this:

1. Risk first: the 76 active suppliers whose status is not active go to the front, because they are receiving orders or payments.
2. Tied-up money next: groups of duplicate materials with stock or orders under more than one code come after, because they cause repeat purchases and inflated inventory.
3. Fields that block processes: the 4,500 active materials without NCM become a separate package, owned by the tax team.
4. Standard before correction: with 55% standardization, the first deliverable is the descriptive standard for the highest-consumption classes. Mass rewriting comes after.
5. Block instead of cleanse: the 20,000 inactive materials leave the detailed scope and become a blocking decision.

This gives you the scope (how many records, in which domains, with which treatment) and the order. It also leaves the ruler ready to measure again at the end.

Where should you start?

1. Pick one domain to start with, usually the one that hurts most today: materials or suppliers.
2. Define the active record criterion and the time window. Write it down.
3. List the critical fields with the teams that use the data.
4. Write the rule for each metric with an explicit numerator and denominator.
5. Run the automatable rules on the entire active base and set aside the sample for manual reviews.
6. Check the registration status of active CNPJs and record the date of the check.
7. Combine the results with the impact on each process and build the priority list.

With this snapshot in hand, it becomes much easier to design and defend a master data cleansing project with a clear scope and order of attack.

Frequently asked questions

How many quality dimensions do I actually need to measure?

For a pre-cleansing baseline, five metrics are usually enough: critical field completeness, duplication, validity (format and official source), description standardization and consistency. Timeliness comes in when the company already has a review routine. Accuracy, in the sense of checking against physical reality, requires going to the item or the supplier and is usually left for spot samples.

What quality target is acceptable?

There is no universal number. The Government Data Quality Framework speaks of data that is fit for purpose, and the target follows the same logic: it depends on the field's impact. For the registration status of a supplier that receives payments, tolerance tends to be minimal. For a secondary descriptive field, a lower target may be enough. Set the target per critical field, with the process owner, once you have the baseline.

How do I count duplicates when the descriptions are different?

Do not compare only the description text. Search by attributes that identify the item: manufacturer and part number, technical characteristics, unit of measure and class. Records that match on these attributes form a suspect group, even with different descriptions. Then count records in the groups (not pairs) and confirm a sample to calibrate the rule.

How often should I measure again after cleansing?

Measure right after the cleansing is delivered, using the same rules, window and criteria as the baseline, so you compare like with like. After that, turn the metrics into a governance routine, at a frequency the company defines, to see whether quality holds or the master data starts to degrade again.

Is a master data quality metric the same as a supplier performance KPI?

No. A performance KPI measures how the supplier delivers: lead time, product quality, service. A master data quality metric measures whether the supplier's record is complete, valid, consistent and free of duplicates. A supplier can deliver very well and still have a CNPJ whose status does not match, and the opposite also happens.

How 4MDG can help

If your company knows its master data has problems but has no numbers yet, the first step is a diagnostic. It is the first of the five stages of our data cleansing service (diagnostic, standardization, deduplication, validation and enrichment, ongoing governance).

• The free diagnostic runs on a real extract of your materials, suppliers and customers, with nothing to install, in a few days.
• It delivers a duplication index with sample duplicate groups, completeness by attribute, registration compliance checked against public sources, description quality, an estimate of gains and a recommended path.
• The material stays with your company whether or not you hire us.

You can request the diagnostic, and the team will assess whether it applies to your case, or get in touch with us.

Sources

• Government Data Quality Hub (UK government). The Government Data Quality Framework. https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework. Accessed on October 5, 2026.
• Receita Federal (Brazil's federal tax authority). CNPJ page, tax guidance, registries (in Portuguese). https://www.gov.br/receitafederal/pt-br/assuntos/orientacao-tributaria/cadastros/cnpj. Accessed on October 5, 2026.