What Makes Good Data?

Context: why “good data” matters

Data quality is not an abstract ideal; it is a practical requirement for reliable analytics, operational processes, and regulatory reporting. When data is inaccurate, incomplete, inconsistent, late, or poorly governed, downstream outcomes degrade: metrics drift, models mislead, teams stop trusting dashboards, and remediation costs rise. A useful starting point (aligned with DAMA-DMBOK and widely cited data quality research) is to treat data quality as fitness for use: “good” means the data is suitable for a specific decision or operational process, with clearly defined expectations and tolerances.

What “good data” means: fitness-for-use and requirements

“Good data” is data that meets explicit, testable requirements for a defined context:

• Use case: What decisions, processes, or products depend on the data?
• Consumers: Who uses it (finance, operations, data science, customers via a data product)?
• Risk and criticality: What happens if the data is wrong or late (regulatory exposure, customer impact, revenue loss)?
• Quality thresholds: What error rates, freshness, and completeness are acceptable? This framing turns data quality from a general aspiration into an engineering and governance practice: define quality expectations, implement controls, monitor continuously, and remediate with clear ownership.

Core data quality dimensions (common set)

Many organizations use a common set of data quality dimensions as a vocabulary for requirements and measurement. The following six dimensions are frequently used in data governance programs and are consistent with how DAMA-DMBOK describes data quality concerns.

• Accuracy: Values correctly represent the real-world entity/event they describe.
  • Example checks: compare to authoritative sources; validate calculated fields; reconcile totals to systems of record.
• Completeness: Required data is present at the needed level of coverage.
  • Example checks: non-null for required fields; record-level coverage (e.g., all stores reporting); event coverage (e.g., all orders have a shipment).
• Consistency: The same concept has the same value/meaning across datasets and systems.
  • Example checks: cross-system reconciliation; consistent code sets; consistent aggregation logic; consistent definitions in the semantic layer.
• Timeliness (freshness): Data is available when needed and reflects the appropriate point-in-time state.
  • Example checks: latency against an SLA; “data is updated by 9:00 AM local time”; event arrival delay distributions.
• Validity (conformance): Data conforms to formats, domains, and business rules.
  • Example checks: schema and type constraints; domain checks (allowed values); referential integrity; business rule validations.
• Uniqueness: Entities/events are represented once where uniqueness is expected (no unintended duplicates).
  • Example checks: duplicate detection; primary key uniqueness; survivorship logic in master data. Important nuance: these dimensions are not exhaustive. Some programs also track dimensions such as integrity, reasonableness, precision, and lineage/traceability. The goal is not to adopt a “perfect list,” but to select a set that supports clear requirements and measurable controls.

From dimensions to measurable rules

A dimension becomes actionable when expressed as a rule that can be tested. A good rule has:

• A clear scope (table/field/metric, business domain, and time window)
• A deterministic definition (what is considered a failure)
• An owner (who fixes it and who approves exceptions)
• A threshold and severity (when it is acceptable to proceed vs. stop) Common rule patterns:
• Schema rules: types, nullability, uniqueness, referential integrity
• Domain rules: allowed values, ranges, patterns (e.g., ISO country codes)
• Cross-field rules: start_date ≤ end_date; currency required when amount present
• Reconciliation rules: totals match a source system within tolerance; row counts align within expected variance
• Freshness rules: max(event_time) within X hours; pipeline completed by X time Metrics and reporting:
• Track failure rate (invalid rows / total rows), coverage (e.g., % of entities represented), and trend over time.
• Separate measurement from acceptance: a dataset can have known issues but still be fit for specific uses if documented and approved.

Data quality in the data lifecycle (DAMA perspective)

DAMA-DMBOK treats data quality management as a continuous discipline that spans the data lifecycle. Practically, quality improves when controls exist at each stage:

• Capture/creation: Prevent defects early (input validation, mandatory fields, reference data, UI constraints).
• Integration and movement: Preserve meaning and integrity (standardize identifiers, manage schema changes, validate mappings, reconcile counts and totals).
• Storage and modeling: Enforce constraints where possible (keys, constraints, canonical models, consistent dimensional definitions).
• Consumption: Provide consistent semantics (metric definitions, semantic layer, governed dashboards, documented transformations).
• Change management: Treat data changes like software changes (versioning, testing, controlled releases, backward compatibility). The principle is “quality by design,” not “quality by inspection.” Monitoring is essential, but prevention and controlled change reduce recurring defects.

Governance and accountability: who owns “good”

Data quality improves when responsibilities are explicit, which aligns with DAMA governance concepts. Key roles (names vary by organization):

• Data Owner: Accountable for data within a domain (approves definitions, quality thresholds, and risk trade-offs).
• Data Steward: Operationalizes policies and rules, manages issue triage, and coordinates remediation.
• Data Custodian / Platform team: Implements technical controls (pipelines, validation, observability, access, retention).
• Data Product Owner (if using data product thinking): Manages the dataset as a product with documented interfaces, SLAs, and consumer support. Artifacts that make ownership workable:
• Data definitions and business glossary (what fields and metrics mean)
• Data contracts between producers and consumers (schemas, SLAs, expectations)
• Issue management workflow (prioritization, root cause, remediation, verification)
• Exception process (when and how rule failures are tolerated with approval)

Practical implementation patterns in modern platforms

Organizations can implement data quality management using a combination of governance process and technical controls.

1) Data profiling to establish baselines

Before setting thresholds, profile the data to understand:

• Value distributions, null rates, duplicates
• Referential integrity gaps
• Outliers and seasonality
• Arrival patterns and latency Profiling helps distinguish true defects from expected variability (e.g., weekend volume changes).

2) Automated testing in the analytics development lifecycle

Treat analytics transformations as software:

• Add unit-like tests for business rules (validity, uniqueness, relationships)
• Add integration-like tests for reconciliations and end-to-end freshness
• Gate deployments based on severity (e.g., fail build for critical uniqueness violations) This aligns with modern Analytics Engineering practices: version control, CI/CD, and tests close to where transformations are defined.

3) Data observability and monitoring

In production, monitor:

• Freshness and pipeline completion SLAs
• Volume anomalies (row counts, event counts)
• Distribution shifts for key fields
• Rule failures with alerting routed to owners Observability does not replace governance; it operationalizes it by turning expectations into signals and actions.

4) Semantic consistency through a governed semantic layer

Many “data quality” incidents are actually semantic defects: teams compute the same metric differently. A semantic layer (or strong metric governance) helps ensure:

• Single definitions for core KPIs
• Consistent filtering, attribution, and time logic
• Reuse of validated metric definitions across BI tools

5) Master data and reference data management

Uniqueness and consistency often require:

• Stable identifiers (customer_id, product_id)
• Matching/merging and survivorship rules
• Governed reference data (code sets, hierarchies) Without master/reference data discipline, duplicates and inconsistent categories propagate across the enterprise.

Common pitfalls and how to avoid them

• Pitfall: “Accuracy” without an authoritative reference.
  • Mitigation: identify systems of record and reconciliation points; document what “truth” means per attribute.
• Pitfall: Measuring quality but not assigning ownership.
  • Mitigation: every rule has an accountable owner; create a triage and remediation workflow with SLAs.
• Pitfall: Treating data quality as only a BI problem.
  • Mitigation: push controls upstream to capture and integration; prevent defects rather than only detect them.
• Pitfall: Overfitting rules that block delivery.
  • Mitigation: classify rules by severity (critical/high/medium); use tolerances; implement an exception process.
• Pitfall: Duplicate metrics and definitions.
  • Mitigation: centralize KPI definitions via a semantic layer or governed metric store; enforce reuse.
• Pitfall: Ignoring timeliness and freshness.
  • Mitigation: define freshness SLAs per use case; monitor and alert; communicate known delays clearly to consumers.

A practical checklist for defining “good data” for a dataset

Use this checklist to translate “good data” into requirements:

• Define the dataset’s purpose and primary consumers.
• Identify critical fields and metrics (the “must be correct” set).
• For each critical element, define quality rules across relevant dimensions (accuracy, completeness, etc.).
• Set thresholds and SLAs (including freshness/latency).
• Assign owners and escalation paths.
• Implement automated validation and monitoring.
• Create an issue workflow and track root causes.
• Publish documentation: definitions, lineage, and known limitations.

Summary: key takeaways

Good data is fit for use, not universally perfect. Quality becomes manageable when organizations define measurable rules using clear dimensions, assign accountable ownership through governance, and implement preventive controls plus continuous monitoring across the data lifecycle. By combining governance artifacts (definitions, ownership, issue management) with modern engineering practices (testing, observability, semantic consistency), teams can sustain trust and reliably deliver value from data.