Real-World Data Quality: Designing Complex Business Rules in HEDDA.IO

How Rule Flows and Conditional Logic Power Enterprise-Grade Validation 

Data quality issues are rarely simple. Whether you’re onboarding data from partners, synchronizing systems, or ingesting raw inputs from external APIs, real-world validation requires conditional logic, domain-specific checks, and structured workflows — not just “is null” filters. 

In HEDDA.IO, you can define such logic using Rulebooks and structured Rule Flows, enabling intelligent branching, reuse, and complete control over how and when records are marked as invalid. 

Let’s walk through a complex business rule flow that mirrors a typical enterprise onboarding scenario. 

Scenario: Onboarding New Customer Records from External Partners 

A global logistics company ingests customer master data from dozens of regional systems. 

The data varies by region, structure, and completeness, and is loaded into a Microsoft Fabric Lakehouse every night for integration with downstream systems. 

The business needs to ensure: 

  • Core customer identifiers are present and valid 
  • Contact details are complete and formatted correctly 
  • Region-specific rules (e.g., VAT, postal formats) are applied 
  • Bad records are flagged but not dropped silently 
  • Valid records are categorized for routing to different downstream systems 

Designing the Rulebook in HEDDA.IO

To build this in HEDDA.IO, we create a Rulebook with a structured Rule Flow consisting of multiple Business Rules that follow conditional paths depending on data quality outcomes.

Rule Flow Execution Logic:
  • Records are passed through Rules sequentially.
  • If a record fails a validation Rule, it is marked invalid and stops processing.
  • If a Rule is marked as a Dataflow Rule, the record is split into logical branches (e.g., “Send to CRM” vs. “Send to Compliance”) but remains valid for further processing.

Rule Structure Example: Customer Onboarding Flow

Let’s look at a breakdown of the Rule Flow:

1. Check Mandatory Fields
a. String Rule: Is Empty on CustomerID, CompanyName, Country
b. Numeric Rule: Is Greater Than 0 on Revenue
c. D
ate Rule: Is After 2010-01-01 on CustomerSince

If any fail → record is invalid

2. Validate Contact Information
a. String Rule: Matches Pattern for email format
b. RDS: Phone Number Validation via libphonenumber
c. Dataflow Rule: Route based on phone number region (US/EU/Other)

Allows splitting by geography without invalidating the row

3. Address Validation (External)
a. RDS: Melissa Address Validation for postal correctness
b. Output: enriched address data + validation flags

Invalid or missing address elements → fail rule

4. Lookup Reference Data
a. Lookup Rule: VAT ID check via internal compliance database
b. Ensures that all EU customers have an active VAT ID in reference table

5. Compliance Check
a. 
String Rule: Contains value from Blacklist for CompanyName
b. 
Internal RDS integration with denied parties list

Match → record marked invalid + event trigger to Azure DevOps

6. Dataflow Rule: Route Records for Processing
a. 
Branch A: Revenue > 1M → route to Strategic Accounts
b. B
ranch B: Region = EU → route to European Processing
c. 
Branch C: Is New Customer → route to Onboarding Workflow

Each branch can lead to different post-validation actions, such as forwarding to a specific team, triggering a notification, or storing in different tables.

 

Rules Types in Use

This workflow uses many of HEDDA.IO’s out-of-the-box rule types:

Rule TypeExamples Used
String RulesIs Empty, Matches Pattern, Contains, Starts With, Character Blacklist 
Numeric RulesIs Greater Than, Equals, In Range 
Date RulesIs Before/After, Is On Weekday (e.g., must be Tuesday) 
Lookup RulesCheck against SQL-based reference tables 
RDS RulesExternal validation via libphonenumber and Melissa Web API 
Dataflow RulesRoute records into different branches without invalidating them 

 

Benefits of This Design

  • Transparency: Every Rule, input, and output is tracked and can be explained to business and compliance teams.
  • Extensibility: New Rules can be added at any point without refactoring the entire flow.
  • Reusability: Rulebooks can be shared across environments (Dev, QA, Prod) and exported via Git.
  • Scalability: The same logic runs across streaming (WebRunner), batch (Fabric), and ad hoc (Excel Add-in) scenarios.
  • Integration-Ready: Rule outcomes can trigger events to systems like Microsoft Teams, Azure Event Grid, or Slack.

Summary

Complex data validation doesn’t have to mean hard-coded scripts or brittle transformations. With HEDDA.IO, you can define Rulebooks and Rule Flows that model real-world business processes with:

  • Logical branching
  • Reusable Rules
  • External integrations (via RDS)
  • Validation at scale, in any data platform

Whether you’re validating customer onboarding, financial transactions, or product master data — HEDDA.IO gives you the structure to build, manage, and evolve your Rules over time.

Hedda.io_primarylogo_orange_white_text

HEDDA.IO is a modern data quality platform that transforms domain knowledge into automated, scalable data validation and governance.

Contact us

A product by

oh22_Logo_weiss_RGB