Published On: August 12th, 2025Categories: Blog

Real-World Data Quality: Designing Complex Business Rules in HEDDA.IO 

How Rule Flows and Conditional Logic Power Enterprise-Grade Validation 

Data quality issues are rarely simple. Whether you’re onboarding data from partners, synchronizing systems, or ingesting raw inputs from external APIs, real-world validation requires conditional logic, domain-specific checks, and structured workflows — not just “is null” filters. 

In HEDDA.IO, you can define such logic using Rulebooks and structured Rule Flows, enabling intelligent branching, reuse, and complete control over how and when records are marked as invalid. 

Let’s walk through a complex business rule flow that mirrors a typical enterprise onboarding scenario. 

Scenario: Onboarding New Customer Records from External Partners 

A global logistics company ingests customer master data from dozens of regional systems. 

The data varies by region, structure, and completeness, and is loaded into a Microsoft Fabric Lakehouse every night for integration with downstream systems. 

The business needs to ensure: 

  • Core customer identifiers are present and valid 
  • Contact details are complete and formatted correctly 
  • Region-specific rules (e.g., VAT, postal formats) are applied 
  • Bad records are flagged but not dropped silently 
  • Valid records are categorized for routing to different downstream systems 

Designing the Rulebook in HEDDA.IO

To build this in HEDDA.IO, we create a Rulebook with a structured Rule Flow consisting of multiple Business Rules that follow conditional paths depending on data quality outcomes.

Rule Flow Execution Logic:
  • Records are passed through Rules sequentially.
  • If a record fails a validation Rule, it is marked invalid and stops processing.
  • If a Rule is marked as a Dataflow Rule, the record is split into logical branches (e.g., “Send to CRM” vs. “Send to Compliance”) but remains valid for further processing.

Rule Structure Example: Customer Onboarding Flow

Let’s look at a breakdown of the Rule Flow:

1. Check Mandatory Fields
a. String Rule: Is Empty on CustomerID, CompanyName, Country
b. Numeric Rule: Is Greater Than 0 on Revenue
c. D
ate Rule: Is After 2010-01-01 on CustomerSince

If any fail → record is invalid

2. Validate Contact Information
a. String Rule: Matches Pattern for email format
b. RDS: Phone Number Validation via libphonenumber
c. Dataflow Rule: Route based on phone number region (US/EU/Other)

Allows splitting by geography without invalidating the row

3. Address Validation (External)
a. RDS: Melissa Address Validation for postal correctness
b. Output: enriched address data + validation flags

Invalid or missing address elements → fail rule

4. Lookup Reference Data
a. Lookup Rule: VAT ID check via internal compliance database
b. Ensures that all EU customers have an active VAT ID in reference table

5. Compliance Check
a.
String Rule: Contains value from Blacklist for CompanyName
b.
Internal RDS integration with denied parties list

Match → record marked invalid + event trigger to Azure DevOps

6. Dataflow Rule: Route Records for Processing
a.
Branch A: Revenue > 1M → route to Strategic Accounts
b. B
ranch B: Region = EU → route to European Processing
c.
Branch C: Is New Customer → route to Onboarding Workflow

Each branch can lead to different post-validation actions, such as forwarding to a specific team, triggering a notification, or storing in different tables.

Rules Types in Use

This workflow uses many of HEDDA.IO’s out-of-the-box rule types:

Rule Type Examples Used
String Rules Is Empty, Matches Pattern, Contains, Starts With, Character Blacklist 
Numeric Rules Is Greater Than, Equals, In Range 
Date Rules Is Before/After, Is On Weekday (e.g., must be Tuesday) 
Lookup Rules Check against SQL-based reference tables 
RDS Rules External validation via libphonenumber and Melissa Web API 
Dataflow Rules Route records into different branches without invalidating them 

Benefits of This Design

  • Transparency: Every Rule, input, and output is tracked and can be explained to business and compliance teams.
  • Extensibility: New Rules can be added at any point without refactoring the entire flow.
  • Reusability: Rulebooks can be shared across environments (Dev, QA, Prod) and exported via Git.
  • Scalability: The same logic runs across streaming (WebRunner), batch (Fabric), and ad hoc (Excel Add-in) scenarios.
  • Integration-Ready: Rule outcomes can trigger events to systems like Microsoft Teams, Azure Event Grid, or Slack.

Summary

Complex data validation doesn’t have to mean hard-coded scripts or brittle transformations. With HEDDA.IO, you can define Rulebooks and Rule Flows that model real-world business processes with:

  • Logical branching
  • Reusable Rules
  • External integrations (via RDS)
  • Validation at scale, in any data platform

Whether you’re validating customer onboarding, financial transactions, or product master data — HEDDA.IO gives you the structure to build, manage, and evolve your Rules over time.

LET’s talk!

 

By the way: we’ll be at the European FabCon in Vienna!

Come find us at booth 20we’re happy to dive deeper, show demos, and exchange ideas on how to bring data quality and Excel together in a meaningful way.

 

Tillmann Eitelberg
Tillmann EitelbergCEO
Tillmann Eitelberg is the CEO of oh22information services GmbH and co-founder of HEDDA.IO. With over 20 years of experience, he is a leading data strategist and a data quality evangelist. He believes the true power of data-driven applications, from robust data integration and business intelligence to advanced data science and AI, can only be unlocked when data quality is a fundamental discipline across the entire data stack.

Tillmann is a regular speaker at international data conferences, where he shares his expertise in building robust data ecosystems. For his profound contributions to the technical community, he has been recognized by Microsoft as a Most Valuable Professional (MVP) for Data Platform for many consecutive years.

WE CREATE

CLEAN DATA EVERY DAY.

GET STARTED