Extend Microsoft Fabric with HEDDA.IO
Microsoft Fabric has significantly transformed the field of data engineering by providing a seamless and comprehensive platform for processing and analyzing large amounts of data. With a combination of advanced tools and integrated workflows, Microsoft Fabric allows data engineers to be more efficient and effective. The new product has already gained widespread recognition and is being used in numerous projects ranging from data integration to processing and analysis.
As a comprehensive data quality solution, HEDDA.IO has from the outset developed a dedicated pyRunner to integrate seamlessly with Spark environments such as Databricks and Azure Synapse. Our platform ensures that our customers’ data integrity and quality remain at the highest level by providing robust mechanisms for detecting and resolving data anomalies. With the launch of Microsoft Fabric, we are pleased to now also offer full support for this revolutionary platform. This enables our customers to maximize the benefits of Microsoft Fabric while ensuring that their data remains reliable and consistent. HEDDA.IO empowers data engineers to effectively navigate complex data landscapes and deliver high quality data products.
With the latest version of our pyRunner, which was released as a beta version in June 2024, we fully support Microsoft Fabric and were able to massively improve runtimes on all Spark environments at the same time. The latest version can be installed as usual via “pip install” in the Spark notebooks or via environments for a whole cluster and used directly. The versions are 100% compatible with each other, eliminating the need for any further changes within the notebooks.
.
In addition to Microsoft Fabric support in the latest pyRunner version, HEDDA.IO now also supports Microsoft Fabric Lakehouse and OneLake as a new external connection. This allows users to perform member searches and lookups in HEDDA.IO directly against data already stored in the Lakehouse or on OneLake.
As a result, HEDDA.IO offers comprehensive integration with Microsoft Fabric and provides full support for the workflows it enables.
.
Tillmann Eitelberg
Tillmann Eitelberg is CEO and co-founder of oh22information services GmbH, which specializes in data management and data governance and offers its own cloud born data quality solution, HEDDA.IO.
Tillmann is a regular speaker at international conferences and an active blogger and podcaster at DECOMPOSE.IO.
He has open sourced several SSIS components and is Co-Author of Power BI for Dummies (German Edition). Since 2013 is Tillmann is awarded as Microsoft Data Platform MVP. He is a user group leader for the PASS Germany RG Rheinland (Cologne) and was a member of the Microsoft Azure Data Community Advisory Board.
