Microsoft Data Fabric's services enable greater efficiency by tying everything together in one SaaS platform. Here's what you need to know:
Author: Corey Biehl
Microsoft Fabric is the latest platform designed to provide a unified data and analytics ecosystem. Fabric has a wide scope, ranging from data science and analytic capabilities to data engineering processing, transformation and ingestion coupled with real-time event monitoring, alerting and BI. Join us as we explore its benefits and how we are achieving success for clients who transition onto this platform.
In a previous series of articles, we discussed the importance of data and its use for establishing well-defined foundations for AI to proliferate. Key elements throughout that series include the need for data quality processes to produce clean, easily understood datasets, automated testing frameworks that accelerate product releases, the benefits and challenges of differing analytic approaches, and overarching governance programs to ensure your data and analytic investments are meaningful, protected and scalable.
User empowerment is a key focus when looking at a unified data platform and Microsoft Fabric addresses this by seamlessly connecting services that significantly reduce the administrative burden often experienced when linking together products into a data solution. Teamed with Copilot, assisting data discovery with suggestion intelligence and component development, and Purview for Data Governance, you can begin to appreciate the enterprise capabilities it brings to the table.
Fabric’s SaaS model provides value through easily connecting data storage in OneLake with workloads such as Data Engineering, Data Science, Data Warehouse, Data Factory, Power BI and Real-Time Intelligence. Additionally, these services are provided on an as-needed basis.
All of this — the ability to scale up/down based on demand, adding additional services as your architecture grows, and allowing for continuous integration and delivery (CICD) – are just a few examples of why Fabric is a great platform to lean into and build upon over time.
Fabric Services
OneLake Data Storage – A unified data platform begins with a unified data storage solution with OneLake. Behind the semantic layer, data is stored using the same approach as Azure with Azure Data Lake Storage (ADLSg2). If you are familiar with Azure services, OneLake can initially be confusing because you can’t “see” it in Azure. This is because Fabric uses Azure services without requiring an Azure account.
In addition to saving costs on subscriptions, you can also save on storage costs themselves with Shortcuts. OneLake provides the ability to have pointers that appear as files but link to other storage services like SQL server, Azure Databricks or other cloud services like Amazon S3, and appear as virtual tables in a Fabric lakehouse.
Data Engineering – This allows developers to build and maintain lakehouse solutions comprising of large data volumes. A lakehouse refers to a type of data architecture that combines both a data lake and warehouse into an abstracted semantic layer. This makes it easier to use notebooks to write code for transformations, ingestion, and definitions. A data pipeline capability exists for processing between lakehouses and is generally a quick and easy way to perform non-complex ELT pipelines.
Data Warehouse – Data Warehouse rovides high-quality performance and scalability of SQL transactions. Ideally used as a gold layer for BI and analysis, it splits up storage and compute capabilities, allowing for independent scaling up/down as needs arise. Unlike traditional data warehouses in SQL Server, Fabric Data Warehouse does not store structured data as in the past and uses Delta Lake storage with the parquet format.
There are two types of compute engines in Fabric: SQL and Spark. For more traditional data teams that are accustomed to SQL models and objects, Fabric’s SQL engines are most appropriate and don’t require a retooling of skills. On the other hand, if your team is focused on working with less structured data, Spark notebooks are an effective method. Ideally, both types are employed in a full architecture solution where one is better suited for Data Warehouse and the other Data Engineering.
Enjoying this insight?
Sign up for our newsletter to receive data-driven insights right to your inbox on a monthly basis.
Data Science – This integrates with Azure Machine Learning to build advanced analytical models. Data scientists can quickly take advantage of data stored in files/lakehouses to create statistical and analytical solutions to guide organizations with predictions and patterns that can be further integrated into visualization tools, such as Power BI or Excel for analysis.
Power BI – Power BI’s data workspaces provide the ability to quickly create and share visualizations that matter with data in Fabric. The ability to analyze data in its workspace or shared across many is natively included in Power BI, providing for a myriad of dashboard and reporting solutions from a team perspective up to an enterprise landscape.
One major update to Power BI is its new ability to instantly update data from Fabric, whether a lakehouse or warehouse — as soon as that data is recorded, it is also available to be visualized, with no need to schedule a refresh. This is a great example of how powerful a unified data fabric can be for enabling other critical services.
Of course, every innovation has its cons alongside the pros. The drawback to this Power BI update is that it requires a new mode called Direct Lake, and because it is a new feature, some caution should be given to adding it, as it has limitations compared to Direct Query or Import modes.
Data Factory – Fabric’s Data Factory is an enterprise ELT service allowing for data preparation, transformation, and ingestion among over 200 native connectors (both on-prem & cloud). Additionally, the upgrade of Power Query as well as Data Factory’s ready-to-use components provide a low-code environment for development, support and enhancements.
Real-Time Intelligence – When batch processes no longer meet organizational requirements, the need for real-time emerges and Fabric provides numerous solutions for data log analysis, streaming data, or event-based activities. Using different services, the ability to process data movement transformations, storage and analytics is all captured and managed with the Real-Time Hub. And, because it’s all part of Fabric, the ability to incorporate this data into dashboards, alerts and other pipelines is available at your fingertips.
Data Governance
Data Governance is the insurance policy protecting your data investment in Fabric. Some areas of focus and consideration are:
Why should you embrace Fabric
What are the areas of access
How long should data be retained
Where do you allow access
When do you empower BI
These are all areas that Governance can address with policies, standards, and education (e.g., data catalogs).
Data governance is an ever-evolving capability and including the capacity to provide self-service autonomy with tools and processes that ensure high data quality and usage emboldens users with ways to do more with less. It’s a crucial part of maintaining the balance between empowerment and chaos, all with the with the necessary guardrails.
Conclusion: Fabric Services and Your Business
The Fabric platform provides many capabilities which previously required more complicated configurations and management to tie into a single solution. Cohesive organization and administration optimization are the major gains from Fabric, and, best of all, they don’t require adoption of a new codebase or learning a new language.
Many of its features are production ready, however, with Microsoft there are always some still in flux, which will naturally decrease over time. Successfully implementing Fabric requires multiple tactical and strategic considerations. Helpfully, Microsoft provides an adoption roadmap, which you can find here. After reviewing the many steps and layers to maneuver, you will see it requires a thoughtful approach.
We also offer several services to build an effective implementation roadmap tailor-made for your organization. To learn more how RevGen can provide thoughtful leadership with Microsoft Fabric contact us today to speak to one of our experts.
Corey Biehl is a technology leader in RevGen’s Analytics & Insights practice. He is passionate about enabling clients with data and analytic solutionsdesigned for strategic alignment of their business.
AI agents are emerging as the next frontier in automation. Unlike traditional AI assistants, these agents can autonomously perform complex tasks, make decisions, and take actions to achieve specified goals.
This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
Strictly Necessary Cookies
Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.
If you disable this cookie, we will not be able to save your preferences. This means that every time you visit this website you will need to enable or disable cookies again.
3rd Party Cookies
This website uses Google Analytics to collect anonymous information such as the number of visitors to the site, and the most popular pages.
Keeping this cookie enabled helps us to improve our website.
Please enable Strictly Necessary Cookies first so that we can save your preferences!