How to prepare a file for ingestion into Azure SQL Data Warehouse using Polybase, when there are NULLs in the data? There are only a few web pages that talk about how Polybase handles NULL values… And most of them are misleading. For example: Polybase to treat the NULL string as Null values and not…
To run ETL on Azure Data Warehouse, it’s a good idea to create a user in a higher resource class. There are a number of ways to authenticate to Azure DW. See – Controlling and granting database access to SQL Database and SQL Data Warehouse The easiest (from a development perspective) is to create an…
In this episode, I had the pleasure of speaking with Klaus Lindinger, Digital Innovation Officer at dataWerks & Partner at Infoteam Sales Process Consulting, based in Frankfurt. Over the past 30 years Klaus has worked with some of the world’s largest software brands. His first introduction to business intelligence, believe it or not, was collecting…
This post is the second in a series of detailed cases studies discussing the ETL strategies that can be used when a Persistent staging layer is included in your Data Warehouse. It’s intended as a reference for developers using Dimodelo Data Warehouse Studio, our Data Warehouse Automation tool to quickly build a Data Warehouse. It’s most…
Why you Need a Persistent Layer 1. All history, all of the time, instead of some history some of the time Including a persistent layer in your architecture is a paradigm shift in how you see the data warehouse. In the popular Kimball methodology, without the persistent layer, the data warehouse layer was responsible for…
The persistent layer contains as set of persistent tables that record the full history of changes to the data of the table/query that is the source of the Persistent table. The source could a source table/file, a source query, another staging table or a view/materialized view in the transform layer. In a persistent table there…
This post is the first in a series of detailed cases studies discussing the ETL strategies that can be used when a Persistent staging layer is included in your Data Warehouse. It’s intended as a reference for developers using Dimodelo Data Warehouse Studio, our Data Warehouse Automation tool to quickly build a Data Warehouse. It’s…
Dimodelo Solutions is proud to announce the release of version 2.15 of Dimodelo Data Warehouse Studio, its data warehouse automation tool for the Microsoft data platform. In this release we have added support for the Microsoft APS platform and SQL Server 2017. We have also redesigned our data management architecture from the ground up, introducing…
In this episode, I had the pleasure of speaking with Chris Webb, a well respected independent consultant based in the UK. He specialises in Analysis Services, MDX, Power Pivot, DAX, Power Query and Power BI in general. He is a widely read blogger having authored more than a 1000 posts over the last 10 years. …
At Dimodelo, we have been hard at work, redesigning the Data Warehouse data management architecture that Dimodelo Data Warehouse Studio (our Data Warehouse Automation tool) generates. Working with our clients we have introduced a Persistent Layer into the data warehouse architecture. “What is a Persistent Layer?” I hear you ask. The persistent layer contains as…
