ETL in the Cloud02-November-2019
Mapping Data Flows include built-in data transformations to address common ETL activities like join, aggregate, pivot, unpivot, split, lookup and sort data. In the event that the out-of-box capabilities don’t address an organization’s requirements, an expression builder can be used that allows developers to customize their ETL solution.
Beyond a simplified developer experience, Azure Data Factory also provides live insights into the data moving through Azure Data Factory pipelines. These insights include metrics like null counts, value distributions, standard deviations, minimum length values, maximum length values, row counts and more.
In addition to ETL telemetry and insights, developers also have access to an interactive visual debug experience that allows for real-time debugging and tracing.
Microsoft is certainly not new to the ETL space. For several releases of SQL Server, Microsoft has included SQL Server Integration Services (SSIS). However, with the shift to cloud computing and Platform as a Service (PaaS) SQL Server offerings, this has left SSIS limited to Infrastructure-as-a-Service (IaaS) or on-premises workloads. Kamil Nowinski, a Microsoft MVP, shared his perspective, on the transition from SSIS to Azure Data Factory Mapping Data Flows, in a recent blog post:
Sam Analytiks motivates, educates and proliferates data for any organisation as a non-profit partner. If you or any of your team member needs a help, surely our consultants will be glad to help you in any case.
Call us at +48-729473572 or email us at firstname.lastname@example.org
#data #datavisualisation #datamodelling #datamart #powerbi #sql #excel #powerquery #azure #azureanalysis