Azure Data Factory
Azure Data Factory is the platform that solves such data scenarios. It is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale.
- Introduction to Azure Data Factory
- Overview of Azure Data Factory
- Components of Azure Data Factory
- Differences between Azure Data Factory and SSIS
- Building Pipelines in Azure Data Factory/li>
- Creating and configuring data pipelines
- Scheduling and triggering pipeline runs
- Monitoring pipeline activity
- Data Integration with Azure Data Factory
- Copying data from various sources (e.g. on-premises, cloud-based) into Azure
- Transforming data using data flowss
- Mapping data to different data models
- Data Transformation in Azure Data Factory
- Using Data Flow to build complex ETL processes
- Mapping and transforming data in Data Flow
- Debugging Data Flow
- Control Flow, Parameters and Variables
- Lookup, If condition and Switch Activities
- Parameters in ADF
- Variables in ADF
- ForEach Activity
- Metadata Based Approach
- Authentication and Access
- Azure Key Vault
- System Assigned and User Managed Identities
- Service Principles
- Data Orchestration and Management
- Managing and coordinating data processing jobs
- Scheduling and triggering data flows and pipelines
- Controlling data access and security
- Performance Optimization and Troubleshooting
- ABest practices for optimizing data transfer and processing performance
- Troubleshooting common issues with Azure Data Factory
- Monitoring and logging for Azure Data Factory