Synapse Pipelines: Orchestration and Development (English)
Description
In this two-day "Synapse Pipelines: Orchestration & Development" training, we delve into both the steering and development, as well as the setup and management of Synapse Pipelines within the Azure platform.
Do you want to execute and control the orchestration, data movement, and ETL/ELT tasks within the Azure platform using Synapse Pipelines? In this Synapse Pipelines: Orchestration and Development training, we focus on orchestration and data movement.
By orchestration, we mean the control of sources, databases, and services. This allows us to specify which task can be performed when.
In addition, we look at data movement: the copying of data to the cloud and within the cloud. This ensures that data is in the right format (e.g., a SQL database, or Parquet files) at the right place (e.g., Azure Data Lake Storage, SQL Database, or Synapse).
This training is designed for anyone responsible for steering and developing ETL and ELT processes with Synapse Pipelines in their organization, as well as those responsible for the setup and management of Synapse Pipelines. This could be because Synapse Pipelines is already being used, or because your organization is moving its data and analytics to Azure. You find it important to understand how Synapse Pipelines works and want to be able to develop in it yourself. In addition, you want to ensure that the setup is solid and want to understand how to use Synapse Pipelines efficiently and securely.
After this two-day training:
- You understand how Synapse Pipelines works
- You have an overview of all components of Synapse Pipelines
- You can set up linked services, datasets, activities, and pipelines yourself
- You know how Synapse Pipelines is structured
- You can securely set up Synapse Pipelines
- You are able to manage the integration runtime
We do this as you are used to at Wortell Smart Learning: under the guidance of an inspiring trainer, using really good material, and very practical. So we do a lot ourselves.
Prerequisites
The training is accessible and can be followed if you have a basic understanding of SQL. However, you should already be somewhat familiar with Azure and Azure DevOps:
- Azure:
- Core Azure Concepts and familiarity with the Azure Portal are important. As preparation, you could follow Microsoft's free e-learning "Azure Fundamentals part 1: Describe core Azure concepts"
- The Azure Data Platform is important. The distinction between the different types of data stores, variations in types of data, and ways of processing are discussed in the one-day training Azure Data Fundamentals (DP-900)
- Azure DevOps is very important for actual development with Synapse Pipelines, and is used for both version control and Continuous Integration and Continuous Delivery (CI/CD) in Synapse Pipelines.
- If you have no experience with Azure DevOps (or at least with Git and another CI/CD tool), you can still follow the training, but some parts may be a bit more challenging.
- Want to get started with Azure DevOps? Take a look at our training CI/CD with Azure DevOps for Data Engineers.
Topics
During the two days, we will lay the foundation for working with Synapse Pipelines and train on the following topics:
- Structure of Synapse Pipelines
- Orchestration and Data Movement in Synapse Pipelines
- Development in Synapse Pipelines
- Setup and management of Synapse Pipelines
- Linked Services, Datasets, Activities and Pipelines in Synapse Pipelines
- Integration Runtimes: setup choices and management
- Private Endpoints in Synapse Pipelines
- Handling triggers
- Synapse Pipelines in Git
- Advanced workflows with dynamic schema and inferred data types
- Monitoring and Logging
- Billing and costs
Study material
In the "Synapse Pipelines: Orchestration & Development" training, we use material that we have developed ourselves at Wortell Smart Learning. We will ensure that you receive all the necessary material in time.
Available dates
There are currently no scheduled dates available. Please contact us for options.
