Load data with data pipelines into SQL database in Microsoft Fabric
Applies to: ✅ SQL database in Microsoft Fabric
In this tutorial, you create a new pipeline that loads sample data from an Azure SQL Database into a SQL database in Fabric.
A data pipeline is a logical grouping of activities that together perform a data ingestion task. Pipelines allow you to manage extract, transform, and load (ETL) activities instead of managing each one individually.
Prerequisites
- You need an existing Fabric capacity. If you don't, start a Fabric trial.
- Create a new workspace or use an existing Fabric workspace.
- Create or use an existing SQL database in Fabric. If you don't have one already, create a new SQL database in Fabric.
- Create or use an existing Azure SQL Database, with data.
- The source Azure SQL Database can be either a single database or a database in an elastic pool.
- If you don't have an Azure SQL Database, create a new single database. Use the Azure SQL Database free offer if you haven't already.
- For this walkthrough, we rely on opening ports on the source Azure SQL Database using the "Allow Azure services and resources to access this server" setting.
- For heightened security, consider a gateway to transfer your data. For more information, see How to access on-premises data sources in Data Factory.
Create data pipeline
- In your workspace, select + New, then More options.
- Under Data Factory, select Data pipeline.
- Once the data pipeline is created, under Start with guidance, choose Copy data assistant.
- In the Choose data source page, select Azure SQL Database.
- Provide authentication for the connection to the source Azure SQL Database.
- For the destination, choose Fabric SQL database from the list in the OneLake catalog.
- Select Next.
Load data
- On the Connect to data destination page, select Load to new table for each table. Verify the mappings for each table. Select Next.
- Review Source and Destination details.
- Check the box next to Start data transfer immediately.
- Select Save + Run.
- In the Activity runs pane, you should see all green checkmarks for successful copy activities. If there are any errors, troubleshooting information is available in the failed row.