Ali Ahmed When working with Microsoft Fabric, one common challenge is keeping the SQL endpoint in sync with the Lakehouse endpoint after a pipeline run. Fabric provides two endpoints for every Lakehouse: Why does the SQL endpoint sometimes lag? The SQL endpoint doesn’t automatically refresh its metadata the moment new data lands in the Lakehouse. Instead, a … Read more
Today I stumbled upon a neat little trick in Power BI’s Power Query Editor, and it’s too good not to share! If you’ve ever loaded data into Power BI and ended up manually renaming columns one by one to clean them up, there’s a much faster way. 💡 You can rename all columns at once … Read more
Staging: Gl: Logging:
List all Error and Comment logging processes, that will be removed if we do not have an SQL server The current logging processes include transaction and error logs. These logs are stored in tables within the staging (stg) and Global Org (GL) databases. GL: Stg:
Data Source Identification: Data Ingestion and Storage: Data Staging: View Creation: Automated Deployment: Snapshot Creation and Storage:
Step 1: Download SQL Files from Azure Storage Step 2: Flatten Directory Structure and Compare Files Using Python Notebook Step 3: Analyse Differences Using Visual Studio Code
Purpose: Transfer SharePoint List data through azure blob storage and into the DataWarehouse Requires: Prerequisites: Process Steps: 4. Data Pipeline 5. Create pipeline parameters. Create two pipeline parameters. 5. Activities Tab 6. Look up Activity Click on the Lookup activity and go to settings, from there if not already set up create a new connection … Read more