Refresh SQL Endpoint Metadata

by

When working with Microsoft Fabric, one common challenge is keeping the SQL endpoint in sync with the Lakehouse endpoint after a pipeline run.

Fabric provides two endpoints for every Lakehouse:

  • Lakehouse endpoint 🏞️
    • Directly tied to the files and tables stored in the Lakehouse.
    • Always reflects the latest state after a notebook ,pipeline/dataflow writes data.
  • SQL endpoint 🗄️
    • An automatically provisioned read-only SQL endpoint for each Lakehouse in Microsoft Fabric.
    • Provides a relational interface for querying Lakehouse tables using T-SQL.
    • Useful for BI tools, dashboards, and ad-hoc SQL queries.

Why does the SQL endpoint sometimes lag?

The SQL endpoint doesn’t automatically refresh its metadata the moment new data lands in the Lakehouse. Instead, a background process in Fabric periodically syncs schema updates.. This can lead to a delay where:

  • The Lakehouse shows the latest tables and schema.
  • The SQL endpoint still reflects the old state.

This lag can cause confusion when running queries immediately after a pipeline finishes.

The Quick Fix: A Notebook to Refresh Metadata

The fastest way to ensure the SQL endpoint is up to date is to run a Fabric Notebook that manually refreshes the metadata. This forces the SQL endpoint to sync with the Lakehouse right away.

How it works

  1. The notebook looks up the Lakehouse (lakehouse_silver in this example).
  2. It retrieves the associated SQL endpoint ID.
  3. It calls the refreshMetadata API to force a sync.
  4. If successful, it waits 60 seconds to give the refresh time to complete.

Benefits

  • ✅ Ensures your T-SQL queries always reflect the latest Lakehouse state.
  • ✅ Eliminates confusion after pipeline runs.
  • ✅ Provides a repeatable, automated way to keep endpoints aligned.

Source: Items – Refresh Sql Endpoint Metadata – REST API (SQLEndpoint) | Microsoft Learn

Leave a Comment