Pipeline.Snapshot

by
Lookup Datamart View Names
ForEach View Name
Set Gold Folder Name
Lookup Contents of SQL Queries
Copy Data to Gold
Handle Copy Failures
Lookup Datamart View Names Retrieves the list of views that need processing
Lookup Datamart View Names Retrieves the list of views that need processing
ForEach View Name Iteration Loops through each datamart view name
Set Gold Folder Name
  • 🟠 If folder_name = snp → Set gold folder name to snapshot
  • 🔵 Else → Set gold folder name dynamically
Lookup Contents of SQL Queries Fetches query definitions for gold processing
Copy Data to Gold Copies staging (STG) data into gold container
Handle Copy Failures If Copy Data fails:
  • ⚠️ Log Error in log.comment_snp
  • 📋 Append Table Name to array_of_file_names
  • 📢 Call upsert_comment stored procedure to log failure reason

Condition: If Errors Exist 🚦 (Validation)

  • Check if array_of_file_names is greater than 0
    • ✅ If No Errors: Log success message
    • ❌ If Errors Found: Log failure message with table count

Leave a Comment