Becoming a data-driven business doesn’t happen overnight. It takes time, money, and effort to develop the combination of knowledge, skills, and technological proficiency needed to weave business intelligence into the cultural fabric of your organisation. It’s a journey: one with many steps and milestones along the way. To reach your destination safely, you need a guide. … Read more
1. Model The Model component in the MVC (Model-View-Controller) design pattern represents the data and business logic of an application. It is responsible for managing the application’s data, processing business rules, and responding to requests for information from other components, such as the View and the Controller. 2. View Displays the data from the Model … Read more
Purpose The get.myview stored procedure API allows Power Applications to interact with the DataWarehouse by performing SELECT queries on specified DataMart views. The results are returned as a JSON array. Security Parameters Required Optional Typical Use URL HTTP Header: cURL Example: PowerBI: PowerShell
Stored Procedure Summary Purpose: The get.myview stored procedure is designed to generate and return a dynamic view of data based on user-defined parameters. It supports complex queries for PowerBI analysis by allowing users to specify various options such as data mart, view name, filtering conditions, and more. Key Features: Example Usage: PowerBI Parameters:DataMart_Host: bmt-dwh-uks-srv-prd.database.windows.netDataMart_Endpoint: bmt_dwh_db_glDataMart_Token: … Read more
The whereClause in the GET.myView function specifies the filter criteria to apply to the DataMart. The syntax of a whereClause is similar to an SQL WHERE clause. Key Details: Additional Rules: Example of a whereClause: In this example, the LOAD_DATE > ‘2024-04-01’ comparison uses double single quotation marks (”) to escape the inner quotes.
Objective: The Snapshot pipeline aims to streamline data analysis and management by creating optimised views from raw data files. Process Overview: Loading parquet files and Delta lake table Creation: View Creation API Access and loading into the data warehouse Conclusion: The Snapshot pipeline offers an efficient solution for transforming raw data into actionable insights. By … Read more
Purpose The Common View for Resource Details, excluding dates or measures, acts as a unified and reliable source for data on project resources. It is accessible to different departments and systems across the organisation. This view supports integration with fact tables for reporting and analysis and provides a limited set of dimension attributes. Use Case … Read more
Purpose The Supplier Invoice (value) fact table is designed to display quantitative data related to invoices generated. This table integrates with dimension tables, such as supplier_details, to provide detailed insights into the financial aspects of projects, including invoicing, revenue, and billing trends. The primary purpose is to support financial analysis and reporting across the organisation. Use … Read more
Efficient data storage and processing are crucial for businesses and organizations dealing with large datasets. Apache Parquet is a popular columnar storage format offering fast query performance and data compression, while CSV is a row-based format that may not be suitable for large-scale processing. This blog post covers how to convert CSV files to Parquet … Read more