Solution Architecture
Infrastructure.OSHrecords
Purpose The Infrastructure.OSHrecords DataMart is designed to provide a structured and centralised repository for occupational health and safety (OHS) records sourced from the myOSH system via API exports. This DataMart enables efficient reporting, compliance tracking, and analysis of safety-related incidents, risk assessments, and other regulatory documentation. By transforming the raw API data into a well-structured … Read more
White Paper: Common Data Model (Datamarts)
Proposal for a white paper on Common Data Model (Datamarts) Proposed Structure for the White Paper 1. Executive Summary 2. Introduction 3. Objectives 4. Proposed Standardised Format 5. Naming Conventions 6. Standardised Structure for Views 7. Metadata Format 8. Implementation Roadmap 9. Benefits 10. Conclusion Naming Suggestions for Views Here are some options for naming … Read more
Release 55
Additions customer_orderline_202501 This release introduces a new version of customer.orderline, evolving from the current beta version. The latest update incorporates a fully JSON-managed field structure, enhancing flexibility, data integrity, and future scalability. Key Changes & Enhancements Impact on Data Consumers Alterations project_details_plus_CAD This update continues the ongoing improvements to mapping accuracy and data consistency within … Read more
What constitutes ‘sensitive personal data’?
In its most basic definition, sensitive data is a specific set of “special categories”: Article 9 of the GDPR prohibits special category data from being processed unless you can rely on an exemption.
Standardised DataMart Model
Core Views and their Standard Structures Each DataMart will be built on six primary views: 1. Details (or Core) View Purpose: Schema: Column Name Data Type Description company VARCHAR Primary entity identifier object_id VARCHAR Unique object identifier (project_id) seq_json JSON Structured metadata reference for linking name VARCHAR Name of the project/entity description TEXT Brief descriptive … Read more
Release 54
Standardisation of DataMart View Queries for SQL OnDemand To enhance query consistency and readiness for SQL OnDemand, we have implemented a major standardisation across all DataMart view queries. Key Enhancements: ✅ Uniform Aliases for Data Sources – All table references now include consistent and meaningful aliases, improving readability and maintainability. ✅ Standard Positioning of — … Read more
Service Level Agreement (SLA) for DataMart availability
This draft framework provides a robust SLA for DataMart availability, balancing business needs, technical feasibility, and stakeholder communication. 1. Availability Target Set a clear uptime percentage that aligns with the criticality of the DataMart to your business processes. 2. Operational Hours Define the expected operational hours of the DataMart. Specify if it needs to be … Read more
Microsoft Fabric Readiness Review
Microsoft Fabric is designed to integrate multiple analytics workloads, including data engineering, data science, and business intelligence, within a unified platform. While our current Medallion structure already has a logical progression of data refinement (Bronze → Silver → Gold), there are key differences in architecture, tooling, and approach that we might consider while emulating a … Read more