Identification of controlled data, its classification, and associated processes.
Data Identification
Inventory of Data Assets
Data Cataloguing: Create a comprehensive inventory of all data assets across your organisation, including databases, data warehouses, data marts, and data lakes.
Data Source Mapping: Identify and map all data sources, including internal systems and third-party sources.
All Data Sources SHALL be described in myBMT
The EA solution SHALL map the Enterprise solutions and interactions
Data Discovery Tools
User Discovery: All data sources that are in-scope are discoverable to authorised users including exploration of data within the export.
myBMT SHALL support data discovery by showing splash views of data container objects
Data Classification
Define Classification Levels
Sensitivity Levels: Establish clear classification levels (e.g., Public, Internal, Confidential, Restricted) based on data sensitivity and regulatory requirements.
Classification Criteria: Define criteria for each classification level, considering factors such as data type, usage, impact of disclosure, and regulatory implications.
Sensitivity Levels SHALL be defined by the DataMart Owner in the DPIA
The schema mvw and dvw SHALL be the public view
Confidential data SHALL be in alternative schema (e.g. fa for financial)
Alternative Schemas SHALL have Restricted access policy applied
Tagging and Labelling
Data Tagging: Implement data tagging mechanisms to label data according to its classification level.
Metadata Management: Use metadata management tools to store and manage classification tags and labels.
Data SHALL be tagged for metadata classification (e.g. Date, Measure, Code/Dim, Sequence)
Data SHOULD be tagged for owner
Data Fields SHOULD be described in myBMT
Associated Processes
Access Control
Role-Based Access Control (RBAC): Implement RBAC to ensure that users have access only to the data necessary for their roles.
Least Privilege Principle: Apply the least privilege principle to minimise access to sensitive data.
There SHALL be two Database access groups Data Admin and Data User
Data Admin SHALL be the System Administrator and DB Owner
Data User SHALL be the public access group
Data User SHALL be member of the Common Data Model and read access to mvw and dvw
Other Read access SHALL be explicitly applied
All users SHALL access via Microsoft Entra/MFA
Azure applications SHALL access via SQL User ([dv] (read/write all), [my] (read all))
The user [bi] SHALL be used to validate user access for Data User
Data Handling Procedures
Data Handling Policies: Develop and enforce data handling policies for each classification level, detailing how data should be accessed, stored, transmitted, and disposed of.
Data Encryption: Use encryption for sensitive data both at rest and in transit.
Data Handling Policy SHALL be described by the Data Owner in the DPIA
Monitoring and Auditing
Activity Monitoring: Implement continuous monitoring of data access and usage to detect unauthorised access and potential breaches.
Audit Trails: Maintain detailed audit trails of data access and modifications for accountability and compliance.
All source data SHALL be archived before processing
Transfer Logs SHALL be applied to each stage of the Pipeline
Execution by external Agent SHALL be recorded to identify the Agent (ReportID) and Calling User
All Data Views (DataMarts) SHALL be checked that
- They open without error
- They contain row data
Data Lifecycle Management
Data Retention Policies: Establish data retention policies based on data classification, specifying how long data should be retained and when it should be archived or deleted.
Data Archiving: Implement archiving procedures for long-term storage of data that is no longer actively used but must be retained for compliance or historical purposes.
All source data SHALL be archived before processing (see Monitoring and Auditing)
All DataMart view SHALL be snapshot every week (on Sunday)
There is currently NO Retention Policy automatically applied in the Data Warehouse archives will be held for at least 6 years IAW the BMT Data Retention Business Procedure
Data Protection and Compliance
Compliance Frameworks: Align your data classification and handling processes with relevant regulatory frameworks (e.g., GDPR).
Data Protection Impact Assessments (DPIAs): Conduct DPIAs for projects involving sensitive data to identify and mitigate risks.
DPIA WILL be completed by the Data Owner and implemented within the Data Warehouse by the Data Engineer
Employee Training and Awareness
Training Programs: Provide regular training for employees on data classification, handling procedures, and security best practices.
Awareness Campaigns: Conduct awareness campaigns to reinforce the importance of data protection and compliance.
KnowHow SHALL describe sufficient knowledge including how to get to DataMart, Who is the owner, what is the intended purpose, Quality dimensions and return values.
Technology and Tools
Data Loss Prevention (DLP)
DLP Solutions: Deploy DLP solutions to monitor, detect, and prevent unauthorised access, use, or transmission of sensitive data.
Policy Enforcement: Configure DLP policies to enforce data classification and handling rules.
All Data Sources SHALL be maintained in Azure
All Code Base SHALL be maintained in DevOPs
Where possible, Production Services SHOULD be updated by deployment from UAT
There is currently NO specific DLP Policy
Data Masking and Anonymisation
Data Masking: Use data masking techniques to obscure sensitive information in non-production environments.
Anonymisation: Apply anonymisation methods to remove personally identifiable information (PII) where possible, reducing the risk of data breaches.
Data Masking WILL be described by the Data Owner in the DPIA
Data Masking SHALL be performed using MD5
Data Linking Keys SHALL be performed using MD5
Data Governance Platforms
Integrated Platforms: Implement integrated data governance platforms that provide tools for data cataloguing, classification, policy management, and compliance tracking.
Workflow Automation: Use workflow automation within these platforms to streamline data classification and handling processes.
The DataMart SHALL be designed to primarily support the performance of report writing using Power BI
myBMT SHALL support the governance of the design of DataMarts
With tickets myBMT SHALL maintain a record of actions performed on DataMarts
With tickets myBMT shall support the workflow for DataMart improvements
