Stripping Tags from Rich Text Imports Using Python

When importing rich text data into a Data Warehouse (DWH), it’s often necessary to remove unwanted HTML tags while preserving essential formatting. This guide outlines a method using Python’s BeautifulSoup library to clean and structure the data efficiently. Why Strip HTML Tags? Rich text from sources like web applications, CMS platforms, and APIs often includes … Read more

Dynamic API Query Construction

The following Power Query (M) script demonstrates how to extract a list of dates from a table, convert them into a formatted string, and use them dynamically in an API request: Step-by-Step Guide Explanation: Application in get.myhistory This approach can be used within get.myhistory to dynamically retrieve historical records by supplying a list of relevant … Read more

Release 58

Additions Customer Details USD ✅ New Addition customer_details_USD – Part of the CAD/USD series of changes.⚠️ Contains Created (Hash) Customer IDs – This will be corrected in the next release. Considerations & Next Steps 🔹 Customer ID Fix Pending – Plan for a smooth transition when correcting the Customer ID field.🔹 UNION-based “mega” list for … Read more

Senior Data Engineer (Grade 4)

SFIA-8 Aligned Overview As a Senior Data Engineer (Grade 4) you design, build, and optimise scalable data platforms, pipelines, and architectures. In this role, you lead data engineering best practices, DevOps, automation, and real-time processing, working closely with Data Architects, Analysts, and Data Scientists to drive business value from data. You play a key role … Read more

Data Analyst/Report Writer (Grade 3)

(Revised for SFIA-8 Alignment) Overview As a Data Analyst / Report Writer (Grade 3) you play a pivotal role in developing insightful reports, dashboards, and data models to support data-driven decision-making. This role involves working closely with the Data Warehouse team, Finance, and Commercial teams, ensuring data integrity, governance, and automation across reporting processes. You … Read more

Data Engineer (Grade 3)

(Revised for SFIA-8 Alignment) Overview As a Data Engineer (Grade 3) you support the development of scalable, efficient, and secure data solutions. You work closely with Data Architects, Analysts, and Business Teams to ensure the effective management of structured and unstructured data across our enterprise. As part of the Data Engineering team, you are responsible … Read more

Introduction to SFIA documentation

The SFIA Framework has become the global common reference for skills and competency for thedigital world.The SFIA website – www.sfia-online.org – is the primary source of information for the SFIA Framework.Additional resources to enable skills and competency development are available along with otherelements of the supporting ecosystem.A number of core documents are available for download … Read more