Review Meetings

Frequency, location and style of meetings, who should attend. To include standups/sprint meetings, code review methods and end of project review. Incorporating an Agile approach forms an integral aspect of the Data Engineering development process. By dividing the development cycle into smaller segments, iterations, and sprints, Agile methodologies enable engineers and data specialists to collaborate … Read more

Project Organisation

People involved and their roles, chains of command and reporting methods, distribution list of SQP. Data Steering Committee Data Stewards Data Specialists Quality Behaviours of Data Specialists Data Engineering Data Mesh Principle The Data Mesh decentralises data ownership by transferring the responsibility from the central data team to the business units that create and consume … Read more

Scope of Task

Scope covered by the project and any documents received. Technical Perspective The Data Warehouse Development Quality Plan describes the creation, organisation, movement, change and storage of data from the perspective of the technical and physical implementation. The scope of the Data Warehouse Development Quality Plan encompasses the following key areas: Business Perspective The business perspective … Read more

Security Requirements

Security specific requirements for the project. Assessment of changes on existing system security. The confidentiality, integrity, and availability (CIA) model is widely used by organisations to implement appropriate security controls and policies, which helps identify key problem areas and the necessary solutions to resolve these issues. Access Control and Authorisation: Role-based access mechanisms and strict … Read more

Avoiding Architectural Debt

In Agile design, teams work in short sprints and prioritise the most important features to be developed first. Value-based prioritisation allows startups to get a Minimal Viable Product (MVP) to market quickly and start gathering feedback from customers. Additionally, and by definition, Agile design encourages safe-to-fail experimentation early on in the product’s lifecycle, giving the designer quick … Read more

How Unarticulated Needs Direct the Evolutionary Path of Agile Designs

The traditional design methods, which assume that developers fully understand the user’s requirements and that the user knows what they want before seeing the product, are often flawed in the real world. Complex systems, such as software products and their users, constantly evolve and adapt to users’ changing needs and preferences. As users interact with a software product, … Read more

A Measure of Good Design

A measure of good design is how dependent higher structures are on lower ones, and this is closely related to the principle of separation of concerns and cohesion. Cohesion is a measure of how closely the elements of a module or component work together to achieve a single, well-defined purpose. When a system is well-designed, the higher structures … Read more

Small Cross-Functional Teams Drive Agile Design

Distributed large teams can face several challenges when producing Agile architecture. One of the main challenges is communication and coordination. Team members may be located in different locations and time zones and have different cultural backgrounds. These factors make having regular meetings, sharing information, and collaborating on design decisions difficult. Additionally, distributed teams may have difficulty building trust and developing a … Read more

Hierarchical Structures and Constant Regeneration

A hierarchical system facilitates evolution and constant regeneration by allowing for modularity and flexibility in its design. The system is divided into several layers, each building upon the one below it. This structure allows individual components within each layer to be modified or replaced without affecting the overall system’s functionality. In software design, this concept … Read more