A data engineer develops and constructs data products and services, and integrates them into systems and business processes.
https://ddat-capability-framework.service.gov.uk/role/data-engineer
You will need the following skills for this role, although the level of expertise for each will vary, depending on the role level.
Data Engineer
A data engineer delivers the designs set by more senior members of the data engineering community.
At this role level, you will:
- implement data flows to connect operational systems, data for analytics and business intelligence (BI) systems
- document source-to-target mappings
- re-engineer manual data flows to enable scaling and repeatable use
- support the build of data streaming systems
- write ETL (extract, transform, load) scripts and code to ensure the ETL process performs optimally
- develop business intelligence reports that can be reused
- build accessible data for analysis
- EO/HEO
Senior Data Engineer
A senior data engineer designs and leads the implementation of data flows to connect operational systems, data for analytics and business intelligence (BI) systems.
At this role level, you will:
- recognise opportunities to reuse existing data flows
- lead the build of data streaming systems
- optimise the code to ensure processes perform optimally
- lead work on database management
- HEO/SEO
Lead Data Engineer
A lead data engineer is responsible for the design and implementation of numerous complex data flows to connect operational systems, data for analytics and business intelligence (BI) systems.
At this role level, you will:
- recognise and share opportunities to reuse existing data flows between teams
- be responsible for the build of data-streaming systems
- co-ordinate teams and set best practice and standards
- apply knowledge of systems integration to your work
- champion data engineering across government
- Grade SEO/G7
Head of Data Engineering
A head of data engineering leads multi-functional delivery teams to deliver robust data services for their department, other government departments and private sector partners.
At this role level, you will:
- inspire best practice for data products and services within your teams
- build data engineering capability by providing technical leadership and career development for the community
- work with other senior team members to identify, plan, develop and deliver data services
- Grade 6/7