CDM Migration Playbook

by

Purpose

This playbook guides the phased migration from existing DataMart views (202409) to CDM-standard views, including core, meta (dates, codes, values), and item tables. It supports the transition to IFS Cloud and ensures a consistent, scalable data model across all domains.

MVP Scope

Domains required for MVP:

  • Project
  • Employee
  • Sales Hierarchy
    • Busopp (Business Opportunity)
    • Order
    • Invoice
  • Supplier & Customer
  • Resource & Person
  • Finance & GeneralLedger
  • Global (via MIS80) & Dates & Programme

Phased Delivery Approach

Phase 1: Live Data Seeding (Apps 10)

  • Recreate vanilla source views from FUNC (e.g. DIM_EMPLOYEE_OL)
  • Seed core, meta, and item tables with current data
  • Validate against existing 202409 views

Phase 2: Historical Backfill (1 Year)

  • Identify archive/export tables with relevant history
  • Load into item/meta tables using natural keys and dates
  • Ensure historical presence aligns with RPV expectations

Phase 3: RPV Framework

  • Implement Last-Seen and Row Presence Verification for *_meta_values and *_item_values
  • Use LAST_SEEN_DATE, IS_ACTIVE, DATE_FROM, DATE_TO attributes

Phase 4: Pipeline Integration

  • Apply upsert patterns (cdm.post_insert, cdm.put_update, cdm.patch_upsert)
  • Schedule refresh logic based on Apps 10 feeds

Phase 5: Cutover to IFS Cloud

  • Switch source logic from 202409/FUNC to IFS Cloud equivalents
  • Revalidate view logic and object structures

Outline Plan and Expectations

To ensure efficient delivery and reduce uncertainty, the migration plan is structured in two key streams:

Stream A: Core / Code / Dates Views (April–May)

These views represent slowly changing dimensions and will be prioritised first. These do not rely on the RPV framework and are essential to underpin the rest of the CDM structure.

  • Workload: Shirt size = Average. Familiar work with manageable assumptions.
  • Domains covered:
    • Group 1: Project & Employee,
    • Group 2: Supplier & Customer
    • Group 3: Sales Hierarchy (BusOpp, Order, Invoice)
    • Group 4: Global
    • Group 5: Utilisation (Person, Resource, project Items)
  • Goal: Deliver consistent core, meta_codes, and meta_dates views for all MVP domains.
  • Progress Management: Weekly delivery dashboard used to track completion and unblock tasks.

Stream B: Value Views (Commencing June)

Value views depend on the RPV MVP being implemented. These involve more complexity, such as incremental loads, historical presence, and tracking business measures.

  • Workload: Shirt size = Large. Requires team collaboration and validated assumptions.
  • Domains covered:
    • Finance: cdm.finance_meta_values
    • GeneralLedger: cdm.generalledger_core, cdm.generalledger_meta_codes, cdm.generalledger_meta_dates, cdm.generalledger_meta_values
    • Sales Hierarchy: cdm.busopp_meta_values, cdm.invoice_meta_values, cdm.order_meta_values, cdm.order_item_meta_value, cdm.order_item_meta_codes, cdm.order_item_meta_dates
    • Project: cdm.project_meta_values, cdm.project_item_meta_values_cost, cdm.project_item_meta_values_expenses, cdm.project_item_meta_values_revenue, cdm.project_item_meta_values_time
    • Employee: cdm.employee_item_meta_values_FTE, cdm.employee_item_meta_values_hours
  • Goal: Deliver robust meta_values and item_meta_values views backed by RPV and audit-ready logic.

As value views are established, their structure and metadata will align closely with the patterns developed during the core/meta phase, reducing future effort and increasing confidence.

Agile Delivery Roadmap (April–July 2025)

Monthly Focus

MonthSprint ThemesKey DeliverablesOutcomes Expected
April🔹 Core, Meta Codes, Meta Dates 🔹 Foundations– Complete View Group 1 & 2 (Project, Employee, Supplier, Customer) – Finalise CDM Delivery Dashboard – Confirm FUNC data mappingsFoundation for MVP domains laid. Visibility in Kanban.
May🔹 Expand Meta Views 🔹 Finalise RPV MVP– Complete View Groups 3–5 (BusOpp, Global, Person, Resource) – Stand up RPV MVP logic in Dev – Start project_item/meta scaffoldingAll core/code/date views scoped. RPV dev ready to support value views.
June🔹 Value Views Development 🔹 RPV Integration– Begin Value Groups 1–4 – RPV applied to meta_values views – Initial testing of incremental loadsValue views scoped, integrated with RPV. Initial testing complete.
July🔹 Value Views Expansion 🔹 Test & Optimise– Complete build of large shirt size views – Performance validation – Documentation and reviewCDM value layer stabilised. Ready for stakeholder visibility and refinement.

Kanban and Ticketing

  • Create tickets per domain/view
  • Use tags: CDM_MIGRATION, MVP_OCT2025
  • Columns:
    • Backlog
    • Source Identified
    • In Progress
    • Ready for RPV
    • Verified
    • Complete (MVP Ready)

Suggested Ticket Fields:

  • Domain (e.g. Employee)
  • View Type (Core, Meta Values, etc.)
  • Source Type (Vanilla, QDIM, QFACT)
  • RPV Required (Yes/No)

View Implementation Steps

📌 Core Steps per View (applies to each domain view)

StepTaskDescription
1Source AlignmentConfirm vanilla source available (FUNC database, DIM_OL) or flag if QDIM/QFACT requires reverse-engineering
2Bronze View CreationRecreate base source query as DIM_XXX_OL for bronze layer
3Core View CreationCreate cdm.[domain]_core 
4Meta ViewsCreate cdm.[domain]_codes, cdm.[domain]_dates, cdm.[domain]_values
5Item View(s)Create cdm.[domain]_item_XXX where appropriate (e.g., employee_absence, project_cost)
6Value Presence Verification (RPV)Implement RPV/last-seen tracking for _values views
7Upsert IntegrationIntegrate cdm.patch_upsert, cdm.post_insert, cdm.put_update into pipelines
8Data VerificationEnsure output replicates or improves on 202409 query results
9Documentation & MetadataRegister in metadata dictionary, update data catalogue, create knowhow page

Governance and Documentation

  • Every CDM view should have an entry in the metadata (attribute lineage) catalogue
  • Use OBJECT_METADATA field for references, lineage, and notes
  • Register transformations and source queries in KnowHow

Optional Enhancements

  • Power BI dashboard for Kanban progress
  • Automation to detect missing last-seen values
  • Scheduled audit of expired but not replaced values

Contacts

  • Data Engineering Lead: [Name]
  • Data Governance: [Name]
  • CDM Architect/Designer: [Name]

Version: Draft 1.1 – July 2025

List of Views

Views Group 1 (Central Domains)

Project

  • cdm.project_core_detail
  • cdm.project_meta_codes
  • cdm.project_meta_dates

Employee

  • cdm.employee_core_detail
  • cdm.employee_meta_codes
  • cdm.employee_meta_dates

Views Group 2 (Associated Domains)

Supplier & Customer

  • cdm.supplier_core_detail
  • cdm.customer_core_detail

Views Group 3 (Sales Hierarchy)

BusOpp

  • cdm.busopp_core_detail
  • cdm.busopp_meta_codes
  • cdm.busopp_meta_dates

Customer

  • cdm.invoice_core_detail
  • cdm.invoice_meta_codes
  • cdm.invoice_meta_dates
  • cdm.order_core_detail
  • cdm.order_meta_codes
  • cdm.order_meta_dates

Views Group 4 (Peripherals)

Global

  • cdm.global_meta_codes_company_helper
  • cdm.global_meta_codes_cost centre
  • cdm.global_meta_codes_programme
  • cdm.dates

Views Group 5 (Utilisation)

Person

  • cdm.person_core_detail
  • cdm.person_meta_codes
  • cdm.person_meta_dates

Resource

  • cdm.resource_core_detail
  • cdm.resource_item_codes
  • cdm.resource_item_dates

Project Items

  • cdm.project_item_codes
  • cdm.project_item_dates

Value Views (require RPV)

Values Group 1 (Finance)

Finance

  • cdm.finance_meta_values

GeneralLedger

  • cdm.generalledger_core_detail
  • cdm.generalledger_meta_codes
  • cdm.generalledger_meta_dates
  • cdm.generalledger_meta_values

Values Group 2 (Sales Hierarchy)

  • cdm.busopp_meta_values
  • cdm.invoice_meta_values
  • cdm.order_meta_values
  • cdm.order_item_values
  • cdm.order_item_codes
  • cdm.order_item_dates

Values Group 3 (Project)

  • cdm.project_meta_values
  • cdm.project_item_values_cost
  • cdm.project_item_values_expenses
  • cdm.project_item_values_revenue
  • cdm.project_item_values_time

Values Group 4 (Employee)

  • cdm.employee_item_values_absence
  • cdm.employee_item_values_expense
  • cdm.employee_item_values_hours

Leave a Comment