Data Engineer
Job requirements
- Work with pricing analysts to identify custom definitions, calculations, transformations, dependencies, assumptions, edge cases, and ambiguities currently implemented in the Gold layer.
- Reverse-engineer analyst-written SQL, spreadsheet logic, and ad hoc calculations to define intended business behaviour.
- Translate legacy logic into clear technical requirements and migration plans.
- Re-implement approved pricing definitions as governed, reusable, documented dbt models in the Silver layer.
- Follow data modelling, naming, testing, version control, and code review standards.
- Build maintainable, scalable, auditable models independent of undocumented analyst knowledge.
- Apply dbt tests for uniqueness, not-null, accepted values, relationships, and business rules.
- Compare migrated Silver-layer results with existing Gold-layer outputs and resolve differences.
- Validate results across representative periods, segments, boundary conditions, and known exceptions.
- Confirm migrations only after parity review and pricing analyst approval.
- Partner with pricing analysts to clarify calculations, confirm intended behaviour, and align on expected results.
- Lead walkthroughs and reviews; document analyst approval before retiring legacy definitions.
- Communicate risks, open questions, dependencies, and decisions to technical and business stakeholders.
- Document business meaning, calculation rules, assumptions, exceptions, source inputs, ownership, lineage, tests, and downstream consumers in dbt and Confluence.
- Ensure documentation enables future engineers and analysts to maintain logic without relying on the original analyst.
- Decommission ad hoc Gold-layer logic after the Silver replacement is validated, approved, adopted, and downstream dependencies are updated.
- Verify retired logic is no longer used and operational documentation reflects the new source of truth.
- 5–8 years of data engineering experience on analytical data platforms.
- Strong SQL skills: joins, window functions, CTEs, aggregations, conditional logic, and optimisation.
- Hands-on dbt experience across models, tests, documentation, and lineage.
- Strong Snowflake experience, including modelling, performance tuning, and warehouse concepts.
- Experience refactoring, migrating, or modernising complex analytical logic.
- Ability to untangle ad hoc SQL and spreadsheet calculations and validate outputs.
- Strong understanding of data quality, reconciliation, testing, and release controls.
- Skilled at working with non-engineering stakeholders to clarify requirements and validate outcomes.
- Excellent communication and documentation habits.
- SQL; Python a plus
- Snowflake
- dbt, dbt Cloud
- Airflow
- Confluence, Jira, Slack
- Data modelling, quality validation, reconciliation, migration planning, stakeholder collaboration, and technical documentation.
Key Responsibilities Logic Discovery and Requirements Analysis
Migration to the Silver Layer
Parity Validation and Quality Assurance
Analyst Collaboration and Sign-Off
Documentation and Lineage
Gold-Layer Cleanup and Cutover
Required Qualifications
Core Skills and Tools