Role Summary
You are an experienced AI Data Engineer, who will work together with AI Agent Engineers to design and create the AI harness, agent framework, and reusable engineering components required to operate a modern data platform based on Apache Iceberg, Trino, and Parquet.
Your primary responsibility is to provide the data engineering know-how behind the agentic execution framework that enables multiple specialized agents to collaborate, execute tasks, maintain state, orchestrate workflows, and deliver end-to-end data engineering capabilities.
This is an exciting opportunity for someone to fuse their strong data engineering background with modern agentic AI automation.
Key Responsibilities
Setting Data Engineering Foundations
- Iceberg table definitions and schemas
- Trino SQL scripts and views
- Python data pipeline code
- Data quality validation suites
- Metadata and lineage documentation
- CI/CD deployment artifacts
- Monitoring and observability configurations
- Technical documentation and runbooks
Supporting Specialized Agents Creation
Supporting AI Agent Engineers as SME to design and validate AI agents to perform the following:
- Discover Iceberg catalogs and schemas
- Profile datasets
- Generate metadata and lineage
- Ingest data from databases, APIs, files, and streams
- Create Parquet datasets
- Register and maintain Iceberg tables
- Generate Trino SQL
- Build transformation workflows
- Create curated and business-ready datasets
- Validate schemas and data integrity
- Detect anomalies
- Generate reconciliation reports
- Analyze Trino workloads
- Recommend partitioning strategies
- Optimize Iceberg tables and Parquet files
- Classify sensitive information
- Manage metadata and lineage
- Apply retention and compliance policies
- Monitor pipelines and agent activities
- Detect failures and bottlenecks
- Generate alerts and operational insights
Skills and Experience
The candidate should demonstrate expertise in:
- Minimum 4++ years of relevant Data Engineering experience.
- Proficiency in Python, including FastAPI, Pydantic and Pandas.
- Experience with Apache Iceberg, Trino, and Parquet-based lakehouse architectures, or their equivaIents.
- Experience building data ingestion, transformation, and orchestration pipelines.
- Strong knowledge of metadata management, data lineage, and data governance practices.
- Familiar with modern data engineering technologies, including DuckDB, dbt, and related engineering tools for data transformation, modelling, and pipeline development.
- Experience developing REST APIs, SDKs, and reusable engineering frameworks.
- Experience with observability, monitoring, tracing, and operational support practices.
- Strong understanding of distributed systems, event-driven architectures, and microservices patterns.
- Working IeveI exposure to Generative AI, Agentic AI, or LLM-based solutions.
- Proven ability to design, develop, and operationalize enterprise-grade AI agents for data discovery, ingestion, transformation, optimization, governance, and observability is a strong bonus.
- Experience designing and implementing AI Harness, multi-agent frameworks, LangChain and LangGraph is a bonus.
- Senior engineers will be expected to have demonstrated evidence of leading technical designs, guiding junior engineering team members, working independently with clients and delivery teams, and translating AI/data requirements into scalable, secure, and deployable solutions.
We are driven by our AEIOU beliefs—Adventure, Excellence, Integrity, Ownership, and Unity—and we seek individuals who embody these values in both their professional and personal lives. We are committed to our Impact: Valuing our clients, Growing our people, and Creating our future.
Together, we make the extraordinary happen.
Learn more about us at ncs.co and visit our LinkedIn career site.
Scam Alert
We are aware of fraudulent job offers and impersonations of NCS recruiters. Phishing emails using convincing-looking but fake addresses are also commonly used to trick you into thinking that they come from official NCS sources.
Please note that all official communications from NCS Group will only be sent from verified corporate email addresses. Always check that the sender’s email address ends with the genuine NCS domain, @ncs.com.sg and beware of extra letters, symbols or misspellings. When in doubt, verify the sender’s identity by contacting us at reachus@ncs.com.sg.