At Divar, millions of people buy, sell and find what they need every day. Behind every data-driven decision and every AI feature in the product is a platform built and run by our team. We're looking for an AI & Big Data Platform Engineer to help build the next generation of that platform: on-premise, open source, and handling millions of events per day.
What the team does
Product teams across Divar use our platform for two missions:
- Data analysis: building pipelines, data products and dashboards to make data-driven decisions. This all runs on-premise on open-source tools such as Kafka, Spark, Trino, Iceberg, Ceph, S3, ClickHouse, Airflow, Metabase, DataHub, JupyterHub and Coder.
- AI in the product: fine-tuning, serving and evaluating models, and calling external AI providers safely. As the platform team, we build the self-service infrastructure behind this, so product teams can do it without managing GPUs or provider accounts themselves. We run our own GPU fleet with vLLM, operate a single AI gateway in front of both our self-hosted models and providers such as OpenAI, Anthropic and Google, and add cloud GPUs when extra capacity is needed.
We don't just move data. We build self-service platforms so that business data analysts, software engineers and ML engineers can ship on their own.
Responsibilities
- Design, build and operate our big data platform (lakehouse, query engines, orchestration) with a focus on reliability, performance and cost.
- Build and run LLM-serving infrastructure on our GPU clusters (vLLM on Kubernetes), including capacity planning, benchmarking and model rollouts.
- Operate and evolve our AI gateway: provider routing and failover, virtual keys and quotas, observability and tracing (Langfuse), and usage and cost tracking for each team.
- Build fine-tuning and MLOps infrastructure: training jobs on on-premise and rented GPUs, experiment tracking and a model registry (MLflow), and a clear path from experiment to production.
- Provide self-service development environments (JupyterHub, Coder) for data analysis and model development.
- Provide infrastructure for data governance, data quality, access control and privacy (e.g. PII anonymization) across the data and AI lifecycle.
- Own monitoring, on-call and incident response for the services you run.
Requirements
- Bachelor's degree in Computer Science, Computer Engineering or a related field, or equivalent experience.
- Proven experience as a Platform, Infrastructure/SRE, Data Infrastructure, MLOps or similar engineer.
- Strong Kubernetes and Docker skills, including running stateful services in production.
- Experience with GitOps and infrastructure as code: ArgoCD, Helm, Ansible, Terraform/Pulumi.
- Hands-on experience with big data technologies such as Kafka, Spark, Trino, Iceberg, Ceph, S3 and with orchestration tools such as Airflow.
- Proficiency in Python.
- A solid understanding of how LLMs work and how they're used in products: inference, context and tokens, prompting, RAG, and the basics of fine-tuning.
- Experience setting up, maintaining and customizing open-source tools (e.g. Airflow, JupyterHub, Coder, DataHub, MLflow, LiteLLM).
- A good understanding of data governance and security best practices.
- Strong problem-solving skills and a proactive approach to technical challenges.
- Clear communication and effective teamwork.
- Ownership of what you build.
Nice to have
- Experience with Go or Java.
- Familiarity with the ML lifecycle, model training workflows and MLOps tools such as MLflow.
- Experience with LLM serving (vLLM, SGLang, TensorRT-LLM) and GPU infrastructure (NVIDIA drivers, GPU scheduling on Kubernetes, multi-GPU inference).
- Observability (Prometheus/VictoriaMetrics, Grafana) and secret management (Vault).
Benefits
- You'll have a big impact. Your work will power data and AI features used by millions of people.
- You'll work on hard, modern problems, from large-scale data to running LLMs on our own GPU fleet.
- You'll be part of a team that values craftsmanship, speed and innovation.
- We offer flexible working hours, health insurance, and a culture that values learning and autonomy.
- You'll work with people who care about doing great work and enjoy it along the way.