AI & Big Data Platform Engineer

At Divar, millions of people buy, sell and find what they need every day. Behind every data-driven decision and every AI feature in the product is a platform built and run by our team. We're looking for an AI & Big Data Platform Engineer to help build the next generation of that platform: on-premise, open source, and handling millions of events per day. What the team does Product teams across Divar use our platform for two missions: Data analysis: building pipelines, data products and dashboards to make data-driven decisions. This all runs on-premise on open-source tools such as Kafka, Spark, Trino, Iceberg, Ceph, S3, ClickHouse, Airflow, Metabase, DataHub, JupyterHub and Coder. AI in the product: fine-tuning, serving and evaluating models, and calling external AI providers safely. As the platform team, we build the self-service infrastructure behind this, so product teams can do it without managing GPUs or provider accounts themselves. We run our own GPU fleet with vLLM, operate a single AI gateway in front of both our self-hosted models and providers such as OpenAI, Anthropic and Google, and add cloud GPUs when extra capacity is needed. We don't just move data. We build self-service platforms so that business data analysts, software engineers and ML engineers can ship on their own. Responsibilities Design, build and operate our big data platform (lakehouse, query engines, orchestration) with a focus on reliability, performance and cost. Build and run LLM-serving infrastructure on our GPU clusters (vLLM on Kubernetes), including capacity planning, benchmarking and model rollouts. Operate and evolve our AI gateway: provider routing and failover, virtual keys and quotas, observability and tracing (Langfuse), and usage and cost tracking for each team. Build fine-tuning and MLOps infrastructure: training jobs on on-premise and rented GPUs, experiment tracking and a model registry (MLflow), and a clear path from experiment to production. Provide self-service development environments (JupyterHub, Coder) for data analysis and model development. Provide infrastructure for data governance, data quality, access control and privacy (e.g. PII anonymization) across the data and AI lifecycle. Own monitoring, on-call and incident response for the services you run. Requirements Bachelor's degree in Computer Science, Computer Engineering or a related field, or equivalent experience. Proven experience as a Platform, Infrastructure/SRE, Data Infrastructure, MLOps or similar engineer. Hands-on experience operating and troubleshooting production services: debugging incidents, finding performance bottlenecks, and making systems reliable. Strong infrastructure fundamentals: Linux, networking, containers and Kubernetes, and infrastructure as code/GitOps practices (e.g. ArgoCD, Helm, Ansible, Terraform). Solid programming skills in Python. Depth in at least one of these areas: Data systems: distributed data processing, streaming or storage (e.g. Kafka, Spark, Trino, Iceberg, Ceph/S3). AI infrastructure: model serving, GPU workloads or training pipelines. Experience setting up, maintaining and customizing open-source software in production. A good understanding of data governance and security best practices. Strong problem-solving skills and a proactive approach to technical challenges. Clear communication and effective teamwork. Ownership of what you build. Nice to have You don't need experience with every tool listed here. These are things we'd be glad to see: Experience with Go or Java. Experience with tools in our stack, such as Airflow, ClickHouse, JupyterHub, Coder, DataHub, MLflow or LiteLLM. Familiarity with the ML lifecycle, model training workflows and MLOps tools such as MLflow. Experience with LLM serving (vLLM, SGLang, TensorRT-LLM) and GPU infrastructure (NVIDIA drivers, GPU scheduling on Kubernetes, multi-GPU inference). Observability (Prometheus/VictoriaMetrics, Grafana) and secret management (Vault). Benefits You'll have a big impact. Your work will power data and AI features used by millions of people. You'll work on hard, modern problems, from large-scale data to running LLMs on our own GPU fleet. You'll be part of a team that values craftsmanship, speed and innovation. We offer flexible working hours, health insurance, and a culture that values learning and autonomy. You'll work with people who care about doing great work and enjoy it along the way.