Lead Data Software Engineer – Spark, Kafka, Kubernetes

About the role

EPAM is seeking a Lead Data Software Engineer to drive the migration of a data analytics platform to a cloud‑agnostic, on‑premises solution. You will lead the design and implementation of streaming and batch pipelines while mentoring a growing team of engineers.

Key responsibilities

• Implement streaming and batch Spark pipelines on Kubernetes.
• Lead development of objects within the data generator framework.
• Oversee deployment and testing in local and development environments.
• Continuously improve the evolving solution and resolve bugs.
• Perform unit testing to ensure high‑quality code.
• Participate in refinement, planning, and demo sessions.
• Guide and mentor team members throughout implementation.

Required profile

• 5+ years of experience with Apache Spark, Confluent/Apache Kafka, and Kubernetes.
• Strong proficiency in Python.
• Experience in data‑engineering and cloud‑agnostic platform migrations.
• Ability to quickly learn new technologies during active projects.
• Independent work style with regular syncs with a team lead.
• Excellent communication skills and proactive attitude.
• Hands‑on technical leadership.
• English proficiency at B2+ level.

Required skills

• Apache Spark
• Confluent/Apache Kafka
• Kubernetes
• Python
• Java (nice to have)
• Kafka Connect, KSQL, Kafka Streams (nice to have)
• Ansible, Argo CD (nice to have)
• Test‑Driven Development (TDD) (nice to have)

What we offer

• Innovative projects for global industry leaders.
• Flexible remote work from anywhere in Kyrgyzstan.
• Opportunities to work abroad up to two months per year.
• Leadership development, certifications, and unlimited learning resources.
• Medical and family care package, referral bonuses, and wellness benefits.