Lead Data Software Engineer with Databricks

About the role

We are looking for a Lead Data Software Engineer with deep Databricks expertise to shape and evolve end‑to‑end traceability solutions across the supply chain. You will provide technical leadership, define architectural standards, and ensure data pipelines, models, and platforms meet regulatory and business needs.

Key responsibilities

• Own technical direction for traceability data platforms, pipelines, and products.
• Define, document, and enforce architectural principles, standards, and best practices.
• Design robust pipelines, quality controls, and proactive monitoring frameworks.
• Guide data engineers on implementation choices, performance optimization, and troubleshooting.
• Contribute to continuous improvement of data quality, lineage, auditability, and operational monitoring.
• Translate regulatory and traceability requirements into technical solutions and data models.
• Collaborate with solution architects, platform teams, and external partners to ensure scalability.
• Support business and sustainability teams in adopting traceability solutions.

Required profile

• 5+ years in data, analytics, or platform engineering roles with leadership or solution ownership experience.
• Strong background in cloud‑based data platforms, preferably Azure.
• Experience in data architecture, data modeling, and design of large‑scale datasets.
• Hands‑on expertise with Databricks, Spark/PySpark, and SQL for transformation pipelines.
• Proficiency in Python, Pandas, NumPy, and ability to review code.
• Familiarity with data visualization tools such as Power BI, Tableau, and Looker.
• Experience designing data quality frameworks, lineage, monitoring, and auditability.
• Knowledge of orchestration tools like Azure Data Factory, Airflow, or AWS Glue.

Required skills

• Azure
• Databricks
• Spark / PySpark
• SQL
• Python
• Pandas
• NumPy
• Power BI
• Tableau
• Looker
• Data lakes
• NoSQL databases
• Relational databases
• Data modeling
• Data architecture
• Data quality frameworks
• Data lineage
• Monitoring & auditability
• Azure Data Factory (ADF)
• Airflow
• AWS Glue