Lead Data Engineer

Lead Data Engineer

Description

We are looking for a highly experienced Lead Data Engineer to lead and guide the technical evolution of our enterprise data lake. This role will combine hands-on development, tactical guidance, team leadership, and partner engagement across our technical team.

English fluency is a MUST for this role! Only candidates with level C1 or C2 will be considered :

A1 Beginner

A2 Elementary

B1 Intermediate

B2 Upper-Intermediate

C1 Advanced

C2 Proficient

Responsibilities

• Lead the implementation of scalable Lakehouse architecture using Delta and medallion architecture
• Elevate data engineering skills across our team through mentorship, collaboration, and exemplification.
• Lead development of scalable data pipelines using incremental ingestion and streaming
• Build and maintain data infrastructure with tools like Databricks, Spark, Airflow 2 in an AWS environment
• Collaborate closely with technical teams across the enterprise to help us transform complex data into actionable insights
• Ensure high data quality standards by leveraging best practice approaches
• Help architect storage layers, metastore / catalog solutions, and performance optimization
• Oversee cloud infrastructure in AWS using Terraform, with a deep understanding of services like S3, IAM, KMS, Glue, Athena, Redshift, SNS / SQS, MSK, and Kinesis.
• Help guide the integration of master data management and governance solutions like Datahub to track the health, lineage, and meaning of data across the enterprise
• Help establish team coding standards and best practices.
• Help refine development and operational processes to drive consistent, predictable quality
• Participate in our on-call schedule to help us ensure availability of critical data services
• Provide technical leadership on DevOps, security, and deployment strategies for AI / ML models

Requirements