Lead Data Engineer

Lead Data Engineer

Description

We are looking for a highly experienced Lead Data Engineer to lead and guide the technical evolution of our enterprise data lake.

This role will combine hands-on development, tactical guidance, team leadership, and partner engagement across our technical team.English fluency is aMUSTfor this role!

Only candidates with levelC1orC2will be considered : A1 Beginner

A2 Elementary

B1 Intermediate

B2 Upper-Intermediate

C1 Advanced

C2 ProficientResponsibilities

Lead the implementation of scalable Lakehouse architecture using Delta and medallion architecture

Elevate data engineering skills across our team through mentorship, collaboration, and exemplification.

Lead development of scalable data pipelines using incremental ingestion and streaming

Build and maintain data infrastructure with tools like Databricks, Spark, Airflow 2 in an AWS environment

Collaborate closely with technical teams across the enterprise to help us transform complex data into actionable insights

Ensure high data quality standards by leveraging best practice approaches

Help architect storage layers, metastore / catalog solutions, and performance optimization

Oversee cloud infrastructure in AWS using Terraform, with a deep understanding of services like S3, IAM, KMS, Glue, Athena, Redshift, SNS / SQS, MSK, and Kinesis.

Help guide the integration of master data management and governance solutions like Datahub to track the health, lineage, and meaning of data across the enterprise

Help establish team coding standards and best practices.

Help refine development and operational processes to drive consistent, predictable quality

Participate in our on-call schedule to help us ensure availability of critical data services

Provide technical leadership on DevOps, security, and deployment strategies for AI / ML modelsRequirements

8+ years of experience in data engineering, including senior / lead experience

Expertise in Lakehouse architectures, Delta / Parquet file formats, hive metastore solutions (Glue, Unity Catalog), and ETL processes (Airflow and Databricks)

Strong Experience with Python and SQL

Strong proficiency in data modeling (OLTP / OLAP)

Strong experience working with AWS cloud infrastructure services

Familiarity with Infrastructure as Code (IAC) using Terraform or CloudFormation

Databricks experience

Strong AWS experience.

Certified AWS Solutions Architect or Data Engineer preferred.

Bachelor's degree in computer science, Information Systems, Data Science, Mathematics, or similar fields, or equivalent experience