Data Solutions Architect

Big Data Solutions ArchitectWe are seeking an experienced Big Data Solutions Architect to join our team.

As a key member of our data engineering group, you will be responsible for designing and implementing scalable data solutions that enable data-driven decision-making across the organization.This role focuses on architecting end-to-end big data pipelines, ensuring data quality, governance, and efficient processing of large datasets.Key Responsibilities : Big Data Architecture : Design and implement scalable, distributed data processing systems using big data technologies (e.g., Hadoop, Spark).

Data Pipelines : Build and optimize ETL / ELT pipelines to handle large-scale data ingestion, transformation, and storage.Data Governance : Establish data governance frameworks, including policies for data security, privacy, and compliance.Quality Control : Develop and enforce data quality standards, leveraging tools to monitor and ensure data accuracy and consistency.Cloud Integration : Design big data solutions on cloud platforms (AWS, GCP, Azure), leveraging cloud-native tools.Collaboration : Work with data engineers, analysts, and business stakeholders to align data architecture with organizational goals.Innovation and Optimization : Stay updated on big data technologies and optimize systems for performance, scalability, and cost-efficiency.Required Skills : Big Data Expertise : Hands-on experience with Hadoop, Spark, Kafka, and other big data frameworks.Data Governance : Knowledge of governance frameworks and tools like Collibra, Alation, or similar.Quality Control : Proficiency in implementing data quality measures and tools (e.g., Apache Griffin, Talend, or Informatica).

Cloud Platforms : Experience with cloud-based data solutions (BigQuery, AWS EMR, Dataproc).

Programming Skills : Proficiency in Python, Java, or Scala for data processing.Database Knowledge : Strong understanding of SQL and NoSQL databases.Problem-solving : Strong analytical skills for troubleshooting and optimizing complex data architectures.Preferred : Certifications in big data or cloud technologies (e.g., GCP Data Engineer, AWS Big Data Specialty).

Experience with MLOps pipelines and integrating AI / ML workflows with big data systems.Knowledge of metadata management and data lineage tools.Familiarity with GDPR, CCPA, and other data privacy regulations.