Associate Distinguished Engineer Solution Architect

Key Responsibilities

• Define and maintain the overall technical architecture for the KM data platform spanning ingestion storage (Databricks/Unity Catalog) transformation security and consumption layers.
• Translate multi-year program roadmaps into phased architectural plans balancing near-term pilot needs against long-term scalability.
• Set architectural standards and guardrails for data modeling pipeline design and catalog registration in partnership with the Senior Data Modeler and Data Engineers.
• Evaluate and recommend platform components tools and integration patterns (e.g. Unity Catalog vs. alternative catalogs orchestration tools vector search/embedding infrastructure).
• Lead technical design reviews and ensure alignment across data engineering security/privacy and downstream product teams (Knowledge Products Research Products).
• Own non-functional requirements: scalability performance security cost and reliability of the platform architecture.
• Serve as the primary technical point of contact for architecture workshops and planning sessions (e.g. multi-day cross-functional architecture planning events).
• Assess technical risk and dependencies across workstreams and flag issues to the Engineering Manager and program leadership.
• Stay current on Databricks platform capabilities and enterprise AI/knowledge management architecture trends bringing recommendations back to the team.

Required Qualifications

• 8 years of experience in data/solution architecture roles with significant hands-on or architectural experience in Databricks/Lakehouse environments.
• Deep understanding of Unity Catalog governance data product design and security classification models.
• Demonstrated experience architecting platforms that handle both structured and unstructured content at scale.
• Experience architecting for AI/LLM-driven consumption patterns (retrieval-augmented generation vector search agent-based data access) is highly valued.
• Strong track record of translating business/program roadmaps into technical architecture and staged delivery plans.
• Excellent stakeholder management skills able to work across engineering product legal/privacy and executive audiences.
• Experience leading architecture reviews and setting technical standards for a growing engineering team.

Preferred Qualifications

• Experience in professional services consulting or other knowledge-intensive industries.
• Familiarity with enterprise search/knowledge platforms (Glean SharePoint ServiceNow) and their integration patterns.
• Experience navigating legal/risk/privacy review processes for data platforms.
• Prior experience standing up a data platform team from a nascent or pilot stage to a scaled production capability. Success Metrics (First 612 Months)
• Documented target-state architecture for the KM data platform validated with key stakeholders (Engineering Manager Data Modeler Product Owners).
• Architectural standards adopted across data modeling and engineering workstreams.
• Clear staged technical roadmap aligned to the broader program timeline (e.g. through 2027).

Qualifications :

Must have skills: Azure Data Factory Data Modeling (Strong) Databricks

Remote Work :

No

Employment Type :

Full-time

Department / Functional Area: Engineering
Experience: years
Vacancy: 1