Sr Data Engineer (Palantir or Databricks)

The work
• Architecture Ownership: Define and own the end‑to‑end architecture for scalable, enterprise-grade data pipelines using Databricks, Spark, and cloud-native technologies.
• Platform Strategy: Drive the technical roadmap for Databricks and/or Palantir Foundry, including best practices, governance, performance optimization, and tooling standards.
• High‑Scale Engineering: Build, optimize, and maintain distributed data processing workflows that support large-scale workloads and mission-critical analytics.
• Integration Leadership: Architect and implement data integration patterns across APIs, databases, SaaS systems, and cloud storage environments.
• Technical Leadership: Lead engineering reviews, mentor junior and mid-level engineers, and establish coding, observability, and reliability standards across the team.
• Cross‑Functional Influence: Partner with data science, analytics, and business teams to define future-state architecture and align data solutions with strategic objectives.
• Data Quality & Reliability: Implement robust data quality frameworks, monitoring, lineage, and observability processes to ensure accuracy and reliability.
• Cloud Expertise: Design and deploy cloud-native data solutions using AWS services (S3, Glue, Lambda, Redshift) or equivalent platforms.

Here’s what you need
• Data engineering experience, including architecting large-scale cloud-native systems
• Hands-on experience with Databricks or Palantir Foundry in enterprise environments
• Expert-level Python, SQL, Spark, and PySpark
• Strong background with ETL/ELT frameworks and orchestration tools (AWS, GCP, or Azure)
• Proven experience designing distributed systems, optimizing Spark jobs, and building complex data models
• Ability to lead technical decisions, influence stakeholders, and drive engineering standards
• Must be a U.S. Citizen

Bonus points if you have
• Active Secret clearance or higher
• Experience working with federal clients or regulated environments
• Experience with ElasticSearch, NiFi, ELK, Kafka, or other COTS/open-source data engineering tools
• Experience with Docker, Kubernetes, or advanced DevOps practices
• Data engineering certifications (Databricks, Palantir Foundry, Azure, GCP, IBM, etc.)
• Experience working in Agile/Scrum environments