Senior Data Engineer/ Architecture (ADF & Databricks)

The Senior Data Engineer will report to the Senior Director of Data & AI and will help ensure UVVC’s implementation and use of technology meets the business needs and supports the company mission. As key member of the Data team, you’ll collaborate with other leaders and their teams across the entire organization.  In this part of your role, you will create opportunities for strategy, leverage and technology direction solutions across UVVC business units.

This position is open to remote candidates. 

Data Pipeline Development & Orchestration

• Design, build, and optimize end-to-end ETL/ELT pipelines using Azure Data Factory and Databricks.
• Develop robust ingestion frameworks for batch and streaming data from APIs, databases, SaaS platforms, and internal systems.
• Create scalable and architecturally sound data transformation frameworks using Delta Lake (medallion architecture), Spark, and SQL, aligned with enterprise Lakehouse standards.

Implement CI/CD, parameterization, triggers, and pipeline automation best practices

Azure Data Platform Engineering

• Architect, manage, and optimize enterprise data environments across ADLS, Azure SQL, and Databricks, including cluster design, cost governance, and workload isolation strategies.
• Conduct performance tuning, cluster scaling, monitoring, and cloud cost optimization.
• Implement DataOps practices including testing, version control, monitoring, and documentation.

 Data Quality, Governance & Reliability

• Build data validation, auditing, and error-handling frameworks to ensure accuracy and consistency.
• Maintain documentation, naming standards, metadata structures, and governance best practices.
• Troubleshoot complex data issues and deliver sustainable technical solutions.

Databricks Architecture & Platform Ownership

• Design and implement enterprise-grade Databricks Lakehouse architecture (Bronze, Silver, Gold layers).