About the role

Job Title: Data Engineer (PySpark / Databricks)

Experience: 5–9 Years Location: Pune (Hybrid – Capco Office)

Job Summary

We are looking for a skilled Data Engineer with strong expertise in PySpark, Databricks, and modern data engineering practices. The ideal candidate will have hands-on experience in building scalable data pipelines, working with large datasets, and leveraging cloud-based data platforms.

Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines Work extensively with PySpark and Apache Spark for large-scale data processing Build and manage workflows using Apache Airflow Develop and optimize data solutions on Databricks (Jobs, Delta Lake) Work with cloud-based data lakes (S3 or equivalent) Write efficient and complex SQL queries for data transformation and analysis Run and manage Spark workloads on EMR Serverless or other managed Spark platforms Ensure data quality, reliability, and performance optimization of pipelines Must Have Skills Strong hands-on experience with PySpark and Apache Spark internals Experience with Databricks (Jobs, Delta Lake) Proficiency in Apache Airflow for workflow orchestration Solid experience building ETL/ELT pipelines at scale Strong SQL skills and experience with Data Warehouse (DWH) systems Experience running Spark workloads on EMR Serverless or managed Spark platforms Hands-on experience with cloud data lakes (S3 or equivalent) Good to Have Skills Experience with Delta Lake / Apache Iceberg Exposure to streaming frameworks (Spark Structured Streaming, Kafka) Familiarity with CI/CD pipelines for data engineering workflows Knowledge of data governance, cataloging, and lineage tools

Skills & Tags

go ai data design

Aplyr's read

Capco is a global management and technology consultancy focused on the financial services sector, attracting professionals with expertise in banking, insurance, and digital transformation.
Synthesized from recent postings & public sources

What's promising

•Capco specializes in financial services, offering tailored solutions for banking and insurance clients.
•The company emphasizes digital transformation, providing opportunities to work with cutting-edge technology.
•Capco's global presence allows for diverse project experiences across different markets.

What to watch

•High-pressure environment due to demanding client expectations in the financial sector.
•Work-life balance can be challenging with frequent travel for client engagements.
•Limited public information about career advancement and internal mobility opportunities.

Why Capco

•Capco's focus on financial services differentiates it from more generalized consultancies.
•The company offers a blend of management and technology consulting, appealing to hybrid skill sets.
•Capco's culture emphasizes innovation in financial technology, attracting tech-savvy professionals.

Aplyr’s read is generated by AI from public sources. Was it useful?