Accolite
Data Engineer – Databricks / PySpark / Spark
Job Description
Job Summary:
We are seeking an experienced Data Engineer with strong hands-on expertise in Databricks, PySpark, and Apache Spark to design and develop scalable data pipelines for analytics and operational workloads. The ideal candidate will work across data ingestion, transformation, optimization, and orchestration in cloud-based environments.
Key Responsibilities:
Design and build ETL/ELT pipelines using Databricks, PySpark, and Spark-based frameworks.
Develop reusable and high-performance data processing workflows for batch and streaming workloads.
Integrate data from multiple structured and unstructured sources into cloud data platforms.
Optimize Spark jobs for performance, reliability, and cost efficiency.
Work with Delta Lake or similar lakehouse architectures for data versioning and ACID compliance.
Collaborate with data architects, analysts, and application teams to deliver clean, reliable datasets.
Write efficient code in PySpark, SQL, and Python.
Implement CI/CD for data workflows and follow best practices in data governance and security.
Troubleshoot data pipeline issues and support production workloads.
Required Qualifications:
5-7 years of experience in data engineering and distributed data processing.
Strong hands-on experience in Databricks, PySpark, and Apache Spark.
Proficient in SQL for transformations, profiling, and validation.
Experience with cloud platforms (AWS/Azure/GCP) and data lake architectures.
Familiarity with Delta Lake, table optimization, and partitioning strategies.
Hands-on experience with Git, CI/CD, and Agile development.
Strong problem-solving and communication skills.
Preferred Skills:
Experience with Airflow/ADF/Luigi for orchestration.
Knowledge of data governance, lineage, and cataloging tools.
Understanding of MLOps or integration with ML pipelines.
Familiarity with Docker or container-based execution.
Jobs at Bangalore
Razorpay
Solutions Consultant
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 13, 2025
KPMG
Associate Consultant- OT Security
KI Professional
Bangalore, Karnataka
Last Date: Dec. 10, 2025
Razorpay
Senior Software Development Engineer
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 13, 2025
KPMG
Assistant Manager - SAP-Basis
KI Professional
Bangalore, Karnataka
Last Date: Dec. 10, 2025
Amazon
Senior Software Engineer, Emerging …
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 16, 2025
Amazon
SLP Specialist, INTL, India Securit…
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 16, 2025
KPMG
Assistant Manager - SAP-GRC
KI Professional
Bangalore, Karnataka
Last Date: Dec. 10, 2025
Razorpay
Consultant - Product Strategy
Freshers/Experienced
Bangalore, Karnataka
Last Date: Dec. 20, 2025
More Jobs at Accolite
Accolite
Data Engineer – Databricks / PySpar…
5 - 7 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Full Stack Developer – .NET & Angul…
7 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Lead Node - Associate Architect
7 - 14 Years Exp.
Gurgaon, Haryana
Last Date: Dec. 13, 2025
Accolite
Murex Back Office / Settlement BA
5 - 8 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
QA Automation Test Engineer
4 - 6 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
AI Engineer – Python / Agentic AI /…
5 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Python Backend Developer (API + AWS)
5 - 7 Years Exp.
Gurgaon, Haryana
Last Date: Dec. 13, 2025
Accolite
Mobile App Test Engineer
5 - 10 Years Exp.
Gurgaon, Haryana
Last Date: Dec. 13, 2025
Accolite
Senior AI Engineer – Python / Agent…
9 - 12 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025
Accolite
Murex Test Manager
8 - 18 Years Exp.
Bangalore, Karnataka
Last Date: Dec. 20, 2025