Manager | Data Engineer | Chennai | Regulatory & Financial Risk

Manager | Data Engineer | Chennai | Regulatory & Financial Risk

• Job requisition ID : 110870
• Location: Chennai
• Entity: Deloitte Touche Tohmatsu India LLP

The team

Deloitte Strategy, Risk & Transaction helps entities mitigate risk while discovering new opportunities to create value. Our end-to-end risk services span all domains, from managing strategic risks in the C-Suite to improving board oversight, and from balancing financial and environmental policies to addressing cyber threats.Learn more about Risk, Regulatory & Forensic)

Your work profile

• We are looking for a skilled Senior Developer with strong expertise in PySpark to design, build, and maintain scalable data processing solutions.

• The ideal candidate will have experience working across the data engineering stack, strong knowledge of distributed computing, and hands-on exposure to SDLC-based project development.

• We are seeking an experienced Python Backend Developer to design, build, and maintain scalable backend systems and APIs.

• The ideal candidate has strong expertise in Python frameworks, database design, and backend performance optimization.

• We are looking for a DevOps Engineer to build, automate, and maintain scalable, secure, and highly available infrastructure.

• The ideal candidate has strong experience in cloud platforms, CI/CD automation, and modern DevOps practices.

Key Responsibilities:

• Design, develop, and optimize large-scale data pipelines using PySpark.

• Build and maintain ETL/ELT workflows to process structured and unstructured data.

• Work with big data technologies such as Hadoop ecosystem, Spark, and Impala.

• Collaborate with cross-functional teams including data scientists, analysts, and product teams.

• Ensure code quality through best practices, code reviews, and version control.

• Optimize job performance, memory management, and execution time in Spark jobs.

• Work on data modeling, transformation, and pipeline orchestration.

• Participate in end-to-end SDLC activities including requirement analysis, design, development, testing, and deployment.

• Troubleshoot and resolve data processing issues and production incidents.

• Contribute to automation, monitoring, and logging frameworks.

• Required Skills and Qualifications:

• Strong hands-on experience with Python and PySpark (mandatory).

• Strong SQL skills including advanced querying and performance tuning.

• Experience with distributed data systems such as Hadoop and Spark ecosystem.

• Experience with Impala, Hive, and SQL Server or other relational databases.

• Strong understanding of data structures, algorithms, and performance optimization.

• Experience with ETL pipeline development and data warehousing concepts.

• Knowledge of SDLC methodologies (Agile/Scrum preferred).

• Familiarity with version control systems such as Git.

• Experience working with large datasets in production environments.

• Knowledge of C# or other programming languages.

• Experience with cloud platforms such as Azure, AWS, or GCP.

• Exposure to workflow orchestration tools like Airflow or Oozie.

• Understanding of data lakes, data governance, and data quality frameworks.

• Experience with CI/CD pipelines.

• Strong analytical and problem-solving abilities.

• Good communication and collaboration skills.

• Ability to work independently and in a fast-paced environment.

• Proactive mindset with attention to detail.

• Preferred Candidate Profile:

• 5-8 Years of Hands-on PySpark developer with real-time project delivery experience.

• Bacheler’s degree in any specialization

• Exposure to end-to-end data pipeline lifecycle.

• Comfortable handling large-scale data processing and optimization challenges.