Lead Big Data Engineer

ABOUT THE ROLE

In this role, you will lead complex data migration and modernization programs, helping organizations transform legacy platforms into scalable cloud-native environments. Leveraging AWS technologies, distributed data processing frameworks, and AI-assisted development tools, you will drive large-scale transformation initiatives that accelerate delivery, improve data quality, and unlock business value from modern data platforms.

RESPONSIBILITIES

• Design and implement data migration workflows and cloud-based data solutions on AWS
• Build and maintain scalable batch and streaming data pipelines using Python and PySpark
• Analyze legacy applications, databases, and data processing workflows to define migration approaches and implementation strategies
• Leverage AI-assisted development tools and LLM-powered solutions to accelerate migration, validation, and automation activities
• Design and implement data quality, reconciliation, and validation frameworks for migration projects
• Develop automation solutions using AWS services such as Step Functions, Lambda, and CloudWatch
• Support modernization initiatives, including migration from legacy technologies to cloud-native platforms
• Collaborate with stakeholders to define requirements, identify risks, and support delivery planning and execution
• Provide technical guidance to engineering teams and promote best practices in data engineering and cloud migration
• Contribute to architecture discussions, technical reviews, and continuous improvement initiatives

REQUIREMENTS

• 7+ years of experience in Data Engineering, Big Data, or Data Migration projects
• Strong proficiency in Python and SQL
• Advanced experience with PySpark and distributed data processing frameworks
• Hands-on experience with AWS services such as Step Functions, S3, Lambda, and CloudWatch
• Strong knowledge of ETL processes and scalable data pipeline development
• Experience leveraging AI-assisted workflows and Large Language Models (LLMs) in engineering activities
• Understanding of migration strategies, legacy system modernization, and cloud transformation initiatives
• Experience with data validation, quality assurance, and migration governance processes
• Proficiency in guiding technical implementation activities and collaborating effectively across engineering teams
• Upper-intermediate or higher level of English
• Experience with video, image, or audio processing pipelines for AI applications (as a plus)
• Experience with IoT data platforms and data processing solutions (advantageous)
• Familiarity with NVIDIA technologies, GPU-accelerated computing, or AI infrastructure (as a win)
• Knowledge of Infrastructure as Code: Terraform, CloudFormation, or similar (as a plus)