Introduction
At IBM, work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some of the world’s most challenging problems? If so, lets talk.

Your Role and Responsibilities
We are seeking an experienced Senior Spark / AWS Engineer to join our team.
As a key member of our technical team, you will be responsible for designing, developing, and deploying scalable, efficient, and reliable Spark-based applications on AWS.
If you have hands-on experience working with AWS, EMR, Airflow, Terraform, and GitLab, and are passionate about Spark development, refactoring, performance tuning, and AWS engineering, we’d love to hear from you!

Responsibilities:
Spark Development
* Design and develop scalable, efficient, and reliable Spark-based applications
* Collaborate with cross-functional teams to understand requirements and deliver high-quality solutions
Spark Refactoring
* Analyze and optimize existing Spark code for performance, scalability, and maintainability
* Refactor code to improve readability, efficiency, and reliability
* Collaborate with the team to ensure smooth code deployments
Spark Performance Tuning
* Identify performance bottlenecks in Spark applications
* Analyze and optimize Spark configurations, such as memory, CPU, and network settings
* Implement optimizations to improve application performance and scalability
AWS Engineering
* Collaborate with cross-functional teams to ensure seamless integration with AWS services
* Ensure AWS security, monitoring, and logging best practices

Required Technical and Professional Expertise
*Strong understanding of Spark: APIs, configurations, performance tuning, and best practices
*Hands-on experience working with AWS: EC2, S3, EMR, Glue, Lambda, API Gateway, etc.
* EMR experience: creating, managing, and optimizing EMR clusters
* Airflow experience: creating, managing, and optimizing Airflow workflows
* Terraform experience: designing and deploying infrastructure as code
* GitLab experience: using GitLab for version control, CI/CD, and collaboration
* Scala knowledge: preferred but not required
* Excellent problem-solving skills: ability to troubleshoot complex technical issues
* Strong communication skills: ability to collaborate with cross-functional teams and communicate technical ideas effectively

Preferred Technical and Professional Expertise

• Relevant applicable certifications