If you are passionate about big data processing and have a strong background in Apache Spark, we would love to hear from you
Position: Spark Engineer
Location: New York, New York, United States
Salary: $107 Per hour
Job Description: We are seeking a highly skilled and experienced Spark Engineer to join our dynamic team. As a Spark Engineer, you will be responsible for designing, developing, and maintaining our big data processing pipelines using Apache Spark. You will work closely with our data scientists and data engineers to build scalable and efficient solutions for our growing data needs.
Key Responsibilities:
- Design, develop, and maintain data processing pipelines using Apache Spark
- Collaborate with data scientists and data engineers to understand business requirements and design efficient solutions
- Develop and implement data quality checks to ensure accuracy and consistency of data
- Optimize data processing pipelines for performance and scalability
- Troubleshoot and debug issues in production environments
- Coordinate with cross-functional teams to ensure timely delivery of projects
- Stay updated with the latest technologies and trends in big data processing and contribute to the improvement of our data infrastructure
- Mentor and provide technical guidance to junior team members
Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- Minimum of 3 years of experience in big data processing using Apache Spark
- Strong programming skills in Java, Scala, or Python
- Experience with cloud-based big data platforms such as AWS, GCP, or Azure
- Knowledge of distributed systems and data processing frameworks such as Hadoop, Hive, or Kafka
- Familiarity with SQL and NoSQL databases
- Proven track record of delivering scalable and efficient solutions for large datasets
- Excellent problem-solving and analytical skills
- Strong communication and collaboration skills
- Ability to work in a fast-paced and dynamic environment
- Experience with Agile methodologies is a plus
Join our team and be a part of our exciting journey towards transforming data into actionable insights.