Cloud AWS {Python I SQL I Data management}  

General Responsibiities:

It will ensure the seamless operation and optimization of data pipelines and platforms under Project Ultron. The role entails troubleshooting data pipeline issues, conducting rigorous testing, coordinating delivery across teams, and collaborating closely with Data Science and Marketing to uphold platform reliability and performance.

Duties and responsibilities:
● Collaborate with the Data Science team to identify, troubleshoot, and resolve data pipeline issues.
● Conduct thorough end-to-end testing of data pipelines across all environments (development, staging, production).
● Prepare and deliver comprehensive Root Cause Analyses (RCAs) for data-related incidents.

Troubleshooting and Firefighting
● Maintain proactive communication with the Marketing team throughout the resolution of bugs and hotfixes.
● Document issues and remediation actions in detailed RCA documentation for transparency and future reference.

Delivery Platform Coordination
● Raise and track prediction data delivery issues across internal and external platforms. ● Coordinate closely with Marketing, Data Science, and Platform Tech Support teams for resolution and alignment.
● Assist in building and maintaining egress pipelines from Project Ultron to delivery platforms.
● Promote adherence to platform engineering best practices and documentation standards.
● Participate in sprint planning and regular stand-ups with cross-functional teams.

Platform Maintenance and Monitoring
● Support the upkeep and monitoring of Project Ultron’s platform infrastructure.
● Identify opportunities for performance tuning, reliability enhancements, and availability improvements.
● Ensures platform integrity, stability, and data delivery continuity across projects.
● Contribute to the automation of monitoring and maintenance tasks.

• Experience in data engineering or platform engineering roles.
• Experience in distributed computing frameworks (e.g., Apache Spark).
• Demonstrated experience in platform operations, data pipeline development, and support.
• Proficient in SQL, Python, PySpark, and Pandas for data manipulation and analysis.
• Hands-on experience with Docker and Kubernetes for containerization and orchestration
• Familiar with Git or similar version control systems.
• Experience working within cloud environments, preferably AWS (e.g., S3, Lambda, EMR, ECS).
• Awareness of industry best practices for monitoring, testing, and debugging data platforms..
• 
  Experience working with SQL is required.
• Experience working with Python is required.