Job Category: Data Engineer/ Architect
Job Type: Remote
Job Description
A leading organization is seeking a Senior Python Engineer with expertise in workflow orchestration, Apache Airflow, and the Astronomer platform. This position will focus on modernizing and enhancing enterprise workflow management capabilities, including a major migration initiative and ongoing optimization of production data pipelines.
The ideal candidate brings strong Python development skills, hands-on experience with Airflow platform upgrades, and a deep understanding of scalable workflow orchestration in cloud environments.
Responsibilities
- Lead the migration of Apache Airflow environments from version 2 to version 3 while ensuring stability and operational continuity.
- Design, develop, and maintain production-grade workflow orchestration solutions using Apache Airflow.
- Create and support Python-based automation, scheduling, and data workflow processes.
- Improve the performance, reliability, and scalability of workflow execution and pipeline operations.
- Troubleshoot orchestration issues, resolve scheduling challenges, and optimize system performance.
- Collaborate with data engineering and platform teams to establish best practices for orchestration, deployment, monitoring, and operational support.
- Implement maintainable and efficient workflow architectures that support business and technical requirements.
- Contribute to continuous improvements across workflow management and data pipeline operations.
Required Experience and Skills
- 5–8 years of professional experience in software engineering, data engineering, or workflow orchestration environments.
- Expert-level Python development experience.
- Hands-on experience migrating and supporting Apache Airflow version 2 to version 3.
- Strong experience working with the Astronomer platform.
- Deep understanding of Airflow DAG development, dependency management, scheduling, and operational best practices.
- Experience building, supporting, and optimizing production data pipelines.
- Experience working with cloud-based platforms and distributed data processing environments.
- Strong troubleshooting and problem-solving skills with a focus on reliability, scalability, and operational excellence.
- Ability to collaborate effectively with engineering and platform teams in a production environment.
Apply for this position
**If you have already submitted your resume for another Job Opening please do not re-apply to a different role. You can email through Contact Us about your interest in other roles.