Job Category: Data Engineer/ Architect
Job Type: Remote
Responsibilities
- Define and execute the data engineering roadmap, balancing new capabilities, platform stability, technical debt reduction, and cost management.
- Establish architectural standards, evaluate cloud services and tools, and build business cases for platform investments.
- Translate emerging capabilities within modern cloud data platforms into actionable engineering plans.
- Lead the design and ongoing operation of a medallion-style data architecture, including raw data ingestion, transformation layers, and curated datasets for analytics.
- Oversee end-to-end data pipelines from ingestion through transformation to delivery, ensuring reliability and scalability.
- Create and enforce development standards using dbt, including model structuring, testing frameworks, and best practices for ELT processes.
- Develop reusable dbt components such as macros, packages, and testing utilities to improve productivity and data quality.
- Manage dbt environments and deployment workflows across development, testing, and production.
- Implement advanced dbt features, including incremental models, snapshots, semantic modeling, and lineage tracking.
- Design and maintain orchestration workflows using DAG-based tools, ensuring pipelines are modular, testable, observable, and fault-tolerant.
- Establish standards for workflow dependencies, retry behavior, alerting, service levels, and backfill processes.
- Integrate orchestration with ingestion frameworks, transformation layers, and downstream systems for seamless execution.
- Govern engineering practices, including version control, code reviews, branching strategies, and release management.
- Build and enforce data quality frameworks with automated validation at each stage of the pipeline.
- Partner with business and technical teams to onboard new data sources and enforce consistent naming, partitioning, and retention rules.
- Monitor and optimize platform usage, including compute and storage efficiency, query performance, and lifecycle management.
- Implement data observability practices to detect anomalies, schema changes, and service-level risks.
- Define service level objectives for critical data assets and lead incident response and root cause analysis.
- Lead, mentor, and grow a team of data engineers, setting clear goals and ensuring delivery against quality standards.
- Act as the escalation point for technical challenges, cross-team dependencies, and operational incidents.
- Manage team performance, staffing, and development to maintain strong delivery and continuity.
Required experience and skills
- Bachelor’s degree or higher in computer science, software engineering, information systems, mathematics, or a related technical field, or equivalent practical experience.
- 10 or more years of hands-on data engineering experience, including tools such as Python, SQL, dbt, Informatica, or DataStage.
- 5 or more years of experience managing and leading data engineering teams.
- Proven ability to deliver scalable ELT data integration solutions in production environments.
- Advanced expertise in SQL, data modeling approaches, data quality practices, orchestration strategies, and performance optimization.
- Strong experience with cloud data platforms such as Snowflake or Databricks, including features like Snowpipe, Spark, and cataloging capabilities.
- Deep experience with dbt, including model development, testing, macros, and package management.
- Experience designing and managing orchestration workflows using tools such as Airflow, Prefect, or Dagster.
- Practical knowledge of data warehouse or lakehouse architecture using layered design patterns such as bronze, silver, and gold.
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
- Experience implementing CI and CD practices for data pipelines using tools like GitHub Actions or dbt Cloud.
- Knowledge of data quality and observability platforms such as Great Expectations or Monte Carlo.
- Demonstrated leadership, coaching, and team development capabilities.
This job is no longer accepting applications.