Data Engineer

iO Associates is hiring for a Data Engineer who can support a data and data pipeline optimization project.

This is a long-term contract position (6-12+ months) and is 100% remote, though applicants do need to be based in the US to be considered.

What You'll Do:

• Identify stakeholders' data needs, define the project scope, and select the right implementation to address stakeholder pain points.
• Bridge the gap between the domain requirements and technology during the tool selection process. Your role involves ensuring that the chosen technology aligns with those requirements and the overall domain strategy.
• Maintaining constant communication with stakeholders regarding project progress, blockers, and expected timelines is crucial.
• Collaborate with stakeholders to continuously automate and improve existing workflows and/or processes.
• Develop, deploy and maintain workflow management tools such as Airflow, Jenkins etc in cloud environments.
• Using cloud technology such as AWS (Lambda, S3, EC2, ECS, DynamoDB, SQS), Kubernetes, Docker, etc.
• Work alongside the infrastructure, data ingestion, data warehouse and analytics teams to manage and optimize ETL and data pipeline parameters, scheduling, monitoring, and alerting.
• Collaborate with other engineers and teams to implement large-scale data processing systems to map and standardize data.
• Maintain and optimize data pipelines, architectures, and data sets
• Build, develop and maintain data models, reporting systems, data automation systems, etc.

Your Background:

• 8+ years of experience as a Data Engineer
• Strong communication skills.
• Strong Snowflake experience
• Strong experience in Python development.
• Experience with a workflow manager - ideally Airflow
• Experience working with AWS (Lambda, S3, EC2, ECS, DynamoDB, SQS)