Data Engineer (m/f/d)
Responsibilities
- Design, implement, and maintain scalable ETL pipelines using AWS Glue (Spark), Python, and PySpark.
- Manage complex data workflows and dependencies, including with AWS Lambda, using Airflow (or similar orchestration tools).
- Integrate and optimize data flows across various AWS services like S3 (Glue Catalog, Athena), Aurora Postgres, Redshift, and Iceberg tables.
Profile
- Proficiency in Python (including Pandas) for data transformations using TDD; hands-on experience with Apache Spark, ideally via AWS Glue
- Experience with AWS services (S3, Glue, Athena, Redshift, Aurora, Lambda, IAM, EventBridge); comfortable with cloud-based architecture, serverless design, and deployment strategies
- Experience in building and managing DAGs in Airflow (or similar tools); familiarity with Lambda-based event-driven architectures
- Proficiency with Git (branching, pull requests, code reviews) and CI/CD pipelines; understanding of release management and automated testing in a multi-environment setup
- Hands-on experience with distributed processing frameworks (like Spark) and large-scale data lake solutions, including Iceberg tables; familiarity with data lake architectures, partitioning strategies, and best practices
- Strong skills in optimizing Spark jobs and SQL queries for performance
- Knowledge of data security practices, including encryption, access controls, and compliance in cloud environments
- SQL knowledge for properly designing and maintaining data schemas for efficient querying and reporting is an advantage
- Familiarity with tools like Terraform or CloudFormation for automated AWS provisioning is an advantage
- Experience with event-driven architectures and working with AWS EventBridge / Lambda / SQS / SNS is an advantage
Benefits
- 100% remote possibility
- Extension of project possible
About Hays
Our many years of recruitment experience and our expert knowledge of the engineering job market make us a strong partner for engineers and technicians. We maintain close relationships with customers across all industries and therefore can provide engineering specialists with exciting projects and attractive job opportunities, drawing on your personal interests and experience. As your professional companion, we support you throughout the entire process, from the first contact to the first day on your new job – all this completely free of charge. Register now and look forward to many interesting and suitable positions and projects.
Staffing process for freelance specialists
- Analysis of qualifications
- Telephone or personal interview
- Contact with customers
- Contract with Hays
- 1
- 2
- 3
- 4
1. Analysis of qualifications
We carry out a detailed analysis of your technical qualifications on the basis of your CV, your application documents and your profile.
Contact at Hays
© Copyright Hays plc, . Das Wort HAYS, die H-Symbole, „Hays Working for your tomorrow" und „Powering the world of work" sowie damit verbundene Logos und Illustrationen sind eingetragene Markenzeichen der Hays PLC. Die H-Symbole sind Originaldesigns, die in vielen Ländern geschützt sind. Alle Rechte vorbehalten.