Data Engineer
Ref: BBBH63023_1739466957Data Engineer
Whitehall Resources currently require an experienced Data Engineer to work with a key client in Sweden
Description:
We are looking for a skilled Data Engineer to join our team and work on cutting-edge autonomous vehicle data solutions. In this role, you will design, develop, and optimize data pipelines, ensuring efficient data processing for autonomous vehicle applications. We exptect you to have a passion for clean code, scalable architecture, and high-performance data solutions.
Key Responsibilities
* Develop and maintain data pipelines using Python, PySpark, and Airflow
* Work with PostgreSQL, NoSQL databases, and on-premise solutions
* Design and implement enhancements for our data lakehouse, with medallion architecture
* Build and optimize ETL processes for upscaling towards large-scale data handling
* Implement CI/CD processes for data pipelines and end-to-end solutions
* Work with Docker, Linux, and Shell scripting
* Collaborate in a GitLab/GitHub environment for version control
* Ensure clean, maintainable, and high-quality code
* Process and manage data in MCAP format for autonomous vehicle applications
* Contribute to autonomous development in our agile team using Jira.
* Handle CAN bus data and support automotive development
* Close collaboration with the data architect contributing with data enginnering solutions
What We’re Looking For
* Strong experience in Python and software development best practices
* Background in data engineering with expertise in data processing
* Experience with GitLab/GitHub, Docker, Linux, and Shell scripting
* Knowledge of CI/CD pipelines and on-prem solutions
* Understanding of ETL workflows, medallion architecture, and data lakehouses
* Experience with MCAP format for data processing in autonomous vehicle systems
* Automotive and autonomous vehicle industry experience or knowledge of CAN bus data is a plus
All of our opportunities require that applicants are eligible to work in the specified country/location, unless otherwise stated in the job description.
Whitehall Resources are an equal opportunities employer who value a diverse and inclusive working environment. All qualified applicants will receive consideration for employment without regard to race, religion, gender identity or expression, sexual orientation, national origin, pregnancy, disability, age, veteran status, or other characteristics.
Apply Now
Your IT Specialist
