Data Engineer (Databricks Expert)
Job description:
**Assignment description:** To support the build activities of our customers Manufacturing Intelligence Platform Proof of Concept (PoC), we are looking to complement our existing team with a skilled Data Engineer. This individual will bring strong Databricks expertise, providing both hands-on development support and input into architectural decisions. **Qualifications:** * Proven experience with Databricks (Python and SQL) and the Medallion architecture. * Solid experience building ETL pipelines to move data from on-premises systems—particularly SQL Server—to cloud environments. * Ability to work with data from different domains (e.g., process data and order data), aligning and aggregating datasets based on timestamps to enable time-series and operational insights. * Experience with both batch and streaming data pipelines. * Familiarity with designing data workflows across Azure Data Factory and Databricks—understanding when and how to use each for orchestration, transformation, and integration tasks. * Ability to work effectively within a cross-functional team during the platform build phase. * DataOps – versioning, CI/CD, dev/test/prod structure This resource will be instrumental in accelerating our prototype development and laying a solid foundation for our data platform. The work can mostly be done remote with a few days on customer premises