Job description:
Project description:
As part of this project, you support the team of the central data platform of a renowned bank. After the successful introduction of Kafka, the focus of this role is on the development and development of data linearity solutions. The aim is to ensure the traceability and transparency of all data flows across the central platform. In particular, OpenLinage and Marquez are used. Their expertise contributes significantly to meeting regulatory requirements and to document data flows efficiently.
Framework parameters:
Start: asap
Duration: until end of 2025, with option for extension
Utilization: full-time
Location: Hannover / Remote (mainly remote 95%, occasional presence days in Hannover)
Budget: Max. hourly rate: 70 €/ h remote, 76 €/ h on site
Tasks:
- Concept, implementation and operation of data lineage solutions in the Kafka environment with OpenLineage and Marquez
- Integration of data lineage into existing Kafka data streams and ETL processes
- Development of interfaces and connectors for automated recording and documentation of data flows
- Cooperation with Data Engineers, Data Stewards and Competence teams to ensure regulatory requirements for data traceability
- Advice in the selection and introduction of appropriate tools and methods for data lineage
- Support for the continuous improvement of the data platform and its documentation
Experience & knowledge:
Required:
- At least 5 years of experience in the development of data platforms and / or streaming solutions
- Knowledge in Apache Kafka
- Experience with Data Linage Tools, ideally OpenLinage and Marquez
- Very good programming knowledge in Java (alternative Python)
- experience with databases (e.g. PostgreSQL, DB2) and the integration of data sources
To be desired:
- experience in the banking environment or in the financial sector
- knowledge of data governance and regulatory requirements (e.g. BCBS 239, DORA)
- familiarity with agile methods
- Flowing German
- and English