Job description:
For our client we are looking for a Data Developer (f/m/d).
Frame data:
Start: November 2024
Duration: 31.03.25 +
- (long term)
Capacity: 100% if possible
Location: Berlin / Remote (1 week Berlin / 3 weeks remote in rotation), up to 50% onsite in peak times
Role:
- The role of a Developer in the Data team within the project will be focusing on providing Data Platform Services like Data Catalog and key components to realize a Data Mesh on the cloud platform.
- This includes increasing level of maturity for well as expanding list of provided services.
- This role is crucial in aiding the deployment of established and standard data architectures across diverse application teams, which are poised to evolve into
specialized Data Domain teams.
- We are looking for a talented and dedicated Software or Data Engineer to join our team, focusing on the developments of capabilities to enable Data Mesh, and offering expertise in leveraging contemporary technologies.
- This involves advising on the utilization of services available on public clouds or developing inhouse services on our internal Kubernetes-based platform.
- In this role, you will be responsible for the development and management of data products, ensuring robust data governance, and establishing comprehensive data cataloging processes.
- This includes the creation, maintenance, and enhancement of cross-cloud managed data services, which are essential for enabling seamless data accessibility and interoperability across different cloud environments.
- The Data Engineer will also work closely with various teams to implement and enforce data governance policies, ensuring data quality, security, and compliance.
Profile Requirements:
- Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
- 5
- years of general IT experience
- 3
- years of Big Data experience
- Proven experience as a Data Engineer with a focus on designing and implementing scalable data architectures.
- Extensive experience in developing and maintaining databases, data lakes, and data warehouses.
- Hands-on experience with ETL processes and data integration from various sources.
- Familiarity with modern data technologies and cloud services.
- Proficient in designing and implementing data models to meet business requirements.
- Experience with Data Mesh
- A keen interest in staying updated on emerging technologies in the data engineering and Data Mesh space.
- Ability to evaluate and recommend the adoption of new tools and technologies.
- Innovative mindset to propose solutions enhancing the organization's data architecture
Skills (must-have):
- Proven hands-on software development experience
- Proficiency in data processing languages such as SQL, Java, Python or Scala
- Knowledge and experience with at least some of the Data technologies/frameworks:
- RDBMS (PostgreSQL/MySql etc.)
- NoSQL Storages (MongoDB, Cassandra, Neo4j etc.)
- Timeseries (InfluxDB, OpenTSDB, TimescaleDB, Prometheus etc.)
- Workflow orchestration (AirFlow/Oozie etc.)
- Data integration/Ingestion (Flume etc) .
- Messaging/Data Streaming (Kafka/RabbitMQ etc.)
- Data Processing (Spark, Flink etc.) and/Or with their Cloud provided counterparts, i.e., Cloud Data/Analytics services (GCP, Azure, AWS)
- Familiarity with reference Big Data architectures (Warehouse, Data Lake, Data Lakehouse) and their implementation.
- Experience in implementing and operating data intensive applications
Strong focus on DataOps/DevOps
- Proficiency in both speech and writing in English (at least C1)
Skills (should-have):
- Deeper K8s skills and experience, e.g. k8s operators’ development experience and/or k8s operators for Big Data technologies)
- In-depth knowledge of best practices in data privacy and data protection
- Proven experience with DataMesh principles in practice
- Data platform development and/or operations experience
- Knowledge and experience in lifecycle management in Data (e.g. CD4ML, MLOps, …)
- Proficiency in German