Our client provides property, agricultural, cyber risk, business, casualty insurance, and reinsurance products.

We are seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will play a crucial role in building and optimizing data pipelines and architectures and integrating data from various sources. You will be responsible for designing, developing, and maintaining data integration solutions using Azure Analytics, Azure Data Factory, Azure Databricks, Azure Pipelines, and other Data Integration (ETL/ELT) Platforms.

Our ideal candidate has a strong background in data engineering and experience with Profisee Master Data Management tool.

Responsibilities

Designing and implementing data integration solutions using Azure Analytics, Azure Data Factory, Azure Databricks, and Azure Pipelines
Developing and maintaining ETL/ELT processes to collect, transform, and load data from various sources into data warehouses
Optimizing data pipelines and architectures to improve efficiency and scalability
Collaborating with cross-functional teams to understand data requirements and provide technical solutions
Troubleshooting and resolving data integration issues
Performing data validation and quality checks to ensure data accuracy and integrity
Keeping up-to-date with industry trends and best practices in data engineering

Requirements

3+ years of experience as a Data Engineer with Data Integration expertise
Azure Analytics, Azure Data Factory, Azure Databricks, Azure Pipelines, Data Integration (ETL/ELT) Platforms, PySpark, Azure Synapse Analytics, Python
Experience developing, and managing end-to-end data pipelines, ETL (Extract, Transform, Load), and ELT (Extract, Load, Transform) solutions
Able to design & build end-to-end data engineering solutions using Azure Synapse Analytics
Solid understanding of data modeling, lakehouse, data warehousing, and data governance principles
Excellent communication and collaboration skills to work effectively with cross-functional Agile teams
Upper-Intermediate level of English, both spoken and written (B2+)

Nice to have

Microsoft SQL Server, Microsoft SQL Server Integration Services, Python, SQL
Understanding of Azure DevOps Pipelines
Understanding Test Driven Design Principles and Unit Test Frameworks such as PyTest