Developing a POC for replacement of the external vendor inside the infrastructure

Setting up required Azure services

Extracting data from data lake (EDL)

Processing data based on application requirements and architecture

Building the service layer to mimic the current existing application

Prioritizing and ensuring high-quality standards at every stage of development

Guaranteeing reliability, availability, performance, and scalability of systems

Collaborating with Developers, Product and Program Management, and senior technical staff to deliver customer-centric solutions.

Providing technical input for new feature requirements, partnering with business owners and architects

Ensuring continuous improvement by staying abreast of industry trends and emerging technologies

At least 3+ years of production experience in Data Software Engineering

Be hands-on with deep expertise in PySpark and Python for Data Engineering

Experience with Azure technologies, including SQL Azure and Microsoft Azure for building scalable data solutions

Solid experience working with Databricks for building data pipelines

Knowledge of Python web services like Django and Flask

Advanced SQL data processing experience for designing and managing database schema, including procedures, triggers, and views

Support applications and systems in a production environment, ensuring timely resolution of issues

Reviewing requirements and translating them into a documented technical design for implementation

Expertise in creating data POCs

Excellent communication skills in spoken and written English, at an upper-intermediate level or higher

Experience with Big Data technologies like Hadoop and Spark Streaming is a plus

Familiarity with machine learning concepts and libraries such as Scikit-learn is an advantage