We are seeking an experienced Lead Data Software Engineer to join our team. This role is integral to the ongoing development of a comprehensive and unified data software engineering framework across all projects within our company. As the Lead Data Software Engineer, you will define and implement data software engineering best practices, ensure the quality and consistency of all deliverables, and promote the uptake of industry best practices across the organization. If you have a demonstrated history of leading Data Software Engineering teams and a passion for process improvement and quality control, we encourage you to apply.

Responsibilities

• Work with prodct and engineering teams to understand requirements and help drive decisions
• Maintain collaboration with architects, technical leads, and key individuals within other functional groups
• Perform analysis of business problems and technical environments and use this in the implementation of quality technical solutions
• Participate in code review and test solutions to ensure they meet best practice specifications
• Write project documentation
• Implement innovative analytical solutions using Spark/PySpark, NoSQL, and other Big Data-related technologies

Requirements

• Experience in software development with Big Data technologies (e.g. administration, configuration management, monitoring, debugging, and performance tuning)
• Strong SQL experience
• Proficiency in Python with the ability to design, develop, and maintain efficient data software solutions
• Strong knowledge of Apache Spark
• Engineering experience and practice in Data Management, Data Storage, Data Visualization, Operation, and Security
• Familiar with concepts of data ingestion pipelines, Data Warehousing, and Data Lakes
• Knowledge and practical experience in the development of scalable, available, and fault-tolerant systems
• Understanding of CI/CD principles and best practices
• Knowledge of Containers and Resource Management systems: Docker, Kubernetes, Yarn