Develop scalable ETL/ELT pipelines leveraging Databricks and DBT

Create data models that enable cost analysis and reporting

Collaborate with Cloud Operations and Finance teams to identify cost-saving opportunities

Automate ingestion of data from various sources such as cloud billing APIs and usage logs

Ensure data quality, governance, and regulatory compliance across datasets

Enhance data processing performance and reduce storage costs in cloud environments

Facilitate offshore delivery and coordinate with international stakeholders

Proven experience of at least 5 years in Data Engineering or related fields

Hands-on expertise with Databricks (Spark, Delta Lake) and DBT for transforming data

Strong skills in SQL and PySpark for effective data manipulation

Solid programming background in Python, including object-oriented design and software engineering principles

Familiarity with cloud platforms like Azure, AWS, or GCP and their cost management solutions

Comprehensive understanding of data warehousing and modern data stack concepts

Experience implementing CI/CD pipelines and using version control tools such as Git

Excellent communication abilities for working within distributed teams

Understanding of FinOps methodologies and cloud cost management techniques

Experience with Databricks system tables to extract platform-specific insights

Proficiency in visualization tools like Power BI, Tableau, or similar

Knowledge of orchestration frameworks such as Databricks Workflows, Airflow, or Prefect

Skills in observability tools and practices including OpenTelemetry, Prometheus, and Grafana