Senior Data Engineer

Work with cutting-edge Microsoft technologies, including Azure Databricks, Azure Data Factory, and Azure SQL in a collaborative environment that values continuous learning. You'll design and build robust analytics capabilities while ensuring high-quality, well-governed data is available across the business for analytics, reporting, and operations.

What You'll Do

Architecture & Design:

• Collaborate with enterprise architects and stakeholders to design secure data solutions
• Define data architecture standards and best practices
• Evaluate and select tools within the Databricks ecosystem

Build & Maintain:

• Design scalable data pipelines using Azure Databricks, Data Factory, and Azure SQL
• Integrate diverse data sources (databases, APIs, files) into centralised repositories
• Implement data orchestration using Apache Airflow
• Develop data transformation processes (cleansing, enrichment, aggregation)

Security & Performance:

• Implement robust security measures and compliance protocols
• Monitor and optimise pipeline performance for cost-effectiveness
• Maintain comprehensive documentation and provide technical support

What You Bring

Essential Requirements:

• Degree in Computer Science, IT, or related field
• Extensive experience with Microsoft Azure services and Databricks Architecture
• Proficiency in SQL, Python, PySpark, Spark, Delta Lake
• Experience with Azure Data Factory, Azure SQL, Azure Data Lake Storage
• Apache Airflow orchestration experience (DAG design, scheduling, monitoring)
• Strong understanding of data warehousing, lakehouse, ETL/ELT, data modelling
• Experience with API integration and authentication methods
• Knowledge of data security best practices

Highly Valued:

• Microsoft or Databricks certifications
• Experience with Fivetran, Terraform, OAuth 2.0
• Knowledge of data governance frameworks
• Big data and cloud platform experience

Personal Qualities:

• Excellent problem-solving and analytical skills
• Strong communication abilities with technical and non-technical teams
• Collaborative mindset with troubleshooting expertise

Why Join Us?

• Flexible Working: Work-life balance with flexible arrangements
• Growth-Focused: Continuous learning opportunities and training resources
• Collaborative Culture: Supportive team environment with knowledge sharing
• Impact: Make meaningful contributions to strategic decision-making
• Technology: Work with the latest Microsoft ecosystem technologies

Ready to Apply?

If you're passionate about data engineering and ready to advance your career in a supportive, innovative environment, we'd love to hear from you.

Apply now!!

Robert Walters endeavours to review all applications in a maximum of five working days. If you have not received correspondence within this timeframe please do not hesitate to contact Bianca Madden on 64 9 374 7300