Data Reliability Engineer

This role combines database development with quality engineering and DevOps practices to proactively identify, prevent, and resolve data issues across complex pipelines and distributed systems.

Database Design & Development for Reliability

• Design, develop, and maintain advanced SQL scripts, stored procedures, triggers, views, and functions to support application logic and data workflows.
• Design, develop, and maintain   automation frameworks   and   custom software tools to monitor and validate data pipelines, storage systems, and analytics platforms.
• Write scalable test harnesses and data validation scripts in languages such as Python or SQL to ensure data accuracy, completeness, and freshness.
• Build utilities and services to identify and remediate data anomalies, schema changes, or pipeline failures.

Data Quality Assurance & Testing 

• Create automated tests for ETL/ELT pipelines, APIs, and streaming services to validate transformations and data integrity across systems.
• Develop   unit, integration, and regression tests   for data flows and processing logic.
• Proactively detect data inconsistencies and ensure quality across staging and production environments.
• Develop and implement data validation, cleansing, and monitoring frameworks to maintain high data quality standards.
• Troubleshoot data anomalies and application issues by analyzing logs, SQL execution plans, and system configurations.
• Lead root cause analysis and resolution of complex databases and reporting issues in collaboration with technical and business teams.

Business Intelligence, Monitoring & Reporting

• Implement monitoring dashboards, alerts, and logs using tools like Azure Monitor to track data pipeline health.
• Automate data checks using orchestration platforms like Airflow, Azure Data Factory, or Synapse Pipelines.
• Establish Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for critical data flows.

Collaboration, Incident Response & Documentation

• Collaborate with data engineers, developers, and product teams to understand system requirements and prevent data issues during development.
• Participate in root cause analysis and incident resolution for data pipeline failures, reporting errors or data quality concerns.
• Contribute to CI/CD pipelines to automate deployment and testing of data services
• Maintain clear documentation of data testing strategies, automation scripts, and reliability metrics.
• Continuously improve reliability engineering practices by contributing to internal standards and best practices.
• Perform regular reviews of data systems for performance, scalability, and quality improvements.

Key Selection Criteria

•    At least five years of relevant professional work experience.

•    Bachelor degree or higher qualification 

•    Proficiency in programming languages such as Python and SQL with a strong focus on test and monitoring automation.

•    Strong understanding of data pipeline orchestration tools (e.g. Airflow, Azure Data Factory, Synapse).

•    Deep knowledge of SQL and experience with data warehousing or distributed data systems.

•    Experience with automated testing frameworks and monitoring tools.

•    Familiarity with DevOps practices and CI/CD pipelines.

•    Strong analytical and debugging skills with attention to data reliability.

•    Outstanding communication and collaboration abilities, especially in cross-functional environments.

•    Expertise in SQL Server development, including complex stored procedures, performance tuning, and advanced query optimization.

•    Demonstrated experience with Microsoft Synapse, DBT, and cloud-based data pipelines.

•    Proficient in developing reports and dashboards using BI tools such as Yellowfin and Power BI.

•    Strong understanding of integrating data from multiple structured and unstructured data sources.

•    Excellent problem-solving and troubleshooting skills with strong attention to detail.

•    Familiarity with GRC (Governance, Risk, and Compliance) systems and data structures is advantageous.