Senior Data Engineer

Senior Data Engineer

Dublin, Ireland | Hybrid | 12-month Contract

About the Company

Our client is a large, global technology-driven organisation operating in a highly regulated, data-rich industry, delivering services at scale to millions of customers worldwide.

About the Role

As a Senior Data Engineer, you will develop and maintain data pipelines that extract, transform, and load (ETL) data from a wide variety of sources into centralised data platforms such as a data warehouse or data lake. You will ensure the smooth, reliable flow of data from source to destination systems while upholding rigorous data quality and integrity standards, working within a highly regulated environment.

Primary Responsibilities

• Data integration: integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers. Handle ingestion, transformation, and consolidation to create a unified, reliable data foundation for analysis and reporting.
• Data transformation and processing: develop transformation routines to clean, normalise, and aggregate data. Handle complex data structures and missing or inconsistent data, preparing data for analysis, reporting, and machine learning use cases.
• Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines.
• Implement data governance and data de-identification frameworks in line with company and regulatory standards.
• Partner with data analytics and product leaders to design best practices and standards for developing and productionising analytic pipelines.
• Partner with infrastructure leaders on architecture approaches to advance the data and analytics platform, including exploring new cloud-based tools and techniques (Azure, Snowflake, and others).
• Monitoring and support: monitor data pipelines and systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling to ensure data integrity and system reliability.

Required Qualifications

• Extensive experience designing data solutions, with a solid understanding of architectural patterns, system design principles, data modelling, and large-scale solution development.
• Extensive hands-on experience developing data processing jobs (PySpark/SQL) demonstrating strong software engineering principles.
• Proven experience with Azure Databricks and Snowflake.
• Experience orchestrating data pipelines using tools such as Azure Data Factory, Airflow, or similar.
• Proven experience working with Spark.
• Advanced SQL skills and deep experience with relational databases (e.g., SQL Server), including optimisation, stored procedures, and performance tuning for high-volume applications.
• Experience with DevOps tooling, Git workflows, and building CI/CD pipelines.
• Experience applying data governance controls, data masking, and data de-identification rules within a highly regulated environment.
• Excellent communication and interpersonal skills, with the ability to convey technical concepts to both technical and non-technical stakeholders.

Preferred Qualifications

• Experience analysing and improving workflows: evaluating process maps and flowcharts, identifying inefficiencies, and recommending enhancements to procedures, technology, and tools.
• Familiarity with languages such as Java or Scala.
• A motivated self-starter who manages their own tasks and takes ownership.
• Experience working in agile/scrum teams.
• Strong Python skills for data manipulation, cleaning, transformation, and analysis of structured data.
• Familiarity with production-quality ML/AI model development and deployment.
• Data architecture experience.

GCS is acting as an Employment Business in relation to this vacancy.