Lead Data Engineer

1 month ago

Limerick, Ireland Cpl Full time

About the job

This client is uniquely positioned to define the future of Health Technology â with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. Itâs all about simplifying healthcare. 


Founded in 2017, they have been committed to transforming the world of healthcare through operational excellence, innovation, and digital transformation. We are uniquely positioned to enable enterprises & organisations across multiple Technology domains including data analytics, digital technologies, automation & AI. Our Claims Processing, Provider Data Operations and Corporate Functions are among the best in the Healthcare Sector.

 

Summary of role: 

We are seeking a Lead Data Engineer with extensive big data frameworks expertise to join our team. The ideal candidate will have a strong background using Amazon Elastic MapReduce (EMR) big data platform with frameworks like Apache Hadoop, Apache Spark. In this role, you will be responsible for designing, developing, and implementing big data processing & streaming solutions on AWS EMR cloud platform leveraging Apache Spark/PySpark, Airflow, MongoDB. 


Primary Duties: 

⢠Design, develop, and process big datasets primarily using Amazon EMR, Apache Spark/ PySpark and Apache Hadoop for large data frames and NoSQL MongoDB, AWS S3 file storage. 

⢠Expertise in distributed data processing techniques and analytics. 

⢠Expertise in GCP big-data solutions such as Cloud Dataproc and GCP BigQuery also advantageous. 

⢠Knowledge of Amazon Managed Workflows for Apache Airflow (MWAA). 

⢠Programming skills with Python (PySpark APIs, Spark Core APIs, Spark MLib), Spark streaming and EMR cluster management techniques. 

⢠Knowledge of Cloud storage resources database, data warehouse, data lake and file storage solutions( AWS S3, GCP GCS, Azure Blob and IBM ICOS) 

⢠Collaborate with cross functional teams to define business requirements and translate them into data design and architecture specifications. 

⢠Analyse source to target systems, data models, and data structures to ensure data quality and consistency across systems. 

⢠Work with AWS resources services, including various data orchestration tools such as Spark/Airflow. 

⢠Knowledge of ML/OPS in deploying ML models. 

⢠Effectively manage version control using Git for large-scale projects, ensuring efficient handling of frequent and ongoing code updates, merges, and branches to maintain code integrity and collaboration among team members. 

⢠Perform data profiling, cleansing, and validation to ensure accurate data transformation and integration. 

⢠Develop and maintain ETL process documentation, including data flow diagrams and data mapping documents. 

⢠Optimise ETL performance through best practices, performance monitoring, and troubleshooting. 

⢠Participate in code reviews, peer feedback sessions, and continuous improvement initiatives. 

⢠Work closely with data warehouse and database administrators for optimal data storage and retrieval. 

⢠Provide technical support and guidance on big data ETL processes and data integration. 

⢠Stay current with industry trends and best practices for big data technologies and cloud infrastructures. 


Requirements: 

⢠Bachelor's degree in Computer Science, Information Systems, or a related field. 

⢠Minimum of 7 years of experience in data engineering roles with primary focus on big data frameworks such as Apache Spark, Hadoop, AWS EMR platform. 

⢠Strong experience with cloud-based data warehouse solutions using Snowflake, AWS Redshift, Google Big Query (GCP), Azure Synapse 

⢠Familiarity with various data & development tools - DBT, DBeaver useful. 

⢠Proficiency in SQL for data querying and transformation. 

⢠Experience with relational databases such as Postgres and SQL Server and NoSQL MongoDB. 

⢠Proficiency in Python or other relevant programming languages. 

⢠Understanding of data modelling concepts and experience in integrating database objects with applications. 

⢠Experience with both AWS and GCP cloud services beneficial. 

⢠Familiarity with version control, CI/CD tools, and best practices. 

⢠Strong analytical and problem-solving skills. 

⢠Excellent communication and collaboration skills, Confluence/JIRA. 

⢠Ability to manage multiple tasks in a fast-paced environment. 

⢠Familiarity with data privacy and security best practices in cloud environments


#LI-EL1

  • Lead CMMS Data Engineer

    1 week ago

    Limerick, Ireland Lilly Full time

    We’re looking for people who are determined to make life better for people around the world. Main Purpose and Objectives of Position: The Maintenance Professional will report into the Maintenance Associate Director. The primary responsibilities of the candidate is to provide support, analysis & direction from the Computerized Maintenance Management...

  • Lead Data Science

    5 days ago

    Limerick, Limerick, Ireland Cpl Full time

    Our Client in Limerick is looking for a Lead Data Scientist to join their team.Primary duties may include but are not limited to:Write correct and clean code with limited guidance while ensuring following best practices.Demonstrate9d ability to effectively collaborate across multiple teams and stakeholders to drive business outcomesAnalyse and classify...

  • Lead Data Science

    21 hours ago

    Limerick, Ireland Cpl Healthcare Full time

    Our Client in Limerick is looking for a Lead Data Scientist to join their team. Primary duties may include but are not limited to: Write correct and clean code with limited guidance while ensuring following best practices. Demonstrate9d ability to effectively collaborate across multiple teams and stakeholders to drive business outcomes Analyse and...

  • Lead Data Science

    5 days ago

    Limerick, Ireland Cpl Full time

    Our Client in Limerick is looking for a Lead Data Scientist to join their team.Primary duties may include but are not limited to:Write correct and clean code with limited guidance while ensuring following best practices.Demonstrate9d ability to effectively collaborate across multiple teams and stakeholders to drive business outcomesAnalyse and classify...

  • Lead Data Scientist

    2 days ago

    Limerick, Ireland Cpl Healthcare Full time

    Job Description: Carelon Global Solutions Ireland (previously known as Legato Health Technologies Ireland) is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare. Founded in 2017 as a...

  • Staff Data engineer

    3 weeks ago

    Limerick, Ireland Stryker Full time

    Why engineering at Stryker? At Stryker we are dedicated to improving lives, with a passion for researching and developing new medical device products. As an engineer at Stryker, you will be proud of the work that you will be doing, using cutting-edge technologies to make healthcare better. Here, you will work in a supportive culture...

  • Lead Data Scientist

    4 hours ago

    Limerick, Ireland Cpl Full time

    Our client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare, they are committed to transforming the world of healthcare through operational excellence, innovation, and digital...

  • Senior Data Engineer

    5 days ago

    Limerick, Ireland Cpl Healthcare Full time

    DESCRIPTION The Role Our client is looking for a Data Engineer to join a fast-paced, innovative, and highly visible team building data-driven digital solutions. The position is based at their Limerick office in Ireland, 3 days a week. Key Responsibilities · Develop a strong understanding of the business, strategic direction, overall organizational goals...

  • Data Analyst Mid, Senior or Lead

    3 weeks ago

    Limerick, Ireland Cpl Healthcare Full time

    About the job This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare. Founded in 2017 as a fully owned subsidiary of a major US Healthcare company, we are committed to...

  • Data Analyst Mid, Senior or Lead

    3 weeks ago

    Limerick, Ireland Cpl Full time

    About the job This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare.  Founded in 2017 as a fully owned subsidiary of a major US Healthcare company, we are...

  • Data Analyst and Data Governance

    4 weeks ago

    Limerick, Ireland Cpl Full time

    Primary duties may include but are not limited to: Teamwork:Work collaboratively within cross functional teams Communication:Good verbal and written communication skills to translate analysis into useful insightsTechnical responsibilities (independently) :Analysing complex healthcare data and building actionable insightsProducing insightful reports and...

  • Data Historian Systems Automation Engineer

    1 week ago

    Limerick, Ireland Lilly Full time

    We’re looking for people who are determined to make life better for people around the world. Overview: At Lilly, we serve an extraordinary purpose. We make a difference for people around the globe by discovering, developing and delivering medicines that help them live longer, healthier, more active lives. Not only do we deliver breakthrough...

  • Senior Data Scientist

    21 hours ago

    Limerick, Ireland Cpl Healthcare Full time

    Our fantastic Client in Limerick City is looking for a Senior Data Scientist to join their team. Primary duties may include but are not limited to: • Write correct and clean code with limited guidance while ensuring following best practices. • Demonstrated ability to effectively collaborate across multiple teams and stakeholders to drive business...

  • Senior Data Scientist

    5 days ago

    Limerick, Limerick, Ireland Cpl Full time

    Our fantastic Client in Limerick City is looking for a Senior Data Scientist to join their team.Primary duties may include but are not limited to: • Write correct and clean code with limited guidance while ensuring following best practices. • Demonstrated ability to effectively collaborate across multiple teams and stakeholders to drive business outcomes...

  • Senior Data Scientist

    5 days ago

    Limerick, Ireland Cpl Full time

    Our fantastic Client in Limerick City is looking for a Senior Data Scientist to join their team.Primary duties may include but are not limited to: • Write correct and clean code with limited guidance while ensuring following best practices. • Demonstrated ability to effectively collaborate across multiple teams and stakeholders to drive...

  • Data Analyst Remote

    4 weeks ago

    Limerick, Ireland Cpl Full time

    Primary duties may include but are not limited to:  Teamwork:Work collaboratively within cross functional teams Communication:Good verbal and written communication skills to translate analysis into useful insightsTechnical responsibilities (independently) :Analysing complex healthcare data and building actionable insightsProducing insightful reports...

  • Lead Full stack NodeJS Developer

    3 weeks ago

    Limerick, Limerick, Ireland Cpl Full time

    NodeJS Developer- LLM- Senior & Lead This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It's all about simplifying healthcare. Founded in 2017, they are committed to transforming the world of healthcare through...

  • Senior Data Scientist NLP

    2 days ago

    Limerick, Ireland Cpl Healthcare Full time

    Primary duties may include but are not limited to: · Write correct and clean code with limited guidance while ensuring following best practices. · Demonstrated ability to effectively collaborate across multiple teams and stakeholders to drive business outcomes · Analyse and classify semi-complex change request and identify and documents system code...

  • Lead Full stack NodeJS Developer

    3 weeks ago

    Limerick, Ireland Cpl Full time

    NodeJS Developer- LLM- Senior & Lead This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare. Founded in 2017, they are committed to transforming the world of healthcare...

  • Reporting and Analytics Lead

    2 weeks ago

    Limerick, Limerick, Ireland J&J Family of Companies Full time

    Reporting and Analytics Lead WDescriptionJob Title: Reporting and Analytics Lead X 2Company: Johnson & Johnson MedTech (Vision)Location: Limerick, IrelandJob Type: Full-timeJohnson & Johnson MedTech is recruiting for a Reporting and Analytics LeadAt Johnson & Johnson, we believe health is everything. Our strength in healthcare innovation empowers us to build...