Lead Data Engineer

Found in: beBee S IE - 1 month ago

Limerick, Ireland Cpl Full time

About the job

This client is uniquely positioned to define the future of Health Technology â with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. Itâs all about simplifying healthcare. 


Founded in 2017, they have been committed to transforming the world of healthcare through operational excellence, innovation, and digital transformation. We are uniquely positioned to enable enterprises & organisations across multiple Technology domains including data analytics, digital technologies, automation & AI. Our Claims Processing, Provider Data Operations and Corporate Functions are among the best in the Healthcare Sector.

 

Summary of role: 

We are seeking a Lead Data Engineer with extensive big data frameworks expertise to join our team. The ideal candidate will have a strong background using Amazon Elastic MapReduce (EMR) big data platform with frameworks like Apache Hadoop, Apache Spark. In this role, you will be responsible for designing, developing, and implementing big data processing & streaming solutions on AWS EMR cloud platform leveraging Apache Spark/PySpark, Airflow, MongoDB. 


Primary Duties: 

⢠Design, develop, and process big datasets primarily using Amazon EMR, Apache Spark/ PySpark and Apache Hadoop for large data frames and NoSQL MongoDB, AWS S3 file storage. 

⢠Expertise in distributed data processing techniques and analytics. 

⢠Expertise in GCP big-data solutions such as Cloud Dataproc and GCP BigQuery also advantageous. 

⢠Knowledge of Amazon Managed Workflows for Apache Airflow (MWAA). 

⢠Programming skills with Python (PySpark APIs, Spark Core APIs, Spark MLib), Spark streaming and EMR cluster management techniques. 

⢠Knowledge of Cloud storage resources database, data warehouse, data lake and file storage solutions( AWS S3, GCP GCS, Azure Blob and IBM ICOS) 

⢠Collaborate with cross functional teams to define business requirements and translate them into data design and architecture specifications. 

⢠Analyse source to target systems, data models, and data structures to ensure data quality and consistency across systems. 

⢠Work with AWS resources services, including various data orchestration tools such as Spark/Airflow. 

⢠Knowledge of ML/OPS in deploying ML models. 

⢠Effectively manage version control using Git for large-scale projects, ensuring efficient handling of frequent and ongoing code updates, merges, and branches to maintain code integrity and collaboration among team members. 

⢠Perform data profiling, cleansing, and validation to ensure accurate data transformation and integration. 

⢠Develop and maintain ETL process documentation, including data flow diagrams and data mapping documents. 

⢠Optimise ETL performance through best practices, performance monitoring, and troubleshooting. 

⢠Participate in code reviews, peer feedback sessions, and continuous improvement initiatives. 

⢠Work closely with data warehouse and database administrators for optimal data storage and retrieval. 

⢠Provide technical support and guidance on big data ETL processes and data integration. 

⢠Stay current with industry trends and best practices for big data technologies and cloud infrastructures. 


Requirements: 

⢠Bachelor's degree in Computer Science, Information Systems, or a related field. 

⢠Minimum of 7 years of experience in data engineering roles with primary focus on big data frameworks such as Apache Spark, Hadoop, AWS EMR platform. 

⢠Strong experience with cloud-based data warehouse solutions using Snowflake, AWS Redshift, Google Big Query (GCP), Azure Synapse 

⢠Familiarity with various data & development tools - DBT, DBeaver useful. 

⢠Proficiency in SQL for data querying and transformation. 

⢠Experience with relational databases such as Postgres and SQL Server and NoSQL MongoDB. 

⢠Proficiency in Python or other relevant programming languages. 

⢠Understanding of data modelling concepts and experience in integrating database objects with applications. 

⢠Experience with both AWS and GCP cloud services beneficial. 

⢠Familiarity with version control, CI/CD tools, and best practices. 

⢠Strong analytical and problem-solving skills. 

⢠Excellent communication and collaboration skills, Confluence/JIRA. 

⢠Ability to manage multiple tasks in a fast-paced environment. 

⢠Familiarity with data privacy and security best practices in cloud environments


#LI-EL1

  • Staff Data engineer

    Found in: Talent IE C2 - 1 week ago

    Limerick, Ireland Stryker Full time

    Why engineering at Stryker? At Stryker we are dedicated to improving lives, with a passion for researching and developing new medical device products. As an engineer at Stryker, you will be proud of the work that you will be doing, using cutting-edge technologies to make healthcare better. Here, you will work in a supportive culture...

  • Staff Data engineer

    2 weeks ago

    Limerick, Ireland Stryker Group Full time

    Why engineering at Stryker? At Stryker we are dedicated to improving lives, with a passion for researching and developing new medical device products. As an engineer at Stryker,you will be proud of the work that you will be doing, using cutting-edge technologies to make healthcare better.Here,you will work in a supportive culture withotherincredibly...

  • Data Analyst Mid, Senior or Lead

    2 weeks ago

    Limerick, Ireland Cpl Healthcare Full time

    About the job This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare. Founded in 2017 as a fully owned subsidiary of a major US Healthcare company, we are committed to...

  • Data Analyst Mid, Senior or Lead

    Found in: beBee jobs IE - 3 days ago

    Limerick, Limerick, Ireland Cpl Full time

    About the job This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It's all about simplifying healthcare. Founded in 2017 as a fully owned subsidiary of a major US Healthcare company, we are committed to transforming...

  • Data Analyst Mid, Senior or Lead

    Found in: beBee S IE - 3 weeks ago

    Limerick, Ireland Cpl Full time

    About the job This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare.  Founded in 2017 as a fully owned subsidiary of a major US Healthcare company, we are...

  • Data Analyst Mid, Senior or Lead

    Found in: beBee S IE - 2 weeks ago

    Limerick, Ireland Cpl Full time

    About the job This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare.  Founded in 2017 as a fully owned subsidiary of a major US Healthcare company, we are...

  • Data Engineer AI HealthTech

    Found in: beBee S IE - 4 weeks ago

    Limerick, Ireland Cpl Full time

    Data Engineer and AI 2Our client is a global healthcare solutions provider who are uniquely positioned to define the future of Health Technology. They are committed to transforming the world of healthcare through operational excellence, innovation, and digital transformation. We are uniquely positioned to enable enterprises & organisations across multiple...

  • Data Analyst and Data Governance

    Found in: beBee S IE - 3 weeks ago

    Limerick, Ireland Cpl Full time

    Primary duties may include but are not limited to: Teamwork:Work collaboratively within cross functional teams Communication:Good verbal and written communication skills to translate analysis into useful insightsTechnical responsibilities (independently) :Analysing complex healthcare data and building actionable insightsProducing insightful reports and...

  • Data Analyst Remote

    Found in: beBee jobs IE - 1 week ago

    Limerick, Limerick, Ireland Cpl Full time

    Primary duties may include but are not limited to: Teamwork:Work collaboratively within cross functional teams Communication:Good verbal and written communication skills to translate analysis into useful insightsTechnical responsibilities (independently) :Analysing complex healthcare data and building actionable insightsProducing insightful reports and...

  • Data Analyst Remote

    Found in: beBee S IE - 3 weeks ago

    Limerick, Ireland Cpl Full time

    Primary duties may include but are not limited to:  Teamwork:Work collaboratively within cross functional teams Communication:Good verbal and written communication skills to translate analysis into useful insightsTechnical responsibilities (independently) :Analysing complex healthcare data and building actionable insightsProducing insightful reports...

  • Data Analyst Remote

    Found in: beBee S IE - 1 week ago

    Limerick, Ireland Cpl Full time

    Primary duties may include but are not limited to:  Teamwork:Work collaboratively within cross functional teams Communication:Good verbal and written communication skills to translate analysis into useful insightsTechnical responsibilities (independently) :Analysing complex healthcare data and building actionable insightsProducing insightful reports...

  • Data Analyst Healthcare

    Found in: beBee S IE - 3 weeks ago

    Limerick, Ireland Cpl Full time

    Our clients uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare.  They are committed to transforming the world of healthcare through operational excellence, innovation, and...

  • Lead Full stack NodeJS Developer

    Found in: beBee jobs IE - 1 week ago

    Limerick, Limerick, Ireland Cpl Full time

    NodeJS Developer- LLM- Senior & Lead This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It's all about simplifying healthcare. Founded in 2017, they are committed to transforming the world of healthcare through...

  • Lead Full stack NodeJS Developer

    Found in: beBee S IE - 4 weeks ago

    Limerick, Ireland Cpl Full time

    NodeJS Developer- LLM- Senior & Lead This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare. Founded in 2017, they are committed to transforming the world of healthcare...

  • Lead Full stack NodeJS Developer

    Found in: beBee S IE - 2 weeks ago

    Limerick, Ireland Cpl Full time

    NodeJS Developer- LLM- Senior & Lead This client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It’s all about simplifying healthcare. Founded in 2017, they are committed to transforming the world of healthcare...

  • Engineering & Design Lead (CSA) - Limerick

    Found in: beBee S IE - 3 weeks ago

    Limerick, County Limerick, Ireland DPS Full time

    Engineering & Design Lead (CSA) - LimerickThe role is to manage the design process with Architect Engineer on site projects, responsible for coordinating capital project related engineering and design activities in alignment with business objectives and Project Delivery Practices. Focused on execution of the implementation phase of Concept (CP), BOD, and...

  • Tech Lead

    Found in: beBee jobs IE - 1 week ago

    Limerick, Limerick, Ireland Cpl Full time

    About the jobThis client is uniquely positioned to define the future of Health Technology – with a profound purpose, to help people live longer and better by predicting and preventing health issues earlier. It's all about simplifying healthcare. Founded in 2017, they are committed to transforming the world of healthcare through operational excellence,...

  • Principal AI/ML Engineer

    Found in: Talent IE C2 - 2 weeks ago

    Limerick, Ireland Analog Devices Full time

    ADI’s Central AI team develops next-generation AI technology that transforms our understanding of the physical world. We develop solutions at multiple tech stack layers, from AI-enabled software applications to deeply embedded AI algorithms. Our mission is to build the Intelligent Edge, where AI transforms how we solve challenging problems by combining...

  • Senior Manufacturing Engineer

    Found in: Talent IE C2 - 2 weeks ago

    Limerick, Ireland Stryker Full time

    Why engineering at Stryker? At Stryker we are dedicated to improving lives, with a passion for researching and developing new medical device products. As an engineer at Stryker, you will be proud of the work that you will be doing, using cutting-edge technologies to make healthcare better. Here, you will work in a supportive culture...

  • Senior Manufacturing Engineer

    1 week ago

    Limerick, Ireland Stryker Group Full time

    Why engineering at Stryker? At Stryker we are dedicated to improving lives, with a passion for researching and developing new medical device products. As an engineer at Stryker,you will be proud of the work that you will be doing, using cutting-edge technologies to make healthcare better.Here,you will work in a supportive culture withotherincredibly...