Big Data Engineer

5 days ago


Dublin, Dublin City, Ireland Vectra Full time

About Vectra

Vectra is the leader in AI-driven threat detection and response for hybrid and multi-cloud enterprises. Our cutting-edge platform empowers security teams to rapidly prioritize, investigate and respond to the most advanced cyber-attacks.

Job Overview

We are seeking an innovative and resourceful data engineer to join our growing team. As a key member of our data science and analytical capabilities, you will assist in optimizing data flows and building out new data pipelines to support our growing product portfolio.

Responsibilities

  • Create, test and maintain optimal data pipeline architectures
  • Identify and develop data set processes for data modeling, minding, and production
  • Build the infrastructure required for optimal extraction, transformation, and loading of large data sets
  • Spark fresh ideas that are infused in our culture and our products
  • Leverage emerging tech and Big Data to ensure delivery of real-time analytics solutions
  • Work on complex, large-scale, group-wide big data projects
  • Perform analysis tasks required to troubleshoot data-related issues and present optimized solutions
  • Interface with other groups including Product Management, UX, Security Research and Data Science to help customers simplify security

Qualifications

  • 6+ years in software development, data engineering, or equivalent
  • Knowledge of software design principles and leading software development practices
  • Willingness to get things done, learn new things, take initiative and challenge existing assumptions and conventions
  • Experience building and deploying to any cloud service (AWS, Azure, GCP, etc.)
  • Production experience with various distributed processing technologies (e.g., Kubernetes, Kafka, Spark, etc.)
  • Comfort in dealing with low-level data storage formats (e.g., Parquet, Avro, Arrow, etc.)
  • Experience programming in any of the following (python, golang, scala)
  • Experience with infrastructure as code (Terraform), automated testing, and CI/CD
  • Knowledge of the following tools (or similar): Git, Jira
  • Experience working with a geographically dispersed team
  • Experience with grooming business-stakeholder requirements into achievable technical goals
  • Ability to work in a collaborative environment
  • B.S or M.S or Ph.D. in Computer Science (or equivalent experience)

  • Big Data Engineer

    3 weeks ago


    Dublin, Dublin City, Ireland Reperio Human Capital Full time

    Big Data Engineer 101043 Desired skills: Big Data Engineer, ETL, Python, Spark, Dublin, Hybrid Hybrid About The Role:   This project is with a big player in the public sector. They are looking to introduce a Big Data Engineer for leveraging data-driven insights. They are committed to innovation, excellence, and making a positive impact...


  • Dublin, Dublin City, Ireland Reperio Human Capital Full time

    Big Data Systems Engineer 100128 Desired skills: Big Data,Cloudera, Hadoop, Python, Hybrid, Dublin Big Data Systems Engineer €75,000 - €85,000 Hybrid (1x per Week) Dublin Reperio have partnered with one of Irelands leading FinTech companies. They are renowned for their innovation within the technology space, providing top...

  • Big Data Engineer

    1 week ago


    Dublin, Dublin City, Ireland Playrix Full time

    Big Data Engineer Playrix is one of the most successful mobile game development companies in the world. In terms of revenue, we rank among the top 3 in the world and are first among all mobile applications in EMEA. Our games have been installed 2.3 billion times (that's a third of the world's population) and over 30 million people play them every day. ...

  • Big Data Architect

    5 days ago


    Dublin, Dublin City, Ireland Amazon Full time

    About the RoleThis is a senior most data engineering role within the Amazon MX Player organization.You will work closely with business stakeholders, software development engineers, and scientist colleagues to design, evangelize, and implement state-of-the-art solutions.Required SkillsStrong analytical, project management, and problem-solving...


  • Dublin, Dublin City, Ireland HCLTech Full time

    Job Title: Associate ManagerHCLTech is seeking an experienced Associate Manager to join their team.About the JobThis mid-senior level position requires a strong background in big data technologies, including Spark, Scala, Hive/HDFS/HQL, and experience with real-time data processing platforms.The ideal candidate will have 6-8 years of relevant experience,...

  • Big Data Engineer

    1 day ago


    Dublin, Dublin City, Ireland Amazon Full time

    Job Summary:We are seeking a highly skilled Big Data Engineer to join our team in the Cloud Infrastructure domain. The ideal candidate will have expertise in designing, building, and maintaining large-scale data processing systems using AWS services.Key Responsibilities:Design and implement scalable data processing systems using AWS services.Collaborate with...


  • Dublin, Dublin City, Ireland Archer Recruitment Full time

    Big Data Solutions SpecialistDrive business growth through data insights by leveraging your expertise in designing, building, and optimizing large-scale data solutions using the latest technologies like Databricks, PySpark, and Azure.Archer Recruitment's FinTech client seeks a Senior Data Engineer to join their rapidly expanding data team and contribute to...


  • Dublin, Dublin City, Ireland Amazon Full time

    About the RoleWe are seeking an experienced AWS Big Data Solutions Expert to join our team. As a Cloud Support Engineer, you will be responsible for providing technical support to our customers, resolving complex issues related to big data and distributed systems.About the Job:Leverage your customer support experience to provide feedback to internal AWS...

  • Big Data Engineer

    2 days ago


    Dublin, Dublin City, Ireland TN Ireland Full time

    Becoming a Data LeaderWe are a UK-based digital bank that aims to fix banking by leveraging cutting-edge technology and innovative solutions. Our data team plays a crucial role in delivering this vision by designing and implementing data-driven solutions that drive business growth and customer satisfaction.About Our Data TeamOur data team consists of...


  • Dublin, Dublin City, Ireland Archer Recruitment Full time

    We are seeking a highly skilled Data Solutions Architect to lead our team in building big data and cloud solutions. As an expert in designing and building data lakes, you'll play a key role in developing new data pipelines that will power our client's future growth.The ideal candidate will have 15+ years of experience as a Senior Data Engineer, deep...


  • Dublin, Dublin City, Ireland Amazon Web Services Emea Sarl (Irish Branch) Full time

    Amazon Web Services is the market leader and technology forerunner in the Cloud business.As a member of the AWS Support team you will be at the forefront of this transformational technology, assisting a global list of companies and developers that are taking advantage of a growing set of services and features to run their mission-critical applications.As a...


  • Dublin, Dublin City, Ireland Archer Recruitment Full time

    Architect Capital Markets + Big Data Design Enterprise-Scale SolutionsAbout the RoleWe are looking for an experienced Solution Architect to lead the design and implementation of enterprise-scale greenfield solutions for a global capital markets leader.As a Solution Architect, you will be responsible for driving the architecture strategy and making key...


  • Dublin, Dublin City, Ireland Flywheel Full time

    About Our CompanyFlywheel's suite of digital commerce solutions enables businesses to accelerate growth across multiple platforms.Our team works closely with clients to provide near real-time performance measurement and improve sales, share, and profit.Job RoleThis Senior Data Engineer position requires a strong background in designing and developing data...


  • Dublin, Dublin City, Ireland Hansevision Gmbh Full time

    Unser Unternehmen ist ein wichtiger Akteur auf dem Gebiet der IT-Lösungen. Wir bieten unseren Kunden innovative und individuelle Lösungen, um ihre Ziele zu erreichen.In deiner Rolle als Big Data-Experte bist du zuständig für die Installation und Konfiguration von Big-Data-Lösungen beim Kunden. Hierbei übernimmst du die Beratung der Kunden bei der...


  • Dublin, Dublin City, Ireland Atrium Uk Full time

    Responsibilities and Requirements:The successful candidate will have a strong background in solution architecture, cloud-based data technologies, and data modeling. They will be responsible for designing and implementing comprehensive data architectures, ensuring scalability, security, and performance across our modern big data analytics platform. Key...

  • Data Engineer

    3 weeks ago


    Dublin, Dublin City, Ireland Cpl Full time

    Cpl are supporting one of our data-driven clients in there search for a Senior Data Engineer. This role involves capturing, integrating, and managing data to support various business functions and drive decision-making processes. In the role you willcross-functionally with product, marketing, and audience teams to optimize existing and new data...

  • Data Engineer

    2 weeks ago


    Dublin, Dublin City, Ireland Luxevision Consulting LLC Full time

    About Us:Luxe Vision Consulting is a Wisconsin-based consulting firm dedicated to connecting top global talent with leading U.S. companies. We specialize in sourcing skilled professionals, conducting rigorous screening and technical assessments, and preparing candidates for opportunities in the U.S. job market.Our mission is to bridge the gap between...


  • Dublin, Dublin City, Ireland Engineeringuk Full time

    You will need to login before you can apply for a job.View more categories View less categories Sector Engineering, Technology Role Professional Contract Type Permanent Hours Full TimeDESCRIPTIONAmazon Web Services is the market leader and technology forerunner in the Cloud business.As a member of the AWS Support team you will be at the forefront of this...


  • Dublin, Dublin City, Ireland Amazon Full time

    **About Us**Amazon is a global leader in e-commerce, cloud computing, and artificial intelligence. Our DISCO team plays a crucial role in helping Amazon Music make sense of its vast amounts of data.As a Data Engineering Manager, you will lead a team of data engineers to design, build, and maintain automated self-service data solutions, data science models,...


  • Dublin, Dublin City, Ireland Amazon Full time

    Job ID: 2821025 | Amazon Web Services EMEA SARL (Irish Branch)Amazon Web Services is the market leader and technology forerunner in the Cloud business. As a member of the AWS Support team you will be at the forefront of this transformational technology, assisting a global list of companies and developers that are taking advantage of a growing set of services...