Senior Platform Engineer

Senior Platform EngineerMillennium Management LLC Dublin, Ireland

Senior Platform Engineer

Senior Platform Engineer with Specialization in HPC, Kubernetes, or Cloud

The team is comprised of extremely talented (but humble) multidisciplinary individuals. We believe that one cannot build a truly great service without the ability to make changes across the stack. We take great care in focusing on solving real business problems, reducing operational overhead and working together as a team.

Role Overview: The Senior Platform Engineer will be responsible for designing, implementing, and maintaining scalable and reliable infrastructure solutions. This role requires deep expertise in one or more of the following areas: High-Performance Computing (HPC), Kubernetes, and cloud platforms (particularly Google Cloud Platform - GCP). The focus will be on solving complex engineering problems, optimizing system performance, and ensuring the reliability and scalability of the infrastructure.

Key Responsibilities:

• Optimize Linux-based systems for performance and reliability.
• Contribute to software engineering efforts, CI/CD, and codebase management.
• Design and manage HPC job scheduling systems.
• Develop and maintain workflow management and batch processing solutions.
• Implement and manage container orchestration using Kubernetes.
• Deploy and manage large-scale storage systems.
• Provision and manage cloud environments, with a focus on GCP.
• Implement and manage cloud-native services and infrastructure on GCP.
• Ensure security and compliance in cloud environments.

Qualifications/Skills Required: We are looking for individuals with experience in one or more of the following areas:

General

• Ability to review and/or extend open-source platforms to satisfy business requirements
• A passion for technology and automation, deep sense of curiosity and willingness to always question
• A passion for in-depth understanding of technology and building large-scale systems.
• Excellent verbal and written communication skills.

HPC Job Scheduling:

• Experience in environments at scale (e.g., billions of jobs per week/month).
• Understanding of cost metrics, preemption, job types, queuing, scheduler, and optimizations.
• Experience with products like HTCondor, Slurm, Spectrum LSF, Nomad, AWS Batch.

Container Orchestration (Kubernetes):

• Experience with Helm, admission/mutation controllers, PVs/PVCs, kube-router, BGP.
• Experience with Docker & registries (e.g., Harbor, Artifactory, GCP Container Registry, AWS Container Registry).
• Mature approach to dealing with operational complexities of the Kubernetes platform.
• Experience with Kubernetes security best practices, including RBAC, network policies, and secrets management.
• Experience with Kubernetes monitoring and logging tools (e.g., Prometheus, Grafana, ELK stack).
• Experience with Kubernetes cluster management and scaling.

Storage Systems:

• Experience deploying and managing petabyte-scale systems supporting varied workloads.
• Mature approach to accessing price/performance, tiering, and backup requirements.
• Experience with products like GPFS, NetApp, Pure, Vast,GCP PDs, or other NVMe-specific products.
• Familiarity with NVMeOF, POSIX, object storage, and various modes of permissioning data.

Linux:

• Extensive experience using configuration management systems (e.g., SaltStack, Ansible, Chef) to automate and manage large-scale Linux environments
• Deep understanding of Linux kernel components (e.g., VFS, scheduler, memory management, networking stack) and their impact on system performance and stability
• Solid troubleshooting experience using advanced tools such as gdb, strace, perf, ltrace, and other OS/application tracing and profiling mechanisms
• Experience with containerization and virtualization technologies, including Docker, LXD/LXC, Kubernetes, KVM, QEMU, and VMware
• Familiarity with modern Linux security mechanisms, including Kerberos, SELinux, AppArmor, and system hardening techniques
• Proficiency in leveraging eBPF for performance monitoring, debugging, and security enforcement in Linux systems

Software Engineering:

• Proficient in one or more programming languages (we use python and go)
• Familiar with git, and CI/CD concepts.
• Comfortable contributing to a large codebase with varied technologies.

Cloud(GCP)

• The chosen candidate will own the infrastructure that makes up our GCP (Google Cloud Platform) offering and will also own the relationship between the firm and Google.
• You will be expected to collaborate with development teams across the firm to optimize cloud application performance and work closely with stakeholders to define cloud strategies and roadmaps. You will provide mentorship and guidance to junior engineers.
• Knowledge of core GCP technologies (Compute, GCS, BigQuery) and an understanding of GCP's underlying architecture.
• Experience with infrastructure-level GCP technologies (VPC Networking, Interconnect, Billing functionality, Image management) and the ability to optimize for performance and cost-efficiency.
• Strong understanding of GCP security technologies (Org Policy, VPC Service Controls, Resource Manager Tags, Firewalls, IAM) and the ability to implement security best practices.
• Knowledge of Terraform and experience managing a CI/CD desired state configuration pipeline for GCP.
• Familiarity with the gcloud toolset and the ability to write Python code against the Google API for automation and integration.
• Familiarity with other cloud platforms (AWS, Azure) and their interoperability with GCP.
• Experience designing and implementing scalable, highly available, and fault-tolerant architectures using GCP services, always following best practices.
• Proficiency in GCP monitoring and logging tools (e.g., Cloud Monitoring, Cloud Logging, Error Reporting, Trace) to ensure system health and performance, with the ability to set up alerts and dashboards.
• Experience of cost management, including monitoring and optimizing cloud spend using GCP's cost management tools (e.g., Billing Reports, Budgets, Recommendations).
• Ability to troubleshoot and resolve complex issues across the GCP stack, leveraging advanced tools and a systematic approach to problem-solving.
• Strong communication and collaboration skills, with the ability to act as a trusted advisor to colleagues, mentor team members, and share knowledge through documentation and training.

More Jobs From Millennium Management LLCBoost your careerFind thousands of job opportunities by signing up to eFinancialCareers today.#J-18808-Ljbffr

---