Cluster Site Reliability Engineer
- Toronto, ON, Canada
- Greater Toronto Area, Canada, Canada
$210,000 – $340,000 USD (US Cologix regions) / equivalent in CA. · Full-time
Applications open in a new tab.
Job details
- Pay
- $210,000 – $340,000 USD (US Cologix regions) / equivalent in CA.
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
Cluster & SRE
Cluster Site Reliability Engineer
Own the physical reality of the platform in one of our seven regions. You bring up new GPU racks, validate InfiniBand fabric end-to-end, and keep the cluster running at the SLA. This is a hands-on role; you will see the hardware.
Apply by email All open roles
Team Cluster & SRE
Location On-site
Apply by email
Hiring manager replies within 5 business days.
The team
About The Team
Cluster SRE is six engineers across the seven regions. Each region has a primary and a secondary; you will be one of those for your region. The team coordinates daily, deploys weekly, and rotates a global pager.
Reports to the head of cluster engineering. Primary on a single region; rotates secondary for one neighboring region.
The role
What you'll do
What we're looking for
Nice to have, not required
In writing, like everything else
We publish bands. We meet them. The number you see on the offer is the same number your future peers got at the same level. We do not negotiate; we level.
Base
$210,000 – $340,000 USD (US Cologix regions) / equivalent in CA.
Equity
Meaningful early-stage equity, refreshed on tenure milestones.
Notes
On-site pay differential at Cologix regions outside SF / NYC / Bay Area is +5–10% to compensate for travel.
How to apply
One email is enough
Send a short note to careers@iframe.ai with the role title in the subject line. Include your CV or LinkedIn, one or two links to work you're proud of, and a sentence on why this role specifically. Hiring managers reply within five business days, regardless of outcome.
A hiring manager reads every email. Reply within five business days.
30–45 minutes. Scope, role, mutual fit. We share the comp band on this call.
3–4 sessions on the same day. Real problems, no homework, no whiteboard riddles.
Same-week offer at the published band for your level. Start dates are flexible.
Equal opportunity
We hire on the work. Race, gender, age, nationality, religion, sexual orientation, disability, and veteran status do not factor into our decisions. We sponsor visas for senior roles in the US, UK, and EU - bring it up on the manager call.
Need an accommodation for the interview process? Mention it in your application or write careers@iframe.ai.
Also open
Other roles you might consider
Ship the next 2× on the open-source model catalog. Triton, CUDA, ROCm. You will publish what you ship.
View role
Lead a paper / quarter on multi-thousand-GPU pre-training. Co-appointment with a partner university available.
View role
Embedded with a small portfolio of reserved-tier accounts. Distributed training perf, NCCL, kernel tuning.
View role
All open roles
One last thing
If this role isn't quite right but you'd be a fit at iframe.ai, write anyway.
Senior engineers and researchers can apply outside the listed roles. The bar is the same. The reply window is the same.
Apply: Cluster Site Reliability Engineer General application
Cluster Site Reliability Engineer
Own the physical reality of the platform in one of our seven regions. You bring up new GPU racks, validate InfiniBand fabric end-to-end, and keep the cluster running at the SLA. This is a hands-on role; you will see the hardware.
Apply by email All open roles
Team Cluster & SRE
Location On-site
- Cologix region of choice (Toronto, Montréal, Columbus, Vancouver, Ashburn)
- IC4–IC6
Apply by email
Hiring manager replies within 5 business days.
The team
About The Team
Cluster SRE is six engineers across the seven regions. Each region has a primary and a secondary; you will be one of those for your region. The team coordinates daily, deploys weekly, and rotates a global pager.
Reports to the head of cluster engineering. Primary on a single region; rotates secondary for one neighboring region.
The role
What you'll do
- 01 Bring up new B200 / B300 / MI300X racks: cabling, ToR config, NCCL/RCCL all-reduce validation, MFU baseline tests.
- 02 Drive InfiniBand fabric to spec - NDR / XDR depending on the rack - and chase residual bit-error budget down to zero.
- 03 Run capacity planning across seven regions: forecast demand, model power and thermal headroom, work with procurement on lead times.
- 04 Own the regional incident response. P1 incidents page within fifteen minutes; resolution target is four hours.
- 05 Build and maintain the bring-up runbook so the second hire after you can do their first rack solo.
- 06 Carry the global pager about one week per six, alongside runtime and customer engineering.
What we're looking for
- Five-plus years operating large compute clusters - supercomputing centers, hyperscaler infra, or HPC at a national lab count.
- Deep InfiniBand and Ethernet RoCE experience: subnet manager tuning, fabric debugging, lossless networking.
- Strong Linux fundamentals: kernel parameters, NUMA topology, kernel cgroup limits.
- Comfort writing Go or Python for tooling. We are not strict about which.
- Calm under load. You will be the named person on a $50M-ARR account when something goes wrong.
Nice to have, not required
- DGX H100 / H200 / B200 bring-up history.
- Experience with NVIDIA Bright / Base Command Manager.
- Bare-metal provisioning systems: Tinkerbell, MAAS, Razor, or in-house equivalents.
- Procurement / vendor management experience.
In writing, like everything else
We publish bands. We meet them. The number you see on the offer is the same number your future peers got at the same level. We do not negotiate; we level.
Base
$210,000 – $340,000 USD (US Cologix regions) / equivalent in CA.
Equity
Meaningful early-stage equity, refreshed on tenure milestones.
Notes
On-site pay differential at Cologix regions outside SF / NYC / Bay Area is +5–10% to compensate for travel.
How to apply
One email is enough
Send a short note to careers@iframe.ai with the role title in the subject line. Include your CV or LinkedIn, one or two links to work you're proud of, and a sentence on why this role specifically. Hiring managers reply within five business days, regardless of outcome.
- 01
A hiring manager reads every email. Reply within five business days.
- 02
30–45 minutes. Scope, role, mutual fit. We share the comp band on this call.
- 03
3–4 sessions on the same day. Real problems, no homework, no whiteboard riddles.
- 04
Same-week offer at the published band for your level. Start dates are flexible.
Equal opportunity
We hire on the work. Race, gender, age, nationality, religion, sexual orientation, disability, and veteran status do not factor into our decisions. We sponsor visas for senior roles in the US, UK, and EU - bring it up on the manager call.
Need an accommodation for the interview process? Mention it in your application or write careers@iframe.ai.
Also open
Other roles you might consider
- Runtime
Ship the next 2× on the open-source model catalog. Triton, CUDA, ROCm. You will publish what you ship.
View role
- Research lab
Lead a paper / quarter on multi-thousand-GPU pre-training. Co-appointment with a partner university available.
View role
- Customer engineering
Embedded with a small portfolio of reserved-tier accounts. Distributed training perf, NCCL, kernel tuning.
View role
All open roles
One last thing
If this role isn't quite right but you'd be a fit at iframe.ai, write anyway.
Senior engineers and researchers can apply outside the listed roles. The bar is the same. The reply window is the same.
Apply: Cluster Site Reliability Engineer General application
More about this job
Responsibilities
Own the physical platform reality in a specific region, including bringing up GPU racks and validating InfiniBand fabric. Manage regional incident response, capacity planning, and maintain bring-up runbooks while participating in a global pager rotation.
Requirements
Requires five-plus years of experience operating large compute clusters such as supercomputing centers or hyperscaler infrastructure. Must have deep expertise in InfiniBand, Ethernet RoCE, Linux fundamentals, and proficiency in Go or Python.
Skills
- InfiniBand
- Linux
- Kubernetes
- Terraform
- Prometheus
- Grafana
- Go
- Python
- NCCL
- RCCL
- RoCE
- Capacity Planning
- Incident Response
- Hardware Bring-up
- NUMA Topology
- Kernel Tuning
Benefits
- Early-stage equity
Visa sponsorship
Detected
Categories
- Technology
- Engineering
- Software
- Data & Analytics
Keywords
- Cluster SRE
- GPU Racks
- InfiniBand
- NDR
- XDR
- NCCL
- RCCL
- Kubernetes
- Terraform
- Prometheus
- Grafana
- Go
- Python
- Linux Kernel
- NUMA
- Cgroup
- B200
- B300
- MI300X
- DGX H100
- DGX H200
- NVIDIA Bright
- Base Command Manager
- Tinkerbell
- MAAS
- Razor
- HPC
- Supercomputing
- Bare-metal Provisioning
- Lossless Networking