Senior Coding, ML & Security Experts - AI Benchmarking
- Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
We’re looking for experienced coding, machine learning or cybersecurity professionals to develop challenging tasks for an AI benchmark. You’ll draw on problems from your own field and turn them into clearly defined challenges that an AI agent must solve from scratch in a Linux terminal.
What you’ll do
- Identify difficult, meaningful problems in your area of expertise.
- Translate those problems into tasks that can be solved in a Linux terminal.
- Design challenges that expose limitations in advanced AI agents. Each task is tested against three leading models, with three attempts per model, and must prove difficult enough that the models mostly fail.
- At least 8 years of relevant professional experience in coding, machine learning or cybersecurity.
- A completed master’s degree or higher. Candidates with a completed bachelor’s degree, at least 10 years of relevant professional experience and a publication may also be considered.
- Applicants must have completed their degrees and must not currently be enrolled in a degree program.
- At least one academic or professional publication.
- Practical comfort working in a Linux terminal.
- Current residence in Argentina, Mexico, Puerto Rico, New Zealand, Canada, the United Kingdom, the United States, Australia, Peru, Colombia, Austria, Ireland, Spain, Luxembourg, Italy, France, Germany, Belgium, the Netherlands, Sweden, Finland, Singapore, Denmark, Norway and Switzerland.
- Task-based project expected to run for the next weeks.
- Onboarding includes two courses and a graded assessment, taking approximately 90 minutes.
- A live onboarding webinar is also required before moving to tasking.
Earn up to $1,350 per task, with opportunities for higher rates based on task quality and submission volume. Tasks take six to seven hours on average, although initial tasks may take longer as you get familiar with the process.
More about this job
Responsibilities
Identify difficult, meaningful problems in coding, machine learning, or cybersecurity and convert them into clearly defined tasks solvable from scratch in a Linux terminal. Design and test benchmark challenges against leading AI models to expose their limitations and ensure the tasks are sufficiently difficult.
Requirements
Applicants need at least eight years of relevant professional experience, a completed master’s degree or higher, and at least one academic or professional publication; candidates with a completed bachelor’s degree, at least ten years of experience, and a publication may also qualify. Applicants must not currently be enrolled in a degree program, must be comfortable using a Linux terminal, and must reside in one of the listed eligible countries.
Skills
- Coding
- Machine Learning
- Cybersecurity
- Task Design
- AI Benchmarking
- Linux Terminal
- Problem Solving
- Challenge Development
- Technical Writing
- Model Evaluation
- Academic Publishing
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Data & Analytics
- Security & Safety
- Science & Research
Keywords
- AI Benchmarking
- Artificial Intelligence
- AI Agents
- Coding
- Machine Learning
- Cybersecurity
- Linux Terminal
- Task-Based Project
- Benchmark Tasks
- Challenge Design
- Problem Solving
- Model Evaluation
- Technical Assessment
- Academic Publication
- Professional Publication