E-commerce Annotation Specialist
- Toronto, ON, Canada
Remote
$40 to $50 CAD / hour
Applications open in a new tab.
Job details
- Pay
- $40 to $50 CAD / hour
- Workplace
- Remote
- Working hours
- 5 hours/week
- Posting language
- English
Original job description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Shopify Shop App Agent Annotator
Type: Contract
Compensation: $40–$50/hour
Location: Remote
Commitment: ~5 hours/week
Role Responsibilities
- Evaluate full shopper-agent conversations, including tool calls and their outcomes.
- Identify the specific turn that received a thumbs-down from the shopper.
- Classify the issue with a primary domain tag and a specific secondary tag.
- Write concise comments explaining the shopper's intent, the agent's actions, and the rationale for the tags.
- Work independently and asynchronously to maintain consistency in tagging across conversations.
Qualifications
Must-Have
- Rule discipline to apply a fixed taxonomy consistently.
- Comfort reading structured traces to understand agent actions.
- Restraint under ambiguity to use the unknown label appropriately.
- Clear, concise written English for explaining labeling decisions.
Preferred
- Consumer e-commerce fluency in orders, tracking, and platform responsibilities.
- Prior annotation, labelling, or model-evaluation work with a defined rubric.
- Experience with the Shop app or other AI shopping assistants.
- Familiarity with customer-support operations and escalation paths.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
More about this job
Responsibilities
The annotation specialist at Mercor will review full shopper-agent chats, including tool calls, find the turn a shopper gave a thumbs-down to, and tag the issue. They will write short comments explaining the shopper's intent and the agent's actions, working independently.
Requirements
Candidates need rule discipline to apply a fixed taxonomy consistently, comfort reading structured traces, and clear written English. Preferred: e-commerce order and tracking knowledge, prior annotation or model-evaluation work, and customer-support experience.
Skills
- Annotation
- Data labelling
- Taxonomy tagging
- Rubric-based evaluation
- Structured trace reading
- Shopify
- E-commerce operations
- Customer-support operations
- Written English
- Consistency in tagging
- Independent asynchronous work
Categories
- Technology
- Data & Analytics
- Customer Service & Support
- Retail
Keywords
- annotation
- data labeling
- AI evaluation
- model evaluation
- rubric
- taxonomy tagging
- Shopify
- Shop app
- AI shopping assistant
- e-commerce
- order tracking
- customer support
- agent traces
- tool calls
- remote contract
- AI training data
Seniority
Entry level
Languages
- English