Professional Services AI Training Specialist - Task Creation & Model Evaluation (Contract)
- Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Posting language
- English
- Posted
Original job description
About Rise Data Labs Rise Data Labs is a leading AI training and data annotation company that specializes in creating high-quality training data for artificial intelligence systems. We work with top AI companies and research institutions to improve machine learning models through expert human annotation and validation. Our team of domain specialists, subject matter experts, and quality assurance professionals work across various fields including accounting, consulting, HR, marketing services, and other professional and business services. We pride ourselves on our attention to detail, domain expertise, and commitment to delivering accurate, high-quality training data.
Job Overview As a Professional Services AI Training Specialist at Rise Data Labs, you will play a crucial role in training and enhancing AI models specifically tailored for the professional services domain. This role goes beyond traditional annotation and labeling - you'll author realistic, economically valuable professional services tasks and rubrics based on real client work, then grade AI-generated deliverables against them. We're especially focused on accounting and audit expertise right now, but are open to related professional service backgrounds as well.
What You'll Do
Job Overview As a Professional Services AI Training Specialist at Rise Data Labs, you will play a crucial role in training and enhancing AI models specifically tailored for the professional services domain. This role goes beyond traditional annotation and labeling - you'll author realistic, economically valuable professional services tasks and rubrics based on real client work, then grade AI-generated deliverables against them. We're especially focused on accounting and audit expertise right now, but are open to related professional service backgrounds as well.
What You'll Do
- Create realistic task scenarios based on real client or engagement work - e.g., reviewing financial statements and audit workpapers, building a consulting deliverable, preparing an HR or compliance report, or drafting a client-facing proposal or analysis
- Supply the kinds of reference materials you'd normally work from (financial statements, engagement letters, project reports, supporting schedules)
- Write detailed rubrics specifying exactly what a correct deliverable requires - accurate figures/formulas, proper file structure, professional formatting
- Grade AI-generated deliverables (often Excel workbooks, Word documents, or PDF reports) against your rubric
- Join calibration sessions with other experts to ensure consistent scoring across the benchmark
- Prior hands-on experience in one or more of: accounting/audit (CPA or equivalent preferred), management or strategy consulting, HR, marketing/business services, or another professional services engagement-based role
- Comfort reviewing detailed client-facing work product line by line
- Prior experience with AI model training, evaluation, or benchmark development is a plus
More about this job
Responsibilities
Create realistic professional services task scenarios and provide relevant reference materials, then develop detailed rubrics defining accurate and professionally formatted deliverables. Evaluate AI-generated documents and workbooks against those rubrics and participate in calibration sessions to support consistent scoring.
Requirements
Candidates should have hands-on experience in accounting or auditing, consulting, HR, marketing or another engagement-based professional services role, with accounting or audit expertise especially relevant. They should be comfortable reviewing detailed client-facing work product line by line; CPA or equivalent credentials are preferred, and AI training, evaluation or benchmark development experience is a plus.
Skills
- Accounting
- Auditing
- Task Scenario Creation
- Rubric Development
- AI Model Training
- Model Evaluation
- Benchmark Development
- Quality Assurance
- Financial Statement Review
- Audit Workpaper Review
- Consulting
- Human Resources
- Marketing Services
- Client-Facing Work Product Review
- Excel
- Professional Writing
Benefits
- Professional Development Opportunities
- Flexible Working Environment
Visa sponsorship
Not detected in the job text
Categories
- Finance & Accounting
- Data & Analytics
- Technology
- Consulting
- Human Resources
Keywords
- AI Training
- Data Annotation
- Model Evaluation
- Benchmark Development
- Accounting
- Auditing
- CPA
- Financial Statements
- Audit Workpapers
- Consulting Deliverables
- Human Resources
- Marketing Services
- Compliance Reports
- Task Scenarios
- Rubric Development
- Reference Materials
- Quality Assurance
- Calibration Sessions
- Excel Workbooks
- Word Documents
- PDF Reports
- Client-Facing Proposals
- Professional Formatting
- Financial Formulas