This is a Rubrics Evaluator (AI Quality Assurance) role with Mercor based in Sydney, NSW, Australia

== Mercor ==

Role Seniority - mid level, senior

More about the Rubrics Evaluator (AI Quality Assurance) role at Mercor

Role Overview

Mercor is seeking experienced enterprise professionals to contribute to an AI evaluation and benchmarking project with a leading AI lab. Contributors will help assess the quality of AI-generated responses, reasoning, and decision-making across complex business and operational scenarios. This opportunity is ideal for professionals with strong analytical judgment and experience navigating enterprise environments such as consulting, finance, law, operations, manufacturing, engineering, or related domains.

Key Responsibilities

• Evaluate AI-generated outputs using structured rubrics and scoring frameworks
• Assess qualitative and quantitative reasoning across enterprise-focused scenarios
• Review business, operational, legal, financial, or strategic analyses for accuracy and clarity
• Identify gaps, inconsistencies, or weak reasoning in AI-generated content
• Provide written justification and feedback to support evaluation decision
  
  

Ideal Qualifications

• Professional degree in a relevant field such as business, law, finance, engineering, operations, or a related discipline
• 4+ years of experience in an enterprise environment
• Background in areas such as consulting, finance, legal operations, manufacturing, operations, engineering, or corporate strategy
• Strong quantitative and qualitative reasoning skills
• Availability through Tuesday, May 19th
• Excellent written communication and attention to detail
• Comfortable evaluating complex business and operational scenarios independently
  
  

Application Process

• Make sure your Mercor profile reflects your latest work experience
• Selected applicants may complete a short evaluation exercise
• Qualified contributors will be contacted with next steps and project details
  
  

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract and Payment Terms

• You will be engaged as an independent contractor.
• This is a fully remote role that can be completed on your own schedule.
• Projects can be extended, shortened, or concluded early depending on needs and performance.
• Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are weekly on Stripe or Wise based on services rendered.
• Please note: We are unable to support H1-B or STEM OPT candidates at this time.
  
  

Before we jump into the responsibilities of the role. No matter what you come in knowing, you’ll be learning new things all the time and the Mercor team will be there to support your growth.

🟢 Please consider applying even if you don't meet 100% of what’s outlined 🟢

Key Responsibilities

• 📝 Evaluating AI-generated outputs
• 🔍 Assessing reasoning
• ✅ Reviewing analyses
  
  

Key Strengths

• 🧠 Analytical judgment
• ✍️ Written communication
• 🔎 Attention to detail
• 💼 Experience in consulting
• 💰 Knowledge of finance
• ⚖️ Familiarity with legal operations
  
  

Why Mercor is partnering with Hatch on this role. Hatch exists to level the playing field for people as they discover a career that’s right for them. So when you apply you have the chance to show more than just your resume.

A Final Note: This is a role with Mercor not with Hatch.