Multimodal AI Evaluation Analyst

Job Description : Multimodal AI Evaluation Analyst

Target Language : English

Company : iMerit Scholars

Position Type : - Remote; Contract ($5 / hour); ~10-40 hours per week

Position Overview :

iMerit Scholars is rapidly expanding our network of skilled AI Evaluation Analysts to help advance next-generation multimodal AI systems.

This opportunity begins with a short paid pilot project (~12 hours total). Successful participants may be invited to continue with ongoing project work (typically 15–40 hours per week), depending on performance and project needs.

As a Multimodal AI Evaluation Analyst, you’ll perform highly detailed evaluations of AI system outputs across text, image, video, and multimodal prompts. Your feedback will directly influence how leading AI systems understand and interact with the world.

Compensation & Work Structure

Rate : $5 / hour

Phase 1 : ~12-hour paid pilot project (completed within a few days)

Phase 2 : Ongoing contract work (typically 15–40 hours / week) for top performers

What You’ll Do

Evaluate and score AI-generated outputs (text, image captions, video descriptions, multimodal prompts)

Assess quality, correctness, coherence, style, and cultural alignment

Identify subtle issues like hallucinations, logical gaps, or bias

Provide detailed written feedback and apply complex evaluation guidelines consistently

Collaborate with Quality Leads and Project Managers to ensure reliability and accuracy

What You Bring

Excellent critical reading and evaluation skills across multiple modalities

Strong English comprehension (CEFR B2 or above)

Familiarity with LLMs, generative AI, and multimodal systems

Exceptional attention to detail and consistency

Comfort with iterative feedback, evolving workflows, and nuanced judgments

Prior experience in data annotation, AI evaluation, or related work (preferred)

Preferred Qualifications

BA degree (preferred) or comparable experience

Experience working with annotation tools or AI data platforms

Understanding of multimodal communication (text-to-image, video narratives, etc.)

Comfort with fast-paced projects and evolving quality standards

Awareness that incidental exposure to sensitive or NSFW content may occur

Why Join iMerit Scholars

Help shape how cutting-edge AI systems understand the world

Work remotely on projects at the forefront of multimodal AI

Gain hands-on experience in applied AI evaluation

Collaborate with a global, expert community of AI trainers

Next Steps

Start by taking the 30-minute skill assessment

Qualified applicants will be invited by email to a paid 12-hour pilot project (includes 2 hours paid training)

Strong results in the pilot may lead to ongoing, flexible remote work (15–40 hrs / week) with iMerit Scholars.