Content Evaluator

4 weeks ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

Overview

Job Opportunity : Content Quality Assurance Specialist.

We are seeking highly analytical and detail-oriented professionals with hands-on experience in testing and evaluating AI-generated content to identify vulnerabilities, assess risks, and ensure compliance with quality standards.

Responsibilities

Conduct rigorous testing of large language models (LLMs) to identify adversarial outputs.

Evaluate and stress-test AI prompts across multiple domains (e.g., finance, healthcare, security) to uncover potential failure modes.

Develop and apply test cases to assess accuracy, bias, toxicity, hallucinations, and misuse potential in AI-generated responses.

Collaborate with data scientists and safety researchers to report risks and suggest mitigations.

Perform manual QA and content validation across model versions, ensuring factual consistency, coherence, and guideline adherence.

Create evaluation frameworks and scoring rubrics for prompt performance and safety compliance.

Document findings, edge cases, and vulnerability reports with high clarity and structure.

Requirements

Proven experience in AI red teaming, LLM safety testing, or adversarial prompt design.

Familiarity with NLP tasks and ethical considerations in generative AI.

Strong background in Quality Assurance, content review, or test case development for AI / ML systems.

Understanding of LLM behaviors, failure modes, and model evaluation metrics.

Excellent critical thinking, pattern recognition, and analytical writing skills.

Ability to work independently, follow detailed evaluation protocols, and meet tight deadlines.

Next Steps

To proceed further in the evaluation process, you will need to complete two assessments : a linguistic and analytical skills assessment and an English proficiency test (C1 or C2 level required).

Once both assessments are successfully completed, you will be eligible for onboarding.

You will also receive an invitation to our internal job platform.

Please take a few minutes to register and complete your profile.

If interested, kindly share your resume at

Seniorities

Entry level

Employment type

Full-time

Job function

Education and Training

Industries

Business Consulting and Services

Referrals increase your chances of interviewing at Innodata Inc. by 2x

Get notified about new Evaluator jobs in Salvador, Bahia, Brazil.

Related roles

Software Engineer, Platform - Salvador, Brazil

Backend Software Engineer (Node, Python, Ruby on Rails or Go!)

Front-End Developer - Remote Work | REF#

Salvador, Bahia, Brazil R$9,

• R$9,
• month ago

We're unlocking community knowledge in a new way.

Experts add insights directly into each article, started with the help of AI.

#J-