Job Title : Content Rater
Location : Av. Maria Coelho Aguiar, 215 - Jardim São Luís, São Paulo - SP,
100% Onsite.
Work from office 9am-4pm BRT.
Salary : BRL 2100 / month
Languages required : Portuguese and English
About the Role
Our project currently seeks a Content Rater. The main goal of this position is to identify and rate various types of content (mostly text, but it can also include audio, video and photos) as per the defined policies and guidelines. The responsibilities include reviewing and rating user / AI-generated content. Ratings should be made within the provided task guidelines and supported by written explanations to enhance rating quality.
Roles and Responsibilities
- Evaluate and rate various content types, including mostly text, but also audio, video and photos.
- Assess AI generated responses for accuracy, factuality, relevance, helpfulness, language usage, and alignment with user intent.
- Conduct research & fact-checking by comparing AI-generated response with other generally available sources (ex : Web search, social media, 3rd party materials) and identify key differences and trends.
- Conduct side-by-side comparisons and identify key differences between responses.
- Rate and stack rank responses& provide clear reasoning while ensuring adherence to guidelines
- Ensure consistency in rating decisions
- Supporting internal training
- Conducting QA / Peer reviews
- Attend customer calls / calibration sessions / calls
- Supporting other projects outside of your core project, as and when required,
- Re-writing prompts or responses
Skills Requirements & Qualifications
English B1 / B2Native in Portuguese language to perform the content ratingUniversity Degree (preferred)1 year of relevant experience (inclusive of journalism, literature, writing or other comparable experience)Basic level computer handling skills (along with a typing speed of 25-30 words per minute)Comfortable with G suite, especially Google SheetsExcellent in verbal & writing skills with the ability to present reasoning in a clear, concise, and logical mannerUnderstanding of various AI models and their capabilities.Knowledge of established guidelines and best practices for content rating.Experience in evaluating and comparing content from different sources.Detail oriented, rigorous, analytical skills to assess content factuality, relevance, and alignment with user intent.Research skills to evaluate content accuracy and identify trustworthy sources.Critical thinking skills to identify biases, inconsistencies, and limitations in AI-generated responses.Attention to detail to review content for grammar, style, and adherence to ethical / legal standards.