David Kennedy Recruitment is working with a leading technology company specialising in artificial intelligence that is seeking to onboard a Labeling & Evaluation Agent to support the development and continuous improvement of its AI models.
The role focuses on evaluating AI-generated outputs, including chatbot responses, and providing high-quality annotations and feedback. You will assess response quality, factual accuracy, relevance, safety and policy compliance, helping improve the performance and reliability of large language models (LLMs).
Position: Labeling & Evaluation Agent (French)
Location: Lisbon, Portugal
Work model: On-site
Employment type: Full-time
DUTIES AND RESPONSIBILITIES:
Evaluate AI-generated responses using detailed quality assessment and annotation guidelines
Label and annotate language model outputs accurately and consistently
Assess generated content for quality, relevance, factual accuracy, safety and policy compliance
Apply complex annotation guidelines across different scenarios and content types
Identify inconsistencies, edge cases and potential issues, escalating them when necessary
Analyse structured text, language data and AI-generated content
Maintain strict confidentiality and comply with project standards and data security requirements
Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement
REQUIREMENTS:
Native-level proficiency in French
English proficiency at B2 level or above
Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms
Excellent analytical and critical thinking skills, with strong attention to detail
Ability to make objective, evidence-based decisions
Ability to interpret and consistently apply complex written guidelines and policies
Strong reading comprehension and written language skills
Ability to evaluate structured text, datasets and AI-generated content
Ability to meet productivity, accuracy and quality targets
Professionalism, integrity and discretion when handling confidential information
BENEFITS:
Modern office in city center with open spaces, easy to access with public transportation
Comprehensive initial training and guidance to make sure you're well prepared and comfortable at a new workplace
Full-time employment in Lisbon
24/7 rotational working schedule
2 rotational days off
Opportunity to work in the rapidly developing field of Generative AI and Large Language Models
International and multicultural working environment
Support from colleagues with coaching and mentoring opportunities
Health and Wellbeing initiatives
Monthly meal allowance
And much more!!!