This role is for one of our clients
We are seeking detail-oriented and analytical Business Analysts – Non-US Raters to support a project focused on evaluating and comparing responses generated by multiple AI chatbots. The role involves reviewing chatbot outputs, applying predefined evaluation criteria, conducting structured assessments, and delivering accurate, well-documented judgments.
Candidates should have strong analytical skills, business analysis experience, attention to detail, and the ability to work independently in a remote, fast-paced environment. This opportunity is suitable for professionals interested in contributing to AI evaluation and improving the quality, relevance, and reliability of AI-generated responses.
Job Details
- Role: Business Analyst – Non-US Raters
- Experience: 3+ years
- Employment Type: Short-term contract
- Contract Duration: 16 weeks
- Location: Argentina, Peru, Mexico, Brazil, Chile, Colombia, India, and Pakistan
- Compensation: USD 15 per hour
- Start Date: Immediate
- Work Mode: Fully remote
- Availability: Full-time commitment of 8 hours per day
- Time Zone Requirement: Minimum 4 hours of overlap with the US Pacific Time (PST) time zone
- Work Schedule: Five working days per week, including weekends when required, with two weekdays off
Requirements
Key Responsibilities
- Review, analyze, and compare responses generated by multiple AI chatbots against predefined evaluation criteria.
- Conduct structured assessments to determine the accuracy, relevance, completeness, clarity, and overall quality of chatbot responses.
- Analyze business requirements and translate them into clear evaluation guidelines and functional specifications.
- Gather, interpret, and document relevant information to support consistent and objective assessments.
- Examine workflows, processes, and data to identify gaps, inconsistencies, and opportunities for improvement.
- Prepare evaluation reports, summaries, and supporting documentation with a strong focus on accuracy.
- Collaborate with project managers and cross-functional teams to clarify requirements and resolve evaluation-related questions.
- Track task progress, quality metrics, and project deliverables while meeting established deadlines.
- Identify recurring response-quality issues and provide actionable observations based on the defined assessment framework.
- Maintain consistency, confidentiality, and high-quality standards throughout the evaluation process.
Required Qualifications and Skills
- Bachelor's degree in Business Administration, Finance, Computer Science, or a related field.
- At least 3 years of experience as a Business Analyst or in a similar analytical role.
- Strong analytical thinking, problem-solving, and decision-making abilities.
- Excellent written communication skills and the ability to explain findings clearly.
- Strong attention to detail and the ability to assess information objectively.
- Proficiency in Microsoft Excel, SQL, Power BI, Tableau, or comparable analytical tools.
- Experience with requirement gathering, process mapping, and business documentation.
- Ability to work independently, manage multiple priorities, and meet strict deadlines.
- Strong documentation skills and the ability to follow structured evaluation guidelines.
- Familiarity with Agile or Scrum methodologies is an advantage.
- Willingness to commit to a full-time, 16-week remote engagement with the required time-zone overlap.
Engagement Benefits
- Opportunity to contribute to projects focused on AI evaluation and response quality.
- Fully remote work arrangement with the flexibility of working from your location.
- Exposure to structured AI assessment workflows and quality evaluation processes.
- Opportunity to contribute to projects with international reach.
- Clearly defined contract duration, hourly compensation, and evaluation responsibilities.
Important Engagement Details
- This is a project-based contract lasting 16 weeks.
- Work will consist of evaluating multiple chatbot responses and submitting a final assessment for each assigned task.
- Candidates must be available for 8 hours per day, five days per week, with four hours of overlap with US Pacific Time.
- The schedule may include weekend work, with two weekdays off each week.
- Candidates are expected to maintain consistent evaluation quality, follow project guidelines, and complete assigned tasks within the required timelines.