This role is for one of our clients
Compensation: $60-$80 per hour
Join a leading AI research organization's advanced GenAI team and contribute your real-world retail expertise to the development and evaluation of next-generation AI models.
We are seeking experienced Retail Specialists with deep industry knowledge and hands-on experience evaluating AI-generated outputs against structured criteria. You will help bring practical retail judgment to AI training data by identifying knowledge gaps, designing realistic retail scenarios, and assessing whether AI-generated responses accurately reflect professional standards and practices.
This is a full-time engagement requiring approximately 35 hours per week, primarily during weekdays. The role is fully remote.
Requirements
2. Key Responsibilities
- Work with research and engineering teams to identify and address knowledge gaps related to retail merchandising, category management, buying, planning, and operations.
- Design challenging, realistic retail tasks that reflect real-world business scenarios and decision-making.
- Develop accurate, well-reasoned solutions based on established retail practices and professional experience.
- Evaluate AI-generated responses using structured rubrics, assessing accuracy, reasoning, judgment, relevance, and completeness.
- Provide clear and actionable written feedback explaining strengths, weaknesses, and areas for improvement.
- Develop and refine evaluation guidelines and scoring frameworks specifically for retail-related tasks.
- Collaborate with other subject-matter experts to maintain consistency, accuracy, and quality across AI training and evaluation datasets.
- Help translate practical retail knowledge and decision-making processes into structured guidance that can be used to improve AI systems.
3. Core Qualifications
- 8+ years of professional retail experience in areas such as merchandising, category management, buying, retail planning, retail operations, or related functions.
- Experience working within a recognized large-scale retail, consumer, e-commerce, or omnichannel organization is strongly preferred.
- Prior hands-on experience evaluating LLM or AI-generated outputs against rubrics or structured scoring criteria is mandatory. Applicants should be prepared to describe this experience.
- Demonstrated career progression and increasing responsibility, such as Category Manager → Senior Manager → Director of Merchandising or an equivalent progression.
- Strong understanding of retail strategy, merchandising processes, category performance, inventory considerations, customer behavior, and operational decision-making.
- Ability to assess complex business scenarios and determine whether recommendations are practical and commercially sound.
- Strong written and verbal communication skills, with the ability to clearly explain professional judgment and reasoning.
- Excellent problem-solving and analytical abilities.
- High attention to detail and consistency when applying evaluation criteria.
- Comfortable collaborating with cross-functional teams and other subject-matter experts.
- Ability to commit reliably to approximately 35 hours per week during weekdays.
4. Ideal Candidate
The ideal candidate is a seasoned retail professional who understands how decisions are made in real-world retail environments and can distinguish between an answer that merely sounds reasonable and one that reflects genuine industry expertise.
Experience across multiple areas of retail, combined with exposure to AI evaluation, training data, structured grading, or similar quality-assurance processes, will be particularly valuable.