Data Annotation Specialist, Generalist

About the role

Cohere is seeking a part‑time independent contractor to improve the quality of its large language models by providing structured feedback on English‑language data across text, image, and structured formats.

Key responsibilities

• Evaluate and rank model outputs, writing clear justifications for judgments.
• Stress‑test and break models by probing for unsafe behavior and capability gaps.
• Create high‑quality prompts, responses, and exemplars for training and evaluation datasets.
• Build and apply rubrics and taxonomies to produce consistent annotations.
• Annotate and correct multimodal data, ensuring high data integrity.
• Participate in calibration exercises and inter‑annotator agreement checks.
• Adapt to new experimental task types as project needs evolve.
• Report on model performance trends and provide evidence‑based feedback.

Required profile

• 1+ years experience in AI data annotation, LLM evaluation, content moderation, research, or a related analytical role.
• Experience applying detailed guidelines to ambiguous content with strong contextual judgment.
• Comfort with ambiguity and willingness to flag unclear edge cases.
• Sharp eye for inconsistencies, model failure modes, and ability to probe models adversarially.
• Excellent written English and strong reading comprehension.
• Strong attention to detail and commitment to accuracy.
• Familiarity with annotation platforms and structured formats such as JSON, CSV/TSV, Markdown, XML, and YAML.
• Effective remote work habits, good time management, and ability to work independently.

Required skills

• JSON
• CSV/TSV
• Markdown
• XML
• YAML