Human Data Strategist
Own the data acquisition pipelines to enhance AI models at Reflection AI, a startup focused on open intelligence.
Posted by employer 1 day ago
First seen on Joblaze 1 day ago
Last verified on the company career page 1 day ago
What you'll build
- Translate model capabilities into data acquisition strategies
- Assess vendors’ capabilities and define task instructions
- Build dashboards tracking data acquisition metrics
- Surface vulnerabilities and implement feedback
- Collaborate with researchers to improve model capabilities
Must have
- 4+ years of experience delivering complex programs
- Understanding of LLM training and evaluation
- Proficiency in SQL and Python
- Clear written communication skills
- Ability to drive impact in dynamic environments
Nice to have
- Experience with annotation or expert data collection
- Experience with human feedback or AI evaluation
Requirements
- Experience
- 4+ years
- Visa
- Sponsorship available
Not disclosed in this posting: compensation, work arrangement.
Benefits
Joblaze summary
The Human Data Strategist at Reflection AI plays a crucial role in transforming human expertise into actionable data acquisition strategies that enhance AI model performance. This position requires proficiency in SQL and Python, alongside a solid understanding of large language model training and evaluation. Ideal candidates will have over four years of experience managing complex programs and a knack for navigating dynamic research environments. Reflection AI emphasizes a collaborative approach, working closely with researchers to ensure data sourcing aligns with evolving model capabilities.
Joblaze insights
Quick facts
- How much experience is required?
- At least 4 years of relevant experience for this Human Data Strategist role.
- What's the tech stack?
- Joblaze extracted these technologies from the posting: Python, SQL.
- Does Reflection AI sponsor work visas for this role?
- Yes — the posting indicates visa sponsorship is available for the right candidate.
- What seniority level is this role?
- Reflection AI targets mid-level candidates for this position.
- Is this full-time or contract?
- Full-time for this Human Data Strategist role at Reflection AI.
From the original posting
Role Overview
The Data Acquisition Team is responsible for ensuring that Reflection has all the data it needs to train the world’s best models. We're looking for a Human Data Strategist to own the pipelines that turn human expertise into better AI models. You will partner closely with researchers, deeply understand their needs, and source from vendors the data that improves our models’ usefulness and performance.
This role combines program ownership with hands-on work on task design, contributor workflows, data quality frameworks, and success metrics. The ideal candidate brings organizational rigor, technical curiosity, and sound judgment about when to move quickly and when to investigate the details.
What You'll Do
Collaborate with research: Translate critical model capabilities, and researchers’ plans for developing them, into executable data acquisition strategies.
Orchestrate providers: Assess vendors’ capabilities, forecast capacity, define task instructions, acceptance criteria and then hold vendors accountable for quality, delivery, and cost.
Track and optimize campaigns: Build dashboards tracking throughput, turnaround time, acceptance rates, rework, and cost per accepted task. Leverage data analytics to investigate bottlenecks and guide improvements.
Ensure alignment: Experiments and priorities evolve. You will surface vulnerabilities early and turn feedback into concrete changes to subsequent sourcing. Work with research to close the loop; tracking how our data acquisitions improve model capabilities
What We're Looking For
4+ years of experience delivering a portfolio of complex programs and prioritizing across changing research needs.
An understanding of LLM training, evaluation, and prompt design, with the ability to learn unfamiliar research concepts quickly.
Proficiency in SQL for querying datasets and building dashboards, and Python for data analysis, API integrations, and workflow automation.
Clear written communication, strong stakeholder judgment, and a willingness to inspect individual tasks and datasets when the headline numbers do not tell the full story.
High agency, bias towards action, and the ability to drive impact in fast-paced, dynamic, and ambiguous environments
Experience with annotation, expert data collection, human feedback, or AI evaluation is a plus.
Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.
Standard company text repeated across Reflection AI's postings is omitted here.
Similar positions
Explore more