Member of Technical Staff (Search Crawler Analyst)
Join Perplexity AI as a Search Crawler Analyst to enhance our crawling pipeline and improve search quality.
Posted by employer 1 day ago
First seen on Joblaze 1 day ago
Last verified on the company career page 1 day ago
Skills & Technologies
What you'll build
- Find and diagnose quality issues in our crawling pipeline
- Train small models that optimize particular aspects of the pipeline
- Build datasets for model training
- Improve page selection algorithms for indexing
- Design and analyze experiments to validate improvements
Must have
- 4+ years of experience as a data analyst, ML engineer, or in a related role
- Strong coding skills
- Experience designing metrics from scratch
- Experience training ML models that shipped to production
Nice to have
- Direct experience working on web crawling or indexing pipelines
Requirements
- Experience
- 4+ years
Not disclosed in this posting: compensation, work arrangement, visa sponsorship.
Joblaze summary
In this role, the individual will focus on diagnosing and resolving quality issues within Perplexity AI's complex crawling pipeline while also enhancing the overall search and answer systems. Key skills include strong coding abilities at a mid-level backend engineer standard, experience in data analysis, and a background in machine learning model training. This position is well-suited for candidates with over four years of relevant experience, particularly those familiar with web crawling or indexing processes. The team operates at the intersection of data analysis and engineering, emphasizing continuous improvement.
Joblaze insights
- Listed yesterday — first seen on Joblaze October 9, 2026. Last confirmed on Perplexity AI's careers page October 9, 2026.
Quick facts
- How much experience is required?
- At least 4 years of relevant experience for this Member of Technical Staff (Search Crawler Analyst) role.
- What's the tech stack?
- Joblaze extracted these technologies from the posting: Backend, Data Analysis, Data Engineering, ML.
- What seniority level is this role?
- Perplexity AI targets mid-level candidates for this position.
- Is this full-time or contract?
- Full-time for this Member of Technical Staff (Search Crawler Analyst) role at Perplexity AI.
From the original posting
The internet is vast, containing trillions of URLs. Perplexity’s crawling and storage system is complex and has multiple stages (URL discovery, crawling, parsing, indexing). Each stage offers many opportunities for improvement and room for intricate bugs. You’ll work at the intersection of data analysis and engineering — designing metrics, building data pipelines, and improving the quality of our search and answer systems.
Responsibilities
Find and diagnose quality issues in our crawling pipeline
Train small models that optimize particular aspects of the pipeline (e.g. parsing quality)
Build datasets for model training, including LLM-as-a-judge labeling pipelines
Improve page selection algorithms for indexing
Design and analyze experiments to validate improvements
Qualifications
4+ years of experience as a data analyst, ML engineer, or in a related role
Strong coding skills: you should be able to write production-grade code at the level of a mid-level backend engineer
Experience designing metrics from scratch
Experience training ML models that shipped to production with measurable metric improvements
Nice to have
Direct experience working on web crawling or indexing pipelines