Join Mirendil as a Member of Technical Staff to shape pretraining data for AI models in a fast-paced startup environment.
Posted by employer 2 days ago
First seen on Joblaze 1 day ago
Last verified on the company career page 1 day ago
What you'll build
Must have
Not disclosed in this posting: years of experience, work arrangement, visa sponsorship.
Benefits
Joblaze summary
In this role, the individual will focus on building and managing data pipelines for pretraining AI models, ensuring the quality and relevance of the training data. Proficiency in Python and experience with large-scale data processing frameworks like Spark or Ray are essential, along with a solid understanding of PyTorch or JAX for running training experiments. This position is suited for someone with a strong background in data engineering and a track record of implementing data changes in pretraining runs. The team is dedicated to advancing AI research by refining data strategies and methodologies.
Joblaze insights
Quick facts
From the original posting
We pretrain our own models and we are scaling to larger models now. Agents handle much of the engineering, but deciding what goes into training still depends on strong human judgment.
You will set the direction for what our models train on and use agents to run the ablations, pipelines, and analyses that test it. You will work closely with the researchers who own architecture and training runs.
Work on building data pipelines for pretraining, including repository-scale code and technical documents.
Get more out of public data through better extraction, filtering, and deduplication of web pages, open-source code, and public datasets.
Decide the data mixture for each run, and design the ablations and data scaling experiments that justify it.
Direct agents to run those experiments and pipelines in parallel, and decide which results to trust.
Extend a code-first mixture into scientific domains, starting from where our models fall short.
Keep the corpus clean as it grows: provenance, decontamination against our evals, and reproducible versions.
You have shipped a data change into a large pretraining run and can show the ablation that justified it.
You have built data pipelines at the scale of trillions of tokens with Spark, Ray, Beam, or similar, and you write strong Python.
You know PyTorch or JAX well enough to run your own training experiments.
You have hands-on experience with at least two of: deduplication, quality classifiers, mixture optimization, data scaling laws, or long-context and code data.
You can tell when an experimental result is real, and you explain your reasoning clearly in writing.
If you’re excited about shaping what our models learn from, and proving it with experiments, we’d love to hear from you.
Standard company text repeated across Mirendil's postings is omitted here.