How We Track AI Hiring Trends at HireIndex

Most job boards are downstream of recruiters. Someone posts a role, it gets distributed, and by the time you see it on LinkedIn or Indeed, thousands of other people have already seen it too. The signal is noisy, the data is shallow, and half the listings are stale.

HireIndex takes a different approach. We go upstream — directly to the applicant tracking systems (ATS) that companies use to publish roles in the first place. Greenhouse, Lever, Workday, and a handful of others host the canonical job posting for most AI-forward companies. If a company is hiring an ML engineer, the listing almost certainly lives on one of those platforms before it gets syndicated anywhere else.

This post is a walkthrough of how we turn that raw data into the clean, city-by-skill index you see on the front page.

The weekly pipeline

Every Monday morning, we run through the following sequence:

1. Scrape. We pull public career pages from a curated list of ~300 companies known to hire for AI/ML roles. These are companies that either publicly disclose AI teams (via published research, earnings calls, or their own blogs) or that we’ve identified through inbound signals like LinkedIn team-growth trends. For each company, we collect every currently open role — not just the ones tagged “AI.”
2. Classify. Job titles are surprisingly inconsistent. “Senior ML Engineer” at one company is “Staff Applied Scientist” at another is “Member of Technical Staff — AI Platform” at a third. We run each title through a keyword-matching pass that maps it to one of thirteen canonical skill categories: Machine Learning Engineer, Data Scientist, MLOps Engineer, AI Research, LLM Engineer, and so on. Roles that don’t fit any category get classified as generic “AI Software Engineer.”
3. Normalize locations. Location strings are even messier than titles. A single role might be listed as “New York, NY; Remote (US); San Francisco, CA — hybrid.” We parse these strings against a regex-based alias map and assign each role to one or more canonical cities. Remote-only roles fall into a dedicated “Remote” bucket.
4. Index by (skill, city) pair. Once every role has canonical skill and city tags, we generate an index: for every combination of skill and city that has at least one matching role, we create a landing page. This is the programmatic backbone of the site — roughly a hundred pages, each listing only the roles that actually match the criteria.
5. Publish. The whole pipeline outputs a single JSON file that the static site generator consumes at build time. Every Monday, a new build ships with fresh data.

The entire process takes about twelve minutes from first scrape to deploy. Zero manual intervention.

What the data reveals

A few patterns have surfaced since we started tracking:

The “AI engineer” title is eating the landscape. Three years ago, most AI-focused roles were titled “ML Engineer” or “Data Scientist.” Today, nearly 40% of the roles we index use the generic “AI Engineer” title — often without meaningful distinction from what used to be called ML Engineer. The title is broader, more marketable, and increasingly disconnected from the underlying technical work.

Remote AI hiring is back. After a post-pandemic pullback in 2023 and 2024, remote-first AI roles have quietly climbed again. About 28% of current listings in our index are remote-eligible, up from ~18% two years ago. This is partly driven by a handful of aggressively-remote companies (Anthropic, Scale AI, Hugging Face), and partly by the simple reality that AI talent is scarce and geographically diffuse.

The “AI research” category has bifurcated. Research scientist roles — the kind that require a PhD and a publication record — are highly concentrated in a shrinking set of well-funded labs. Meanwhile, “research engineer” titles are proliferating at series-B and series-C startups that want research credibility but can’t compete on compensation for traditional research scientists. If you have strong engineering skills and an interest in research, the research-engineer path is dramatically less competitive than the research-scientist path.

Skill-specific demand is surprisingly localized. We expected to see uniform AI hiring across major tech hubs. Instead, the data shows clear specialization. Tokyo is heavily weighted toward robotics and applied AI. London has a disproportionate share of LLM-engineering roles tied to a handful of UK-based foundation-model labs. Singapore is becoming a regional hub for MLOps and AI infrastructure work tied to financial services.

What we don’t do

A few deliberate choices about what this site isn’t:

We don’t syndicate listings. Every role on HireIndex links directly back to the company’s own career page. You apply there. We don’t run an applicant tracking layer, don’t collect resumes, and don’t charge companies to post.

We don’t track compensation. Salary data is either self-reported (unreliable) or scraped from disclosure-mandated jurisdictions (biased toward a few US states). We’d rather show you no data than misleading data.

We don’t score candidates. The scoring we do is at the company level — how intensely a given company is investing in AI hiring, measured by headcount growth, role diversity, and team-page signals. That’s a signal for job seekers trying to decide where to focus, not a tool for companies to filter applicants.

What’s next

Coverage expansion is the main focus for the next couple of months. We’re targeting 500 companies by end of Q2, with better coverage of European and APAC markets (the current index skews heavily US). We’re also building a dashboard that shows hiring velocity over time — which companies are ramping up, which are slowing down, and how that correlates with public signals like earnings calls and funding announcements.

If you’re job-hunting in AI or ML right now, bookmark the skill-city page that matches what you’re looking for. Every Monday, it’ll have fresh roles.