Data Health — 2026-09
We publish the state of the job market, so we also publish the state of our own data: what it covers, where it is weak, and what we have actually verified. Last built 2026-09-14.
0.55%probed links dead (23,080 checked)
9.9 dmedian crawl freshness
7.23%postings with no publish date
0.72%duplicate URLs
Corpus
| Unique postings in this edition | 6,564,432 |
| Median listing age | 26 days |
| State a salary amount not validated | 12.4% |
| Remote-friendly | 3.1% |
| Listed more than 60 days ago listing age, not a liveness check | 34.6% |
Field coverage & date sanity
| Sample examined 400 leaf files | 217,558 |
| Publish date missing | 7.23% |
| Publish date in the future | 0.03% |
| Timestamp sentinel (1970 epoch bug) | 7.23% |
| Listed more than 2 years ago | 11.74% |
| Last-seen before publish date impossible; source date is bad | 2.43% |
| Location empty | 2.91% |
| Job description empty | 0.02% |
| Duplicate URLs | 0.72% |
| Duplicate (ats, id) | 0.97% |
| Source label vs URL host mismatch label heuristic, not a hard error | 20.67% |
Freshness (crawler cycle)
| Median days since last seen | 9.9 d |
| 95th percentile days since last seen | 18.5 d |
Ghost signal — publish date × our own crawl
Our un-copyable measurement: the source's publish date crossed with the date we last saw the posting in that source. A posting that is months old yet still re-crawled every week is the classic "evergreen" / ghost-job listing.
| Listed more than 90 days ago | 33.6% |
| Re-seen by our crawl within 7 days | 46.98% |
| Both — old AND still re-crawled | 17.07% |
| Share of 90-day-old postings still re-crawled the headline ghost figure | 50.79% |
Link liveness — measured, one by one
We probe the employer's own page for a sample of postings (HEAD, then a small ranged GET when HEAD is inconclusive), and scan the body for "no longer accepting applications". These are measurements, not estimates.
| Total probed | 23,080 |
| Alive | 97.73% |
| Dead (404/410/451 or closed notice) | 0.55% |
| Blocked by a bot wall (403/429) unknown, not counted as dead | 0.82% |
| Probe: liveness.db 10.83% dead | 240 |
| Probe: liveness_prop.db 5.33% dead | 300 |
| Probe: liveness_prop2.db 1.0% dead | 300 |
| Probe: liveness_strat2.db 5.83% dead | 240 |
| Probe: overdrive:du-liv-20260914 1.1% dead | 2,000 |
| Probe: overdrive:du-liv-20k 0.23% dead | 20,000 |
Dead links by source
| dark 1,471 probed | 0.41% |
| smartrecruiters 134 probed | 0.0% |
| workable 116 probed | 0.0% |
| dark 16,959 probed | 0.02% |
| workable 470 probed | 0.0% |
| crelate 450 probed | 0.22% |
| smartrecruiters 417 probed | 0.48% |
| paycom 250 probed | 0.0% |
| bamboohr 219 probed | 0.0% |
| lever 208 probed | 3.37% |
| breezy 188 probed | 0.0% |
| phenom 135 probed | 0.0% |
| ukg 132 probed | 0.0% |
| greenhouse 113 probed | 0.0% |
How to read this. "Still listed" is a listing-age measure from the source-provided publish date — it is not a verification that a role is open; the liveness probes are that. A "blocked" probe is unknown, not dead, and is excluded from the dead rate. Our crawl revisits the whole index roughly every 2–3 weeks, so very fresh postings may not be seen yet. Numbers are regenerated each month by scripts/data_audit.py, scripts/liveness_check.py and the overdrive full-corpus run.