Public data.

01 / Start here

Take a copy.

Or with the tools, which also give you a query shell over it:

1git clone https://github.com/elliottdehn/open-jobs
2cd open-jobs
3uv run tools/jobs.py export

Needs uv, one line to install: curl -LsSf https://astral.sh/uv/install.sh | sh (or brew install uv; on Windows winget install astral-sh.uv).

The full export puts every open posting on your disk, with description text, available vectors, and company fields. One jobs file per applicant tracking system, plus boards/ for the company records.

The command saves to work/export/2026-09-10/ and makes jobs and boards available as DuckDB views. The files also live at exports/2026-09-10/jobs/. Each new export gets its own date; the previous day stays up for one more night.

Or click, one file at a time

Each file is every open posting on one applicant tracking system, as of 2026-09-10, parquet, no account. Company records for each are in the table below.

A few things to ask it

Each is one command. jobs and boards are views over the export you just pulled.

uv run tools/jobs.py sql "SELECT count(*) postings, count(DISTINCT slug) career_sites, count(*) FILTER (embedding IS NOT NULL) with_vectors FROM jobs"

uv run tools/jobs.py sql "SELECT title, url, published_at::date AS says_posted FROM jobs WHERE published_at < now() - INTERVAL 1 YEAR ORDER BY 3 LIMIT 20"

uv run tools/jobs.py sql "SELECT date_trunc('month', published_at)::date AS month, count(*) FROM jobs GROUP BY 1 ORDER BY 1 DESC LIMIT 12"

uv run tools/jobs.py sql "SELECT title, count(*) n FROM jobs GROUP BY 1 ORDER BY 2 DESC LIMIT 25"

uv run tools/jobs.py sql "SELECT ats, slug, count(*) open_roles FROM jobs GROUP BY 1, 2 ORDER BY 3 DESC LIMIT 25"

uv run tools/jobs.py sql "SELECT ats, round(100.0 * count(*) FILTER (regexp_matches(content, '[$][0-9]{2,3},[0-9]{3}')) / count(*), 1) pct_with_pay FROM jobs GROUP BY 1 ORDER BY 2 DESC"

uv run tools/jobs.py sql "SELECT title, url FROM jobs WHERE content ILIKE '%kubernetes%' AND title ILIKE '%engineer%' LIMIT 20"

uv run tools/jobs.py sql "WITH q AS (SELECT embedding FROM jobs WHERE title ILIKE '%data engineer%' AND embedding IS NOT NULL LIMIT 1) SELECT title, round(list_cosine_similarity(embedding, (SELECT embedding FROM q)), 3) sim FROM jobs WHERE embedding IS NOT NULL ORDER BY 2 DESC LIMIT 20"

Inside each jobs file

Embedding recipe: text-embedding-3-small:1536:v3.

ats, slug, id, title, location, url, departments[], published_at, updated_at, content, detail_status, content_hash, first_seen_at, last_seen_at, changed_at, removed_at, is_open, enrich_status, enriched_at, enrichment_json, embed_status, embed_model, embedding FLOAT[1536]

Too big? The ledger has every posting's dates and status without text or vectors in 1.4 GB. The group files have text plus vectors for open postings in pieces of a few MB.

import duckdb
base = "https://backend.dehnbostele.workers.dev/data/exports/2026-09-10/jobs/"
ats = ["ashby", "bamboohr", "breezy", "comeet", "cornerstone", "crelate", "dark", "dayforce", "eightfold", "gohire", "governmentjobs", "greenhouse", "icims", "jazzhr", "jibe", "jobscore", "jobvite", "join", "lever", "oraclecloud", "paycom", "paylocity", "personio", "phenom", "pinpoint", "recruitee", "recruiterbox", "smartrecruiters", "softgarden", "successfactors", "taleo", "teamtailor", "ukg", "usajobs", "workable", "workday"]
jobs = duckdb.read_parquet([f"{base}{a}.parquet" for a in ats])
duckdb.sql("SELECT ats, count(*) FROM jobs GROUP BY 1 ORDER BY 2 DESC")

02 / Daily changes

Keep it current.

Each night publishes one diff, diffs/<yesterday>__<today>/, with one row per event: op is added, removed, changed, changed_prev (the previous version of a changed row), or carried. Added and changed rows carry the posting's full fields. Lite parts drop the embedding and raw JSON; full parts keep them. Every diff names its parent's content hash, so a chain of diffs verifies. Apply the diff whose from equals the export you hold.

Removed rows say why: closed (the site dropped it), left_dataset, or unknown.

import duckdb, json, urllib.request
idx = json.load(urllib.request.urlopen("https://backend.dehnbostele.workers.dev/data/diffs/index.json"))
d = idx["entries"][-1]                       # the newest diff; idx["head"] is the export it produces
lite = [f"https://backend.dehnbostele.workers.dev/data/{d['lite']['dir']}{p['file']}" for p in d["lite"]["parts"]]
ev = duckdb.read_parquet(lite)
duckdb.sql("SELECT op, count(*) FROM ev GROUP BY 1")

For a database, use the change feed

The same changes as hashed pages of newline-delimited JSON: upserts and removes, replayed in order. A generation ID records your place. The protocol and reference consumer cover verification, checkpoints, and recovery.

03 / Looking back

The record over time.

Ledger: every posting ever recorded

One row per posting the crawler has ever seen, open or removed, as of that day, with no text and no vectors: ats, slug, id, title, location, url, published_at, content_hash, first_seen_at, last_seen_at, changed_at, removed_at, is_open, detail_status, embed_status. first_seen_at is the crawler's own first sighting, which the job board cannot re-stamp; removed_at is when the site stopped listing it. This is the file for posting lifetimes and survival curves. Newer days supersede older ones; read all parts of a day together. Index: ledger/index.json.

Every diff since the start

Kept indefinitely and never rewritten once listed. Index with counts, per-part sha256, and parent hashes: diffs/index.json.

04 / Similarity search

The search index.

The current snapshot, arranged for similarity search on your own machine. Download a few groups of related postings, or walk the tree with byte-range reads.

Client estimators

These files are rewritten nightly under the same names. Layout and API documentation ↗