legalize-pipeline

The engine behind legalize.dev -- converts official legislation into version-controlled Markdown in Git.

Each law is a file. Each reform is a commit. Every country is a repo.

How it works

Architecture

src/legalize/
  fetcher/              # Country-specific data fetching
    base.py               Abstract interfaces (LegislativeClient, NormDiscovery, TextParser, MetadataParser)
    es/                   Spain (BOE API)
      client.py             HTTP client with rate limiting, caching
      discovery.py          Norm discovery via catalog + sumarios
      parser.py             BOE XML -> Block/NormMetadata
    fr/                   France (LEGI XML dump)
      client.py             Local XML dump reader
      discovery.py          Filesystem-based discovery
      parser.py             LEGI XML -> Block/NormMetadata
    de/                   Germany (gesetze-im-internet.de)
      client.py             GIIClient: ZIP download + XML extraction
      discovery.py          TOC XML discovery (~6900 laws)
      parser.py             gii-norm XML -> Block/NormMetadata
    se/                   Sweden (SFSR / Riksdag)
      client.py             Riksdag API client
      discovery.py          SFS catalog discovery
      parser.py             Swedish XML -> Block/NormMetadata
    ad/                   Andorra (BOPA)
    ar/                   Argentina (InfoLEG)
    at/                   Austria (RIS OGD API)
    be/                   Belgium (Justel)
    cl/                   Chile (BCN / LeyChile)
    cz/                   Czech Republic (e-Sbírka)
    dk/                   Denmark (Retsinformation)
    ee/                   Estonia (Riigi Teataja)
    fi/                   Finland (Finlex AKN)
    gr/                   Greece (ET)
    ie/                   Ireland (Irish Statute Book)
    it/                   Italy (Normattiva AKN)
    lt/                   Lithuania (TAR / data.gov.lt)
    lu/                   Luxembourg (Legilux)
    lv/                   Latvia (likumi.lv HTML scraping with sitemap discovery)
    nl/                   Netherlands (BWB / wetten.overheid.nl)
    no/                   Norway (Lovdata)
    pl/                   Poland (Dziennik Ustaw / Sejm ELI)
    pt/                   Portugal (DRE SQLite dump)
    sk/                   Slovakia (Slov-Lex)
    ua/                   Ukraine (data.rada.gov.ua)
    uy/                   Uruguay (IMPO)
  transformer/          # Generic: XML -> Markdown
    xml_parser.py         Bloque/Version extraction, reform timeline
    markdown.py           Bloque -> Markdown (CSS class mapping)
    frontmatter.py        YAML frontmatter rendering
    slug.py               norm_to_filepath() -> {country_dir}/{id}.md
  committer/            # Generic: Markdown -> git commits
    git_ops.py            Git operations with historical dates
    message.py            Commit message formatting (6 types)
    author.py             Author from git config (whoever runs the pipeline)
  state/                # Pipeline state tracking
    store.py              Last processed summary, run history
  countries.py          # Country registry (lazy import dispatch)
  config.py             # Config + CountryConfig from config.yaml
  models.py             # Domain models (generic, multi-country)
  storage.py            # Save XML + JSON to data/ (intermediate cache)
  layout.py             # Directory layout per country (Format Spec v0.4)
  pipeline.py           # Generic orchestration (fetch, commit, bootstrap, daily, reprocess)

See ARCHITECTURE.md for the module-by-module reference.

Prerequisites

Python 3.12+
Git

Quick start

git clone https://github.com/legalize-dev/legalize-pipeline.git
cd legalize-pipeline

pip install -e ".[dev]"

# Run tests
pytest tests/ -v

# Lint
ruff check src/ tests/

CLI

All commands use a unified --country / -c flag:

# Fetch laws to data/ (does not touch git)
legalize fetch -c es --catalog             # Spain: full BOE catalog
legalize fetch -c fr --all --legi-dir /path # France: all codes from LEGI dump
legalize fetch -c se --all                  # Sweden: all statutes from SFSR
legalize fetch BOE-A-1978-31229             # Single law by ID

# Generate git commits from local data/ (does not download)
legalize commit -c es --all
legalize commit -c fr --all

# Full pipeline: fetch + commit
legalize bootstrap                          # Spain (default)
legalize bootstrap -c fr --legi-dir /path   # France
legalize bootstrap -c se                    # Sweden

# Daily incremental update
legalize daily -c es --date 2026-03-28

# Reprocess specific norms
legalize reprocess -c es --reason "bug fix" BOE-A-1978-31229

# Pipeline status
legalize status

Adding a new country

Create fetcher/{code}/ with client.py, discovery.py, parser.py
Implement the 4 interfaces from fetcher/base.py:
- LegislativeClient -- fetch raw data
- NormDiscovery -- discover all laws in catalog
- TextParser -- parse into Bloque objects
- MetadataParser -- parse into NormaMetadata
Register in countries.py REGISTRY
Add countries: section to config.yaml

See adding-a-country/ for the full walkthrough.

Countries

Source of truth for this table: REGISTRY in src/legalize/countries.py (which countries have a fetcher) and the daily-update.yml / monthly-update-*.yml workflow matrices (which of those run on a schedule). Status is derived from the latter, not hand-maintained — see OPERATIONS.md for what each value means operationally and why a handful of countries are unscheduled on purpose.

Review is the other half, and it is the honest one: whether the country has been through a deep review against the current adding-a-country/ playbook. It matters because every fetcher in this repo was written between 2026-03-30 and 2026-04-23, and the playbook that gates a country today was written in late August 2026 — the Step 7 five-dimension quality gate, TEXT_STATE, the declared LAYOUT (spec v0.4), legalize push and the re-emission procedure all landed after the last fetcher. No country was onboarded under it. So a review here is not a re-read: it is the first time that gate is applied.

Value	Means
Reviewed	Passed the current playbook's gate, and whatever it found has been fixed in the published corpus.
Pending re-issue	Reviewed, and the review proved the published corpus is wrong. Per the commit-integrity rule these defects cannot be patched forward — the corpus has to be re-emitted. The linked issue carries the list.
—	Not reviewed yet. Says nothing about whether the country publishes: an unreviewed country can be perfectly current, and a reviewed one can still be waiting on a re-issue.

Unlike Status, this column cannot be derived — it is a human fact and is kept by hand. It changes a handful of times per country in the project's whole life, and each cell carries the date or the issue that backs it, so a bare ✅ is never enough.

Country	Status	Review	Source	Repo	Maintainer
Andorra	Daily	—	BOPA	legalize-ad	—
Argentina	Monthly	—	InfoLEG	legalize-ar	—
Austria	Daily	—	RIS	legalize-at	—
Belgium	Daily	—	Justel	legalize-be	—
Chile	Unscheduled	—	BCN	legalize-cl	—
Colombia	Monthly	—	SUIN-Juriscol	legalize-co	—
Czech Republic	Daily	—	e-Sbírka	legalize-cz	—
Denmark	Unscheduled	—	Retsinformation	legalize-dk	—
Estonia	Daily	—	Riigi Teataja	legalize-ee	—
European Union	Daily	—	EUR-Lex	legalize-eu	—
Finland	Daily	—	Finlex	legalize-fi	—
France	Unscheduled	—	Legifrance	legalize-fr	—
Germany	Daily	—	gesetze-im-internet.de	legalize-de	—
Greece	Daily	—	ET	legalize-gr	—
Ireland	Unscheduled	—	Irish Statute Book	legalize-ie	—
Italy	Daily	—	Normattiva	legalize-it	—
Latvia	Daily	—	likumi.lv	legalize-lv	—
Liechtenstein	Unscheduled	—	Lilex	legalize-li	—
Lithuania	Daily	—	TAR	legalize-lt	—
Luxembourg	Daily	—	Legilux	legalize-lu	—
Netherlands	Daily	—	BWB	legalize-nl	—
Norway	Unscheduled	—	Lovdata	legalize-no	—
Poland	Daily	—	Dziennik Ustaw	legalize-pl	—
Portugal	Daily	Reviewed 2026-08-26	DRE	legalize-pt	—
Romania	Unscheduled	—	Portalul Legislativ	legalize-ro	—
Slovakia	Unscheduled	—	Slov-Lex	legalize-sk	—
Spain	Daily	Pending re-issue (#106)	BOE	legalize-es	@EnriqueLop
Sweden	Daily	—	Riksdag	legalize-se	—
Switzerland	Monthly	—	Fedlex	legalize-ch	—
Ukraine	Unscheduled	—	Rada	legalize-ua	—
United Kingdom	Daily	—	legislation.gov.uk	legalize-uk	@florinungur
United States	Unscheduled	—	OLRC	legalize-us	—
Uruguay	Unscheduled	—	IMPO	legalize-uy	—

South Korea (legalize-kr, maintained by @9bow) is live but, unlike every country above, is not built by this pipeline — it has no entry in REGISTRY and isn't part of this table for that reason. It's the deliberate exception, not a second pattern to follow: a new country goes into the pipeline as fetcher/{cc}/ unless the source genuinely can't be brought in that way.

Want to add your country? See adding-a-country/.

Contributing

We welcome contributions, especially new country parsers. See CONTRIBUTING.md and adding-a-country/. If you want to look after a country long-term, MAINTAINERS.md explains the federated model.

Operations

Keeping the published repos current — the publication calendar, what to do when a scheduled run fails, and the slow-source mitigation — is documented in OPERATIONS.md.