LegalOSS82.1ktracked
Case Law & Legal Data

legalize-pipeline

legalize-dev/legalize-pipeline

Spanish legislation as a git repo

legalize-pipeline

CI Python 3.12+ License: MIT

The engine behind legalize.dev -- converts official legislation into version-controlled Markdown in Git.

Each law is a file. Each reform is a commit. Every country is a repo.

How it works

Pipeline diagram

Public repos (output)

Country Repo Source
Andorra legalize-ad BOPA
Argentina legalize-ar InfoLEG
Austria legalize-at RIS (Bundeskanzleramt)
Belgium legalize-be Justel (Belgisch Staatsblad)
Chile legalize-cl BCN (LeyChile)
Czech Republic legalize-cz e-Sbírka
Denmark legalize-dk Retsinformation
Estonia legalize-ee Riigi Teataja
Finland legalize-fi Finlex
France legalize-fr LEGI (Legifrance)
Germany legalize-de gesetze-im-internet.de
Greece legalize-gr ET (Ethniko Typografeio)
Ireland legalize-ie Irish Statute Book
Italy legalize-it Normattiva
Latvia legalize-lv likumi.lv (Latvijas Vēstnesis)
Lithuania legalize-lt TAR (data.gov.lt)
Luxembourg legalize-lu Legilux
Netherlands legalize-nl BWB (wetten.overheid.nl)
Norway legalize-no Lovdata
Poland legalize-pl Dziennik Ustaw (Sejm ELI)
Portugal legalize-pt DRE (Diário da República)
Slovakia legalize-sk Slov-Lex
Spain legalize-es BOE
Sweden legalize-se SFSR (Riksdag)
Ukraine legalize-ua Rada (data.rada.gov.ua)
Uruguay legalize-uy IMPO

Architecture

src/legalize/
  fetcher/              # Country-specific data fetching
    base.py               Abstract interfaces (LegislativeClient, NormDiscovery, TextParser, MetadataParser)
    es/                   Spain (BOE API)
      client.py             HTTP client with rate limiting, caching
      discovery.py          Norm discovery via catalog + sumarios
      parser.py             BOE XML -> Block/NormMetadata
    fr/                   France (LEGI XML dump)
      client.py             Local XML dump reader
      discovery.py          Filesystem-based discovery
      parser.py             LEGI XML -> Block/NormMetadata
    de/                   Germany (gesetze-im-internet.de)
      client.py             GIIClient: ZIP download + XML extraction
      discovery.py          TOC XML discovery (~6900 laws)
      parser.py             gii-norm XML -> Block/NormMetadata
    se/                   Sweden (SFSR / Riksdag)
      client.py             Riksdag API client
      discovery.py          SFS catalog discovery
      parser.py             Swedish XML -> Block/NormMetadata
    ad/                   Andorra (BOPA)
    ar/                   Argentina (InfoLEG)
    at/                   Austria (RIS OGD API)
    be/                   Belgium (Justel)
    cl/                   Chile (BCN / LeyChile)
    cz/                   Czech Republic (e-Sbírka)
    dk/                   Denmark (Retsinformation)
    ee/                   Estonia (Riigi Teataja)
    fi/                   Finland (Finlex AKN)
    gr/                   Greece (ET)
    ie/                   Ireland (Irish Statute Book)
    it/                   Italy (Normattiva AKN)
    lt/                   Lithuania (TAR / data.gov.lt)
    lu/                   Luxembourg (Legilux)
    lv/                   Latvia (likumi.lv HTML scraping with sitemap discovery)
    nl/                   Netherlands (BWB / wetten.overheid.nl)
    no/                   Norway (Lovdata)
    pl/                   Poland (Dziennik Ustaw / Sejm ELI)
    pt/                   Portugal (DRE SQLite dump)
    sk/                   Slovakia (Slov-Lex)
    ua/                   Ukraine (data.rada.gov.ua)
    uy/                   Uruguay (IMPO)
  transformer/          # Generic: XML -> Markdown
    xml_parser.py         Bloque/Version extraction, reform timeline
    markdown.py           Bloque -> Markdown (CSS class mapping)
    frontmatter.py        YAML frontmatter rendering
    slug.py               norm_to_filepath() -> {country_dir}/{id}.md
  committer/            # Generic: Markdown -> git commits
    git_ops.py            Git operations with historical dates
    message.py            Commit message formatting (6 types)
    author.py             Author from git config (whoever runs the pipeline)
  state/                # Pipeline state tracking
    store.py              Last processed summary, run history
  countries.py          # Country registry (lazy import dispatch)
  config.py             # Config + CountryConfig from config.yaml
  models.py             # Domain models (generic, multi-country)
  storage.py            # Save XML + JSON to data/ (intermediate cache)
  pipeline.py           # Generic orchestration (fetch, commit, bootstrap, daily, reprocess)

Prerequisites

  • Python 3.12+
  • Git

Quick start

git clone https://github.com/legalize-dev/legalize-pipeline.git
cd legalize-pipeline

pip install -e ".[dev]"

# Run tests
pytest tests/ -v

# Lint
ruff check src/ tests/

CLI

All commands use a unified --country / -c flag:

# Fetch laws to data/ (does not touch git)
legalize fetch -c es --catalog             # Spain: full BOE catalog
legalize fetch -c fr --all --legi-dir /path # France: all codes from LEGI dump
legalize fetch -c se --all                  # Sweden: all statutes from SFSR
legalize fetch BOE-A-1978-31229             # Single law by ID

# Generate git commits from local data/ (does not download)
legalize commit -c es --all
legalize commit -c fr --all

# Full pipeline: fetch + commit
legalize bootstrap                          # Spain (default)
legalize bootstrap -c fr --legi-dir /path   # France
legalize bootstrap -c se                    # Sweden

# Daily incremental update
legalize daily -c es --date 2026-03-28

# Reprocess specific norms
legalize reprocess -c es --reason "bug fix" BOE-A-1978-31229

# Pipeline status
legalize status

Adding a new country

  1. Create fetcher/{code}/ with client.py, discovery.py, parser.py
  2. Implement the 4 interfaces from fetcher/base.py:
    • LegislativeClient -- fetch raw data
    • NormDiscovery -- discover all laws in catalog
    • TextParser -- parse into Bloque objects
    • MetadataParser -- parse into NormaMetadata
  3. Register in countries.py REGISTRY
  4. Add countries: section to config.yaml

See ADDING_A_COUNTRY.md for the full walkthrough.

Countries

Country Status Source Maintainer Repo
Andorra Live BOPA legalize-ad
Argentina Live InfoLEG legalize-ar
Austria Live RIS legalize-at
Belgium Live Justel legalize-be
Chile Live BCN legalize-cl
Czech Republic Live e-Sbírka legalize-cz
Denmark Fetcher ready Retsinformation legalize-dk
Estonia Live Riigi Teataja legalize-ee
Finland Live Finlex legalize-fi
France Live Legifrance legalize-fr
Germany Live gesetze-im-internet.de legalize-de
Greece Live ET legalize-gr
Ireland Fetcher ready Irish Statute Book legalize-ie
Italy Fetcher ready Normattiva legalize-it
Latvia Live likumi.lv legalize-lv
Lithuania Live TAR legalize-lt
Luxembourg Live Legilux legalize-lu
Netherlands Live BWB legalize-nl
Norway Live Lovdata legalize-no
Poland Live Dziennik Ustaw legalize-pl
Portugal Live DRE legalize-pt
Slovakia Fetcher ready Slov-Lex legalize-sk
South Korea Planned law.go.kr @9bow
Spain Live BOE @EnriqueLop legalize-es
Sweden Live Riksdag legalize-se
Ukraine Live Rada legalize-ua
United Kingdom Live legislation.gov.uk @florinungur legalize-uk
United States Live OLRC legalize-us
Uruguay Live IMPO legalize-uy

Want to add your country? See ADDING_A_COUNTRY.md.

Contributing

We welcome contributions, especially new country parsers. See CONTRIBUTING.md and ADDING_A_COUNTRY.md. If you want to look after a country long-term, MAINTAINERS.md explains the federated model.

License

MIT