Site copy now resolves through CONTENT_DIR, alongside PARLIAMENT_CONFIG and
PROMPTS_DIR. All three let a deployment keep its own wording, branding and contact
details outside the repository, so they never appear in a diff against upstream —
which is what keeps a production fork mergeable.
parliament.yaml's site.title was "rixdagen.se". That is the deployment's identity,
not Sweden's data, so upstream now ships a neutral default. The Swedish user guide
keeps its content but loses the personal contact details and site-specific wording;
a deployment ships its own guide by pointing CONTENT_DIR at it.
Fixes /api/guide, which still read ../user-guide.md after that file moved to
content/sv/ and had been returning 404.
Deliberately kept: the author metadata in pyproject.toml and the copyright line in
LICENSE, which are attribution rather than configuration, and the README's note that
this runs in production as rixdagen.se, which is provenance worth stating.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`python -m ingest.cli fetch|load|sync` now exists — README, PORTING.md, the systemd
example and pyproject's console script have all been referencing it.
This replaces six scripts that the rename had left genuinely broken: the speech
loader's INSERT listed `year, year` as two columns, and both loaders read the source
JSON using plenum's column names rather than the Riksdag's field names, so a sync
would have written empty rows without erroring. Rewriting on top of the adapter was
safer than repairing them, because the adapter makes the source-name/column-name
boundary explicit rather than leaving it to be remembered.
Two source quirks found by running it over real archives rather than reasoning about
it: a few records carry a bare string where a child element belongs, which took the
whole load down on an AttributeError; and party codes appear in both cases, so
`parties` held {S,s} and any filter on 'S' would have silently missed half the
documents. Both are handled in the adapter, where source messiness belongs.
Also adds doc_type to schema.sql. It was only in the migration, so a fresh install
lacked a column the loader writes.
Verified end to end against real archives: 400 speeches and 400 documents ingested
into a clean database, producing 3,742 authors and 2,229 proposals, with search
vectors built by the triggers and full-text queries returning hits.
Post-load steps (debate ids, embeddings, summaries) still run separately via
scripts/debates.py and scripts/make_embeddings.py; wiring them into `sync` is
tracked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The project could only run from /home/lasse/riksdagen: 22 files hardcoded that
path, mostly as an `os.chdir(...)` + `sys.path.append(...)` prelude. Those now
derive the root from the file's own location via bootstrap.py, and bulk data
resolves against PLENUM_DATA_DIR.
requirements.txt was from March 2023 and listed Streamlit, altair and pydeck while
omitting fastapi, uvicorn, psycopg2 and pgvector. Replaced with a pyproject.toml
derived from the actual import graph, plus ruff and pytest config.
Postgres opened its connection pool in __init__, so `import backend.app` failed
outright without a reachable database — breaking test collection and any tooling
that merely imports the app. The pool is now opened on first use behind a lock.
systemd units and an nginx site move to deploy/examples/ with __PROJECT_ROOT__ and
__DOMAIN__ placeholders. Real values belong in the deployment's own deploy/prod/,
which is a path upstream never writes to, so the two cannot conflict on merge.
Removed: scripts/migrate_keys.py (an already-executed Arango migration referenced
nowhere) and a dead `import talks2db` in download_talks.py that pointed at a file
excluded from this repo.
Verified: `import backend.app` now succeeds with neither a database nor any network,
and all rewritten scripts import from a foreign working directory with their module
docstrings intact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seeded via an explicit allow-list (see /home/lasse/plenum-seed.sh) rather than by
deleting files from a copy, so nothing sensitive can survive by omission.
Excluded: WireGuard backup + client config, the plaintext DB password in admin.py,
Arango credentials in scripts/notes.md, .claude/settings.json, a 113 MB log,
providers.yaml (private endpoint), the Arango/ChromaDB-era scripts, the duplicated
claude-design-system frontend copy, and assorted screenshots and one-off planning docs.
297 tracked files / 49 MB of history -> 161 files / 2.1 MB.
Recovered 14 database migrations that the old .gitignore's `*.sql` rule had been
hiding from version control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lasse Edfast