recovery.sh migrated data out of ArangoDB. This project abandoned Arango before the
extraction, and no one cloning plenum has an Arango server to migrate from — it was
carried over by inertia.
scripts/deploy_debate_embeddings.sh and deploy_summary_embeddings.sh were one-off
rollout scripts for features whose columns are now part of the base schema, so a fresh
install already has them.
scripts/correct_arguments.py was a single corrective pass over a specific extraction
error made by a specific small model on Swedish speeches from 2002 onward. Not
reusable, and not something a new deployment should run.
_postgres/rename_map.py and scripts/generate_rename_migration.py produced the
Swedish-to-English migration. That migration is written, applied, and its rollback
verified; the generator has no remaining use and the map described a schema that no
longer exists.
Root __init__.py was empty and made the repository look like an importable package,
which it is not.
scripts/test_auth_e2e.py moved to tests/ — it is a test, and putting it in scripts/
implied it was an operational tool.
Kept deliberately: the eval harness and its analysis scripts, which are a documented
feature rather than leftovers, and the embed/summarise/debate pipeline scripts, which
any deployment needs after an ingest run.
talks -> speeches ("talks" reads as conference talks to everyone outside this
project), motions -> documents with a new doc_type column so bills, written
questions and committee reports can share the table later, and 79 columns from
anforandetext -> text, intressent_id -> person_id, valkrets -> constituency,
lydelse -> text, and so on.
_postgres/rename_map.py is the single source of truth. The migration, its rollback,
and the code rewrite are all derived from it, so they cannot drift apart. Hand-
writing a rollback is how you end up with one that fails halfway through.
Two identifiers could not be renamed mechanically and were done by reading the
queries: dok_id means the protocol document in talks but the primary key in
motions, and year stays a calendar year in speeches while becoming session_year in
documents. The latter is aliased in SQL so the JSON field stays `year` and the
frontend contract is unchanged.
Values are never translated. Bifall and Avslag stay as published; parliament.yaml
glosses them. A research tool must not silently rewrite the record.
The migration is guarded by an existence check, so the same file is a no-op on a
fresh database and does the work on an existing one — one schema definition in the
world. ALTER TABLE ... RENAME is catalog-only, so the millions of HNSW-indexed
vectors are untouched.
The trigger functions are recreated explicitly, because plpgsql bodies are stored
as opaque text and do not follow renames: they would have compiled fine and then
failed at the next INSERT. They now read the text-search config from a database
setting rather than hardcoding 'swedish'.
Verified: migration round-trips to a byte-identical schema across columns, indexes
and triggers; re-running is a no-op; triggers repopulate search_vector with working
Swedish stemming on INSERT and UPDATE; and the renamed code runs real searches
against a migrated database holding 5,000 rows of production data, with plain,
prefix and exclusion syntax all working. Frontend type errors went from 9 to 7 —
the rename fixed two and introduced none.
Not yet done, and tracked: legacy compatibility views, the shim for SQL replayed
from saved snapshots, tool-name aliases, and updating docs to the new names.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seeded via an explicit allow-list (see /home/lasse/plenum-seed.sh) rather than by
deleting files from a copy, so nothing sensitive can survive by omission.
Excluded: WireGuard backup + client config, the plaintext DB password in admin.py,
Arango credentials in scripts/notes.md, .claude/settings.json, a 113 MB log,
providers.yaml (private endpoint), the Arango/ChromaDB-era scripts, the duplicated
claude-design-system frontend copy, and assorted screenshots and one-off planning docs.
297 tracked files / 49 MB of history -> 161 files / 2.1 MB.
Recovered 14 database migrations that the old .gitignore's `*.sql` rule had been
hiding from version control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lasse Edfast