This project moved to Postgres some time ago, but ~380 lines of SQL-to-AQL
translation and AQL syntax validation were still sitting in utils.py, reachable
from nothing. utils.py is now just TextChunker, which is the only part anything
imports. Also drops a stray prompt draft that had been left in backend/services/,
and the _normalize_arango_search_args name, which never touched Arango.
Worth knowing separately: the Arango server is still running and still
authenticating, so the riksdagsgruppen account is a live door into a system nothing
here uses any more. Deleting that account is simpler than rotating its password.
The env_manager account is a different one and other projects still depend on it.
.gitignore now excludes .claude/ entirely rather than just settings.local.json.
settings.json can carry credentials inside allowlisted command strings, which is
exactly how a live database password ended up committed to the predecessor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
talks -> speeches ("talks" reads as conference talks to everyone outside this
project), motions -> documents with a new doc_type column so bills, written
questions and committee reports can share the table later, and 79 columns from
anforandetext -> text, intressent_id -> person_id, valkrets -> constituency,
lydelse -> text, and so on.
_postgres/rename_map.py is the single source of truth. The migration, its rollback,
and the code rewrite are all derived from it, so they cannot drift apart. Hand-
writing a rollback is how you end up with one that fails halfway through.
Two identifiers could not be renamed mechanically and were done by reading the
queries: dok_id means the protocol document in talks but the primary key in
motions, and year stays a calendar year in speeches while becoming session_year in
documents. The latter is aliased in SQL so the JSON field stays `year` and the
frontend contract is unchanged.
Values are never translated. Bifall and Avslag stay as published; parliament.yaml
glosses them. A research tool must not silently rewrite the record.
The migration is guarded by an existence check, so the same file is a no-op on a
fresh database and does the work on an existing one — one schema definition in the
world. ALTER TABLE ... RENAME is catalog-only, so the millions of HNSW-indexed
vectors are untouched.
The trigger functions are recreated explicitly, because plpgsql bodies are stored
as opaque text and do not follow renames: they would have compiled fine and then
failed at the next INSERT. They now read the text-search config from a database
setting rather than hardcoding 'swedish'.
Verified: migration round-trips to a byte-identical schema across columns, indexes
and triggers; re-running is a no-op; triggers repopulate search_vector with working
Swedish stemming on INSERT and UPDATE; and the renamed code runs real searches
against a migrated database holding 5,000 rows of production data, with plain,
prefix and exclusion syntax all working. Frontend type errors went from 9 to 7 —
the rename fixed two and introduced none.
Not yet done, and tracked: legacy compatibility views, the shim for SQL replayed
from saved snapshots, tool-name aliases, and updating docs to the new names.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Everything that made this Sweden-only in code now lives in one configuration file:
party list and colours, chamber activity types, the Postgres text-search
dictionary, the party-code pattern used to verify attributions, identifier shapes,
source URLs, the person-photo URL template, embedding model and dimension, and the
site copy.
info.py is deleted. Its party_colors_lighten table is gone too — the tint is now
computed from the party colour, so there is no second palette to keep in sync. Its
dead select_columns and css strings go with it.
The stylesheet no longer carries per-party rules. It had --party-M through
--party-NYD plus twenty [data-party="..."] selectors, which no amount of
configuration could adapt: CSS cannot read a dict. App.tsx now publishes the
configured palette as custom properties from /api/meta, and one color-mix rule
covers every party in any country.
The author's email and Twitter handle are out of the source entirely. Explainer
copy moved to content/sv/*.md, and contact details come from `site.contact`, which
upstream ships empty for a deployment to fill in via its own PARLIAMENT_CONFIG.
Two startup assertions added, both for failures that are otherwise silent: a
text-search config mismatch returns near-zero rows with no error, and a vector
column that disagrees with embeddings.dimension fails deep inside pgvector with a
message that never mentions configuration.
Verified: five searches spanning phrase, exclusion and prefix syntax — exercising
all three tsquery builders that changed — return byte-identical payloads against
production, with differing hit counts and first hits proving the queries really
differ. Frontend builds, and tsc reports the same 9 pre-existing errors as before,
none new.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The project could only run from /home/lasse/riksdagen: 22 files hardcoded that
path, mostly as an `os.chdir(...)` + `sys.path.append(...)` prelude. Those now
derive the root from the file's own location via bootstrap.py, and bulk data
resolves against PLENUM_DATA_DIR.
requirements.txt was from March 2023 and listed Streamlit, altair and pydeck while
omitting fastapi, uvicorn, psycopg2 and pgvector. Replaced with a pyproject.toml
derived from the actual import graph, plus ruff and pytest config.
Postgres opened its connection pool in __init__, so `import backend.app` failed
outright without a reachable database — breaking test collection and any tooling
that merely imports the app. The pool is now opened on first use behind a lock.
systemd units and an nginx site move to deploy/examples/ with __PROJECT_ROOT__ and
__DOMAIN__ placeholders. Real values belong in the deployment's own deploy/prod/,
which is a path upstream never writes to, so the two cannot conflict on merge.
Removed: scripts/migrate_keys.py (an already-executed Arango migration referenced
nowhere) and a dead `import talks2db` in download_talks.py that pointed at a file
excluded from this repo.
Verified: `import backend.app` now succeeds with neither a database nor any network,
and all rewritten scripts import from a foreign working directory with their module
docstrings intact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seeded via an explicit allow-list (see /home/lasse/plenum-seed.sh) rather than by
deleting files from a copy, so nothing sensitive can survive by omission.
Excluded: WireGuard backup + client config, the plaintext DB password in admin.py,
Arango credentials in scripts/notes.md, .claude/settings.json, a 113 MB log,
providers.yaml (private endpoint), the Arango/ChromaDB-era scripts, the duplicated
claude-design-system frontend copy, and assorted screenshots and one-off planning docs.
297 tracked files / 49 MB of history -> 161 files / 2.1 MB.
Recovered 14 database migrations that the old .gitignore's `*.sql` rule had been
hiding from version control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lasse Edfast