Umsetzungspläne: Kundenportal, Ticketsystem, Lead-Automatik
Plan 1 Kundenportal (Gitea-Grundlage, Umzug, Nachtragen, Livegang), Plan 2 Ticketsystem (Dialog, Knöpfe), Plan 3 Lead-Automatik (neue Lead-Repos in WEBklar über das System-Konto). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ClaudeandClaude Opus 5.5 committed
1 parent
3edb5f8bb2
commit
a3faa1aff1
3 files changed
+1804
No files matched your search
@@ -0,0 +1,343 @@
|
||||
# Plan 3 – Lead-Automatik: neue Lead-Repos in WEBklar, Zugang über das System-Konto
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Ziel:** Die Lead-Automatik legt neue Lead-Previews in der Organisation WEBklar an und arbeitet mit dem Schlüssel `lead-automatik` des Kontos `webklar-system`. Bestehende Lead-Repos bei knso werden weiter unter ihrem bisherigen Pfad bedient.
|
||||
|
||||
**Aufbau:**
|
||||
|
||||
- **Neues Modul `repo_paths.py`:** reine Funktionen, die entscheiden:
|
||||
- wem ein Lead-Repo gehört,
|
||||
- wie Pfad, Web-Adresse und Git-Adresse lauten,
|
||||
- über welche Gitea-Schnittstelle ein Repo angelegt wird,
|
||||
- ob es einen Einzel-Webhook braucht.
|
||||
|
||||
Es wird mit pytest geprüft.
|
||||
- **Bestehende Dateien:** `worker.py`, `processor.py`, `enrich.py`, `backfill_leads.py` und `migrate_previews.py` nutzen diese Funktionen statt `GITEA_USER/<repo_name>`.
|
||||
- **Unveränderter Start:** Ohne neue Einstellungen verhält sich alles wie heute. Der Code geht deshalb zuerst live, die Umstellung folgt danach über `config.env`.
|
||||
|
||||
**Technik:**
|
||||
|
||||
- Python 3 (System), pytest 6.2.5
|
||||
- systemd-Dienste `webklar-automation-hook` und `webklar-automation-processor`
|
||||
- Cron-Lauf `worker.py` um 06:30 Uhr (Europe/Berlin)
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-09-27-gitea-organisation-design.md` im Kundenportal-Repo. Maßgeblich sind die Abschnitte 8, 9, 11 und 12.
|
||||
|
||||
**Voraussetzung:** Plan 1 ist live (dort L5). Erst dann erzeugen Pushes des System-Kontos keine Arbeitsblätter mehr. Mit dem alten Portal-Code bekämen Lead-Tickets sonst Arbeitsblätter auf „webklar-system“.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
**Werte, die wörtlich gelten**
|
||||
|
||||
- **Neue Schlüssel in `config.env`:**
|
||||
- `GITEA_AUTH_USER=webklar-system`. Das ist der Benutzer in Git-Adressen.
|
||||
- `GITEA_NEW_REPO_OWNER=WEBklar`
|
||||
- **Unveränderte Schlüssel:** `GITEA_USER=knso` bleibt, als Besitzer der bestehenden Lead-Repos. `TEMPLATE_OWNER` und `TEMPLATE_REPO` bleiben ebenfalls.
|
||||
- **`GITEA_TOKEN`** wird der Schlüssel `lead-automatik` des Kontos `webklar-system`, mit den Rechten `write:repository,write:organization,read:user`.
|
||||
- **Voreinstellungen:**
|
||||
- Fehlt `GITEA_AUTH_USER`, gilt `GITEA_USER`.
|
||||
- Fehlt `GITEA_NEW_REPO_OWNER`, gilt `GITEA_USER`.
|
||||
- Ohne die neuen Schlüssel ist das Verhalten also genau wie heute.
|
||||
- **Besitzer eines Lead-Repos** (`lead_repo_owner`), in dieser Reihenfolge:
|
||||
1. `lead["repo_owner"]`, falls gesetzt,
|
||||
2. sonst der Besitzer aus `lead["repo_url"]` (`https://host/<besitzer>/<repo>`),
|
||||
3. sonst `GITEA_NEW_REPO_OWNER`.
|
||||
|
||||
Das präzisiert Spec 8 („bestehende Leads gelten als knso“): Alle bestehenden Leads mit Repo tragen `repo_url` auf knso, und nur Leads ohne Repo kommen neu nach WEBklar.
|
||||
- **Einzel-Webhook:** Er wird nur dann **nicht** angelegt, wenn der Besitzer `GITEA_NEW_REPO_OWNER` ist und dieser von `GITEA_USER` abweicht. Das ist der Fall der Organisation, in dem ihr eigener Webhook greift.
|
||||
|
||||
**Sicherheit und Betrieb**
|
||||
|
||||
- Schlüssel stehen nie in Logs, Tests oder Ausgaben.
|
||||
- `config.env` wird nie committet.
|
||||
- Der Verarbeitungsdienst arbeitet alle 5 Minuten. Umgestellt wird darum nur, während er gestoppt ist (siehe Livegang).
|
||||
- Nach jeder Code-Änderung werden beide Dienste neu gestartet, weil sie die Module beim Start laden.
|
||||
|
||||
**Git**
|
||||
|
||||
- `/opt/webklar-automation` ist bisher kein Git-Repo. Task 1 legt ein **lokales** Repo ohne Remote an, nur für Nachvollziehbarkeit und Review. `.gitignore` schützt Geheimnisse und Daten.
|
||||
- Immer `git add <Dateien>`, nie `-A`.
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. **Alter Lead mit `repo_url` auf knso, ohne `repo_owner`.** Weitere Pushes, etwa aus der Anreicherung, gehen weiter nach `knso/<repo>`. Test: Task 2, „alter Lead bleibt bei knso“.
|
||||
2. **Ein neuer Lead wird erneut verarbeitet, und sein Repo existiert schon in WEBklar (409).** Das Repo wird weiterverwendet, ohne Einzel-Webhook. Tests: Task 2 (`needs_repo_webhook`) und Task 3 (Prüfschritt).
|
||||
3. **Git-Adresse.** Benutzer ist `GITEA_AUTH_USER`, der Pfad kommt aber vom Besitzer des Leads, nicht vom Benutzer. Test: Task 2, `git_url`.
|
||||
4. **Ohne die neuen Einstellungen** bleiben Pfade, Anlage über `/user/repos` und Einzel-Webhook genau wie heute. Test: Task 2, „ohne neue Einstellungen wie heute“.
|
||||
5. **Die Vorlage liegt bei knso** (`TEMPLATE_OWNER`) und wird mit dem System-Konto geklont. Geprüft wird das im Livegang, Probelauf L3.
|
||||
|
||||
---
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Aufgabe |
|
||||
|---|---|
|
||||
| `.gitignore` (neu) | schließt `config.env*`, Daten, Logs und Sicherungen aus |
|
||||
| `repo_paths.py` (neu) | `lead_repo_owner`, `repo_full`, `web_url`, `git_url`, `repo_create_path`, `needs_repo_webhook` |
|
||||
| `tests/test_repo_paths.py` (neu) | Tests |
|
||||
| `worker.py` | Einstellungen, `ensure_repo(repo_name, firma, owner)`, Git- und Web-Adressen, Lookups, `_mark_processed(lead_id, repo_url, repo_owner)` |
|
||||
| `processor.py`, `enrich.py` | Adressen mit Besitzer |
|
||||
| `backfill_leads.py`, `migrate_previews.py` | Pfade über `W.lead_repo_full(lead)` |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Lokales Git-Repo und Ausgangsstand
|
||||
|
||||
- [ ] **Step 1: Write `.gitignore`**
|
||||
|
||||
```
|
||||
config.env*
|
||||
*.bak*
|
||||
logs/
|
||||
queue/
|
||||
status/
|
||||
tmp/
|
||||
backups/
|
||||
master*.json
|
||||
master*.xlsx
|
||||
standorte.*
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create the repo and commit the baseline**
|
||||
|
||||
```bash
|
||||
cd /opt/webklar-automation && git init -q
|
||||
git add .gitignore *.py DOKUMENTATION.md ROUTINE_ANREICHERUNG_PROMPT.txt fallback-assets
|
||||
git -c user.name=Claude -c user.email=claude@webklar.local commit -qm "Ausgangsstand Lead-Automatik"
|
||||
git ls-files | grep -c "config.env\|master\|\.bak"
|
||||
```
|
||||
|
||||
Expected: `0`. `git status --short` shows no `.py` file as untracked.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Pfad-Regeln als reine Funktionen
|
||||
|
||||
**Files:**
|
||||
- Create: `repo_paths.py`, `tests/test_repo_paths.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `lead_repo_owner(lead: dict, legacy_owner: str, new_owner: str) -> str`
|
||||
- `repo_full(owner: str, repo_name: str) -> str`
|
||||
- `web_url(base_url: str, owner: str, repo_name: str) -> str`
|
||||
- `git_url(base_url: str, auth_user: str, token: str, owner: str, repo_name: str) -> str`
|
||||
- `repo_create_path(owner: str, auth_user: str, legacy_owner: str) -> str | None`. The cases, checked in this order and ignoring case:
|
||||
1. `owner == auth_user` → `'/user/repos'`,
|
||||
2. `owner == legacy_owner` → `None`, meaning the repo must already exist,
|
||||
3. otherwise → `'/orgs/<owner>/repos'`.
|
||||
- `needs_repo_webhook(owner: str, new_owner: str, legacy_owner: str) -> bool`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
B = "https://git.webklar.com"
|
||||
def test_alter_lead_bleibt_bei_knso():
|
||||
lead = {"repo_name": "preview-x", "repo_url": B + "/knso/preview-x"}
|
||||
assert lead_repo_owner(lead, "knso", "WEBklar") == "knso"
|
||||
def test_repo_owner_hat_vorrang():
|
||||
assert lead_repo_owner({"repo_owner": "WEBklar", "repo_url": B + "/knso/x"}, "knso", "WEBklar") == "WEBklar"
|
||||
def test_neuer_lead_kommt_nach_webklar():
|
||||
assert lead_repo_owner({"repo_name": "preview-neu", "repo_url": ""}, "knso", "WEBklar") == "WEBklar"
|
||||
def test_ohne_neue_einstellungen_wie_heute():
|
||||
assert lead_repo_owner({"repo_name": "preview-neu"}, "knso", "knso") == "knso"
|
||||
assert repo_create_path("knso", "knso", "knso") == "/user/repos"
|
||||
assert needs_repo_webhook("knso", "knso", "knso") is True
|
||||
def test_adressen():
|
||||
assert repo_full("WEBklar", "preview-x") == "WEBklar/preview-x"
|
||||
assert web_url(B, "WEBklar", "preview-x") == B + "/WEBklar/preview-x"
|
||||
assert git_url(B, "webklar-system", "TOKEN", "knso", "preview-x") == "https://webklar-system:TOKEN@git.webklar.com/knso/preview-x.git"
|
||||
def test_anlegen():
|
||||
assert repo_create_path("WEBklar", "webklar-system", "knso") == "/orgs/WEBklar/repos"
|
||||
assert repo_create_path("knso", "webklar-system", "knso") is None
|
||||
assert repo_create_path("Webklar-System", "webklar-system", "knso") == "/user/repos"
|
||||
def test_einzel_webhook():
|
||||
assert needs_repo_webhook("WEBklar", "WEBklar", "knso") is False
|
||||
assert needs_repo_webhook("webklar", "WEBklar", "knso") is False
|
||||
assert needs_repo_webhook("knso", "WEBklar", "knso") is True
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails**
|
||||
|
||||
Run: `cd /opt/webklar-automation && python3 -m pytest -q tests`
|
||||
Expected: FAIL (`ModuleNotFoundError: repo_paths`)
|
||||
|
||||
- [ ] **Step 3: Implement `repo_paths.py`**
|
||||
|
||||
- It uses only the standard library and loads no configuration.
|
||||
- `git_url` builds the host from `base_url` without the scheme.
|
||||
- The owner is extracted from `repo_url` with `^https?://[^/]+/([^/]+)/[^/]+/?$`.
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes**
|
||||
|
||||
Run: `python3 -m pytest -q tests`
|
||||
Expected: `7 passed`
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add repo_paths.py tests/test_repo_paths.py && git -c user.name=Claude -c user.email=claude@webklar.local commit -qm "Pfad-Regeln für Lead-Repos mit Tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: worker.py umstellen
|
||||
|
||||
**Files:**
|
||||
- Modify: `worker.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `repo_paths` (Task 2)
|
||||
- Produces, for the other files:
|
||||
- `W.GITEA_AUTH_USER`
|
||||
- `W.NEW_REPO_OWNER`
|
||||
- `W.lead_repo_owner(lead) -> str`
|
||||
- `W.lead_repo_full(lead) -> str`
|
||||
- `W.lead_git_url(owner, repo_name) -> str`
|
||||
- `W.lead_web_url(owner, repo_name) -> str`
|
||||
- `W.ensure_repo(repo_name, firma, owner)`
|
||||
- `W._mark_processed(lead_id, repo_url, repo_owner="")`
|
||||
|
||||
- [ ] **Step 1: Implement**
|
||||
|
||||
1. **Settings:** `GITEA_AUTH_USER = cfg.get("GITEA_AUTH_USER") or GITEA_USER` and `NEW_REPO_OWNER = cfg.get("GITEA_NEW_REPO_OWNER") or GITEA_USER`. Add the four `lead_*` wrappers, which fill the `repo_paths` functions with these values.
|
||||
2. **`ensure_repo(repo_name, firma, owner)`:**
|
||||
- `path = repo_create_path(owner, GITEA_AUTH_USER, GITEA_USER)`.
|
||||
- If `path` is `None`, check `GET /repos/<owner>/<repo_name>`. On 200, reuse the repo; otherwise raise `RuntimeError("Repo %s/%s fehlt und kann dort nicht angelegt werden" % (owner, repo_name))`.
|
||||
- Otherwise POST to `path`. As today, 409 means reuse.
|
||||
- Only call `ensure_repo_webhook(repo_full(owner, repo_name))` if `needs_repo_webhook(owner, NEW_REPO_OWNER, GITEA_USER)`.
|
||||
3. **Git and web addresses:**
|
||||
- The push URL in the fill step (today line 531) becomes `lead_git_url(owner, repo_name)`, so that function gets `owner` as a parameter.
|
||||
- `get_template_cache` uses `GITEA_AUTH_USER` as the user; owner and repo stay `TEMPLATE_OWNER`/`TEMPLATE_REPO`.
|
||||
- Every `"%s/%s/%s" % (GITEA_URL, GITEA_USER, repo_name)` becomes `lead_web_url(owner, repo_name)`.
|
||||
4. **Lookups:** In the three functions at the former lines 331, 373 and 415, use `lead_repo_full(lead)`.
|
||||
5. **`process_lead`:**
|
||||
- `owner = lead_repo_owner(lead)`, set `lead["repo_owner"] = owner`, then `ensure_repo(repo_name, firma, owner)`.
|
||||
- `_mark_processed(lead_id, repo_url, owner)` also stores `repo_owner` in `master.json`.
|
||||
- The caller around the former line 621 passes the owner along.
|
||||
|
||||
- [ ] **Step 2: Check**
|
||||
|
||||
```bash
|
||||
python3 -m py_compile worker.py && python3 -m pytest -q tests
|
||||
grep -n "GITEA_USER, repo_name\|GITEA_USER, (lead" worker.py
|
||||
```
|
||||
|
||||
Expected: compiles, `7 passed`, and grep finds nothing.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add worker.py && git -c user.name=Claude -c user.email=claude@webklar.local commit -qm "worker: Repo-Besitzer pro Lead, System-Konto für Git-Adressen"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: processor.py, enrich.py und Wartungsskripte umstellen
|
||||
|
||||
**Files:**
|
||||
- Modify: `processor.py`, `enrich.py`, `backfill_leads.py`, `migrate_previews.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: the `W.*` functions from Task 3
|
||||
|
||||
- [ ] **Step 1: Implement**
|
||||
|
||||
- **`processor.py`:**
|
||||
- `_repo_url(owner, repo_name)` returns `worker.lead_git_url(owner, repo_name)`.
|
||||
- `_clone_repo(owner, repo_name)`.
|
||||
- In `process_lead`: `owner = worker.lead_repo_owner(lead)`, then `worker.ensure_repo(repo_name, firma, owner)`, `worker._mark_processed(lead_id, repo_url, owner)`, and `repo_url = worker.lead_web_url(owner, repo_name)`.
|
||||
- **`enrich.py`:**
|
||||
- `_repo_push_url(owner, repo_name)` and `_clone_repo(owner, repo_name)`.
|
||||
- `_apply_to_repo(owner, repo_name, p, ctx=None)`. The caller determines `owner = worker.lead_repo_owner(lead)` inside the `master_lock` block, together with `repo_name`.
|
||||
- In `get_file`, the API path uses `worker.lead_repo_owner(lead)` instead of `worker.GITEA_USER`.
|
||||
- `repo_url` (the former line 1049) becomes `worker.lead_web_url(owner, repo_name)`.
|
||||
- **`backfill_leads.py` and `migrate_previews.py`:** `W.lead_repo_full(lead)` instead of `"%s/%s" % (W.GITEA_USER, repo_name)`.
|
||||
|
||||
- [ ] **Step 2: Check**
|
||||
|
||||
```bash
|
||||
python3 -m py_compile processor.py enrich.py backfill_leads.py migrate_previews.py hook_server.py && python3 -m pytest -q tests
|
||||
grep -n "GITEA_USER" processor.py enrich.py backfill_leads.py migrate_previews.py
|
||||
```
|
||||
|
||||
Expected: compiles, `7 passed`. `GITEA_USER` appears in these four files nowhere, or only in comments.
|
||||
|
||||
- [ ] **Step 3: Restart with unchanged `config.env` (the code alone changes nothing)**
|
||||
|
||||
With the user's OK:
|
||||
|
||||
```bash
|
||||
systemctl restart webklar-automation-hook webklar-automation-processor
|
||||
systemctl is-active webklar-automation-hook webklar-automation-processor
|
||||
```
|
||||
|
||||
Expected: `active` twice, and the log files in `logs/` show no traceback in the next 10 minutes.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add processor.py enrich.py backfill_leads.py migrate_previews.py && git -c user.name=Claude -c user.email=claude@webklar.local commit -qm "processor/enrich: Adressen mit Repo-Besitzer des Leads"
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Review**
|
||||
|
||||
Run a fresh code review of `git diff <Ausgangsstand>..HEAD` against Spec 8 and this plan (`superpowers:requesting-code-review`).
|
||||
|
||||
---
|
||||
|
||||
## Livegang – nur mit OK des Nutzers, erst nach Plan 1 L5
|
||||
|
||||
**L0 Sicherung**
|
||||
|
||||
```bash
|
||||
cp /opt/webklar-automation/config.env /opt/webklar-automation/config.env.bak-gitea-org
|
||||
```
|
||||
|
||||
**L1 Schlüssel und Einstellungen.** The processor is stopped, so no lead is in the middle of processing:
|
||||
|
||||
```bash
|
||||
systemctl stop webklar-automation-processor
|
||||
docker exec -u git webklar-gitea gitea admin user generate-access-token --username webklar-system --token-name lead-automatik --scopes write:repository,write:organization,read:user --raw \
|
||||
| python3 -c 'import re,sys; t=sys.stdin.read().strip(); p="/opt/webklar-automation/config.env"; s=open(p).read(); s=re.sub(r"^GITEA_TOKEN=.*$", "GITEA_TOKEN="+t, s, flags=re.M); open(p,"w").write(s.rstrip("\n")+"\nGITEA_AUTH_USER=webklar-system\nGITEA_NEW_REPO_OWNER=WEBklar\n")'
|
||||
grep -c "^GITEA_AUTH_USER=webklar-system$\|^GITEA_NEW_REPO_OWNER=WEBklar$" /opt/webklar-automation/config.env
|
||||
systemctl start webklar-automation-processor && systemctl restart webklar-automation-hook
|
||||
```
|
||||
|
||||
Expected: `2`, and both services `active`. The token only passes through the pipe.
|
||||
|
||||
**L2 Zugang prüfen:**
|
||||
|
||||
```bash
|
||||
cd /opt/webklar-automation && python3 -c "import worker as W; c,u=W.gitea_api('GET','/user'); print(c,u.get('login'),u.get('is_admin'))"
|
||||
```
|
||||
|
||||
Expected: `200 webklar-system True`.
|
||||
|
||||
**L3 Probelauf mit Wegwerf-Repo.** Only with explicit OK:
|
||||
|
||||
1. With the worker functions, create `WEBklar/preview-probe-gitea-org` (`W.ensure_repo('preview-probe-gitea-org', 'Probe', W.NEW_REPO_OWNER)`), then `W.set_repo_file('WEBklar/preview-probe-gitea-org', 'index.html', '<h1>Probe</h1>', 'Probe')`.
|
||||
2. Clone the template once with `W.get_template_cache()`. This checks that the system account can read `knso/<TEMPLATE_REPO>`.
|
||||
3. Expected:
|
||||
- the repo exists in WEBklar,
|
||||
- `GET /repos/WEBklar/preview-probe-gitea-org/hooks` returns `[]`,
|
||||
- within a minute the portal has a `websiteProjects` entry `WEBklar/preview-probe-gitea-org` with status `synced`, created by the organisation webhook,
|
||||
- no new arbeitsblatt.
|
||||
4. Clean up:
|
||||
- `DELETE /repos/WEBklar/preview-probe-gitea-org`,
|
||||
- delete the `websiteProjects` entry with a script in the portal container,
|
||||
- check that both are gone.
|
||||
|
||||
**L4 Beobachten.** Check the next real lead: its repo is created in WEBklar, the preview deploys, and there is no per-repo webhook. Update the memory note.
|
||||
|
||||
**Rückweg:**
|
||||
|
||||
```bash
|
||||
cp /opt/webklar-automation/config.env.bak-gitea-org /opt/webklar-automation/config.env
|
||||
systemctl restart webklar-automation-hook webklar-automation-processor
|
||||
```
|
||||
|
||||
The code can stay: without the new keys it behaves as it does today. Remove the `lead-automatik` token in Gitea under Settings → Applications of the account, as an admin.
|
||||
Reference in new issue
Block a user