Compare commits
33
Commits
edef85c7b3
...
54f307d12a
No files matched your search
@@ -0,0 +1,17 @@
|
||||
# Task 2 Report: Projekterkennung
|
||||
Implemented runner/src/detect.js (detectProject, detectFromDir) + runner/test/detect.test.js (19 tests).
|
||||
TDD: RED = ERR_ASSERTION / module not found (detect.js missing); first GREEN attempt had 1 failure (devCmd null for build kind) -> fixed; final: runner npm test 27/27 pass.
|
||||
|
||||
## Return shape
|
||||
{ kind, pm, install, buildCmd, devCmd, outputDirs, devPort, [index], [hint], [error] }
|
||||
- buildCmd/devCmd are set whenever the script exists (also for kind 'build' -> devCmd = dev fallback after failed build, per spec).
|
||||
- static: install [], buildCmd/devCmd null, outputDirs [], `index` = start file (opts.index > previewJson.index > 'index.html'); `hint` (German) when neither index nor scripts found.
|
||||
- Forced mode without script: `{ kind:'build', ..., buildCmd:null, error:'Kein build-Skript in package.json' }` / `{ kind:'dev', ..., devCmd:null, error:'Kein dev-Skript in package.json' }`.
|
||||
- auto: index && !build => static; build => build; dev => dev; else static + hint.
|
||||
- Invalid package.json JSON => no scripts, never throws. devPort int 1024-65535 else 5173; output/index must be relative without '..' else ignored.
|
||||
## Commands
|
||||
npm: ci --include=dev (package-lock/npm-shrinkwrap) or install --include=dev --no-audit --no-fund; pnpm/yarn/bun: `<pm> install --frozen-lockfile`; build `<pm> run build`.
|
||||
dev: npm/pnpm `run dev -- --host 0.0.0.0 --port N`; yarn/bun `run dev --host 0.0.0.0 --port N` (no `--`, they pass args through).
|
||||
Note: pnpm/yarn in node:20 need corepack (`corepack enable`) — not handled here (later task).
|
||||
## detectFromDir(dir, opts)
|
||||
fs adapter (has = isFile); reads .webklar-preview.json as previewJson unless opts.previewJson given; tested with temp dir.
|
||||
@@ -0,0 +1,191 @@
|
||||
# Preview 2.0 (Preview-Runner) – Umsetzungsplan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Builds und Dev-Server für Previews laufen in abgeschotteten Containern eines neuen Dienstes
|
||||
`webklar-preview-runner`; Modus und Variablen pro Projekt im WOMS.
|
||||
|
||||
**Architecture:** Neuer Node-Dienst in `runner/` (eigenes package.json, keine Portal-Imports, spricht Docker
|
||||
über die CLI). Das Portal (`server/`) schickt Aufträge per HTTP an den Runner, hält Zustand, Variablen
|
||||
(verschlüsselt) und Protokolle und schreibt die Traefik-Router. WOMS bekommt einen Preview-Block am Projekt.
|
||||
|
||||
**Tech Stack:** Node 20/24 (ESM, `node --test`), Docker-CLI, Express (Portal), React/Vite (WOMS).
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-09-30-preview-runner-design.md` (Commit 1e9c2bc)
|
||||
|
||||
**Worktrees:** Portal `/home/webklar/apps/kundenbereich.webklar.com/dev-preview` (Branch `feature/preview-runner`).
|
||||
WOMS: `git -C /home/webklar/apps/ticket.webklar.com/build worktree add -b feature/preview-runner ../dev-preview origin/test`,
|
||||
danach `git ls-files -z node_modules | xargs -0 git update-index --skip-worktree` und eigenes `npm install`
|
||||
(getrackte node_modules – nie `git add -A`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Modi exakt `auto` | `build` | `dev` | `off`; leer = `auto`. Vorgabe bei Zuordnung: Art website/webapp ⇒ `auto`, sonst `off`.
|
||||
- Zustände exakt `online` | `building` | `failed` | `sleeping` | `starting` | `off`.
|
||||
- Build: `--memory 1g --cpus 1 --pids-limit 512 --cap-drop ALL --security-opt no-new-privileges --user 1000:1000 --network preview-sandbox`, Zeitlimit 15 min, genau 1 gleichzeitig.
|
||||
- Dev: gleiches Profil mit `--memory 512m`, Name `preview-dev-<sub>`, max. 3, Leerlauf 20 min, Bereitschaft max. 180 s, Standard-Port 5173.
|
||||
- Schutzschalter: MemAvailable < 1,5 GB ⇒ `503 { error: 'Server ausgelastet' }`, nichts Neues starten.
|
||||
- Runner-Auth: `Authorization: Bearer <PREVIEW_RUNNER_TOKEN>`; API-Port 8080, Proxy-Port 8081; kein veröffentlichter Port.
|
||||
- Runner-Eingaben: Subdomain `^[a-z0-9-]{1,63}$`, Repo `^[\w.-]+/[\w.-]+$`, sha `^[0-9a-f]{7,40}$`, Branch `^[\w./-]{1,100}$`.
|
||||
- Variablen: Name `^[A-Z_][A-Z0-9_]*$`, max. 30, Wert ≤ 4096 Byte, AES-256-GCM, Format `v1:<iv>:<tag>:<ct>` (base64), Schlüssel `PREVIEW_ENV_KEY` (32 Byte base64).
|
||||
- Protokoll: letzte 300 Zeilen je Lauf, letzte 5 Läufe je Projekt, Variablenwerte (≥ 4 Zeichen) ⇒ `•••`.
|
||||
- Build-Fehler ⇒ vorhandener Preview-Ordner bleibt; Erfolg ⇒ atomar über `<sub>.new` + rename.
|
||||
- `PREVIEW_RUNNER=off` (Vorgabe) ⇒ Portal verhält sich exakt wie heute.
|
||||
- Texte deutsch; kein `confirm()`. Nichts pushen/live schalten ohne OK des Nutzers.
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. Build scheitert bei bestehender Preview ⇒ alte Dateien bleiben unverändert online. → Task 4
|
||||
2. 4. Dev-Start bei 3 laufenden ⇒ der am längsten ungenutzte stoppt, nicht der gerade angefragte. → Task 5
|
||||
3. Böswillige Eingaben (Subdomain `../x`, Repo mit `;`) ⇒ 400, kein Docker-Aufruf. → Task 1
|
||||
4. Variablenwert erscheint im Build-Protokoll ⇒ im gespeicherten Protokoll maskiert. → Task 7
|
||||
5. `PREVIEW_RUNNER=off` ⇒ `runDeploy` ruft keinen Runner, alte Tests unverändert grün. → Task 9
|
||||
|
||||
---
|
||||
|
||||
## Runner (`runner/`)
|
||||
|
||||
### Task 1: Gerüst, Eingabeprüfung, Auth, `/status`
|
||||
|
||||
**Files:** Create `runner/package.json` (type module, `"test": "node --test test/*.test.js"`, keine Laufzeit-Abhängigkeiten),
|
||||
`runner/src/config.js`, `runner/src/validate.js`, `runner/src/server.js`, `runner/src/index.js`, `runner/Dockerfile`
|
||||
(`FROM node:20-alpine`, `apk add --no-cache docker-cli git`), Test `runner/test/server.test.js`.
|
||||
|
||||
**Interfaces – Produces:**
|
||||
- `loadConfig(env) ⇒ { token, giteaUrl, giteaToken, previewsRoot, workRoot, cacheRoot, apiPort:8080, proxyPort:8081, minFreeBytes: 1.5*1024**3, buildTimeoutMs: 15*60e3, devIdleMs: 20*60e3, devMax:3, devReadyMs:180e3 }`
|
||||
- `validateJob(body) ⇒ { ok:true, job } | { ok:false, error }` (Regeln aus Global Constraints; `mode` ∈ auto|build|dev; `env` Objekt mit gültigen Namen)
|
||||
- `createApiServer({ config, jobs, dev, mem }) ⇒ http.Server` mit Routen `POST /jobs/build`, `GET /jobs/:runId`, `POST /dev/:sub/start|stop`, `GET /status`; ohne/mit falschem Token ⇒ 401; ungültig ⇒ 400.
|
||||
- `readMemAvailable(path='/proc/meminfo') ⇒ bytes`
|
||||
|
||||
- [ ] **Step 1: Tests** – 401 ohne Token; 400 für Subdomain `../x`, Repo `a/b;rm`, sha `xyz`; 200 `/status` mit `{ dev:[], queue:0, memAvailable }`; `readMemAvailable` aus Beispieltext.
|
||||
- [ ] **Step 2–4:** FAIL → implementieren (reines `node:http`) → PASS
|
||||
- [ ] **Step 5:** `git commit -m "Runner: Geruest, Eingabepruefung, Auth"`
|
||||
|
||||
### Task 2: Erkennung
|
||||
|
||||
**Files:** Create `runner/src/detect.js`, Test `runner/test/detect.test.js`
|
||||
|
||||
**Interfaces – Produces:** `detectProject(files: { has(path):bool, read(path):string|null }, { mode, index, previewJson }) ⇒ { kind: 'static'|'build'|'dev', pm: 'npm'|'pnpm'|'yarn'|'bun', install: string[], buildCmd: string[]|null, devCmd: string[]|null, outputDirs: string[], devPort: number }`
|
||||
Regeln laut Spec 1.2 „Erkennung“; `mode:'build'` erzwingt build (Fehler, wenn kein Build-Skript), `mode:'dev'` erzwingt dev; `auto`: static ⇒ build ⇒ dev. `outputDirs` = `[previewJson.output]` oder `['dist','build','out','web-build']`. npm ohne Lock ⇒ `npm install --include=dev --no-audit --no-fund`, mit Lock ⇒ `npm ci --include=dev`. devCmd z. B. `['npm','run','dev','--','--host','0.0.0.0','--port','5173']`.
|
||||
|
||||
- [ ] **Step 1: Tests** – HTML ohne package.json ⇒ static; Vite mit build ⇒ build, pm npm; pnpm-lock ⇒ pnpm; nur `dev`-Skript ⇒ dev; index.html + build-Skript ⇒ build; `mode:'dev'` erzwingt dev; `previewJson.devPort` 3000 ⇒ devPort 3000 und im devCmd.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren → PASS → `git commit -m "Runner: Projekterkennung"`
|
||||
|
||||
### Task 3: Docker-Adapter
|
||||
|
||||
**Files:** Create `runner/src/docker.js`, Test `runner/test/docker.test.js`
|
||||
|
||||
**Interfaces – Produces:** `buildRunArgs({ image, name, workDir, cacheDir, env, cmd, memory, network }) ⇒ string[]` (enthält exakt das Sicherheitsprofil aus Global Constraints, `--rm`, `-w /work`, Mounts nur `workDir:/work` und `cacheDir:/cache`, `-e NAME` mit Werten per `env` des Kindprozesses, nie auf der Kommandozeile);
|
||||
`devRunArgs(...)` (ohne `--rm`, `-d`, Name `preview-dev-<sub>`, Label `webklar.preview=<sub>`);
|
||||
`createDocker({ exec = spawnDocker }) ⇒ { run(args, { env, timeoutMs, onLine }) ⇒ {code}, stop(name), rm(name), ps(label) ⇒ names[], inspectIp(name) }`.
|
||||
|
||||
- [ ] **Step 1: Tests** – Argumentliste enthält jede Sicherheitsoption; kein Variablenwert in args; Zeitlimit ⇒ Kill und `code:124`; `ps` parst Ausgabe.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren (child_process.spawn) → PASS → `git commit -m "Runner: Docker-Adapter mit Sicherheitsprofil"`
|
||||
|
||||
### Task 4: Build-Aufträge
|
||||
|
||||
**Files:** Create `runner/src/jobs.js`, Test `runner/test/jobs.test.js`
|
||||
|
||||
**Interfaces:** Consumes Task 2 `detectProject`, Task 3 `createDocker`. Produces `createJobs({ config, docker, fs, git, mem, now }) ⇒ { submit(job) ⇒ runId, get(runId) ⇒ { state:'queued'|'running'|'done'|'failed', detected, sha, log:string[], error, result:'static'|'build'|'dev' } }`.
|
||||
Ablauf: Warteschlange, genau 1 aktiv; Schutzschalter vor Start (bleibt queued, erneuter Versuch alle 30 s); `git clone --depth 1 --branch <branch>` mit Token-URL (Token nie im Log); Erkennung; static ⇒ kopieren; build ⇒ install+build im Container, Ausgabeordner suchen, nach `<sub>.new` kopieren, alten Ordner ersetzen; Fehler ⇒ alter Ordner unberührt, `failed` (bei `auto` mit dev-Skript ⇒ Ergebnis `dev`, kein Fehler); Log-Ringpuffer 300 Zeilen; erledigte Läufe 1 h im Speicher. Cache je Repo unter `cacheRoot/<owner>__<repo>` (als `/cache`, Paketmanager-Cache dorthin), `pruneCache()` täglich: Einträge > 14 Tage und, solange gesamt > 5 GB, die ältesten löschen.
|
||||
|
||||
- [ ] **Step 1: Tests** – `pruneCache` löscht > 14 Tage und hält 5 GB ein; zwei Aufträge laufen nacheinander; Build-Fehler lässt vorhandenen Ordner byte-gleich; Erfolg ersetzt atomar; Token nicht im Log; wenig Speicher ⇒ bleibt queued; auto+dev-Skript nach Build-Fehler ⇒ result `dev`; Log auf 300 Zeilen gekürzt.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren → PASS → `git commit -m "Runner: Build-Auftraege mit Warteschlange"`
|
||||
|
||||
### Task 5: Dev-Server-Verwaltung
|
||||
|
||||
**Files:** Create `runner/src/dev.js`, Test `runner/test/dev.test.js`
|
||||
|
||||
**Interfaces – Produces:** `createDev({ config, docker, jobs, mem, now }) ⇒ { ensure(sub, spec) ⇒ { state:'starting'|'online', target? }, touch(sub), stop(sub), list() ⇒ [{sub, lastUsed, state}], sweep() }`.
|
||||
Specs (Repo, sha, env, devCmd, devPort) setzt das Portal per Build-Auftrag mit Ergebnis dev bzw. `POST /dev/:sub/start` (Body wie build-Job). Max. 3 ⇒ LRU-Verdrängung; `sweep()` (jede Minute) stoppt nach 20 min ohne `touch`; Bereitschaft = TCP-Verbindung zum Port des Containers (über Container-IP im Netz `preview-sandbox`), max. 180 s, sonst Zustand `failed` mit Log. Dev-Container bekommen zusätzlich `__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS=.project.webklar.com` (Basis-Host aus config). `ensure` mit neuem sha für laufenden sub ⇒ Container neu starten.
|
||||
|
||||
- [ ] **Step 1: Tests** – 4. Start verdrängt ältesten; `sweep` nach 21 min stoppt, nach 19 min nicht; wenig Speicher ⇒ Fehler `Server ausgelastet`; gleicher sub zweimal ⇒ ein Container.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren → PASS → `git commit -m "Runner: Dev-Server bei Bedarf"`
|
||||
|
||||
### Task 6: Proxy mit Startseite
|
||||
|
||||
**Files:** Create `runner/src/proxy.js`, Modify `runner/src/index.js` (API + Proxy + sweep starten), Test `runner/test/proxy.test.js`
|
||||
|
||||
**Interfaces – Produces:** `createProxy({ config, dev }) ⇒ http.Server`. Host `<sub>.<baseHost>` ⇒ `dev.touch` + Weiterleitung (HTTP und `upgrade`/WebSocket) an `target`; nicht online ⇒ `dev.ensure` anstoßen und HTML-Seite „Preview startet …“ (Status 503, `Retry-After: 3`, `<meta http-equiv="refresh" content="3">`), nach Fehler Seite „Preview konnte nicht starten“. Unbekannter Host ⇒ 404.
|
||||
|
||||
- [ ] **Step 1: Tests** – laufendes Ziel (lokaler Test-HTTP-Server) wird durchgereicht inkl. WebSocket-Echo; nicht laufend ⇒ 503 mit Starttext; fremder Host ⇒ 404.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren → PASS (`cd runner && npm test`) → `git commit -m "Runner: Proxy und Startseite"`
|
||||
|
||||
## Portal (`server/`)
|
||||
|
||||
### Task 7: Variablen-Verschlüsselung und Maskierung
|
||||
|
||||
**Files:** Create `server/services/previewEnv.js`, Test `test/previewEnv.test.js`
|
||||
|
||||
**Interfaces – Produces:** `encryptEnv(obj, key) ⇒ string`, `decryptEnv(str, key) ⇒ obj`, `validateEnvChange(current, { set, remove }) ⇒ { ok, env | error }`, `envNames(str, key) ⇒ string[]`, `maskLog(text, env) ⇒ string`, `envKey() ⇒ Buffer|null` (aus `process.env.PREVIEW_ENV_KEY`).
|
||||
|
||||
- [ ] **Step 1: Tests** – Round-Trip; falscher Schlüssel ⇒ Fehler; manipulierter Tag ⇒ Fehler; 31. Variable ⇒ Fehler; Name `abc` ⇒ Fehler; Wert 4097 Byte ⇒ Fehler; `maskLog` ersetzt Werte ≥ 4 Zeichen, lässt kürzere.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren (node:crypto) → PASS → `git commit -m "Preview: Variablen verschluesselt"`
|
||||
|
||||
### Task 8: Schema-Skript + Konfiguration
|
||||
|
||||
**Files:** Create `scripts/preview-runner-setup.mjs` (Muster `scripts/projektarten-setup.mjs` inkl. `limit 200`), Modify `server/config.js` (`previewRunner: { enabled: process.env.PREVIEW_RUNNER === 'on', url: process.env.PREVIEW_RUNNER_URL || 'http://webklar-preview-runner:8080', token: process.env.PREVIEW_RUNNER_TOKEN || '' }`), Test `test/previewRunnerSetup.test.js`.
|
||||
Attribute laut Spec 1.1 (`previewMode` 10, `previewEnv` 20000, `previewState` 20, `previewDetected` 40, `previewBuiltSha` 40, `previewBuiltAt` 40, `previewError` 500; alle optional).
|
||||
|
||||
- [ ] **Step 1–5:** Test (7 Schlüssel, Größen, fehlende werden angelegt) → FAIL → implementieren → PASS → `git commit -m "Preview: Schema-Skript und Runner-Konfiguration"`
|
||||
|
||||
### Task 9: Runner-Anbindung in `runDeploy`
|
||||
|
||||
**Files:** Create `server/services/previewRunnerClient.js`, `server/services/previewLogs.js`, Modify `server/services/previewDeploy.js` (`runDeploy`), Test `test/previewRunner.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- `previewRunnerClient`: `submitBuild(job) ⇒ { runId }`, `getRun(runId)`, `startDev(sub, job)`, `stopDev(sub)`, `status()`; Fehler tragen `.status`.
|
||||
- `previewLogs`: `saveRunLog(projectId, lines, env) ⇒ runFile` (maskiert, behält 5), `listRunLogs(projectId)`, `readRunLog(projectId, runFile)` unter `upload-data/preview-logs/`.
|
||||
- `runDeploy(repoFullName, branch, previewConfig, { project } = {})`: bei `config.previewRunner.enabled` ⇒ Modus aus `project.previewMode` (leer = auto; `off` ⇒ nichts bauen, `previewState:'off'`), Variablen entschlüsseln, Auftrag senden, `previewState:'building'`, bis Ende pollen (alle 2 s, max. 16 min), Ergebnis in Projekt schreiben (`previewState` online|failed|sleeping, `previewDetected`, `previewBuiltSha`, `previewBuiltAt`, `previewError`), Log speichern, danach `syncPreviewTls()`. Ergebnis dev und Dev-Container läuft (`status()`) ⇒ `startDev` mit neuem sha (Neustart mit neuem Code). Aufrufer (Webhook, `updateWebsiteProject`) übergeben das Projekt-Dokument.
|
||||
- Schalter aus ⇒ bisheriger Code unverändert.
|
||||
|
||||
- [ ] **Step 1: Tests** – Schalter aus ⇒ kein Runner-Aufruf (Fake-Runner zählt 0) und `previewDeploy.test.js` bleibt grün; an ⇒ Auftrag mit entschlüsselten Variablen; Ergebnis failed ⇒ `previewState:'failed'`, Fehler gespeichert, Ordner unangetastet; Ergebnis dev ⇒ `sleeping`; Log maskiert und nur 5 behalten; Modus off ⇒ kein Auftrag.
|
||||
- [ ] **Step 2–5:** FAIL → implementieren → PASS → `git commit -m "Preview: Builds ueber den Runner"`
|
||||
|
||||
### Task 10: Traefik-Router für Dev
|
||||
|
||||
**Files:** Modify `server/services/previewDeploy.js` (`syncPreviewTls`), Test in `test/previewDeploy.test.js`
|
||||
|
||||
`syncPreviewTls()` nimmt zusätzlich alle Subdomains, deren Projekt `previewState` ∈ {sleeping, starting, online} **und** `previewDetected` beginnt mit `dev` (auch ohne Ordner), und schreibt für diese `service: preview-runner@docker` (HTTPS und HTTP-Redirect wie gehabt, Middleware-Regel für öffentlich/nicht öffentlich unverändert). Appwrite nicht erreichbar ⇒ Dev-Liste aus der letzten Datei übernehmen (wie `publicSubdomainsFromHostsFile`).
|
||||
|
||||
- [ ] **Step 1–5:** Tests (dev-Projekt ⇒ runner-Service; static ⇒ nginx; Appwrite weg ⇒ vorherige Dev-Hosts bleiben) → FAIL → implementieren → PASS → `git commit -m "Preview: Router fuer Dev-Server"`
|
||||
|
||||
### Task 11: Admin-API und Modus-Vorgabe
|
||||
|
||||
**Files:** Create `server/routes/admin/preview.js` (eingehängt unter `/api/admin/website-projects` vor den bestehenden `/:id`-Routen oder als eigener Router mit `requireAppwriteStaff`), Modify `server/services/projectKindAssign.js` (Modus-Vorgabe aus Art: website/webapp ⇒ `auto`, sonst `off`, nur wenn `previewMode` leer), Test `test/previewAdmin.test.js`
|
||||
|
||||
Routen laut Spec 1.1. `GET …/preview` ⇒ `{ mode, state, detected, builtSha, builtAt, error, envNames, logs:[{runFile, at}] }`; `PATCH …/preview { mode }` (400 bei ungültig) ⇒ Neubau anstoßen (nicht bei off; bei off Dev stoppen); `PUT …/preview/env` ⇒ `validateEnvChange`, speichern, Neubau; ohne `PREVIEW_ENV_KEY` ⇒ 503 `{ error: 'Preview-Variablen sind nicht eingerichtet' }`; `POST …/rebuild`; `POST …/dev/start|stop`; `GET …/logs/:runFile` (Dateiname streng prüfen). Schalter aus ⇒ Schreib-Routen 409 `{ error: 'Preview-Runner ist nicht aktiv' }`, GET funktioniert.
|
||||
|
||||
- [ ] **Step 1–5:** Tests (401 ohne JWT, Modus-Validierung, env ohne Schlüssel ⇒ 503, Werte nie in Antwort, Log-Pfad `../x` ⇒ 400, Zuordnen setzt Vorgabe) → FAIL → implementieren → PASS (`npm test` komplett) → `git commit -m "Preview: Admin-API und Modus-Vorgabe"`
|
||||
|
||||
## WOMS
|
||||
|
||||
### Task 12: Preview-API-Client und Anzeige-Logik
|
||||
|
||||
**Files:** Create `src/lib/previewApi.js` (Muster `projectKindsApi.js`, Fehler mit `.status`/`.data`), `src/lib/previewView.js`, Test `tests/unit/previewView.test.mjs`
|
||||
|
||||
**Interfaces – Produces:** `previewStatus(p) ⇒ { color:'green'|'yellow'|'red'|'grey', text }` mit exakten Texten: online „Online seit TT.MM. HH:MM (Build <sha7>)“, building „Baut gerade …“, sleeping „Dev-Server schläft, startet beim nächsten Aufruf“, starting „Dev-Server startet …“, failed „Build fehlgeschlagen – letzte funktionierende Version ist online“, off „Keine Preview“; `detectedLabel(detected) ⇒` z. B. `build:vite:npm` ⇒ „Build (Vite, npm)“, `dev:npm` ⇒ „Dev-Server (npm)“, `static` ⇒ „Statisch“; `MODE_OPTIONS` = Automatisch/Build/Dev-Server/Aus.
|
||||
|
||||
- [ ] **Step 1–5:** Tests → FAIL → implementieren → PASS → `git commit -m "Preview: Anzeige-Logik und API-Client"`
|
||||
|
||||
### Task 13: Preview-Block in Projektkarte, Bearbeiten-Dialog und Services
|
||||
|
||||
**Files:** Create `src/components/PreviewBlock.jsx`, Modify `src/components/AssignedProjectCard.jsx`, `src/pages/ProjectsPage.jsx` (Bearbeiten-Modal), `src/pages/ServicesPage.jsx` (Punkt je Zeile aus `previewState`, Portal liefert das Feld in `/api/admin/services` – dort in Task 11 ergänzen, falls fehlend)
|
||||
|
||||
Block laut Spec Abschnitt 2 (Modus, Status, Knöpfe, Protokoll-Auswahl der 5 Läufe, Variablen-Tabelle mit `•••`, Hinweis `VITE_`/`NEXT_PUBLIC_`). 409 „Preview-Runner ist nicht aktiv“ ⇒ Block zeigt Hinweis statt Knöpfen. Handy-tauglich (390 px).
|
||||
|
||||
- [ ] **Step 1:** implementieren · **Step 2:** `npm test` + Build grün · **Step 3:** `git commit -m "Preview: Block im WOMS"`
|
||||
|
||||
### Task 14: Oberflächen-Check (Playwright, Fixture)
|
||||
|
||||
WOMS-dist mit gemockten Portal-/Appwrite-Antworten: Modus wechseln (PATCH-Body), Variablen setzen (Werte danach `•••`), Protokoll öffnen, 409-Hinweis, Services-Punkt, 390 px ohne horizontales Scrollen. Screenshots nach `/root/reports/preview-runner-2026-09-30/`.
|
||||
|
||||
- [ ] **Step 1:** Checks grün · **Step 2:** Ergebnis im Ledger
|
||||
|
||||
## Livegang (jeder Schritt mit OK des Nutzers)
|
||||
|
||||
- **L1** Sicherung `docker-compose.yml`, `.env`, `deploy.sh`. `.env`: `PREVIEW_RUNNER_TOKEN` (32 Byte zufällig), `PREVIEW_ENV_KEY` (32 Byte base64). `docker network create preview-sandbox`. Compose-Dienst `webklar-preview-runner` (build `kundenbereich.webklar.com/build/runner`, Netze `webklar-network` + `preview-sandbox`, Mounts Docker-Socket, `/var/www/previews`, Volumes work/cache, Traefik-Labels Service `preview-runner` Port 8081, Env nur Token/Gitea-Deploy-Token/URL). Portal-Compose: `PREVIEW_RUNNER`, `PREVIEW_RUNNER_TOKEN`, `PREVIEW_ENV_KEY`. `deploy.sh`: nach dem Portal-Neustart `docker compose … up -d --build webklar-preview-runner`, wenn sich `runner/` geändert hat. Schema-Skript (`--check` zuerst).
|
||||
- **L2** Portal pushen (`PREVIEW_RUNNER=off`), Runner-Container starten, `/status` aus dem Portal-Container prüfen.
|
||||
- **L3** Echte Probe mit drei Wegwerf-Repos (HTML, Vite-Build, nur `dev`) in `WEBklar/` → aufräumen. Dann `PREVIEW_RUNNER=on`, die 12 Previews einzeln neu bauen, jede Seite vorher/nachher per HTTP-Status + Hash der `index.html` vergleichen.
|
||||
- **L4** WOMS pushen (`feature/preview-runner:test`).
|
||||
- **L5** Live-Test des Nutzers mit Dev-Modus an einem echten Projekt. Rückweg: `PREVIEW_RUNNER=off` + Portal neu starten.
|
||||
@@ -0,0 +1,135 @@
|
||||
# Preview 2.0 – Preview-Runner, Modi, Variablen – Design
|
||||
|
||||
Stand: 2026-09-30 · Status: vom Nutzer im Chat freigegeben (Abschnitte 1–3), Spec zur Durchsicht
|
||||
Teilprojekt **2** von 3 (1 Projektarten – live seit 2026-09-30; 3 Website-Analyse – eigene Spec).
|
||||
Regel des Nutzers: bauen → live → Nutzer testet live → erst dann das nächste Teilprojekt.
|
||||
Recherche: `/root/reports/analyse-funktion-2026-09-30/projekttypen-preview.md`.
|
||||
|
||||
## Ziel
|
||||
|
||||
Previews starten automatisch, auch für vibe-gecodete Projekte, die man mit `npm run dev` startet.
|
||||
Builds laufen **nicht mehr im Portal-Container** (der Appwrite-, Gitea-, SMTP- und Stripe-Zugangsdaten hat),
|
||||
sondern in Wegwerf-Containern ohne Zugangsdaten. Projekte können Preview-Variablen bekommen, ohne dass
|
||||
eine `.env` im Repo liegt.
|
||||
|
||||
## Entscheidungen (vom Nutzer)
|
||||
|
||||
| # | Entscheidung |
|
||||
|---|---|
|
||||
| V1 | Modus **pro Projekt**: `auto` (Vorauswahl) · `build` · `dev` · `off`. `auto` = erst statisch/Build, Dev-Server nur als Ausweg. |
|
||||
| V2 | Neuer Dienst **Preview-Runner** als eigener Container (Variante A). Nur er darf Docker-Container starten. |
|
||||
| V3 | **Preview-Variablen pro Projekt** im WOMS, verschlüsselt gespeichert, nur an Build/Dev dieses Projekts. |
|
||||
| V4 | Build-Fehler ⇒ **letzte funktionierende Version bleibt online** (keine Platzhalterseite mehr). |
|
||||
| V5 | Dev-Server starten beim ersten Aufruf, stoppen nach **20 min** ohne Aufruf, **max. 3** gleichzeitig. |
|
||||
| V6 | Kundenportal unverändert (außer der Seite „Preview startet …“ bei schlafendem Dev-Server). |
|
||||
|
||||
## 1. Aufbau
|
||||
|
||||
```
|
||||
Gitea-Push ──▶ Portal (webhook) ──Auftrag──▶ Preview-Runner ──▶ Docker
|
||||
│ │ ├─ Build-Container (Wegwerf)
|
||||
│ │ └─ Dev-Container (bei Bedarf)
|
||||
│ ▼
|
||||
│ /var/www/previews/<sub> (fertige Dateien)
|
||||
▼
|
||||
Traefik: <sub>.project.webklar.com ──preview-chain──▶ preview-nginx (static/build)
|
||||
└────▶ Runner-Proxy (dev)
|
||||
```
|
||||
|
||||
### 1.1 Portal (bleibt Chef)
|
||||
- Neue Attribute an `websiteProjects` (Schema-Skript `scripts/preview-runner-setup.mjs`, idempotent,
|
||||
Attributliste mit `limit 200` lesen):
|
||||
`previewMode` (string 10: auto|build|dev|off; leer = auto), `previewEnv` (string 20000, verschlüsselt),
|
||||
`previewState` (string 20: online|building|failed|sleeping|starting|off), `previewDetected` (string 40,
|
||||
z. B. `build:vite:npm`, `static`, `dev:npm`), `previewBuiltSha` (string 40), `previewBuiltAt` (ISO),
|
||||
`previewError` (string 500).
|
||||
- Vorauswahl des Modus bei neuer Zuordnung aus dem Projektarten-Katalog (`projectKinds.js`):
|
||||
Art website/webapp ⇒ `auto`, alle anderen ⇒ `off`. Bestehende Projekte ohne Wert gelten als `auto`.
|
||||
- `server/services/previewDeploy.js` bekommt einen Schalter `config.previewRunner.enabled`
|
||||
(`PREVIEW_RUNNER=on|off`, Vorgabe off). `off` = bisheriger Weg unverändert. `on` = Aufträge an den Runner.
|
||||
- `syncPreviewTls()` schreibt für Projekte im Modus dev (oder `auto` mit Ergebnis dev) den Router auf den
|
||||
Service `preview-runner@docker` statt `preview-nginx@docker`; Middleware `preview-chain` bleibt gleich
|
||||
(öffentliche Subdomains wie heute ohne Middleware).
|
||||
- Verschlüsselung `server/services/previewEnv.js`: AES-256-GCM, Schlüssel `PREVIEW_ENV_KEY` (32 Byte base64)
|
||||
aus `/home/webklar/apps/.env`; Format `v1:<iv>:<tag>:<ciphertext>` (base64). Namen `^[A-Z_][A-Z0-9_]*$`,
|
||||
max. 30 Variablen, Wert max. 4096 Byte. Ohne Schlüssel ⇒ Variablen-Funktion aus, Fehlermeldung im WOMS.
|
||||
- Protokolle: `upload-data/preview-logs/<projektId>/<zeitstempel>.log`, letzte 300 Zeilen je Lauf,
|
||||
die letzten 5 Läufe je Projekt, ältere löschen. Variablenwerte (≥ 4 Zeichen) werden vor dem Speichern
|
||||
durch `•••` ersetzt.
|
||||
- Admin-API (requireAppwriteStaff):
|
||||
`GET /api/admin/website-projects/:id/preview` (Modus, Status, erkannt, Namen der Variablen, Protokoll-Liste),
|
||||
`PATCH …/preview` `{ mode }`, `PUT …/preview/env` `{ set: {NAME: wert}, remove: [NAME] }`,
|
||||
`POST …/preview/rebuild`, `POST …/preview/dev/start|stop`, `GET …/preview/logs/:runId`.
|
||||
Änderung von Variablen oder Modus ⇒ automatisch Neubau bzw. Dev-Neustart.
|
||||
|
||||
### 1.2 Preview-Runner (neuer Container `webklar-preview-runner`)
|
||||
- Code im Portal-Repo unter `runner/` (eigenes `package.json`, Node 20, keine Abhängigkeit zum Portal-Code).
|
||||
Compose-Dienst in `/home/webklar/apps/docker-compose.yml`, Image `node:20-alpine` + Docker-CLI,
|
||||
Mount `build/runner` (read-only), `/var/run/docker.sock`, `/var/www/previews`, Cache-Volume.
|
||||
`deploy.sh` startet den Runner neu, wenn sich `runner/` geändert hat.
|
||||
- Netze: `webklar-network` (für Portal und Traefik) und `preview-sandbox` (Dev-Container). **Kein**
|
||||
veröffentlichter Port.
|
||||
- Umgebung: nur `PREVIEW_RUNNER_TOKEN`, `GITEA_DEPLOY_TOKEN` (read:repository), Gitea-URL, Limits.
|
||||
**Keine** Appwrite-/SMTP-/Stripe-Zugangsdaten.
|
||||
- HTTP-API (Bearer `PREVIEW_RUNNER_TOKEN`, Port 8080 nur intern):
|
||||
`POST /jobs/build` `{ projectId, repoFullName, branch, sha, subdomain, mode, env, index }` ⇒ `{ runId }`;
|
||||
`GET /jobs/:runId` ⇒ `{ state, detected, sha, log, error }`;
|
||||
`POST /dev/:subdomain/start|stop`; `GET /status` (laufende Dev-Server, Warteschlange, freier Speicher).
|
||||
Nur diese festen Aufträge – keine freien Docker-Befehle.
|
||||
- **Erkennung (`auto`)**: `index.html` im Wurzelverzeichnis (oder `index` aus `.webklar-preview.json`) und
|
||||
kein `build`-Skript ⇒ `static`; sonst `build`-Skript vorhanden ⇒ Build; Build fehlgeschlagen oder kein
|
||||
Build-Skript, aber `dev`-Skript ⇒ `dev`; sonst `static` mit Hinweis. Paketmanager aus Lock-Datei:
|
||||
`pnpm-lock.yaml` ⇒ pnpm, `yarn.lock` ⇒ yarn, `bun.lockb`/`bun.lock` ⇒ bun, sonst npm (`npm ci`, ohne Lock
|
||||
`npm install`). Ausgabeordner: `dist`, `build`, `out`, `web-build` (wie heute) bzw. aus `.webklar-preview.json`
|
||||
(`output`).
|
||||
- **Build-Container**: `node:20` (bun: `oven/bun`), `--memory 1g --cpus 1 --pids-limit 512 --cap-drop ALL
|
||||
--security-opt no-new-privileges --user 1000:1000 --network preview-sandbox`, Zeitlimit 15 min, genau **einer**
|
||||
gleichzeitig (Warteschlange). Quelle per `git clone --depth 1` mit Deploy-Token in ein Arbeitsverzeichnis;
|
||||
Cache `node_modules` je Repo (Volume, max. 5 GB gesamt, Einträge > 14 Tage löschen). Erfolg ⇒ Ausgabe atomar
|
||||
nach `/var/www/previews/<sub>` (in `<sub>.new` kopieren, dann umbenennen). Fehler ⇒ vorhandener Ordner bleibt.
|
||||
- **Dev-Container**: gleiches Sicherheitsprofil, `--memory 512m`, Name `preview-dev-<sub>`, nur im Netz
|
||||
`preview-sandbox`. Start `npm run dev -- --host 0.0.0.0 --port 5173` (bzw. passender Paketmanager; Port aus
|
||||
`.webklar-preview.json` `devPort` überschreibbar); Vite bekommt `__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS=.project.webklar.com`.
|
||||
Bereit, sobald der Port antwortet (max. 180 s). Leerlauf 20 min ⇒ stoppen (Container entfernen, Cache bleibt).
|
||||
Max. 3; beim 4. weicht der am längsten ungenutzte. Neuer Push ⇒ Code aktualisieren und neu starten.
|
||||
- **Runner-Proxy** (Port 8081, Traefik-Service `preview-runner`): leitet `Host: <sub>.project.webklar.com`
|
||||
inkl. WebSocket (Hot-Reload) an `preview-dev-<sub>:<port>` weiter; läuft keiner ⇒ Start anstoßen und Seite
|
||||
„Preview startet …“ (deutsch, Neuladen alle 3 s, nach 180 s Fehlermeldung „Preview konnte nicht starten“).
|
||||
- **Schutzschalter**: freier Speicher (`/proc/meminfo` MemAvailable des Hosts) < 1,5 GB ⇒ keine neuen Builds
|
||||
oder Dev-Starts; Antwort `503 { error: 'Server ausgelastet' }`, Build bleibt in der Warteschlange.
|
||||
|
||||
## 2. WOMS
|
||||
|
||||
Block **„Preview“** auf der Projektkarte im Ticket (`TicketProjects`/`AssignedProjectCard`) und im
|
||||
„Bearbeiten“-Dialog unter Projekte:
|
||||
1. **Modus**: Automatisch · Build · Dev-Server · Aus; darunter „Automatisch → Build (Vite, npm)“ o. ä. aus
|
||||
`previewDetected`.
|
||||
2. **Status**: 🟢 „Online seit TT.MM. HH:MM (Build <sha7>)“ · 🟡 „Baut gerade …“ / „Dev-Server schläft,
|
||||
startet beim nächsten Aufruf“ · 🔴 „Build fehlgeschlagen – letzte funktionierende Version ist online“.
|
||||
Knöpfe: Neu bauen · Protokoll ansehen · bei Dev: Jetzt starten / Stoppen.
|
||||
3. **Protokoll**: letzte 300 Zeilen, aufklappbar, letzte 5 Läufe wählbar.
|
||||
4. **Preview-Variablen**: Tabelle Name/Wert, Werte nach dem Speichern nur `•••`; Hinweis „Alles mit `VITE_` /
|
||||
`NEXT_PUBLIC_` ist im Browser sichtbar“.
|
||||
5. **Services-Seite**: Preview-Punkt (grün/gelb/rot/grau) je Projektzeile.
|
||||
|
||||
## 3. Sicherheit, Speicher, Tests, Livegang
|
||||
|
||||
Sicherheit und Grenzen wie in Abschnitt 1.2. Zusätzlich: Runner-Eingaben streng prüfen (Subdomain
|
||||
`^[a-z0-9-]{1,63}$`, Repo `^[\w.-]+/[\w.-]+$`, sha `^[0-9a-f]{7,40}$`), Pfade nur unterhalb der festen Wurzeln.
|
||||
|
||||
**Tests**: Runner-Unit-Tests mit Schein-Docker (Auftragsprüfung, Warteschlange, Erkennung, Leerlauf-Stopp,
|
||||
Verdrängung, Schutzschalter, Proxy inkl. Startseite); Portal (Verschlüsselung, Maskierung, Modus-Vorgaben,
|
||||
TLS-Router für dev, Schalter off = altes Verhalten); WOMS (Preview-Block-Logik, Playwright gegen Fixtures);
|
||||
echter Durchlauf auf dem Server mit drei Beispiel-Repos (HTML, Vite-Build, nur `dev`).
|
||||
|
||||
**Livegang** (jeder Schritt mit OK des Nutzers):
|
||||
L1 Netz `preview-sandbox`, Schlüssel `PREVIEW_RUNNER_TOKEN` + `PREVIEW_ENV_KEY` in `.env`, Runner-Dienst in
|
||||
Compose, Schema-Skript · L2 Portal pushen mit `PREVIEW_RUNNER=off` · L3 Schalter `on`, 12 Previews einzeln neu
|
||||
bauen und vorher/nachher vergleichen · L4 WOMS pushen · L5 Live-Test des Nutzers mit Dev-Modus.
|
||||
Rückweg jederzeit: `PREVIEW_RUNNER=off` + Portal neu starten.
|
||||
|
||||
## Nicht Teil dieser Spec
|
||||
|
||||
Datenbanken/Backends als Beiwagen-Container, Next.js-SSR mit eigenem Start-Modus (`start`), gemeinsame
|
||||
Standard-Variablen (Variante C), Previews für Handy-Apps (Expo-Web), Website-Analyse (Teilprojekt 3),
|
||||
„Art ändern“ aus Teilprojekt 1.
|
||||
@@ -0,0 +1,156 @@
|
||||
# Preview-Runner: Livegang-Anleitung (R18)
|
||||
|
||||
> **Umgesetzt am 2026-10-01 (L1), abweichend von diesem Vorschlag:** Der Runner laeuft wie das Portal
|
||||
> als `node:20-alpine` mit schreibgeschuetzt eingehaengtem `build/runner` (`/runner:ro`,
|
||||
> `command: apk add docker-cli git && node src/index.js`) statt eines gebauten Images; `deploy.sh`
|
||||
> macht bei Aenderungen in `runner/` und leerer Warteschlange nur `docker restart webklar-preview-runner`
|
||||
> (Status-Abfrage ueber die Nicht-Sandbox-Adresse, nicht 127.0.0.1). Git-Zugang: `GITEA_DEPLOY_TOKEN`
|
||||
> (nur lesen). Firewall: `/usr/local/sbin/preview-sandbox-firewall.sh` + systemd-Dienst
|
||||
> `preview-sandbox-firewall.service` (Bridge-Name fest `br-preview`), Einzelregeln statt der
|
||||
> fehlerhaften kombinierten Zeile unten. Massgeblich ist die echte Konfiguration auf dem Server.
|
||||
|
||||
Stand: Vorschlag aus der Abschluss-Prüfung. Nichts davon ist im Repo umgesetzt, `deploy.sh`
|
||||
(`/home/webklar/apps/kundenbereich.webklar.com/deploy.sh`) liegt außerhalb des Repos und wird hier
|
||||
nur als Vorschlag beschrieben. Reihenfolge laut Spezifikation: L1 (Netz, Geheimnisse, Compose,
|
||||
Schema-Skript) vor L2 (Portal mit `PREVIEW_RUNNER=off`), dann L3 (`PREVIEW_RUNNER=on`).
|
||||
|
||||
## 1. Netzwerk `preview-sandbox` (VOR dem ersten Start anlegen)
|
||||
|
||||
```sh
|
||||
docker network create --subnet 172.30.0.0/24 preview-sandbox
|
||||
```
|
||||
|
||||
Der Runner verweigert den Start, wenn er keine IPv4-Adresse außerhalb dieses Subnetzes hat.
|
||||
`PREVIEW_SANDBOX_SUBNET` muss exakt dem Subnetz entsprechen. Das Subnetz nicht in Compose anders
|
||||
definieren (Netz als `external: true` einbinden).
|
||||
|
||||
Verzeichnisse auf dem Host anlegen (root-eigen genügt, der Runner setzt Besitzer 1000 selbst):
|
||||
|
||||
```sh
|
||||
mkdir -p /srv/webklar-preview-runner/work /srv/webklar-preview-runner/cache /var/www/previews
|
||||
```
|
||||
|
||||
## 2. Compose-Dienst (in `/home/webklar/apps/docker-compose.yml`)
|
||||
|
||||
Der Image-Build-Kontext ist `kundenbereich.webklar.com/build/runner` (Dockerfile im Repo unter
|
||||
`runner/`). Hostpfade stehen im Container unter dem IDENTISCHEN Pfad (R6).
|
||||
|
||||
```yaml
|
||||
preview-runner:
|
||||
build:
|
||||
context: ./kundenbereich.webklar.com/build/runner
|
||||
image: webklar-preview-runner:latest
|
||||
container_name: webklar-preview-runner
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
mem_limit: 256m
|
||||
# keine ports: - erreichbar nur im Docker-Netz webklar-network (Traefik) und vom Portal
|
||||
environment:
|
||||
- PREVIEW_RUNNER_TOKEN=${PREVIEW_RUNNER_TOKEN}
|
||||
- GITEA_URL=https://git.webklar.com
|
||||
- GITEA_TOKEN=${GITEA_SYSTEM_TOKEN} # lesender Token, der die WEBklar-Repos sehen darf
|
||||
- PREVIEW_BASE_HOST=project.webklar.com
|
||||
- PREVIEWS_ROOT=/var/www/previews
|
||||
- RUNNER_WORK_ROOT=/srv/webklar-preview-runner/work
|
||||
- RUNNER_CACHE_ROOT=/srv/webklar-preview-runner/cache
|
||||
- PREVIEW_SANDBOX_SUBNET=172.30.0.0/24
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
- /var/www/previews:/var/www/previews
|
||||
- /srv/webklar-preview-runner:/srv/webklar-preview-runner
|
||||
networks:
|
||||
- webklar-network
|
||||
- preview-sandbox
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.docker.network=webklar-network
|
||||
- traefik.http.services.preview-runner.loadbalancer.server.port=8081
|
||||
# nur Dienst, kein Router: die Router erzeugt previewDeploy.syncPreviewTls (preview-runner@docker)
|
||||
|
||||
networks:
|
||||
preview-sandbox:
|
||||
external: true
|
||||
```
|
||||
|
||||
(Den `networks:`-Block in die bestehende Datei einfügen, nicht doppelt anlegen.) Keine Appwrite-,
|
||||
SMTP- oder Stripe-Variablen und kein `PREVIEW_ENV_KEY` im Runner.
|
||||
|
||||
Erster Start: `docker compose build preview-runner && docker compose up -d preview-runner`.
|
||||
|
||||
## 3. Portal-Umgebung (Dienst `kundenbereich-server`, Variablen einzeln auflisten)
|
||||
|
||||
```yaml
|
||||
- PREVIEW_RUNNER=${PREVIEW_RUNNER:-off} # L2: off, L3: on
|
||||
- PREVIEW_RUNNER_URL=http://webklar-preview-runner:8080
|
||||
- PREVIEW_RUNNER_TOKEN=${PREVIEW_RUNNER_TOKEN}
|
||||
- PREVIEW_ENV_KEY=${PREVIEW_ENV_KEY} # 32 Byte, Base64 (openssl rand -base64 32)
|
||||
```
|
||||
|
||||
Das Verzeichnis `upload-data/preview-logs` muss persistent sein (liegt in `./build/`).
|
||||
|
||||
## 4. Firewall für die Sandbox-Bridge (Livegang-Schritt, kein Code)
|
||||
|
||||
Zuerst den Bridge-Namen ermitteln: `BR=br-$(docker network inspect preview-sandbox -f '{{.Id}}' | cut -c1-12)`.
|
||||
|
||||
```sh
|
||||
# Sandbox -> Host (INPUT): alles verwerfen, bereits bestehende Antworten erlauben
|
||||
iptables -I INPUT -i $BR -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
|
||||
iptables -A INPUT -i $BR -j DROP
|
||||
|
||||
# Sandbox -> andere Ziele (DOCKER-USER, wird vor Docker-Regeln ausgewertet)
|
||||
iptables -I DOCKER-USER -s 172.30.0.0/24 -d 169.254.169.254 -j DROP # Hetzner-Metadaten
|
||||
iptables -I DOCKER-USER -s 172.30.0.0/24 -d 10.0.0.0/8 -j DROP
|
||||
iptables -I DOCKER-USER -s 172.30.0.0/24 -d 192.168.0.0/16 -j DROP
|
||||
# 172.16.0.0/12 (andere Docker-Netze) sperren, ausser dem eigenen /24; Runner-Zugriff wird nicht gebraucht
|
||||
iptables -I DOCKER-USER -s 172.30.0.0/24 -d 172.30.0.0/24 -j ACCEPT # Dev-zu-Dev im eigenen Netz (R13, bekannt)
|
||||
iptables -I DOCKER-USER -s 172.30.0.0/24 ! -d 172.30.0.0/24 -d 172.16.0.0/12 -j DROP
|
||||
```
|
||||
|
||||
Reihenfolge beachten: `-I` setzt vorne ein, die ACCEPT-Regel für das eigene /24 muss vor dem
|
||||
172.16/12-DROP stehen (also zuletzt mit `-I` eingefügt, wie oben die Zeilen in umgekehrter
|
||||
Wirkungsreihenfolge; nach dem Einrichten mit `iptables -S DOCKER-USER` prüfen). DNS läuft über den
|
||||
Docker-Resolver (127.0.0.11) und bleibt erlaubt, Ausgang ins Internet (npm, Registries) ebenfalls.
|
||||
Regeln persistent machen (z. B. `iptables-persistent`), sonst gehen sie beim Neustart verloren.
|
||||
|
||||
## 5. deploy.sh-Erweiterung (Vorschlag, bewusst NICHT eingespielt)
|
||||
|
||||
Voraussetzung: der Build-Container (`kundenbereich-build`) bekommt zusätzlich
|
||||
`/home/webklar/apps/docker-compose.yml:/compose/docker-compose.yml:ro` eingehängt und hat
|
||||
`docker-cli` samt Compose-Plugin. Der Runner wird nur neu gebaut und neu erstellt, wenn sich
|
||||
`runner/` geändert hat UND `GET /status` eine leere Warteschlange meldet (sonst würden laufende
|
||||
Builds/Dev-Server beendet); andernfalls überspringen und protokollieren, der nächste Deploy versucht
|
||||
es erneut. Der Runner wird VOR dem Portal neu gestartet, damit dessen Dev-Registrierung gleich
|
||||
klappt (der Abgleich im Portal holt Verpasstes ohnehin alle 2 Minuten nach).
|
||||
|
||||
Einfügen direkt vor dem Neustart des Portals; vor dem `git reset --hard` den alten Stand merken
|
||||
(`OLD_HEAD=$(git rev-parse HEAD 2>/dev/null || true)`):
|
||||
|
||||
```sh
|
||||
# --- Preview-Runner (nur wenn runner/ geaendert) ---
|
||||
if [ -S /var/run/docker.sock ] && command -v docker >/dev/null 2>&1 && [ -f /compose/docker-compose.yml ]; then
|
||||
NEW_HEAD=$(git rev-parse HEAD)
|
||||
if [ -z "${OLD_HEAD}" ] || ! git diff --quiet "${OLD_HEAD}" "${NEW_HEAD}" -- runner 2>/dev/null; then
|
||||
QUEUE=$(docker exec webklar-preview-runner sh -c \
|
||||
'wget -qO- --header "Authorization: Bearer $PREVIEW_RUNNER_TOKEN" http://127.0.0.1:8080/status' 2>/dev/null \
|
||||
| sed -n 's/.*"queue":\([0-9]*\).*/\1/p')
|
||||
if [ -z "${QUEUE}" ] || [ "${QUEUE}" = "0" ]; then
|
||||
echo "Preview-Runner: Image neu bauen und Container neu erstellen..."
|
||||
docker build -t webklar-preview-runner:latest /app/runner \
|
||||
&& docker compose -f /compose/docker-compose.yml --project-directory /home/webklar/apps \
|
||||
up -d --no-build --no-deps preview-runner \
|
||||
|| echo "Preview-Runner: Aktualisierung fehlgeschlagen (Portal-Deploy laeuft weiter)"
|
||||
else
|
||||
echo "Preview-Runner: Warteschlange nicht leer (${QUEUE}) - Aktualisierung uebersprungen, naechster Deploy versucht es erneut"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
Hinweise: Ist der Runner noch nicht angelegt (`docker exec` schlägt fehl, `QUEUE` leer), wird gebaut
|
||||
und erstellt, das ist beim ersten Mal gewollt. `wget` ist im Alpine-Image enthalten. Die
|
||||
Status-Abfrage bitte vor dem Einspielen einmal von Hand gegenprüfen.
|
||||
|
||||
## 6. Rückfall
|
||||
|
||||
`PREVIEW_RUNNER=off` und Portal neu starten: Der bisherige Weg läuft unverändert. Dokumente behalten
|
||||
`previewMode`/`previewState`, die Dev-Router verschwinden beim nächsten TLS-Sync.
|
||||
@@ -0,0 +1,6 @@
|
||||
FROM node:20-alpine
|
||||
RUN apk add --no-cache docker-cli git
|
||||
WORKDIR /app
|
||||
COPY package.json ./
|
||||
COPY src ./src
|
||||
CMD ["node","src/index.js"]
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"name": "webklar-preview-runner",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"start": "node src/index.js",
|
||||
"test": "node --test test/*.test.js"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
// Konfiguration aus Umgebungsvariablen. Pfade sind HOST-Pfade (R6): der Runner mountet sie identisch.
|
||||
export function loadConfig(env = process.env) {
|
||||
return {
|
||||
token: env.PREVIEW_RUNNER_TOKEN || '',
|
||||
giteaUrl: env.GITEA_URL || '',
|
||||
giteaToken: env.GITEA_TOKEN || '',
|
||||
baseHost: env.PREVIEW_BASE_HOST || 'project.webklar.com',
|
||||
previewsRoot: env.PREVIEWS_ROOT || '/var/www/previews',
|
||||
workRoot: env.RUNNER_WORK_ROOT || '/srv/webklar-preview-runner/work',
|
||||
cacheRoot: env.RUNNER_CACHE_ROOT || '/srv/webklar-preview-runner/cache',
|
||||
sandboxSubnet: env.PREVIEW_SANDBOX_SUBNET || '172.30.0.0/24',
|
||||
apiPort: 8080,
|
||||
proxyPort: 8081,
|
||||
minFreeBytes: 1.5 * 1024 ** 3,
|
||||
buildTimeoutMs: 15 * 60e3,
|
||||
devIdleMs: 20 * 60e3,
|
||||
devMax: 3,
|
||||
devReadyMs: 180e3,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
// Reine Projekterkennung (Spec 1.2 „Erkennung“): static | build | dev, Paketmanager, Befehle.
|
||||
import { statSync, readFileSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
|
||||
const DEFAULT_OUTPUTS = ['dist', 'build', 'out', 'web-build'];
|
||||
const DEFAULT_PORT = 5173;
|
||||
|
||||
const safeRel = (p) => typeof p === 'string' && p !== '' && p.length <= 200 && !p.startsWith('/')
|
||||
&& !p.includes('\0') && !p.split(/[\\/]/).includes('..');
|
||||
|
||||
function scriptsOf(files) {
|
||||
try {
|
||||
const txt = files.read('package.json');
|
||||
if (txt == null) return {};
|
||||
const s = JSON.parse(txt)?.scripts;
|
||||
return s && typeof s === 'object' && !Array.isArray(s) ? s : {};
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
const hasScript = (scripts, name) => typeof scripts[name] === 'string' && scripts[name].trim() !== '';
|
||||
|
||||
function detectPm(files) {
|
||||
if (files.has('pnpm-lock.yaml')) return 'pnpm';
|
||||
if (files.has('yarn.lock')) return 'yarn';
|
||||
if (files.has('bun.lockb') || files.has('bun.lock')) return 'bun';
|
||||
return 'npm';
|
||||
}
|
||||
|
||||
function installCmd(pm, files) {
|
||||
if (pm === 'npm') {
|
||||
return files.has('package-lock.json') || files.has('npm-shrinkwrap.json')
|
||||
? ['npm', 'ci', '--include=dev']
|
||||
: ['npm', 'install', '--include=dev', '--no-audit', '--no-fund'];
|
||||
}
|
||||
if (pm === 'yarn' && files.has('.yarnrc.yml')) return ['yarn', 'install', '--immutable']; // Yarn Berry
|
||||
return [pm, 'install', '--frozen-lockfile'];
|
||||
}
|
||||
|
||||
// Nur npm braucht `--` vor den Skript-Argumenten, pnpm/yarn/bun reichen sie direkt durch.
|
||||
function devCmdFor(pm, port) {
|
||||
const args = ['--host', '0.0.0.0', '--port', String(port)];
|
||||
return pm === 'npm' ? [pm, 'run', 'dev', '--', ...args] : [pm, 'run', 'dev', ...args];
|
||||
}
|
||||
|
||||
export function detectProject(files, { mode, index, previewJson } = {}) {
|
||||
const pj = previewJson && typeof previewJson === 'object' && !Array.isArray(previewJson) ? previewJson : {};
|
||||
const scripts = scriptsOf(files);
|
||||
const pm = detectPm(files);
|
||||
const devPort = Number.isInteger(pj.devPort) && pj.devPort >= 1024 && pj.devPort <= 65535 ? pj.devPort : DEFAULT_PORT;
|
||||
const outputDirs = safeRel(pj.output) ? [pj.output] : [...DEFAULT_OUTPUTS];
|
||||
const indexFile = [index, pj.index].find(safeRel) || 'index.html';
|
||||
const hasIndex = files.has(indexFile);
|
||||
const hasBuild = hasScript(scripts, 'build');
|
||||
const hasDev = hasScript(scripts, 'dev');
|
||||
const m = mode === 'build' || mode === 'dev' ? mode : 'auto';
|
||||
|
||||
const base = { pm, install: installCmd(pm, files), buildCmd: hasBuild ? [pm, 'run', 'build'] : null, devCmd: hasDev ? devCmdFor(pm, devPort) : null, outputDirs, devPort };
|
||||
const asBuild = () => ({ kind: 'build', ...base });
|
||||
const asDev = () => ({ kind: 'dev', ...base });
|
||||
const asStatic = (hint) => ({ kind: 'static', ...base, install: [], buildCmd: null, devCmd: null, outputDirs: [], index: indexFile, ...(hint ? { hint } : {}) });
|
||||
|
||||
if (m === 'build') return hasBuild ? asBuild() : { ...asBuild(), error: 'Kein build-Skript in package.json' };
|
||||
if (m === 'dev') return hasDev ? asDev() : { ...asDev(), error: 'Kein dev-Skript in package.json' };
|
||||
|
||||
if (hasIndex && !hasBuild) return asStatic();
|
||||
if (hasBuild) return asBuild();
|
||||
if (hasDev) return asDev();
|
||||
return asStatic(`Hinweis: Weder ${indexFile} noch ein build-/dev-Skript gefunden – Inhalt wird unverändert als statische Seite bereitgestellt.`);
|
||||
}
|
||||
|
||||
// Dateisystem-Adapter für detectProject; previewJson wird aus .webklar-preview.json gelesen, wenn nicht übergeben.
|
||||
export function detectFromDir(dir, opts = {}) {
|
||||
const abs = (p) => join(dir, p);
|
||||
const files = {
|
||||
has: (p) => { try { return statSync(abs(p)).isFile(); } catch { return false; } },
|
||||
read: (p) => { try { return readFileSync(abs(p), 'utf8'); } catch { return null; } },
|
||||
};
|
||||
let previewJson = opts.previewJson;
|
||||
if (previewJson === undefined) {
|
||||
try { previewJson = JSON.parse(files.read('.webklar-preview.json')); } catch { previewJson = {}; }
|
||||
}
|
||||
return detectProject(files, { ...opts, previewJson });
|
||||
}
|
||||
@@ -0,0 +1,206 @@
|
||||
// Dev-Server bei Bedarf: Container mit `npm run dev` starten, bei Leerlauf stoppen, max. devMax gleichzeitig (LRU-Verdrängung).
|
||||
import net from 'node:net';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { join } from 'node:path';
|
||||
import * as nodeFs from 'node:fs/promises';
|
||||
import { devRunArgs } from './docker.js';
|
||||
import { readMemAvailable } from './server.js';
|
||||
|
||||
const LOG_LINES = 300;
|
||||
const MAX_LINE = 2000;
|
||||
const POLL_MS = 1000;
|
||||
const RETRY_MS = 60e3;
|
||||
const UID = 1000;
|
||||
const PREFIX = 'preview-dev-';
|
||||
const LIVE_CHECK_MS = 10e3;
|
||||
// I2: Stand der Startparameter (Variablen, Branch, Modus): Aenderung => Neustart
|
||||
const cfgHash = (job) => createHash('sha256').update(JSON.stringify([job.branch, job.mode, Object.entries(job.env || {}).sort(([a], [b]) => (a < b ? -1 : 1))])).digest('hex');
|
||||
const cacheName = (repo) => repo.replace('/', '__');
|
||||
|
||||
async function chownR(p) {
|
||||
const st = await nodeFs.lstat(p);
|
||||
await nodeFs.lchown(p, UID, UID);
|
||||
if (st.isDirectory()) for (const e of await nodeFs.readdir(p)) await chownR(join(p, e));
|
||||
}
|
||||
|
||||
// R7: Container laufen als 1000:1000, also gehören Arbeitsordner und Cache dieser Nutzer-ID.
|
||||
const defaultPrepareDirs = async (workDir, cacheDir) => {
|
||||
await nodeFs.mkdir(cacheDir, { recursive: true });
|
||||
await chownR(workDir);
|
||||
await chownR(cacheDir);
|
||||
};
|
||||
|
||||
// TCP-Verbindung zum Dev-Server (R5: Docker-DNS-Name im Netz preview-sandbox).
|
||||
export function tcpConnect(host, port, timeoutMs = 2000) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const s = net.connect({ host, port });
|
||||
s.setTimeout(timeoutMs);
|
||||
s.on('connect', () => { s.destroy(); resolve(true); });
|
||||
s.on('timeout', () => { s.destroy(); reject(new Error('Zeitüberschreitung')); });
|
||||
s.on('error', (e) => { s.destroy(); reject(e); });
|
||||
});
|
||||
}
|
||||
|
||||
export function createDev({
|
||||
config, docker, jobs, mem = readMemAvailable, now = Date.now, connect = tcpConnect,
|
||||
setTimer = (fn, ms) => setTimeout(fn, ms).unref?.() ?? 0, prepareDirs = defaultPrepareDirs,
|
||||
rmDir = (p) => nodeFs.rm(p, { recursive: true, force: true }),
|
||||
}) {
|
||||
const specs = new Map(); // R12: letzte Job-Spezifikation je sub (bleibt auch im Zustand 'sleeping')
|
||||
const entries = new Map(); // sub ⇒ { sub, state, sha, lastUsed, log, port, job, release, promise, cancelled }
|
||||
const sleep = (ms) => new Promise((r) => setTimer(r, ms));
|
||||
const isRunning = (e) => e.state === 'online' || e.state === 'starting';
|
||||
|
||||
function masker(job) {
|
||||
const secrets = [];
|
||||
if (config.giteaToken) secrets.push(config.giteaToken);
|
||||
for (const v of Object.values(job?.env || {})) if (typeof v === 'string' && v.length >= 4) secrets.push(v);
|
||||
secrets.sort((a, b) => b.length - a.length);
|
||||
return (line) => { let s = String(line).slice(0, MAX_LINE); for (const x of secrets) s = s.split(x).join('•••'); return s; };
|
||||
}
|
||||
|
||||
// Container entfernen, Cache freigeben; Zielzustand 'sleeping' (Leerlauf/Verdrängung) oder Eintrag löschen.
|
||||
async function shutdown(e, finalState) {
|
||||
e.cancelled = true;
|
||||
e.release?.(); e.release = null;
|
||||
if (finalState === 'off') { if (entries.get(e.sub) === e) entries.delete(e.sub); } else { e.state = finalState; e.port = null; }
|
||||
// Nur der Eintrag, der den Container angelegt hat, entfernt ihn (Name ist je sub geteilt). Läuft `docker run` noch,
|
||||
// räumt start() nach der Rückkehr selbst auf.
|
||||
if (!e.stopping) {
|
||||
e.stopping = (async () => {
|
||||
if (!e.created) return;
|
||||
try { const r = await docker.rm(PREFIX + e.sub); if (r && r.code) console.error('dev rm', e.sub, 'Exit-Code', r.code); } catch (err) { console.error('dev rm', e.sub, err); }
|
||||
})();
|
||||
}
|
||||
return e.stopping;
|
||||
}
|
||||
|
||||
async function start(e, job) {
|
||||
const name = PREFIX + e.sub;
|
||||
const mask = masker(job);
|
||||
const log = (l) => { e.log.push(mask(l)); if (e.log.length > LOG_LINES) e.log.splice(0, e.log.length - LOG_LINES); };
|
||||
const fail = async (msg) => {
|
||||
if (e.cancelled) return;
|
||||
log(msg);
|
||||
e.release?.(); e.release = null;
|
||||
e.state = 'failed'; e.port = null;
|
||||
e.failedAt = now();
|
||||
if (e.created) { try { await docker.rm(name); } catch { /* egal */ } }
|
||||
};
|
||||
try {
|
||||
if (e.cancelled) return; // vor dem Start abgebrochen: nichts einreihen (ein laufendes prepareWorkspace ließe sich nicht abbrechen)
|
||||
const workDir = join(config.workRoot, 'dev', e.sub);
|
||||
const cacheDir = join(config.cacheRoot, cacheName(job.repo));
|
||||
// Bleibt ein Rest: ist prepareWorkspace schon gestartet, belegt es den Build-Slot bis zum Ende, auch nach Abbruch.
|
||||
const prep = await jobs.prepareWorkspace(job, { onLine: log, workDir });
|
||||
if (e.cancelled) return;
|
||||
const d = prep.detected;
|
||||
if (!d?.devCmd) return await fail('Kein dev-Skript in package.json');
|
||||
await prepareDirs(prep.workDir ?? workDir, cacheDir);
|
||||
if (e.cancelled) return;
|
||||
const cmd = d.pm === 'pnpm' || d.pm === 'yarn' ? ['corepack', ...d.devCmd] : d.devCmd; // R9
|
||||
const args = devRunArgs({
|
||||
image: d.pm === 'bun' ? 'oven/bun:1' : 'node:20', sub: e.sub, workDir, cacheDir,
|
||||
workRoot: config.workRoot, cacheRoot: config.cacheRoot, cmd, memory: '512m',
|
||||
});
|
||||
// Feste Variable zuletzt, damit Projektvariablen sie nicht überschreiben; Werte laufen per --env-file (R10).
|
||||
const env = { ...job.env, __VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS: `.${config.baseHost}` };
|
||||
e.created = true;
|
||||
const r = await docker.run(args, { env, onLine: (l) => log(l) });
|
||||
if (e.cancelled) { await docker.rm(name).catch(() => {}); return; }
|
||||
if (r.code !== 0) return await fail(`Container konnte nicht gestartet werden (Exit-Code ${r.code})`);
|
||||
e.port = d.devPort;
|
||||
const deadline = now() + config.devReadyMs;
|
||||
while (!e.cancelled) {
|
||||
let ok = false;
|
||||
try { ok = await connect(name, d.devPort); } catch { /* noch nicht bereit */ }
|
||||
if (e.cancelled) return;
|
||||
if (ok) { e.state = 'online'; e.lastUsed = now(); return; }
|
||||
if (now() >= deadline) break;
|
||||
await sleep(POLL_MS);
|
||||
}
|
||||
if (e.cancelled) return;
|
||||
await docker.run(['logs', '--tail', '100', name], { onLine: (l) => log(l) }).catch(() => {});
|
||||
await fail(`Dev-Server nach ${Math.round(config.devReadyMs / 1000)} s nicht erreichbar`);
|
||||
} catch (err) {
|
||||
await fail(err?.message || 'Unbekannter Fehler');
|
||||
}
|
||||
}
|
||||
|
||||
async function ensure(sub, job) {
|
||||
specs.set(sub, job);
|
||||
const cur = entries.get(sub);
|
||||
// I7: ein abgestuerzter/beendeter Container darf nicht als online gelten
|
||||
if (cur && cur.state === 'online' && docker.running && now() - (cur.checkedAt || 0) >= LIVE_CHECK_MS) {
|
||||
let alive = true;
|
||||
try { alive = await docker.running(PREFIX + sub); } catch { /* unklar: als lebendig behandeln */ }
|
||||
cur.checkedAt = now();
|
||||
if (!alive && entries.get(sub) === cur && cur.state === 'online') { console.error('dev', sub, 'Container laeuft nicht mehr - Neustart'); await shutdown(cur, 'sleeping'); }
|
||||
}
|
||||
if (cur && cur.sha === job.sha && cur.cfg === cfgHash(job)) {
|
||||
if (cur.state === 'failed' && now() - cur.failedAt < RETRY_MS) return { state: 'failed' }; // nicht öfter als alle 60 s neu versuchen
|
||||
if (cur.state === 'online') { cur.lastUsed = now(); return { state: 'online', target: `http://${PREFIX}${sub}:${cur.port}` }; }
|
||||
if (cur.state === 'starting') { cur.lastUsed = now(); return { state: 'starting' }; }
|
||||
}
|
||||
let free = 0;
|
||||
try { free = mem(); } catch { /* Fehler = zu wenig Speicher */ }
|
||||
if (!(free >= config.minFreeBytes)) throw Object.assign(new Error('Server ausgelastet'), { status: 503 });
|
||||
const stops = [];
|
||||
if (cur) stops.push(shutdown(cur, 'sleeping'), cur.promise); // neuer sha oder failed/sleeping: frisch starten, erst nach dem alten Eintrag
|
||||
const others = [...entries.values()].filter((e) => e !== cur && isRunning(e)).sort((a, b) => a.lastUsed - b.lastUsed);
|
||||
while (others.length >= config.devMax) stops.push(shutdown(others.shift(), 'sleeping')); // LRU
|
||||
const e = { sub, state: 'starting', sha: job.sha, cfg: cfgHash(job), checkedAt: now(), lastUsed: now(), log: [], port: null, release: jobs.retainCache(job.repo), promise: null, cancelled: false, created: false, stopping: null, failedAt: 0 };
|
||||
entries.set(sub, e);
|
||||
e.promise = Promise.allSettled(stops).then(() => start(e, job));
|
||||
return { state: 'starting' };
|
||||
}
|
||||
|
||||
// R12: Spezifikation merken, ohne zu starten. Läuft der sub mit anderem sha, wird nach den üblichen Regeln neu gestartet.
|
||||
async function register(sub, job) {
|
||||
const cur = entries.get(sub);
|
||||
if (cur && isRunning(cur) && (cur.sha !== job.sha || cur.cfg !== cfgHash(job))) return ensure(sub, job);
|
||||
specs.set(sub, job);
|
||||
if (cur && cur.sha === job.sha && cur.cfg === cfgHash(job) && (cur.state === 'online' || cur.state === 'starting' || cur.state === 'failed')) return { state: cur.state };
|
||||
return { state: 'sleeping' };
|
||||
}
|
||||
// R12: Start mit der gemerkten Spezifikation (Proxy kennt nur die Subdomain).
|
||||
async function wake(sub) {
|
||||
const job = specs.get(sub);
|
||||
if (!job) return { state: 'unknown' };
|
||||
return ensure(sub, job);
|
||||
}
|
||||
|
||||
// I7: Proxy meldet einen nicht erreichbaren Container: als schlafend markieren und entfernen (naechster Aufruf startet neu).
|
||||
async function markDead(sub) {
|
||||
const e = entries.get(sub);
|
||||
if (e && e.state === 'online') await shutdown(e, 'sleeping');
|
||||
}
|
||||
const touch = (sub) => { const e = entries.get(sub); if (e) e.lastUsed = now(); };
|
||||
async function stop(sub) {
|
||||
specs.delete(sub); // 'off': Spezifikation vergessen, bis das Portal neu registriert
|
||||
const e = entries.get(sub);
|
||||
if (!e) return;
|
||||
await shutdown(e, 'off');
|
||||
await e.promise.catch(() => {});
|
||||
if (!entries.has(sub)) await rmDir(join(config.workRoot, 'dev', sub)).catch(() => {}); // Arbeitsordner aufräumen
|
||||
}
|
||||
// R18: Subdomains mit gemerkter Spezifikation (auch schlafende); das Portal registriert fehlende nach.
|
||||
const known = () => [...specs.keys()];
|
||||
const list = () => [...entries.values()].map((e) => ({ sub: e.sub, state: e.state, lastUsed: e.lastUsed, sha: e.sha }));
|
||||
function status(sub) {
|
||||
const e = entries.get(sub);
|
||||
return e ? { state: e.state, sha: e.sha, lastUsed: e.lastUsed, log: [...e.log] } : { state: 'off', log: [] };
|
||||
}
|
||||
async function sweep() {
|
||||
const t = now();
|
||||
await Promise.all([...entries.values()].filter((e) => e.state === 'online' && t - e.lastUsed > config.devIdleMs).map((e) => shutdown(e, 'sleeping')));
|
||||
}
|
||||
// Zustand liegt nur im Speicher: nach Runner-Neustart übrig gebliebene Dev-Container entfernen.
|
||||
async function cleanupOrphans() {
|
||||
const names = (await docker.ps('webklar.preview')).filter((n) => n.startsWith(PREFIX));
|
||||
for (const r of await Promise.allSettled(names.map((n) => docker.rm(n)))) if (r.status === 'rejected') console.error('orphan rm', r.reason);
|
||||
}
|
||||
const idle = () => Promise.all([...entries.values()].map((e) => e.promise)).then(() => {});
|
||||
|
||||
return { ensure, register, wake, touch, markDead, known, stop, list, status, sweep, cleanupOrphans, idle };
|
||||
}
|
||||
@@ -0,0 +1,193 @@
|
||||
// Docker-Adapter: baut `docker run`-Argumente mit festem Sicherheitsprofil und kapselt die docker-CLI (spawn, nie Shell).
|
||||
import { spawn } from 'node:child_process';
|
||||
import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
|
||||
const SUB_RE = /^[a-z0-9-]{1,63}$/;
|
||||
const ENV_NAME_RE = /^[A-Z_][A-Z0-9_]*$/;
|
||||
const NAME_RE = /^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,127}$/;
|
||||
const NETWORK = 'preview-sandbox'; // fest (R10), kein Parameter
|
||||
const IMAGES = ['node:20', 'oven/bun:1'];
|
||||
const MEMORIES = ['1g', '512m'];
|
||||
const MAX_LINE = 64 * 1024;
|
||||
// Feste, unkritische Umgebung im Container (R9): Caches landen im gemounteten Cache-Verzeichnis.
|
||||
const FIXED_ENV = ['HOME=/cache/home', 'npm_config_cache=/cache/npm', 'COREPACK_HOME=/cache/corepack'];
|
||||
const FIXED_NAMES = new Set(['HOME', 'COREPACK_HOME']);
|
||||
|
||||
// Host-Pfad: absolut, keine Sonderzeichen für -v, keine '..'-Segmente, unterhalb der Wurzel (R10).
|
||||
function checkPath(p, root, what) {
|
||||
if (typeof p !== 'string' || typeof root !== 'string' || !p.startsWith('/') || /[:,\0\n\r]/.test(p)
|
||||
|| p.split('/').includes('..') || !(p === root || p.startsWith(root.replace(/\/+$/, '') + '/'))) {
|
||||
throw new Error(`Ungültiger Pfad (${what})`);
|
||||
}
|
||||
}
|
||||
|
||||
function checkEnv(env) {
|
||||
for (const [k, v] of Object.entries(env || {})) {
|
||||
if (!ENV_NAME_RE.test(k) || FIXED_NAMES.has(k)) throw new Error(`Ungültiger Variablenname: ${k}`);
|
||||
if (/[\0\n\r]/.test(String(v))) throw new Error(`Ungültiger Wert für ${k}`);
|
||||
}
|
||||
}
|
||||
|
||||
function securityArgs({ image, memory, name, workDir, cacheDir, workRoot, cacheRoot, env }) {
|
||||
if (!IMAGES.includes(image)) throw new Error('Image nicht erlaubt');
|
||||
if (!MEMORIES.includes(memory)) throw new Error('Ungültiges Speicherlimit');
|
||||
if (typeof name !== 'string' || !NAME_RE.test(name)) throw new Error('Ungültiger Containername');
|
||||
checkPath(workDir, workRoot, 'workDir');
|
||||
checkPath(cacheDir, cacheRoot, 'cacheDir');
|
||||
checkEnv(env); // Werte selbst laufen nie über die Kommandozeile, siehe run() (--env-file)
|
||||
const args = [
|
||||
'--name', name,
|
||||
'--init',
|
||||
'--memory', memory,
|
||||
'--memory-swap', memory,
|
||||
'--cpus', '1',
|
||||
'--pids-limit', '512',
|
||||
'--cap-drop', 'ALL',
|
||||
'--security-opt', 'no-new-privileges',
|
||||
'--user', '1000:1000',
|
||||
'--network', NETWORK,
|
||||
'-v', `${workDir}:/work`,
|
||||
'-v', `${cacheDir}:/cache`,
|
||||
'-w', '/work',
|
||||
];
|
||||
for (const e of FIXED_ENV) args.push('-e', e);
|
||||
return args;
|
||||
}
|
||||
|
||||
// name ist Pflicht (Aufrufer: 'preview-build-<runId>'), damit ein Zeitlimit den Container sicher entfernen kann.
|
||||
export function buildRunArgs({ image = 'node:20', name, workDir, cacheDir, workRoot, cacheRoot, env, cmd = [], memory = '1g' }) {
|
||||
return ['run', '--rm', ...securityArgs({ image, memory, name, workDir, cacheDir, workRoot, cacheRoot, env }), image, ...cmd];
|
||||
}
|
||||
|
||||
export function devRunArgs({ image = 'node:20', sub, workDir, cacheDir, workRoot, cacheRoot, env, cmd = [], memory = '512m' }) {
|
||||
if (typeof sub !== 'string' || !SUB_RE.test(sub)) throw new Error('Ungültige Subdomain');
|
||||
return [
|
||||
'run', '-d', '--restart', 'no', '--label', `webklar.preview=${sub}`,
|
||||
...securityArgs({ image, memory, name: `preview-dev-${sub}`, workDir, cacheDir, workRoot, cacheRoot, env }),
|
||||
image, ...cmd,
|
||||
];
|
||||
}
|
||||
|
||||
export const spawnDocker = (args, opts) => spawn('docker', args, { stdio: ['ignore', 'pipe', 'pipe'], ...opts });
|
||||
|
||||
function lineSplitter(onLine) {
|
||||
let buf = '';
|
||||
return {
|
||||
push(chunk) {
|
||||
buf += chunk;
|
||||
const parts = buf.split(/\r?\n/);
|
||||
buf = parts.pop().slice(0, MAX_LINE);
|
||||
for (const l of parts) onLine(l.length > MAX_LINE ? l.slice(0, MAX_LINE) : l);
|
||||
},
|
||||
flush() { if (buf !== '') onLine(buf); buf = ''; },
|
||||
};
|
||||
}
|
||||
|
||||
const checkName = (n) => { if (typeof n !== 'string' || !NAME_RE.test(n)) throw new Error('Ungültiger Containername'); };
|
||||
|
||||
// Name aus dem Optionsteil (vor dem Image) – nie aus dem Container-Befehl.
|
||||
function optionName(args) {
|
||||
const end = args.findIndex((a) => IMAGES.includes(a));
|
||||
const opts = end === -1 ? args : args.slice(0, end);
|
||||
const i = opts.indexOf('--name');
|
||||
return i !== -1 && NAME_RE.test(opts[i + 1] || '') ? opts[i + 1] : null;
|
||||
}
|
||||
|
||||
export function createDocker({ exec = spawnDocker, baseEnv = process.env, envDir = tmpdir(), rmCapMs = 5000 } = {}) {
|
||||
// CLI-Umgebung NUR aus PATH/DOCKER_HOST des Runners; Job-Variablen gelangen ausschließlich per --env-file (R10).
|
||||
const minimalBase = {};
|
||||
for (const k of ['PATH', 'DOCKER_HOST']) if (baseEnv[k] !== undefined) minimalBase[k] = baseEnv[k];
|
||||
|
||||
function writeEnvFile(env) {
|
||||
checkEnv(env);
|
||||
const dir = mkdtempSync(join(envDir, 'preview-env-'));
|
||||
const file = join(dir, 'env');
|
||||
writeFileSync(file, Object.entries(env).map(([k, v]) => `${k}=${v}\n`).join(''), { mode: 0o600 });
|
||||
return { file, cleanup: () => { try { rmSync(dir, { recursive: true, force: true }); } catch { /* egal */ } } };
|
||||
}
|
||||
|
||||
// docker rm -f mit kurzer Obergrenze abwarten.
|
||||
function forceRemove(name) {
|
||||
return new Promise((resolve) => {
|
||||
const t = setTimeout(resolve, rmCapMs);
|
||||
try {
|
||||
const c = exec(['rm', '-f', name], { env: minimalBase });
|
||||
c.on?.('error', () => { clearTimeout(t); resolve(); });
|
||||
c.on?.('close', () => { clearTimeout(t); resolve(); });
|
||||
} catch { clearTimeout(t); resolve(); }
|
||||
});
|
||||
}
|
||||
|
||||
// onLine(zeile, 'out' | 'err')
|
||||
function run(args, { env = {}, timeoutMs, onLine } = {}) {
|
||||
let envFile = null;
|
||||
let finalArgs = args;
|
||||
if (Object.keys(env).length > 0) {
|
||||
if (args[0] !== 'run') return Promise.reject(new Error('Variablen nur für docker run'));
|
||||
try { envFile = writeEnvFile(env); } catch (e) { return Promise.reject(e); }
|
||||
finalArgs = ['run', '--env-file', envFile.file, ...args.slice(1)];
|
||||
}
|
||||
return new Promise((resolve, reject) => {
|
||||
let child;
|
||||
try { child = exec(finalArgs, { env: { ...minimalBase } }); } catch (e) { envFile?.cleanup(); reject(e); return; }
|
||||
const split = lineSplitter((l) => { if (onLine) onLine(l, 'out'); });
|
||||
const split2 = lineSplitter((l) => { if (onLine) onLine(l, 'err'); });
|
||||
child.stdout?.setEncoding?.('utf8');
|
||||
child.stderr?.setEncoding?.('utf8');
|
||||
child.stdout?.on('data', (d) => split.push(String(d)));
|
||||
child.stderr?.on('data', (d) => split2.push(String(d)));
|
||||
let done = false;
|
||||
let timedOut = false;
|
||||
let timer;
|
||||
const finish = (res) => {
|
||||
if (done) return;
|
||||
done = true;
|
||||
clearTimeout(timer);
|
||||
envFile?.cleanup();
|
||||
split.flush(); split2.flush();
|
||||
resolve(res);
|
||||
};
|
||||
child.on('error', (e) => {
|
||||
if (done) return;
|
||||
if (timedOut) return; // Zeitlimit-Pfad löst selbst auf (124)
|
||||
done = true; clearTimeout(timer); envFile?.cleanup(); reject(e);
|
||||
});
|
||||
child.on('close', (code) => { if (!timedOut) finish({ code: code ?? 1 }); });
|
||||
if (timeoutMs > 0) {
|
||||
timer = setTimeout(async () => {
|
||||
if (done) return;
|
||||
timedOut = true;
|
||||
try { child.kill('SIGKILL'); } catch { /* schon beendet */ }
|
||||
const name = optionName(args);
|
||||
if (name) await forceRemove(name);
|
||||
finish({ code: 124 });
|
||||
}, timeoutMs);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async function capture(args) {
|
||||
const out = [];
|
||||
const r = await run(args, { onLine: (l, st) => { if (st === 'out') out.push(l); } });
|
||||
return { code: r.code, lines: out };
|
||||
}
|
||||
|
||||
return {
|
||||
run,
|
||||
async ps(label) {
|
||||
const { code, lines } = await capture(['ps', '-a', '--filter', `label=${label}`, '--format', '{{.Names}}']);
|
||||
if (code !== 0) throw new Error(`docker ps fehlgeschlagen (${code})`);
|
||||
return lines.map((l) => l.trim()).filter((l) => NAME_RE.test(l));
|
||||
},
|
||||
// Laeuft der Container? (inspect; fehlt er oder ist beendet, dann false)
|
||||
async running(name) {
|
||||
checkName(name);
|
||||
const { code, lines } = await capture(['inspect', '-f', '{{.State.Running}}', name]);
|
||||
return code === 0 && lines.some((l) => l.trim() === 'true');
|
||||
},
|
||||
async rm(name) { checkName(name); return capture(['rm', '-f', name]); },
|
||||
async stop(name) { checkName(name); return capture(['rm', '-f', name]); },
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { loadConfig } from './config.js';
|
||||
import { createApiServer } from './server.js';
|
||||
import { createDocker } from './docker.js';
|
||||
import { createJobs } from './jobs.js';
|
||||
import { createDev } from './dev.js';
|
||||
import { createProxy } from './proxy.js';
|
||||
import os from 'node:os';
|
||||
import { listenAddresses } from './listen.js';
|
||||
|
||||
const config = loadConfig(process.env);
|
||||
const docker = createDocker();
|
||||
const realJobs = createJobs({ config, docker });
|
||||
// Die API antwortet auf POST /jobs/build mit { runId }.
|
||||
const jobs = { submit: (job) => ({ runId: realJobs.submit(job) }), get: realJobs.get, queueLength: realJobs.queueLength };
|
||||
setInterval(() => realJobs.pruneCache().catch((e) => console.error('pruneCache', e)), 24 * 3600e3).unref();
|
||||
const dev = createDev({ config, docker, jobs: realJobs });
|
||||
// Zustand der Dev-Server liegt nur im Speicher: Überbleibsel entfernen, dann jede Minute Leerlauf prüfen.
|
||||
dev.cleanupOrphans().catch((e) => console.error('cleanupOrphans', e));
|
||||
setInterval(() => dev.sweep().catch((e) => console.error('sweep', e)), 60e3).unref();
|
||||
|
||||
// R13: nur auf Adressen außerhalb des Sandbox-Netzes lauschen (kein Zugriff aus Dev-Containern am Traefik-Auth vorbei).
|
||||
const addrs = listenAddresses(os.networkInterfaces(), config.sandboxSubnet);
|
||||
for (const addr of addrs) {
|
||||
createApiServer({ config, jobs, dev }).listen(config.apiPort, addr, () => console.log(`Preview-Runner API auf ${addr}:${config.apiPort}`));
|
||||
createProxy({ config, dev }).listen(config.proxyPort, addr, () => console.log(`Preview-Runner Proxy auf ${addr}:${config.proxyPort}`));
|
||||
}
|
||||
@@ -0,0 +1,315 @@
|
||||
// Build-Aufträge: Einzel-Slot-Warteschlange, Klonen, Erkennung, Installation/Build im Container, atomare Veröffentlichung.
|
||||
import crypto from 'node:crypto';
|
||||
import { spawn } from 'node:child_process';
|
||||
import * as nodeFs from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, dirname, sep } from 'node:path';
|
||||
import { buildRunArgs } from './docker.js';
|
||||
import { detectFromDir } from './detect.js';
|
||||
import { readMemAvailable } from './server.js';
|
||||
|
||||
const LOG_LINES = 300;
|
||||
const MAX_LINE = 2000;
|
||||
const KEEP_MS = 60 * 60e3;
|
||||
const MEM_RETRY_MS = 30e3;
|
||||
const DAY = 24 * 3600e3;
|
||||
const UID = 1000;
|
||||
|
||||
// ---- Dateisystem-Helfer (überschreibbar für Tests) ----
|
||||
async function chownR(p) {
|
||||
const st = await nodeFs.lstat(p);
|
||||
await nodeFs.lchown(p, UID, UID);
|
||||
if (st.isDirectory()) for (const e of await nodeFs.readdir(p)) await chownR(join(p, e));
|
||||
}
|
||||
|
||||
// Kopiert nur Verzeichnisse und reguläre Dateien (keine Symlinks, nie .git), Rechte 755/644 (R7).
|
||||
async function copyTree(fs, src, dst, { skipGit = false } = {}) {
|
||||
await fs.mkdir(dst, { recursive: true });
|
||||
await fs.chmod(dst, 0o755);
|
||||
for (const e of await fs.readdir(src, { withFileTypes: true })) {
|
||||
if (skipGit && e.name === '.git') continue;
|
||||
const s = join(src, e.name);
|
||||
const d = join(dst, e.name);
|
||||
if (e.isDirectory()) await copyTree(fs, s, d);
|
||||
else if (e.isFile()) { await fs.copyFile(s, d); await fs.chmod(d, 0o644); }
|
||||
}
|
||||
}
|
||||
|
||||
async function dirSize(fs, p) {
|
||||
const st = await fs.lstat(p);
|
||||
if (!st.isDirectory()) return st.size;
|
||||
let sum = 0;
|
||||
for (const e of await fs.readdir(p)) sum += await dirSize(fs, join(p, e));
|
||||
return sum;
|
||||
}
|
||||
|
||||
const defaultFs = { ...nodeFs, chownR, copyTree, dirSize: (p) => dirSize(nodeFs, p) };
|
||||
|
||||
// ---- git (R11) ----
|
||||
const CLONE_MAX_MS = 5 * 60e3;
|
||||
export function createGit(config, { spawnFn = spawn } = {}) {
|
||||
return function git({ repo, branch, dir, onLine, timeoutMs = CLONE_MAX_MS }) {
|
||||
const b64 = Buffer.from(`x-access-token:${config.giteaToken}`).toString('base64');
|
||||
const url = `${String(config.giteaUrl).replace(/\/+$/, '')}/${repo}.git`;
|
||||
const args = ['-c', 'http.lowSpeedLimit=1000', '-c', 'http.lowSpeedTime=60', 'clone', '--depth', '1', '--branch', branch, '--', url, dir];
|
||||
const env = { PATH: process.env.PATH || '/usr/bin:/bin', GIT_TERMINAL_PROMPT: '0', GIT_CONFIG_NOSYSTEM: '1', HOME: tmpdir(),
|
||||
// Token nur über die Umgebung des git-Prozesses, nie in argv/.git/config (R11)
|
||||
GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: 'http.extraHeader', GIT_CONFIG_VALUE_0: `Authorization: Basic ${b64}` };
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawnFn('git', args, { env, stdio: ['ignore', 'pipe', 'pipe'] });
|
||||
let timedOut = false;
|
||||
const timer = setTimeout(() => { timedOut = true; try { child.kill('SIGKILL'); } catch { /* beendet */ } reject(new Error('git clone: Zeitlimit überschritten')); }, Math.max(1000, Math.min(timeoutMs, CLONE_MAX_MS)));
|
||||
const feed = (s) => { let buf = ''; s.setEncoding('utf8'); s.on('data', (d) => { const parts = (buf + d).split(/\r?\n|\r/); buf = parts.pop(); parts.forEach((l) => l && onLine?.(l)); }); s.on('end', () => buf && onLine?.(buf)); };
|
||||
feed(child.stdout); feed(child.stderr);
|
||||
child.on('error', (e) => { clearTimeout(timer); reject(e); });
|
||||
child.on('close', (code) => { clearTimeout(timer); if (!timedOut) { if (code === 0) resolve(); else reject(new Error(`git clone fehlgeschlagen (Exit-Code ${code})`)); } });
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
const withCorepack = (pm, cmd) => (pm === 'pnpm' || pm === 'yarn' ? ['corepack', ...cmd] : cmd); // R9
|
||||
const imageFor = (pm) => (pm === 'bun' ? 'oven/bun:1' : 'node:20');
|
||||
const cacheName = (repo) => repo.replace('/', '__');
|
||||
|
||||
export function createJobs({ config, docker, fs: fsOver = {}, git, mem = readMemAvailable, now = Date.now, setTimer = (fn, ms) => setTimeout(fn, ms).unref?.() ?? 0 }) {
|
||||
const fs = { ...defaultFs, ...fsOver };
|
||||
const gitFn = git || createGit(config);
|
||||
const runs = new Map();
|
||||
const queue = [];
|
||||
let active = false;
|
||||
let retry = false;
|
||||
const activeCaches = new Map(); // Cache-Name ⇒ Anzahl Halter
|
||||
|
||||
// ---- Warteschlange mit genau einem aktiven Slot und Speicherwächter ----
|
||||
function pump() {
|
||||
if (active || queue.length === 0) return;
|
||||
let free = 0;
|
||||
try { free = mem(); } catch { /* Fehler = zu wenig Speicher */ }
|
||||
if (!(free >= config.minFreeBytes)) {
|
||||
if (!retry) { retry = true; setTimer(() => { retry = false; pump(); }, MEM_RETRY_MS); }
|
||||
return;
|
||||
}
|
||||
active = true;
|
||||
const task = queue.shift();
|
||||
Promise.resolve().then(task).finally(() => { active = false; pump(); });
|
||||
}
|
||||
const enqueue = (fn) => new Promise((resolve, reject) => {
|
||||
queue.push(() => fn().then(resolve, reject));
|
||||
pump();
|
||||
});
|
||||
|
||||
function purge() {
|
||||
const t = now();
|
||||
for (const [id, r] of runs) if (r.finishedAt && t - r.finishedAt > KEEP_MS) runs.delete(id);
|
||||
}
|
||||
|
||||
// ---- Log: Ringpuffer, Token/Variablenwerte maskiert ----
|
||||
function masker(job) {
|
||||
const secrets = [];
|
||||
const tok = config.giteaToken;
|
||||
if (tok) secrets.push(tok, Buffer.from(`x-access-token:${tok}`).toString('base64'), Buffer.from(`${tok}:`).toString('base64'));
|
||||
for (const v of Object.values(job?.env || {})) if (typeof v === 'string' && v.length >= 4) secrets.push(v);
|
||||
secrets.sort((a, b) => b.length - a.length);
|
||||
return (line) => {
|
||||
let s = String(line).slice(0, MAX_LINE);
|
||||
for (const x of secrets) s = s.split(x).join('•••');
|
||||
return s.replace(/(Authorization:\s*)(Basic|Bearer)\s+\S+/gi, '$1$2 •••').replace(/(https?:\/\/)[^/\s@]+@/g, '$1•••@');
|
||||
};
|
||||
}
|
||||
const makeLog = (log, mask) => (line) => { log.push(mask(line)); if (log.length > LOG_LINES) log.splice(0, log.length - LOG_LINES); };
|
||||
|
||||
// ---- Container-Schritt ----
|
||||
async function containerStep({ job, id, step, cmd, pm, workDir, cacheDir, env, onLine, deadline }) {
|
||||
await fs.mkdir(cacheDir, { recursive: true });
|
||||
const t = new Date(now());
|
||||
await fs.utimes(cacheDir, t, t).catch(() => {}); // Alter = letzte Nutzung
|
||||
await fs.chownR(workDir);
|
||||
await fs.chownR(cacheDir); // R7
|
||||
const remaining = Math.max(1000, deadline - now());
|
||||
const args = buildRunArgs({ image: imageFor(pm), name: `preview-build-${id}-${step}`, workDir, cacheDir, workRoot: config.workRoot, cacheRoot: config.cacheRoot, cmd: withCorepack(pm, cmd), memory: '1g' });
|
||||
return docker.run(args, { env, timeoutMs: remaining, onLine: (l) => onLine(l) });
|
||||
}
|
||||
const failText = (what, code, limit) => (code === 124 ? `${what}: Zeitlimit von ${Math.round(limit / 60e3)} Minuten überschritten` : `${what} fehlgeschlagen (Exit-Code ${code})`);
|
||||
|
||||
// Klonen + Erkennung + Installation (R1); Aufrufer hält den Slot.
|
||||
async function prepare(job, { onLine, workDir, id = crypto.randomUUID(), deadline = now() + config.buildTimeoutMs }) {
|
||||
const cacheDir = join(config.cacheRoot, cacheName(job.repo));
|
||||
await fs.rm(workDir, { recursive: true, force: true });
|
||||
await fs.mkdir(dirname(workDir), { recursive: true });
|
||||
await gitFn({ repo: job.repo, branch: job.branch, dir: workDir, onLine, timeoutMs: Math.max(1000, deadline - now()) });
|
||||
const detected = detectFromDir(workDir, { mode: job.mode, index: job.index });
|
||||
if (detected.error || detected.kind === 'static' || !detected.install?.length) return { workDir, detected };
|
||||
let r = await containerStep({ job, id, step: 'install', cmd: detected.install, pm: detected.pm, workDir, cacheDir, onLine, deadline });
|
||||
// I8: `npm ci` scheitert bei nicht passender Lock-Datei; wie der bisherige Build-Weg auf `npm install` ausweichen.
|
||||
if (r.code !== 0 && detected.pm === 'npm' && detected.install[1] === 'ci') {
|
||||
onLine('npm ci fehlgeschlagen – Wiederholung mit npm install');
|
||||
r = await containerStep({ job, id, step: 'install2', cmd: ['npm', 'install', '--include=dev', '--no-audit', '--no-fund'], pm: detected.pm, workDir, cacheDir, onLine, deadline });
|
||||
}
|
||||
if (r.code !== 0) return { workDir, detected, error: failText('Installation', r.code, config.buildTimeoutMs) };
|
||||
return { workDir, detected };
|
||||
}
|
||||
|
||||
// Cache vor pruneCache schützen; release() gibt ihn frei (idempotent). Zählend, mehrere Halter je Repo möglich.
|
||||
function retainCache(repo) {
|
||||
const n = cacheName(repo);
|
||||
activeCaches.set(n, (activeCaches.get(n) || 0) + 1);
|
||||
let done = false;
|
||||
return () => {
|
||||
if (done) return;
|
||||
done = true;
|
||||
const c = (activeCaches.get(n) || 1) - 1;
|
||||
if (c <= 0) activeCaches.delete(n); else activeCaches.set(n, c);
|
||||
};
|
||||
}
|
||||
async function holdCache(repo, fn) {
|
||||
const release = retainCache(repo);
|
||||
try { return await fn(); } finally { release(); }
|
||||
}
|
||||
|
||||
// ---- Atomare Veröffentlichung (R11) ----
|
||||
async function publish(job, srcDir, { skipGit, onLine }) {
|
||||
const target = join(config.previewsRoot, job.sub);
|
||||
const fresh = `${target}.new`;
|
||||
const old = `${target}.old`;
|
||||
await fs.rm(fresh, { recursive: true, force: true });
|
||||
// Abgebrochener früherer Tausch: fehlt <sub>, ist <sub>.old die letzte gute Kopie – zurückholen.
|
||||
const exists = (p) => fs.lstat(p).then(() => true, () => false);
|
||||
if (!(await exists(target)) && (await exists(old))) await fs.rename(old, target);
|
||||
await fs.rm(old, { recursive: true, force: true });
|
||||
let hadOld = true;
|
||||
try {
|
||||
await fs.mkdir(config.previewsRoot, { recursive: true });
|
||||
await fs.copyTree(fs, srcDir, fresh, { skipGit });
|
||||
try { await fs.rename(target, old); } catch (e) { if (e.code !== 'ENOENT') throw e; hadOld = false; }
|
||||
try { await fs.rename(fresh, target); } catch (e) {
|
||||
if (hadOld) await fs.rename(old, target).catch((e2) => onLine?.(`Rollback fehlgeschlagen: ${e2.message}`));
|
||||
throw e;
|
||||
}
|
||||
} catch (e) {
|
||||
await fs.rm(fresh, { recursive: true, force: true }).catch(() => {});
|
||||
throw e;
|
||||
}
|
||||
// Ab hier ist der Preview live; Aufräumen darf den Lauf nicht scheitern lassen.
|
||||
await fs.rm(old, { recursive: true, force: true }).catch((e) => onLine?.(`Aufräumen fehlgeschlagen: ${e.message}`));
|
||||
}
|
||||
|
||||
async function findOutput(workDir, dirs) {
|
||||
const realWork = await fs.realpath(workDir);
|
||||
for (const d of dirs) {
|
||||
const p = join(workDir, d);
|
||||
try {
|
||||
const st = await fs.lstat(p);
|
||||
if (!st.isDirectory()) continue;
|
||||
const real = await fs.realpath(p);
|
||||
if (real === realWork || !real.startsWith(realWork + sep)) continue;
|
||||
return p;
|
||||
} catch { /* nicht vorhanden */ }
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---- Ablauf eines Build-Laufs ----
|
||||
const execute = (run, job) => holdCache(job.repo, () => executeRun(run, job));
|
||||
async function executeRun(run, job) {
|
||||
const onLine = makeLog(run.log, masker(job));
|
||||
const workDir = join(config.workRoot, 'build', run.runId);
|
||||
const deadline = now() + config.buildTimeoutMs;
|
||||
run.state = 'running';
|
||||
let final;
|
||||
try {
|
||||
const prep = await prepare(job, { onLine, workDir, id: run.runId, deadline });
|
||||
const d = prep.detected;
|
||||
run.detected = d;
|
||||
if (d.error) throw new Error(d.error);
|
||||
if (prep.error) throw new Error(prep.error);
|
||||
if (d.kind === 'static') {
|
||||
if (d.hint) onLine(d.hint);
|
||||
await publish(job, workDir, { skipGit: true, onLine });
|
||||
run.result = 'static';
|
||||
} else if (d.kind === 'dev') {
|
||||
run.result = 'dev';
|
||||
} else {
|
||||
let problem = null;
|
||||
const cacheDir = join(config.cacheRoot, cacheName(job.repo));
|
||||
const r = await containerStep({ job, id: run.runId, step: 'build', cmd: d.buildCmd, pm: d.pm, workDir, cacheDir, env: job.env, onLine, deadline });
|
||||
if (r.code !== 0) problem = failText('Build', r.code, config.buildTimeoutMs);
|
||||
let out = null;
|
||||
if (!problem) {
|
||||
out = await findOutput(workDir, d.outputDirs);
|
||||
if (!out) problem = `Kein Ausgabeordner gefunden (gesucht: ${d.outputDirs.join(', ')})`;
|
||||
}
|
||||
if (problem) {
|
||||
if (job.mode === 'auto' && d.devCmd) { onLine(`${problem} – Dev-Server wird verwendet`); run.result = 'dev'; }
|
||||
else throw new Error(problem);
|
||||
} else {
|
||||
await publish(job, out, { skipGit: false, onLine });
|
||||
run.result = 'build';
|
||||
}
|
||||
}
|
||||
final = 'done';
|
||||
} catch (e) {
|
||||
final = 'failed';
|
||||
run.result = undefined;
|
||||
run.error = masker(job)(e?.message || 'Unbekannter Fehler');
|
||||
onLine(run.error);
|
||||
}
|
||||
// Endzustand erst nach dem Aufräumen veröffentlichen: wer 'done'/'failed' sieht, sieht ein bereinigtes workDir und finishedAt.
|
||||
await fs.rm(workDir, { recursive: true, force: true }).catch(() => {});
|
||||
run.finishedAt = now();
|
||||
run.state = final;
|
||||
}
|
||||
|
||||
function submit(job) {
|
||||
purge();
|
||||
const runId = crypto.randomUUID();
|
||||
const run = { runId, state: 'queued', detected: null, sha: job.sha, log: [], error: null, result: null, finishedAt: 0 };
|
||||
runs.set(runId, run);
|
||||
enqueue(() => execute(run, job));
|
||||
return runId;
|
||||
}
|
||||
|
||||
function get(runId) {
|
||||
purge();
|
||||
const r = runs.get(runId);
|
||||
if (!r) return null;
|
||||
const { finishedAt, ...pub } = r; // eslint-disable-line no-unused-vars
|
||||
return { ...pub, log: [...r.log] };
|
||||
}
|
||||
|
||||
// Für Dev (Task 5): gleicher Slot, Klon + Installation in workDir.
|
||||
function prepareWorkspace(job, { onLine = () => {}, workDir } = {}) {
|
||||
const mask = masker(job);
|
||||
const log = (l) => onLine(mask(l));
|
||||
return enqueue(() => holdCache(job.repo, async () => {
|
||||
const r = await prepare(job, { onLine: log, workDir });
|
||||
if (r.error) throw new Error(r.error);
|
||||
return { workDir: r.workDir, detected: r.detected };
|
||||
}));
|
||||
}
|
||||
|
||||
async function pruneCache() {
|
||||
const root = config.cacheRoot;
|
||||
const maxAge = config.cacheMaxAgeMs ?? 14 * DAY;
|
||||
const maxBytes = config.cacheMaxBytes ?? 5 * 1024 ** 3;
|
||||
let names = [];
|
||||
try { names = await fs.readdir(root); } catch { return; }
|
||||
const items = [];
|
||||
for (const n of names) {
|
||||
if (activeCaches.has(n)) continue;
|
||||
try { const p = join(root, n); const st = await fs.lstat(p); items.push({ p, mtime: st.mtimeMs, size: await fs.dirSize(p) }); } catch { /* weg */ }
|
||||
}
|
||||
let kept = [];
|
||||
for (const it of items) {
|
||||
if (now() - it.mtime > maxAge) await fs.rm(it.p, { recursive: true, force: true }); else kept.push(it);
|
||||
}
|
||||
kept.sort((a, b) => a.mtime - b.mtime);
|
||||
let total = kept.reduce((s, i) => s + i.size, 0);
|
||||
for (const it of kept) {
|
||||
if (total <= maxBytes) break;
|
||||
await fs.rm(it.p, { recursive: true, force: true });
|
||||
total -= it.size;
|
||||
}
|
||||
}
|
||||
|
||||
return { submit, get, queueLength: () => queue.length + (active ? 1 : 0), prepareWorkspace, retainCache, pruneCache };
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
// R13: API und Proxy nur auf Adressen binden, die NICHT im Sandbox-Netz liegen (Dev-Container laufen fremden Code).
|
||||
import os from 'node:os';
|
||||
|
||||
const ipToInt = (ip) => ip.split('.').reduce((a, o) => ((a << 8) | Number(o)) >>> 0, 0);
|
||||
|
||||
export function inSubnet(ip, subnet) {
|
||||
const [base, bits] = subnet.split('/');
|
||||
const n = Number(bits);
|
||||
if (!/^\d+\.\d+\.\d+\.\d+$/.test(base) || !Number.isInteger(n) || n < 0 || n > 32) throw new Error(`Ungültiges Sandbox-Netz: ${subnet}`);
|
||||
const mask = n === 0 ? 0 : (~0 << (32 - n)) >>> 0;
|
||||
return ((ipToInt(ip) & mask) >>> 0) === ((ipToInt(base) & mask) >>> 0);
|
||||
}
|
||||
|
||||
export function listenAddresses(interfaces = os.networkInterfaces(), subnet, { allowLoopback = false } = {}) {
|
||||
const out = [];
|
||||
for (const list of Object.values(interfaces)) {
|
||||
for (const i of list || []) {
|
||||
if (i.family !== 'IPv4' && i.family !== 4) continue;
|
||||
if (i.internal && !allowLoopback) continue;
|
||||
if (inSubnet(i.address, subnet)) continue;
|
||||
if (!out.includes(i.address)) out.push(i.address);
|
||||
}
|
||||
}
|
||||
if (!out.length) throw new Error(`Keine Listen-Adresse außerhalb des Sandbox-Netzes ${subnet} gefunden – Runner startet nicht.`);
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
// Reverse-Proxy für Dev-Previews: Host <sub>.<baseHost> ⇒ laufender Dev-Container (R5: Docker-DNS im Netz preview-sandbox).
|
||||
// Keine Authentifizierung hier: die Hosts liegen hinter Traefiks preview-chain (ForwardAuth). Weitergeleitet wird
|
||||
// ausschließlich an das Ziel, das dev.wake/dev.ensure liefert (preview-dev-<sub>).
|
||||
import http from 'node:http';
|
||||
import { isValidSub } from './validate.js';
|
||||
|
||||
const esc = (s) => String(s).replace(/[&<>"']/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c]));
|
||||
|
||||
function page(title, text, { refresh = false, hint = '' } = {}) {
|
||||
return `<!doctype html><html lang="de"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">${refresh ? '<meta http-equiv="refresh" content="3">' : ''}<title>${esc(title)}</title><style>body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;font-family:system-ui,sans-serif;background:#f5f6f8;color:#1c2330}main{max-width:26rem;padding:2rem;text-align:center}h1{font-size:1.25rem;margin:0 0 .5rem}p{margin:.25rem 0;color:#5b6575}</style></head><body><main><h1>${esc(title)}</h1><p>${esc(text)}</p>${hint ? `<p>${esc(hint)}</p>` : ''}</main></body></html>`;
|
||||
}
|
||||
|
||||
const PAGES = {
|
||||
starting: page('Preview startet …', 'Der Dev-Server wird gestartet. Die Seite lädt automatisch neu.', { refresh: true }),
|
||||
failed: page('Preview konnte nicht starten', 'Der Dev-Server ist nicht hochgekommen.', { hint: 'Details stehen im Protokoll im WOMS.' }),
|
||||
unknown: page('Preview nicht verfügbar', 'Diese Preview ist gerade nicht verfügbar.'),
|
||||
busy: page('Preview nicht verfügbar', 'Der Server ist gerade ausgelastet. Bitte später erneut versuchen.'),
|
||||
};
|
||||
|
||||
function html(res, status, body, extra = {}) {
|
||||
res.writeHead(status, { 'Content-Type': 'text/html; charset=utf-8', 'Content-Length': Buffer.byteLength(body), 'Cache-Control': 'no-store', ...extra });
|
||||
res.end(body);
|
||||
}
|
||||
|
||||
const HOP = ['connection', 'keep-alive', 'proxy-authorization', 'proxy-connection', 'te', 'trailer', 'transfer-encoding', 'upgrade'];
|
||||
// allowHalfOpen: ein vom Client geschlossener Socket ist zwar nicht 'destroyed', aber nicht mehr lesbar
|
||||
const gone = (s) => s.destroyed || !s.readable;
|
||||
const TOUCH_MS = 60e3;
|
||||
|
||||
// Hop-by-hop-Header entfernen (samt der in Connection genannten Tokens). Auf dem Upgrade-Pfad bleiben Upgrade/Connection: Upgrade.
|
||||
export function stripHop(headers, { upgrade = false } = {}) {
|
||||
const out = { ...headers };
|
||||
const listed = String(headers.connection || '').split(',').map((t) => t.trim().toLowerCase()).filter(Boolean);
|
||||
for (const k of Object.keys(out)) if (HOP.includes(k.toLowerCase()) || listed.includes(k.toLowerCase())) delete out[k];
|
||||
if (upgrade) { out.connection = 'Upgrade'; out.upgrade = headers.upgrade; }
|
||||
return out;
|
||||
}
|
||||
|
||||
export function createProxy({ config, dev, request = http.request, upstreamTimeoutMs = 120e3, now = Date.now }) {
|
||||
// Sub aus dem Host-Header: exakt <sub>.<baseHost>, klein geschrieben, Port entfernt.
|
||||
function subOf(hostHeader) {
|
||||
const host = String(hostHeader || '').replace(/:\d+$/, '');
|
||||
const suffix = `.${config.baseHost}`;
|
||||
if (host !== host.toLowerCase() || !host.endsWith(suffix)) return null;
|
||||
const sub = host.slice(0, -suffix.length);
|
||||
return isValidSub(sub) ? sub : null;
|
||||
}
|
||||
|
||||
function fwdHeaders(req, upgrade = false) {
|
||||
const h = stripHop(req.headers, { upgrade });
|
||||
const prior = req.headers['x-forwarded-for'];
|
||||
const ip = req.socket.remoteAddress || '';
|
||||
h['x-forwarded-for'] = prior ? `${prior}, ${ip}` : ip;
|
||||
h['x-forwarded-host'] = req.headers.host || '';
|
||||
h['x-forwarded-proto'] = req.headers['x-forwarded-proto'] || 'https';
|
||||
return h;
|
||||
}
|
||||
|
||||
// Ergebnis: { target } wenn online, sonst { state }.
|
||||
async function resolve(sub) {
|
||||
let r;
|
||||
try { r = await dev.wake(sub); } catch (e) { if (e?.status !== 503) console.error('Proxy wake', sub, e); return { state: e?.status === 503 ? 'busy' : 'unknown' }; }
|
||||
if (r.state === 'online' && r.target) return { target: new URL(r.target) };
|
||||
return { state: r.state };
|
||||
}
|
||||
|
||||
const server = http.createServer(async (req, res) => {
|
||||
try {
|
||||
const sub = subOf(req.headers.host);
|
||||
if (!sub) return html(res, 404, page('Nicht gefunden', 'Diese Adresse gibt es nicht.'));
|
||||
const r = await resolve(sub);
|
||||
if (!r.target) {
|
||||
const key = PAGES[r.state] ? r.state : 'unknown';
|
||||
return html(res, 503, PAGES[key], key === 'starting' ? { 'Retry-After': '3' } : {});
|
||||
}
|
||||
dev.touch(sub);
|
||||
const up = request({
|
||||
hostname: r.target.hostname, port: r.target.port, method: req.method, path: req.url, headers: fwdHeaders(req),
|
||||
}, (ures) => {
|
||||
res.writeHead(ures.statusCode, stripHop(ures.headers));
|
||||
ures.pipe(res);
|
||||
ures.on('error', () => res.destroy());
|
||||
});
|
||||
let timedOut = false;
|
||||
up.setTimeout(upstreamTimeoutMs, () => { timedOut = true; up.destroy(); });
|
||||
up.on('error', (err) => {
|
||||
if (['ECONNREFUSED', 'ENOTFOUND', 'EHOSTUNREACH', 'EAI_AGAIN'].includes(err?.code)) dev.markDead?.(sub)?.catch?.(() => {});
|
||||
if (!res.headersSent) {
|
||||
const body = timedOut ? 'Die Preview antwortet nicht rechtzeitig (504).' : 'Die Preview ist gerade nicht erreichbar (502).';
|
||||
res.writeHead(timedOut ? 504 : 502, { 'Content-Type': 'text/plain; charset=utf-8', 'Content-Length': Buffer.byteLength(body) });
|
||||
res.end(body);
|
||||
} else res.destroy();
|
||||
});
|
||||
res.on('close', () => { if (!res.writableFinished) up.destroy(); });
|
||||
req.pipe(up);
|
||||
} catch (e) {
|
||||
console.error('Proxy-Fehler:', e);
|
||||
if (!res.headersSent) html(res, 500, page('Fehler', 'Interner Fehler.'));
|
||||
else res.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
server.on('upgrade', async (req, socket, head) => {
|
||||
socket.on('error', () => socket.destroy());
|
||||
try {
|
||||
const sub = subOf(req.headers.host);
|
||||
if (!sub) return socket.end('HTTP/1.1 404 Not Found\r\nConnection: close\r\n\r\n');
|
||||
const r = await resolve(sub);
|
||||
if (gone(socket)) return socket.destroy(); // Client hat sich während des Wartens verabschiedet
|
||||
if (!r.target) return socket.end('HTTP/1.1 503 Service Unavailable\r\nConnection: close\r\n\r\n');
|
||||
dev.touch(sub);
|
||||
const up = request({
|
||||
hostname: r.target.hostname, port: r.target.port, method: req.method, path: req.url, headers: fwdHeaders(req, true),
|
||||
});
|
||||
up.on('response', (ures) => { // Upstream lehnt das Upgrade ab: Antwort durchreichen und schließen
|
||||
let raw = `HTTP/1.1 ${ures.statusCode} ${ures.statusMessage}\r\n`;
|
||||
for (let i = 0; i < ures.rawHeaders.length; i += 2) raw += `${ures.rawHeaders[i]}: ${ures.rawHeaders[i + 1]}\r\n`;
|
||||
socket.write(`${raw}\r\n`);
|
||||
ures.pipe(socket);
|
||||
});
|
||||
up.on('upgrade', (ures, usock, uhead) => {
|
||||
if (gone(socket)) { usock.destroy(); up.destroy(); return; }
|
||||
usock.on('error', () => { usock.destroy(); socket.destroy(); });
|
||||
// allowHalfOpen-Sockets bleiben nach 'end' offen: bei Ende oder Schließen einer Seite beide zerstören
|
||||
const both = () => { usock.destroy(); socket.destroy(); };
|
||||
for (const ev of ['end', 'close']) { socket.on(ev, both); usock.on(ev, both); }
|
||||
let raw = `HTTP/1.1 ${ures.statusCode} ${ures.statusMessage}\r\n`;
|
||||
for (let i = 0; i < ures.rawHeaders.length; i += 2) raw += `${ures.rawHeaders[i]}: ${ures.rawHeaders[i + 1]}\r\n`;
|
||||
socket.write(`${raw}\r\n`);
|
||||
if (uhead?.length) socket.write(uhead);
|
||||
if (head?.length) usock.write(head);
|
||||
// Offene Verbindung (z. B. HMR) hält den Dev-Server wach: höchstens einmal pro Minute touch
|
||||
let last = now();
|
||||
const keep = () => { const t = now(); if (t - last >= TOUCH_MS) { last = t; dev.touch(sub); } };
|
||||
usock.on('data', keep); socket.on('data', keep);
|
||||
usock.pipe(socket);
|
||||
socket.pipe(usock);
|
||||
});
|
||||
up.on('error', () => socket.destroy());
|
||||
socket.on('close', () => up.destroy());
|
||||
up.end();
|
||||
} catch (e) {
|
||||
console.error('Proxy-Upgrade-Fehler:', e);
|
||||
socket.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
return server;
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
import http from 'node:http';
|
||||
import crypto from 'node:crypto';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { validateJob, isValidSub, isValidRunId } from './validate.js';
|
||||
|
||||
const MAX_BODY = 256 * 1024;
|
||||
|
||||
export function readMemAvailable(path = '/proc/meminfo') {
|
||||
const m = /^MemAvailable:\s+(\d+)\s+kB/m.exec(readFileSync(path, 'utf8'));
|
||||
return m ? Number(m[1]) * 1024 : 0;
|
||||
}
|
||||
|
||||
function tokenOk(header, token) {
|
||||
if (!token) return false;
|
||||
const m = /^Bearer (.+)$/.exec(header || '');
|
||||
if (!m) return false;
|
||||
const a = crypto.createHash('sha256').update(m[1]).digest();
|
||||
const b = crypto.createHash('sha256').update(token).digest();
|
||||
return crypto.timingSafeEqual(a, b);
|
||||
}
|
||||
|
||||
function send(res, status, obj) {
|
||||
const body = JSON.stringify(obj);
|
||||
res.writeHead(status, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) });
|
||||
res.end(body);
|
||||
}
|
||||
|
||||
function readJson(req) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
let tooBig = false;
|
||||
req.on('data', (c) => {
|
||||
if (tooBig) return; // Rest verwerfen, damit die Antwort noch ausgeliefert wird
|
||||
size += c.length;
|
||||
if (size > MAX_BODY) { tooBig = true; chunks.length = 0; return; }
|
||||
chunks.push(c);
|
||||
});
|
||||
req.on('end', () => {
|
||||
if (tooBig) return reject(Object.assign(new Error('Anfrage zu groß'), { status: 413 }));
|
||||
const raw = Buffer.concat(chunks).toString('utf8');
|
||||
if (!raw) return resolve({});
|
||||
try { resolve(JSON.parse(raw)); } catch { reject(Object.assign(new Error('Ungültiges JSON'), { status: 400 })); }
|
||||
});
|
||||
req.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
export function createApiServer({ config, jobs, dev, mem = readMemAvailable }) {
|
||||
return http.createServer(async (req, res) => {
|
||||
try {
|
||||
if (!tokenOk(req.headers.authorization, config.token)) return send(res, 401, { error: 'Nicht autorisiert' });
|
||||
const url = new URL(req.url, 'http://x');
|
||||
const parts = url.pathname.split('/').filter(Boolean);
|
||||
const method = req.method;
|
||||
|
||||
if (method === 'GET' && url.pathname === '/status') {
|
||||
return send(res, 200, { dev: dev.list(), known: dev.known ? dev.known() : [], queue: jobs.queueLength(), memAvailable: mem() });
|
||||
}
|
||||
if (method === 'POST' && url.pathname === '/jobs/build') {
|
||||
const v = validateJob(await readJson(req));
|
||||
if (!v.ok) return send(res, 400, { error: v.error });
|
||||
return send(res, 200, jobs.submit(v.job));
|
||||
}
|
||||
if (method === 'GET' && parts[0] === 'jobs' && parts.length === 2) {
|
||||
if (!isValidRunId(parts[1])) return send(res, 400, { error: 'Ungültige Lauf-ID' });
|
||||
const run = jobs.get(parts[1]);
|
||||
return run ? send(res, 200, run) : send(res, 404, { error: 'Lauf nicht gefunden' });
|
||||
}
|
||||
if (method === 'PUT' && parts[0] === 'dev' && parts.length === 2) {
|
||||
const sub = parts[1];
|
||||
if (!isValidSub(sub)) return send(res, 400, { error: 'Ungültige Subdomain' });
|
||||
const v = validateJob(await readJson(req));
|
||||
if (!v.ok) return send(res, 400, { error: v.error });
|
||||
if (v.job.sub !== sub) return send(res, 400, { error: 'Subdomain stimmt nicht überein' });
|
||||
return send(res, 200, { state: (await dev.register(sub, v.job)).state });
|
||||
}
|
||||
if (method === 'POST' && parts[0] === 'dev' && parts.length === 3 && ['start', 'stop'].includes(parts[2])) {
|
||||
const sub = parts[1];
|
||||
if (!isValidSub(sub)) return send(res, 400, { error: 'Ungültige Subdomain' });
|
||||
if (parts[2] === 'stop') {
|
||||
await dev.stop(sub);
|
||||
return send(res, 200, { ok: true });
|
||||
}
|
||||
const v = validateJob(await readJson(req));
|
||||
if (!v.ok) return send(res, 400, { error: v.error });
|
||||
if (v.job.sub !== sub) return send(res, 400, { error: 'Subdomain stimmt nicht überein' });
|
||||
return send(res, 200, await dev.ensure(sub, v.job));
|
||||
}
|
||||
return send(res, 404, { error: 'Nicht gefunden' });
|
||||
} catch (e) {
|
||||
const status = e.status || 500;
|
||||
if (status >= 500) console.error('Runner-Fehler:', e);
|
||||
if (!res.headersSent) send(res, status, { error: status === 500 ? 'Interner Fehler' : e.message });
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
// Strikte Eingabeprüfung für Runner-Jobs.
|
||||
const RE_SUB = /^[a-z0-9-]{1,63}$/;
|
||||
const RE_REPO = /^[\w.-]+\/[\w.-]+$/;
|
||||
const RE_SHA = /^[0-9a-f]{7,40}$/;
|
||||
const RE_BRANCH = /^[\w./-]{1,100}$/;
|
||||
const RE_ENV = /^[A-Z_][A-Z0-9_]*$/;
|
||||
const MODES = ['auto', 'build', 'dev'];
|
||||
|
||||
export const isValidRunId = (s) => typeof s === 'string' && /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(s);
|
||||
export const isValidSub = (s) => typeof s === 'string' && RE_SUB.test(s);
|
||||
|
||||
export function validateJob(body) {
|
||||
if (!body || typeof body !== 'object' || Array.isArray(body)) return { ok: false, error: 'Ungültige Anfrage' };
|
||||
const { sub, repo, sha, branch } = body;
|
||||
if (!isValidSub(sub)) return { ok: false, error: 'Ungültige Subdomain' };
|
||||
if (typeof repo !== 'string' || !RE_REPO.test(repo) || repo.split('/').some((x) => x === '.' || x === '..' || x.startsWith('-'))) return { ok: false, error: 'Ungültiges Repository' };
|
||||
if (typeof sha !== 'string' || !RE_SHA.test(sha)) return { ok: false, error: 'Ungültiger Commit' };
|
||||
if (typeof branch !== 'string' || !RE_BRANCH.test(branch) || branch.startsWith('-') || branch.includes('..') || branch.endsWith('/') || branch.endsWith('.lock')) return { ok: false, error: 'Ungültiger Branch' };
|
||||
const mode = body.mode === undefined || body.mode === '' ? 'auto' : body.mode;
|
||||
if (!MODES.includes(mode)) return { ok: false, error: 'Ungültiger Modus' };
|
||||
let index;
|
||||
if (body.index !== undefined && body.index !== '') {
|
||||
const i = body.index;
|
||||
if (typeof i !== 'string' || i.length > 200 || i.startsWith('/') || i.split(/[\\/]/).includes('..') || i.includes('\0')) {
|
||||
return { ok: false, error: 'Ungültiger Startseiten-Pfad' };
|
||||
}
|
||||
index = i;
|
||||
}
|
||||
const env = {};
|
||||
if (body.env !== undefined) {
|
||||
if (!body.env || typeof body.env !== 'object' || Array.isArray(body.env)) return { ok: false, error: 'Ungültige Variablen' };
|
||||
const names = Object.keys(body.env);
|
||||
if (names.length > 30) return { ok: false, error: 'Zu viele Variablen (max. 30)' };
|
||||
for (const n of names) {
|
||||
const v = body.env[n];
|
||||
if (!RE_ENV.test(n)) return { ok: false, error: `Ungültiger Variablenname: ${n}` };
|
||||
if (typeof v !== 'string' || Buffer.byteLength(v) > 4096) return { ok: false, error: `Ungültiger Wert für ${n}` };
|
||||
env[n] = v;
|
||||
}
|
||||
}
|
||||
// Whitelist: keine ungeprüften Client-Felder im Job
|
||||
const job = { sub, repo, sha, branch, mode, env };
|
||||
if (index !== undefined) job.index = index;
|
||||
return { ok: true, job };
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtempSync, writeFileSync, mkdirSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { detectProject, detectFromDir } from '../src/detect.js';
|
||||
|
||||
const fake = (map) => ({
|
||||
has: (p) => Object.hasOwn(map, p),
|
||||
read: (p) => (Object.hasOwn(map, p) ? map[p] : null),
|
||||
});
|
||||
const pkg = (scripts) => JSON.stringify({ scripts });
|
||||
|
||||
test('HTML ohne package.json ⇒ static', () => {
|
||||
const r = detectProject(fake({ 'index.html': '<html>' }), {});
|
||||
assert.equal(r.kind, 'static');
|
||||
assert.equal(r.buildCmd, null);
|
||||
assert.equal(r.devCmd, null);
|
||||
});
|
||||
|
||||
test('Vite mit build ⇒ build, npm', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'vite build', dev: 'vite' }) }), { mode: 'auto' });
|
||||
assert.equal(r.kind, 'build');
|
||||
assert.equal(r.pm, 'npm');
|
||||
assert.deepEqual(r.install, ['npm', 'install', '--include=dev', '--no-audit', '--no-fund']);
|
||||
assert.deepEqual(r.buildCmd, ['npm', 'run', 'build']);
|
||||
assert.deepEqual(r.outputDirs, ['dist', 'build', 'out', 'web-build']);
|
||||
assert.equal(r.devPort, 5173);
|
||||
});
|
||||
|
||||
test('npm mit Lock ⇒ npm ci', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'x' }), 'package-lock.json': '{}' }), {});
|
||||
assert.deepEqual(r.install, ['npm', 'ci', '--include=dev']);
|
||||
});
|
||||
|
||||
test('pnpm-lock ⇒ pnpm', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'x', dev: 'y' }), 'pnpm-lock.yaml': '' }), {});
|
||||
assert.equal(r.pm, 'pnpm');
|
||||
assert.deepEqual(r.install, ['pnpm', 'install', '--frozen-lockfile']);
|
||||
assert.deepEqual(r.buildCmd, ['pnpm', 'run', 'build']);
|
||||
assert.deepEqual(r.devCmd, ['pnpm', 'run', 'dev', '--host', '0.0.0.0', '--port', '5173']);
|
||||
});
|
||||
|
||||
test('yarn Berry (.yarnrc.yml) ⇒ --immutable', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'x' }), 'yarn.lock': '', '.yarnrc.yml': '' }), {});
|
||||
assert.deepEqual(r.install, ['yarn', 'install', '--immutable']);
|
||||
});
|
||||
|
||||
test('yarn.lock und bun.lockb/bun.lock', () => {
|
||||
assert.deepEqual(detectProject(fake({ 'package.json': pkg({ build: 'x' }), 'yarn.lock': '' }), {}).install, ['yarn', 'install', '--frozen-lockfile']);
|
||||
for (const lock of ['bun.lockb', 'bun.lock']) {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'x' }), [lock]: '' }), {});
|
||||
assert.equal(r.pm, 'bun');
|
||||
assert.deepEqual(r.install, ['bun', 'install', '--frozen-lockfile']);
|
||||
assert.deepEqual(r.buildCmd, ['bun', 'run', 'build']);
|
||||
}
|
||||
});
|
||||
|
||||
test('nur dev-Skript ⇒ dev', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ dev: 'vite' }) }), {});
|
||||
assert.equal(r.kind, 'dev');
|
||||
assert.equal(r.buildCmd, null);
|
||||
assert.deepEqual(r.devCmd, ['npm', 'run', 'dev', '--', '--host', '0.0.0.0', '--port', '5173']);
|
||||
});
|
||||
|
||||
test('index.html + build-Skript ⇒ build', () => {
|
||||
const r = detectProject(fake({ 'index.html': 'x', 'package.json': pkg({ build: 'vite build' }) }), {});
|
||||
assert.equal(r.kind, 'build');
|
||||
});
|
||||
|
||||
test('index.html + nur dev-Skript ⇒ static (kein build)', () => {
|
||||
const r = detectProject(fake({ 'index.html': 'x', 'package.json': pkg({ dev: 'vite' }) }), {});
|
||||
assert.equal(r.kind, 'static');
|
||||
});
|
||||
|
||||
test('mode dev erzwingt dev, auch mit build', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'b', dev: 'd' }) }), { mode: 'dev' });
|
||||
assert.equal(r.kind, 'dev');
|
||||
assert.ok(r.devCmd);
|
||||
});
|
||||
|
||||
test('mode dev ohne dev-Skript ⇒ Fehler', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ build: 'b' }) }), { mode: 'dev' });
|
||||
assert.equal(r.kind, 'dev');
|
||||
assert.equal(r.error, 'Kein dev-Skript in package.json');
|
||||
});
|
||||
|
||||
test('mode build ohne build-Skript ⇒ Fehler', () => {
|
||||
const r = detectProject(fake({ 'index.html': 'x', 'package.json': pkg({ dev: 'd' }) }), { mode: 'build' });
|
||||
assert.equal(r.kind, 'build');
|
||||
assert.equal(r.error, 'Kein build-Skript in package.json');
|
||||
});
|
||||
|
||||
test('devPort 3000 aus previewJson landet im devCmd', () => {
|
||||
const r = detectProject(fake({ 'package.json': pkg({ dev: 'd' }) }), { previewJson: { devPort: 3000 } });
|
||||
assert.equal(r.devPort, 3000);
|
||||
assert.deepEqual(r.devCmd.slice(-2), ['--port', '3000']);
|
||||
});
|
||||
|
||||
test('ungültiger devPort ⇒ 5173', () => {
|
||||
for (const p of [80, 70000, 3000.5, '3000', null]) {
|
||||
assert.equal(detectProject(fake({ 'package.json': pkg({ dev: 'd' }) }), { previewJson: { devPort: p } }).devPort, 5173);
|
||||
}
|
||||
});
|
||||
|
||||
test('previewJson.output überschreibt, unsicher wird ignoriert', () => {
|
||||
const f = fake({ 'package.json': pkg({ build: 'b' }) });
|
||||
assert.deepEqual(detectProject(f, { previewJson: { output: 'public/site' } }).outputDirs, ['public/site']);
|
||||
for (const o of ['../x', '/etc', 'a/../b', '', 5]) {
|
||||
assert.deepEqual(detectProject(f, { previewJson: { output: o } }).outputDirs, ['dist', 'build', 'out', 'web-build']);
|
||||
}
|
||||
});
|
||||
|
||||
test('index aus opts/previewJson: eigene Startseite ⇒ static', () => {
|
||||
const f = fake({ 'site/home.html': 'x', 'package.json': pkg({ dev: 'd' }) });
|
||||
const r = detectProject(f, { previewJson: { index: 'site/home.html' } });
|
||||
assert.equal(r.kind, 'static');
|
||||
assert.equal(r.index, 'site/home.html');
|
||||
assert.equal(detectProject(f, { index: '../evil.html' }).kind, 'dev');
|
||||
});
|
||||
|
||||
test('kaputtes package.json wirft nicht', () => {
|
||||
const r = detectProject(fake({ 'package.json': '{kaputt' }), {});
|
||||
assert.equal(r.kind, 'static');
|
||||
assert.ok(r.hint);
|
||||
});
|
||||
|
||||
test('weder index noch Skripte ⇒ static mit Hinweis', () => {
|
||||
const r = detectProject(fake({ 'README.md': 'x' }), {});
|
||||
assert.equal(r.kind, 'static');
|
||||
assert.match(r.hint, /Hinweis|index|Startseite/i);
|
||||
});
|
||||
|
||||
test('detectFromDir liest das Dateisystem', () => {
|
||||
const d = mkdtempSync(join(tmpdir(), 'detect-'));
|
||||
writeFileSync(join(d, 'package.json'), pkg({ build: 'vite build' }));
|
||||
writeFileSync(join(d, 'pnpm-lock.yaml'), '');
|
||||
writeFileSync(join(d, '.webklar-preview.json'), JSON.stringify({ output: 'out2' }));
|
||||
assert.equal(detectFromDir(d, {}).pm, 'pnpm');
|
||||
assert.deepEqual(detectFromDir(d, {}).outputDirs, ['out2']);
|
||||
const e = mkdtempSync(join(tmpdir(), 'detect-'));
|
||||
mkdirSync(join(e, 'index.html'));
|
||||
assert.equal(detectFromDir(e, {}).kind, 'static');
|
||||
assert.equal(detectFromDir(join(e, 'nix'), {}).kind, 'static');
|
||||
});
|
||||
@@ -0,0 +1,354 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { createDev } from '../src/dev.js';
|
||||
|
||||
const MIN = 60e3;
|
||||
const GB = 1024 ** 3;
|
||||
|
||||
function setup({ mem = 10 * GB, connect, detected, prep, rmImpl, alive } = {}) {
|
||||
const config = { workRoot: '/w/work', cacheRoot: '/w/cache', baseHost: 'project.webklar.com', minFreeBytes: 1.5 * GB, devIdleMs: 20 * MIN, devMax: 3, devReadyMs: 180e3, giteaToken: 'tok_GEHEIM' };
|
||||
const clock = { t: 1_000_000 };
|
||||
const rmDirs = []; const runs = []; const removed = []; const held = new Map(); const prepared = [];
|
||||
let psResult = [];
|
||||
const docker = {
|
||||
run: async (args, opts) => { if (args[0] === 'run') runs.push({ args, opts }); else opts?.onLine?.('container-log-zeile', 'out'); return { code: 0 }; },
|
||||
rm: async (n) => { removed.push(n); if (rmImpl) await rmImpl(n); return { code: 0, lines: [] }; },
|
||||
ps: async () => psResult,
|
||||
...(alive ? { running: async (n) => alive(n) } : {}),
|
||||
};
|
||||
const jobs = {
|
||||
prepareWorkspace: async (job, o) => { prepared.push({ job, o }); if (prep) return prep(job, o); o.onLine?.('npm install ok'); return { workDir: o.workDir, detected: detected ?? { pm: 'npm', devCmd: ['npm', 'run', 'dev', '--', '--host', '0.0.0.0', '--port', '5173'], devPort: 5173 } }; },
|
||||
retainCache: (repo) => { held.set(repo, (held.get(repo) || 0) + 1); let d = false; return () => { if (!d) { d = true; held.set(repo, held.get(repo) - 1); } }; },
|
||||
};
|
||||
const connects = [];
|
||||
const dev = createDev({
|
||||
config, docker, jobs, mem: () => (typeof mem === 'function' ? mem() : mem), now: () => clock.t,
|
||||
connect: connect ?? (async (h, p) => { connects.push([h, p]); return true; }),
|
||||
setTimer: (fn, ms) => { clock.t += ms; queueMicrotask(fn); return 0; },
|
||||
prepareDirs: async () => {},
|
||||
rmDir: async (p) => { rmDirs.push(p); },
|
||||
});
|
||||
return { rmDirs, dev, clock, runs, removed, held, prepared, connects, config, setPs: (v) => { psResult = v; } };
|
||||
}
|
||||
const job = (sub, o = {}) => ({ sub, repo: `own/${sub}`, sha: 'abcdef1', branch: 'main', mode: 'dev', env: { API_KEY: 'geheim-wert' }, ...o });
|
||||
|
||||
test('start: starting sofort, dann online mit Docker-DNS-Ziel; Container mit Fixed-Env per env-Datei', async () => {
|
||||
const s = setup();
|
||||
assert.deepEqual(await s.dev.ensure('a', job('a')), { state: 'starting' });
|
||||
await s.dev.idle();
|
||||
assert.deepEqual(s.connects[0], ['preview-dev-a', 5173]);
|
||||
assert.equal(s.runs.length, 1);
|
||||
const { args, opts } = s.runs[0];
|
||||
assert.ok(args.includes('preview-dev-a') && args.includes('512m') && args.includes('-d'));
|
||||
assert.ok(args.includes('/w/work/dev/a:/work'));
|
||||
assert.equal(opts.env.__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS, '.project.webklar.com');
|
||||
assert.equal(opts.env.API_KEY, 'geheim-wert');
|
||||
assert.ok(!args.join(' ').includes('geheim-wert'));
|
||||
assert.deepEqual(await s.dev.ensure('a', job('a')), { state: 'online', target: 'http://preview-dev-a:5173' });
|
||||
assert.equal(s.runs.length, 1);
|
||||
});
|
||||
|
||||
test('gleicher sub zweimal während Start ⇒ ein Container', async () => {
|
||||
const s = setup();
|
||||
const [r1, r2] = await Promise.all([s.dev.ensure('a', job('a')), s.dev.ensure('a', job('a'))]);
|
||||
assert.equal(r1.state, 'starting'); assert.equal(r2.state, 'starting');
|
||||
await s.dev.idle();
|
||||
assert.equal(s.runs.length, 1); assert.equal(s.prepared.length, 1);
|
||||
});
|
||||
|
||||
test('pnpm läuft über corepack, bun nutzt bun-Image', async () => {
|
||||
const s = setup({ detected: { pm: 'pnpm', devCmd: ['pnpm', 'run', 'dev'], devPort: 4000 } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
const a = s.runs[0].args;
|
||||
assert.deepEqual(a.slice(-4), ['corepack', 'pnpm', 'run', 'dev']);
|
||||
assert.deepEqual(s.connects[0], ['preview-dev-a', 4000]);
|
||||
const b = setup({ detected: { pm: 'bun', devCmd: ['bun', 'run', 'dev'], devPort: 5173 } });
|
||||
await b.dev.ensure('a', job('a')); await b.dev.idle();
|
||||
assert.ok(b.runs[0].args.includes('oven/bun:1'));
|
||||
});
|
||||
|
||||
test('4. Start verdrängt den am längsten ungenutzten', async () => {
|
||||
const s = setup();
|
||||
for (const n of ['a', 'b', 'c']) { await s.dev.ensure(n, job(n)); await s.dev.idle(); s.clock.t += MIN; }
|
||||
s.dev.touch('a'); // b ist jetzt der Älteste
|
||||
s.clock.t += MIN;
|
||||
await s.dev.ensure('d', job('d')); await s.dev.idle();
|
||||
assert.ok(s.removed.includes('preview-dev-b'));
|
||||
assert.ok(!s.removed.includes('preview-dev-a'));
|
||||
assert.equal(s.held.get('own/b'), 0);
|
||||
const states = Object.fromEntries(s.dev.list().map((e) => [e.sub, e.state]));
|
||||
assert.equal(states.d, 'online'); assert.equal(states.b, 'sleeping'); assert.equal(states.a, 'online');
|
||||
assert.equal(s.dev.list().filter((e) => e.state === 'online').length, 3);
|
||||
});
|
||||
|
||||
test('sweep: nach 21 min gestoppt, nach 19 min nicht', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
s.clock.t += 19 * MIN; await s.dev.sweep();
|
||||
assert.equal(s.dev.status('a').state, 'online'); assert.equal(s.removed.length, 0);
|
||||
s.clock.t += 2 * MIN; await s.dev.sweep();
|
||||
assert.equal(s.dev.status('a').state, 'sleeping'); assert.deepEqual(s.removed, ['preview-dev-a']);
|
||||
assert.equal(s.held.get('own/a'), 0);
|
||||
});
|
||||
|
||||
test('touch verlängert die Leerlaufzeit', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
s.clock.t += 19 * MIN; s.dev.touch('a'); s.clock.t += 19 * MIN; await s.dev.sweep();
|
||||
assert.equal(s.dev.status('a').state, 'online');
|
||||
});
|
||||
|
||||
test('wenig Speicher ⇒ Fehler Server ausgelastet (503), nichts gestartet', async () => {
|
||||
const s = setup({ mem: 1 * GB });
|
||||
await assert.rejects(() => s.dev.ensure('a', job('a')), (e) => e.message === 'Server ausgelastet' && e.status === 503);
|
||||
assert.equal(s.prepared.length, 0); assert.equal(s.runs.length, 0);
|
||||
});
|
||||
|
||||
test('Speicherwächter gilt nicht für bereits laufenden sub', async () => {
|
||||
let m = 10 * GB;
|
||||
const s = setup({ mem: () => m });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
m = 1 * GB;
|
||||
assert.equal((await s.dev.ensure('a', job('a'))).state, 'online');
|
||||
});
|
||||
|
||||
test('neuer sha ⇒ Neustart mit neuem Code', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.deepEqual(await s.dev.ensure('a', job('a', { sha: '1234567' })), { state: 'starting' });
|
||||
await s.dev.idle();
|
||||
assert.equal(s.runs.length, 2); assert.equal(s.prepared.length, 2);
|
||||
assert.ok(s.removed.includes('preview-dev-a'));
|
||||
assert.equal(s.dev.status('a').sha, '1234567'); assert.equal(s.dev.status('a').state, 'online');
|
||||
assert.equal(s.held.get('own/a'), 1);
|
||||
});
|
||||
|
||||
test('Bereitschafts-Zeitlimit ⇒ failed, Container entfernt, Log vorhanden, Cache frei', async () => {
|
||||
const s = setup({ connect: async () => { throw new Error('ECONNREFUSED'); } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
const st = s.dev.status('a');
|
||||
assert.equal(st.state, 'failed');
|
||||
assert.ok(s.removed.includes('preview-dev-a'));
|
||||
assert.ok(st.log.some((l) => l.includes('180')));
|
||||
assert.ok(st.log.includes('container-log-zeile'));
|
||||
assert.equal(s.held.get('own/a'), 0);
|
||||
});
|
||||
|
||||
test('ohne dev-Skript ⇒ failed „Kein dev-Skript“', async () => {
|
||||
const s = setup({ detected: { pm: 'npm', devCmd: null, devPort: 5173 } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.equal(s.dev.status('a').state, 'failed');
|
||||
assert.ok(s.dev.status('a').log.some((l) => l.includes('Kein dev-Skript')));
|
||||
assert.equal(s.runs.length, 0); assert.equal(s.held.get('own/a'), 0);
|
||||
});
|
||||
|
||||
test('Vorbereitungsfehler ⇒ failed, Cache frei', async () => {
|
||||
const s = setup({ prep: async () => { throw new Error('Installation fehlgeschlagen (Exit-Code 1)'); } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.equal(s.dev.status('a').state, 'failed');
|
||||
assert.equal(s.held.get('own/a'), 0);
|
||||
});
|
||||
|
||||
test('Cache wird gehalten solange der Container läuft, stop gibt frei', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.equal(s.held.get('own/a'), 1);
|
||||
await s.dev.stop('a');
|
||||
assert.equal(s.held.get('own/a'), 0);
|
||||
assert.deepEqual(s.removed.filter((n) => n === 'preview-dev-a').length >= 1, true);
|
||||
assert.equal(s.dev.status('a').state, 'off');
|
||||
assert.deepEqual(s.dev.list(), []);
|
||||
});
|
||||
|
||||
test('stop während des Starts: nie online, kein Container', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a'));
|
||||
await s.dev.stop('a'); await s.dev.idle();
|
||||
assert.equal(s.dev.status('a').state, 'off');
|
||||
assert.equal(s.held.get('own/a'), 0);
|
||||
assert.equal(s.runs.length, 0); // prepare läuft hier evtl. schon (nicht abbrechbar), der Container nie
|
||||
});
|
||||
|
||||
test('Protokoll maskiert Variablenwerte und ist auf 300 Zeilen begrenzt', async () => {
|
||||
const s = setup({ prep: async (j, o) => { for (let i = 0; i < 400; i++) o.onLine(`zeile ${i} geheim-wert`); return { workDir: o.workDir, detected: { pm: 'npm', devCmd: ['npm', 'run', 'dev'], devPort: 5173 } }; } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
const log = s.dev.status('a').log;
|
||||
assert.equal(log.length, 300);
|
||||
assert.ok(log.some((l) => l.includes('zeile 399 •••')));
|
||||
assert.ok(!log.some((l) => l.includes('geheim-wert')));
|
||||
});
|
||||
|
||||
test('Start: verwaiste preview-dev-*-Container werden entfernt', async () => {
|
||||
const s = setup();
|
||||
s.setPs(['preview-dev-x', 'preview-dev-y', 'preview-build-zzz', 'anderer']);
|
||||
await s.dev.cleanupOrphans();
|
||||
assert.deepEqual(s.removed, ['preview-dev-x', 'preview-dev-y']);
|
||||
});
|
||||
|
||||
const tick = () => new Promise((r) => setImmediate(r));
|
||||
|
||||
test('stop während Verdrängungs-rm: kein Cache-Leck, kein prepare, Arbeitsordner entfernt', async () => {
|
||||
let gate; const opened = new Promise((r) => { gate = r; });
|
||||
let slow = false;
|
||||
const s = setup({ rmImpl: async () => { if (slow) await opened; } });
|
||||
for (const n of ['a', 'b', 'c']) { await s.dev.ensure(n, job(n)); await s.dev.idle(); s.clock.t += MIN; }
|
||||
slow = true;
|
||||
await s.dev.ensure('d', job('d')); // verdrängt a, wartet auf dessen rm
|
||||
await tick();
|
||||
const stopping = s.dev.stop('d');
|
||||
gate(); await stopping; await s.dev.idle();
|
||||
assert.equal(s.held.get('own/d'), 0);
|
||||
assert.ok(!s.prepared.some((p) => p.job.sub === 'd'));
|
||||
assert.equal(s.dev.status('d').state, 'off');
|
||||
assert.ok(s.rmDirs.includes('/w/work/dev/d'));
|
||||
});
|
||||
|
||||
test('sha-Wechsel während des Starts: genau ein Container des neuen sha, alter entfernt neuen nie', async () => {
|
||||
let release; const gate = new Promise((r) => { release = r; });
|
||||
let first = true;
|
||||
const s = setup({ prep: async (j, o) => { if (first) { first = false; await gate; } return { workDir: o.workDir, detected: { pm: 'npm', devCmd: ['npm', 'run', 'dev'], devPort: 5173 } }; } });
|
||||
await s.dev.ensure('a', job('a'));
|
||||
await s.dev.ensure('a', job('a', { sha: '1234567' }));
|
||||
release(); await s.dev.idle();
|
||||
assert.equal(s.runs.length, 1);
|
||||
assert.equal(s.removed.length, 0); // alter Eintrag hat nie einen Container angelegt
|
||||
assert.equal(s.dev.status('a').sha, '1234567'); assert.equal(s.dev.status('a').state, 'online');
|
||||
assert.equal(s.held.get('own/a'), 1);
|
||||
});
|
||||
|
||||
test('gleichzeitig sweep + ensure(neuer sha): ein Container, neuer bleibt online', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
s.clock.t += 21 * MIN;
|
||||
const p = Promise.all([s.dev.sweep(), s.dev.ensure('a', job('a', { sha: '1234567' }))]);
|
||||
await p; await s.dev.idle();
|
||||
assert.equal(s.runs.length, 2);
|
||||
assert.equal(s.dev.status('a').state, 'online'); assert.equal(s.dev.status('a').sha, '1234567');
|
||||
assert.equal(s.held.get('own/a'), 1);
|
||||
// das letzte rm gehört dem alten Container und liegt vor dem zweiten run
|
||||
assert.ok(s.removed.every((n) => n === 'preview-dev-a'));
|
||||
});
|
||||
|
||||
test('failed-Eintrag mit gleichem sha: Neuversuch frühestens nach 60 s', async () => {
|
||||
let ok = false;
|
||||
const s = setup({ connect: async () => { if (!ok) throw new Error('nein'); return true; } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.equal(s.dev.status('a').state, 'failed');
|
||||
const prepared = s.prepared.length;
|
||||
assert.deepEqual(await s.dev.ensure('a', job('a')), { state: 'failed' });
|
||||
assert.equal(s.prepared.length, prepared);
|
||||
s.clock.t += 61e3; ok = true;
|
||||
assert.deepEqual(await s.dev.ensure('a', job('a')), { state: 'starting' });
|
||||
await s.dev.idle();
|
||||
assert.equal(s.dev.status('a').state, 'online');
|
||||
});
|
||||
|
||||
test('cleanupOrphans: ein fehlschlagendes rm stoppt die übrigen nicht', async () => {
|
||||
const s = setup({ rmImpl: async (n) => { if (n === 'preview-dev-x') throw new Error('kaputt'); } });
|
||||
s.setPs(['preview-dev-x', 'preview-dev-y']);
|
||||
const orig = console.error; console.error = () => {};
|
||||
try { await s.dev.cleanupOrphans(); } finally { console.error = orig; }
|
||||
assert.deepEqual(s.removed, ['preview-dev-x', 'preview-dev-y']);
|
||||
});
|
||||
|
||||
test('register: speichert Spezifikation ohne Start (sleeping); wake startet damit', async () => {
|
||||
const s = setup();
|
||||
assert.deepEqual(await s.dev.register('a', job('a')), { state: 'sleeping' });
|
||||
assert.equal(s.runs.length, 0); assert.equal(s.prepared.length, 0);
|
||||
assert.deepEqual(await s.dev.wake('a'), { state: 'starting' });
|
||||
await s.dev.idle();
|
||||
assert.equal(s.runs.length, 1);
|
||||
assert.deepEqual(await s.dev.wake('a'), { state: 'online', target: 'http://preview-dev-a:5173' });
|
||||
});
|
||||
|
||||
test('wake: unbekannter sub ⇒ unknown, nach stop ebenfalls', async () => {
|
||||
const s = setup();
|
||||
assert.deepEqual(await s.dev.wake('x'), { state: 'unknown' });
|
||||
await s.dev.register('a', job('a'));
|
||||
await s.dev.stop('a');
|
||||
assert.deepEqual(await s.dev.wake('a'), { state: 'unknown' });
|
||||
});
|
||||
|
||||
test('register: läuft mit gleichem sha ⇒ unverändert; mit anderem sha ⇒ Neustart; wake nutzt neue Spezifikation', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.deepEqual(await s.dev.register('a', job('a')), { state: 'online' });
|
||||
assert.equal(s.runs.length, 1);
|
||||
assert.deepEqual(await s.dev.register('a', job('a', { sha: '1234567' })), { state: 'starting' });
|
||||
await s.dev.idle();
|
||||
assert.equal(s.runs.length, 2);
|
||||
assert.equal(s.prepared[1].job.sha, '1234567');
|
||||
assert.ok(s.removed.includes('preview-dev-a'));
|
||||
});
|
||||
|
||||
test('register: schlafender sub mit neuem sha startet nicht, wake nutzt das neue sha', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
s.clock.t += 21 * MIN; await s.dev.sweep();
|
||||
assert.equal(s.dev.status('a').state, 'sleeping');
|
||||
assert.deepEqual(await s.dev.register('a', job('a', { sha: '7654321' })), { state: 'sleeping' });
|
||||
assert.equal(s.runs.length, 1);
|
||||
await s.dev.wake('a'); await s.dev.idle();
|
||||
assert.equal(s.runs.length, 2);
|
||||
assert.equal(s.prepared[1].job.sha, '7654321');
|
||||
});
|
||||
|
||||
test('I2: gleicher sha, geänderte Variablen ⇒ register/ensure starten neu', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
assert.equal(s.runs.length, 1);
|
||||
// gleiche Variablen (auch andere Reihenfolge): kein Neustart
|
||||
assert.equal((await s.dev.register('a', job('a', { env: { API_KEY: 'geheim-wert' } }))).state, 'online');
|
||||
assert.equal(s.runs.length, 1);
|
||||
assert.equal((await s.dev.register('a', job('a', { env: { API_KEY: 'neuer-wert' } }))).state, 'starting');
|
||||
await s.dev.idle();
|
||||
assert.equal(s.runs.length, 2);
|
||||
assert.ok(s.removed.includes('preview-dev-a'));
|
||||
assert.equal(s.runs[1].opts.env.API_KEY, 'neuer-wert');
|
||||
// Branch-Wechsel ebenfalls
|
||||
await s.dev.ensure('a', job('a', { env: { API_KEY: 'neuer-wert' }, branch: 'dev' })); await s.dev.idle();
|
||||
assert.equal(s.runs.length, 3);
|
||||
});
|
||||
|
||||
test('I7: beendeter Container ⇒ sleeping, entfernt und per wake neu gestartet', async () => {
|
||||
let up = true;
|
||||
const s = setup({ alive: () => up });
|
||||
await s.dev.wake('a').catch(() => {}); // unbekannt
|
||||
await s.dev.register('a', job('a'));
|
||||
await s.dev.wake('a'); await s.dev.idle();
|
||||
assert.equal(s.runs.length, 1);
|
||||
up = false;
|
||||
s.clock.t += 11e3;
|
||||
assert.deepEqual(await s.dev.wake('a'), { state: 'starting' });
|
||||
await s.dev.idle();
|
||||
assert.ok(s.removed.includes('preview-dev-a'));
|
||||
assert.equal(s.runs.length, 2);
|
||||
});
|
||||
|
||||
test('I7: Lebendigkeitsprüfung höchstens alle 10 s', async () => {
|
||||
let checks = 0;
|
||||
const s = setup({ alive: () => { checks++; return true; } });
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
for (let i = 0; i < 5; i++) await s.dev.ensure('a', job('a'));
|
||||
assert.equal(checks, 0);
|
||||
s.clock.t += 11e3;
|
||||
await s.dev.ensure('a', job('a')); await s.dev.ensure('a', job('a'));
|
||||
assert.equal(checks, 1);
|
||||
});
|
||||
|
||||
test('I7: markDead entfernt den Container und setzt sleeping', async () => {
|
||||
const s = setup();
|
||||
await s.dev.ensure('a', job('a')); await s.dev.idle();
|
||||
await s.dev.markDead('a');
|
||||
assert.ok(s.removed.includes('preview-dev-a'));
|
||||
assert.equal(s.dev.list().find((e) => e.sub === 'a').state, 'sleeping');
|
||||
assert.equal((await s.dev.wake('a')).state, 'starting');
|
||||
});
|
||||
|
||||
test('R18: known liefert Subdomains mit Spezifikation (auch ohne Eintrag), stop vergisst sie', async () => {
|
||||
const s = setup();
|
||||
await s.dev.register('a', job('a'));
|
||||
assert.deepEqual(s.dev.known(), ['a']);
|
||||
await s.dev.stop('a');
|
||||
assert.deepEqual(s.dev.known(), []);
|
||||
});
|
||||
@@ -0,0 +1,223 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync, existsSync, statSync } from 'node:fs';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import { PassThrough } from 'node:stream';
|
||||
import { buildRunArgs, devRunArgs, createDocker, spawnDocker } from '../src/docker.js';
|
||||
|
||||
const base = { image: 'node:20', name: 'preview-build-1', workDir: '/srv/w/build/1', cacheDir: '/srv/c/repo', workRoot: '/srv/w', cacheRoot: '/srv/c', cmd: ['npm', 'ci'] };
|
||||
const pair = (args, flag, val) => args.some((a, i) => a === flag && args[i + 1] === val);
|
||||
|
||||
test('buildRunArgs: komplettes Sicherheitsprofil', () => {
|
||||
const a = buildRunArgs(base);
|
||||
assert.deepEqual(a.slice(0, 2), ['run', '--rm']);
|
||||
assert.ok(a.includes('--init'));
|
||||
for (const [f, v] of [['--memory', '1g'], ['--memory-swap', '1g'], ['--name', 'preview-build-1'], ['--cpus', '1'], ['--pids-limit', '512'], ['--cap-drop', 'ALL'], ['--security-opt', 'no-new-privileges'], ['--user', '1000:1000'], ['--network', 'preview-sandbox'], ['-w', '/work'], ['-v', '/srv/w/build/1:/work'], ['-v', '/srv/c/repo:/cache'], ['-e', 'HOME=/cache/home'], ['-e', 'npm_config_cache=/cache/npm'], ['-e', 'COREPACK_HOME=/cache/corepack']]) {
|
||||
assert.ok(pair(a, f, v), `${f} ${v}`);
|
||||
}
|
||||
assert.equal(a.filter((x) => x === '-v').length, 2);
|
||||
assert.deepEqual(a.slice(-3), ['node:20', 'npm', 'ci']);
|
||||
});
|
||||
|
||||
test('buildRunArgs: Variablenwerte nie in args, ungültige Werte/Namen abgelehnt', () => {
|
||||
const a = buildRunArgs({ ...base, env: { API_KEY: 'geheim-wert' } });
|
||||
assert.ok(!a.join(' ').includes('geheim-wert'));
|
||||
assert.throws(() => buildRunArgs({ ...base, env: { 'a b': '1' } }));
|
||||
assert.throws(() => buildRunArgs({ ...base, env: { A: 'x\0y' } }));
|
||||
assert.throws(() => buildRunArgs({ ...base, env: { A: 'x\ny' } }));
|
||||
assert.throws(() => buildRunArgs({ ...base, env: { HOME: 'x' } }));
|
||||
});
|
||||
|
||||
test('buildRunArgs: Name Pflicht, Memory/Image Allowlist, Network fest', () => {
|
||||
assert.throws(() => buildRunArgs({ ...base, name: undefined }));
|
||||
assert.throws(() => buildRunArgs({ ...base, name: '-x' }));
|
||||
assert.throws(() => buildRunArgs({ ...base, memory: '64g' }));
|
||||
assert.throws(() => buildRunArgs({ ...base, image: 'evil/img' }));
|
||||
assert.ok(buildRunArgs({ ...base, image: 'oven/bun:1', memory: '512m' }));
|
||||
assert.ok(pair(buildRunArgs({ ...base, network: 'host' }), '--network', 'preview-sandbox'));
|
||||
});
|
||||
|
||||
test('Pfade: absolut, unter Wurzel, keine Sonderzeichen', () => {
|
||||
for (const workDir of ['rel/x', '/etc', '/srv/w/../x', '/srv/w/a:b', '/srv/w/a,b', '/srv/w/a\nb', '/srv/wx/y', '/srv/w/a\0b', '/srv/w/x/../../etc']) {
|
||||
assert.throws(() => buildRunArgs({ ...base, workDir }), workDir);
|
||||
}
|
||||
assert.throws(() => buildRunArgs({ ...base, cacheDir: '/srv/w/x' }));
|
||||
assert.throws(() => buildRunArgs({ ...base, workRoot: undefined }));
|
||||
});
|
||||
|
||||
test('devRunArgs: detached, Name, Label, ohne --rm, 512m', () => {
|
||||
const a = devRunArgs({ ...base, sub: 'demo-1', memory: undefined, cmd: ['npm', 'run', 'dev'] });
|
||||
assert.ok(!a.includes('--rm'));
|
||||
assert.ok(a.includes('-d'));
|
||||
assert.ok(pair(a, '--name', 'preview-dev-demo-1'));
|
||||
assert.ok(pair(a, '--label', 'webklar.preview=demo-1'));
|
||||
assert.ok(pair(a, '--restart', 'no'));
|
||||
assert.ok(pair(a, '--memory', '512m'));
|
||||
assert.ok(pair(a, '--cap-drop', 'ALL') && pair(a, '--user', '1000:1000'));
|
||||
assert.deepEqual(a.slice(-4), ['node:20', 'npm', 'run', 'dev']);
|
||||
});
|
||||
|
||||
test('devRunArgs: Subdomain wird validiert', () => {
|
||||
for (const sub of ['', 'A', 'a/b', '-x;', 'x'.repeat(64), undefined]) assert.throws(() => devRunArgs({ ...base, sub }), sub);
|
||||
});
|
||||
|
||||
// Fake-Kindprozess
|
||||
function fakeChild() {
|
||||
const c = new EventEmitter();
|
||||
c.stdout = new PassThrough(); c.stderr = new PassThrough();
|
||||
c.killed = []; c.kill = (s) => { c.killed.push(s); };
|
||||
return c;
|
||||
}
|
||||
|
||||
test('run: streamt Zeilen, CLI-Umgebung nur minimalBase, Werte nur in env-file (0600, danach gelöscht)', async () => {
|
||||
const calls = [];
|
||||
let content; let mode; let filePath;
|
||||
const child = fakeChild();
|
||||
const exec = (args, opts) => {
|
||||
calls.push({ args, opts });
|
||||
const i = args.indexOf('--env-file');
|
||||
filePath = args[i + 1]; content = readFileSync(filePath, 'utf8'); mode = statSync(filePath).mode & 0o777;
|
||||
return child;
|
||||
};
|
||||
const docker = createDocker({ exec, baseEnv: { PATH: '/bin', DOCKER_HOST: 'unix:///x', PREVIEW_RUNNER_TOKEN: 'leak' } });
|
||||
const lines = [];
|
||||
const p = docker.run(['run', '--rm', 'img'], { env: { K: 'v', DOCKER_HOST: 'x', PATH: 'y' }, onLine: (l) => lines.push(l) });
|
||||
child.stdout.write('eins\nzw');
|
||||
child.stderr.write('fehler\n');
|
||||
child.stdout.write('ei\n');
|
||||
child.stdout.end(); child.stderr.end();
|
||||
await new Promise((r) => setImmediate(r));
|
||||
child.emit('close', 3);
|
||||
assert.deepEqual(await p, { code: 3 });
|
||||
assert.deepEqual(lines.sort(), ['eins', 'fehler', 'zwei']);
|
||||
assert.deepEqual(calls[0].opts.env, { PATH: '/bin', DOCKER_HOST: 'unix:///x' });
|
||||
assert.equal(content, 'K=v\nDOCKER_HOST=x\nPATH=y\n');
|
||||
assert.equal(mode, 0o600);
|
||||
assert.ok(!calls[0].args.join(' ').includes('=v'));
|
||||
assert.equal(calls[0].args[0], 'run');
|
||||
assert.equal(existsSync(filePath), false);
|
||||
});
|
||||
|
||||
test('run: NUL/Newline im Wert ⇒ Fehler vor dem Start', async () => {
|
||||
let n = 0;
|
||||
const docker = createDocker({ exec: () => { n++; return fakeChild(); } });
|
||||
await assert.rejects(docker.run(['run'], { env: { A: 'x\0' } }));
|
||||
await assert.rejects(docker.run(['run'], { env: { A: 'x\ny' } }));
|
||||
assert.equal(n, 0);
|
||||
});
|
||||
|
||||
test('run: Zeitlimit ⇒ Kill, docker rm -f, code 124', async () => {
|
||||
const calls = [];
|
||||
const child = fakeChild();
|
||||
const docker = createDocker({ rmCapMs: 30, exec: (args) => { calls.push(args); return calls.length === 1 ? child : Object.assign(fakeChild(), { __rm: true }); } });
|
||||
const p = docker.run(['run', '--name', 'job-9', 'node:20', 'x', '--name', 'fake'], { timeoutMs: 20 });
|
||||
const r = await p;
|
||||
assert.deepEqual(r, { code: 124 });
|
||||
assert.deepEqual(child.killed, ['SIGKILL']);
|
||||
assert.deepEqual(calls[1], ['rm', '-f', 'job-9']);
|
||||
});
|
||||
|
||||
test('Zeitlimit: Name nur aus Optionsteil, ohne Namen kein rm', async () => {
|
||||
const calls = [];
|
||||
const docker = createDocker({ rmCapMs: 10, exec: (args) => { calls.push(args); return fakeChild(); } });
|
||||
assert.deepEqual(await docker.run(['run', 'node:20', 'x', '--name', 'fake'], { timeoutMs: 10 }), { code: 124 });
|
||||
assert.equal(calls.length, 1);
|
||||
});
|
||||
|
||||
test('ps: nur stdout, nur gültige Namen', async () => {
|
||||
const exec = () => {
|
||||
const c = fakeChild();
|
||||
setImmediate(() => { c.stderr.write('Warnung: foo\n'); c.stdout.write('preview-dev-a\n'); c.stdout.end(); c.stderr.end(); c.emit('close', 0); });
|
||||
return c;
|
||||
};
|
||||
assert.deepEqual(await createDocker({ exec }).ps('webklar.preview'), ['preview-dev-a']);
|
||||
});
|
||||
|
||||
test('lineSplitter: überlange Zeilen werden gekappt', async () => {
|
||||
const child = fakeChild();
|
||||
const docker = createDocker({ exec: () => child });
|
||||
const lines = [];
|
||||
const p = docker.run(['ps'], { onLine: (l) => lines.push(l) });
|
||||
child.stdout.write('a'.repeat(200000) + '\n');
|
||||
child.stdout.end(); child.stderr.end();
|
||||
await new Promise((r) => setImmediate(r));
|
||||
child.emit('close', 0);
|
||||
await p;
|
||||
assert.ok(lines[0].length <= 65536);
|
||||
});
|
||||
|
||||
test('ps/stop/rm', async () => {
|
||||
const calls = [];
|
||||
const exec = (args) => {
|
||||
calls.push(args);
|
||||
const c = fakeChild();
|
||||
setImmediate(() => { c.stdout.write('preview-dev-a\npreview-dev-b\n\n'); c.stdout.end(); c.stderr.end(); c.emit('close', 0); });
|
||||
return c;
|
||||
};
|
||||
const docker = createDocker({ exec });
|
||||
assert.deepEqual(await docker.ps('webklar.preview'), ['preview-dev-a', 'preview-dev-b']);
|
||||
assert.deepEqual(calls[0], ['ps', '-a', '--filter', 'label=webklar.preview', '--format', '{{.Names}}']);
|
||||
await docker.stop('preview-dev-a');
|
||||
await docker.rm('preview-dev-a');
|
||||
assert.deepEqual(calls[1], ['rm', '-f', 'preview-dev-a']);
|
||||
assert.deepEqual(calls[2], ['rm', '-f', 'preview-dev-a']);
|
||||
assert.equal(typeof spawnDocker, 'function');
|
||||
});
|
||||
|
||||
test('rm/stop: ungültiger Name wird abgelehnt', async () => {
|
||||
const docker = createDocker({ exec: () => { throw new Error('nicht aufrufen'); } });
|
||||
await assert.rejects(docker.rm('--all'));
|
||||
await assert.rejects(docker.stop('a b'));
|
||||
});
|
||||
|
||||
test('Zeitlimit: close direkt nach kill ändert Ergebnis nicht, rm -f wird vor Auflösung abgewartet', async () => {
|
||||
const events = [];
|
||||
const child = fakeChild();
|
||||
child.kill = (s) => { child.killed.push(s); setImmediate(() => child.emit('close', null)); };
|
||||
const rmChild = fakeChild();
|
||||
let calls = 0;
|
||||
const docker = createDocker({ rmCapMs: 1000, exec: (args) => {
|
||||
if (calls++ === 0) return child;
|
||||
events.push('rm-start');
|
||||
setTimeout(() => { events.push('rm-done'); rmChild.emit('close', 0); }, 40);
|
||||
return rmChild;
|
||||
} });
|
||||
const r = await docker.run(['run', '--name', 'job-2', 'node:20', 'x'], { timeoutMs: 10 });
|
||||
events.push('resolved');
|
||||
assert.deepEqual(r, { code: 124 });
|
||||
assert.deepEqual(events, ['rm-start', 'rm-done', 'resolved']);
|
||||
});
|
||||
|
||||
test('spawn-Fehler ⇒ env-file wird entfernt, run lehnt ab', async () => {
|
||||
let filePath;
|
||||
const child = fakeChild();
|
||||
const docker = createDocker({ exec: (args) => { filePath = args[args.indexOf('--env-file') + 1]; return child; } });
|
||||
const p = docker.run(['run', 'node:20'], { env: { K: 'v' } });
|
||||
assert.equal(existsSync(filePath), true);
|
||||
child.emit('error', new Error('ENOENT'));
|
||||
await assert.rejects(p, /ENOENT/);
|
||||
assert.equal(existsSync(filePath), false);
|
||||
});
|
||||
|
||||
test('Zeitlimit mit env-file: Datei wird entfernt', async () => {
|
||||
let filePath;
|
||||
const child = fakeChild();
|
||||
child.kill = () => setImmediate(() => child.emit('close', null));
|
||||
const docker = createDocker({ rmCapMs: 10, exec: (args) => { if (args[0] === 'run') { filePath = args[args.indexOf('--env-file') + 1]; return child; } return fakeChild(); } });
|
||||
assert.deepEqual(await docker.run(['run', '--name', 'j', 'node:20'], { env: { K: 'v' }, timeoutMs: 10 }), { code: 124 });
|
||||
assert.equal(existsSync(filePath), false);
|
||||
});
|
||||
|
||||
test('ps: mit -a, damit beendete Container aufgeräumt werden', async () => {
|
||||
const calls = [];
|
||||
const exec = (args) => { calls.push(args); const c = fakeChild(); setImmediate(() => { c.stdout.end(); c.stderr.end(); c.emit('close', 0); }); return c; };
|
||||
await createDocker({ exec }).ps('webklar.preview');
|
||||
assert.deepEqual(calls[0].slice(0, 2), ['ps', '-a']);
|
||||
});
|
||||
|
||||
test('running: inspect true/false/fehlend', async () => {
|
||||
const mk = (out, code) => () => { const c = fakeChild(); setImmediate(() => { if (out) c.stdout.write(out); c.stdout.end(); c.stderr.end(); c.emit('close', code); }); return c; };
|
||||
assert.equal(await createDocker({ exec: mk('true\n', 0) }).running('preview-dev-a'), true);
|
||||
assert.equal(await createDocker({ exec: mk('false\n', 0) }).running('preview-dev-a'), false);
|
||||
assert.equal(await createDocker({ exec: mk('', 1) }).running('preview-dev-a'), false);
|
||||
});
|
||||
@@ -0,0 +1,256 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, existsSync, readdirSync, statSync, utimesSync, symlinkSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import { PassThrough } from 'node:stream';
|
||||
import { createJobs, createGit } from '../src/jobs.js';
|
||||
|
||||
const TOKEN = 'tok_SECRET_123456';
|
||||
const B64 = Buffer.from(`x-access-token:${TOKEN}`).toString('base64');
|
||||
|
||||
function setup({ files, dockerImpl, mem = () => 10 * 1024 ** 3, cfg = {}, git } = {}) {
|
||||
const root = mkdtempSync(join(tmpdir(), 'jobs-'));
|
||||
const config = { giteaUrl: 'https://git.x', giteaToken: TOKEN, previewsRoot: join(root, 'previews'), workRoot: join(root, 'work'), cacheRoot: join(root, 'cache'), minFreeBytes: 1000, buildTimeoutMs: 900000, ...cfg };
|
||||
mkdirSync(config.previewsRoot, { recursive: true });
|
||||
const calls = [];
|
||||
const chowned = [];
|
||||
const timers = [];
|
||||
const docker = { run: async (args, opts) => { calls.push({ args, opts }); return dockerImpl ? dockerImpl(args, opts) : { code: 0 }; } };
|
||||
const gitFake = git || (async ({ dir }) => { for (const [f, c] of Object.entries(files || {})) { mkdirSync(join(dir, f, '..'), { recursive: true }); writeFileSync(join(dir, f), c); } mkdirSync(join(dir, '.git'), { recursive: true }); writeFileSync(join(dir, '.git', 'config'), 'x'); });
|
||||
const jobs = createJobs({ config, docker, git: gitFake, mem, fs: { chownR: async (p) => { chowned.push(p); } }, setTimer: (fn, ms) => timers.push({ fn, ms }) });
|
||||
return { root, config, calls, chowned, timers, jobs };
|
||||
}
|
||||
const job = (o = {}) => ({ sub: 'demo', repo: 'own/rep', sha: 'abcdef1', branch: 'main', mode: 'auto', env: {}, ...o });
|
||||
const wait = async (jobs, id) => { for (let i = 0; i < 400; i++) { const r = jobs.get(id); if (r.state === 'done' || r.state === 'failed') return r; await new Promise((r2) => setTimeout(r2, 5)); } throw new Error('timeout'); };
|
||||
const workOf = (args) => args.find((a) => a.endsWith(':/work')).split(':')[0];
|
||||
const PKG = (extra = {}) => JSON.stringify({ scripts: { build: 'vite build', ...extra } });
|
||||
|
||||
test('static: kopiert ohne .git, Rechte 755/644', async () => {
|
||||
const s = setup({ files: { 'index.html': 'hi', 'a/b.css': 'x' } });
|
||||
const r = await wait(s.jobs, s.jobs.submit(job()));
|
||||
assert.equal(r.state, 'done'); assert.equal(r.result, 'static');
|
||||
const t = join(s.config.previewsRoot, 'demo');
|
||||
assert.equal(readFileSync(join(t, 'index.html'), 'utf8'), 'hi');
|
||||
assert.ok(!existsSync(join(t, '.git')));
|
||||
assert.equal(statSync(join(t, 'a')).mode & 0o777, 0o755);
|
||||
assert.equal(statSync(join(t, 'a/b.css')).mode & 0o777, 0o644);
|
||||
assert.ok(!existsSync(join(s.config.workRoot, 'build', r.runId)));
|
||||
assert.equal(s.calls.length, 0);
|
||||
});
|
||||
|
||||
test('build: Install+Build im Container (corepack für pnpm), Erfolg ersetzt Ordner atomar', async () => {
|
||||
const s = setup({
|
||||
files: { 'package.json': PKG(), 'pnpm-lock.yaml': '' },
|
||||
dockerImpl: (args) => { if (args.includes('build')) { mkdirSync(join(workOf(args), 'dist'), { recursive: true }); writeFileSync(join(workOf(args), 'dist', 'index.html'), 'NEU'); } return { code: 0 }; },
|
||||
});
|
||||
mkdirSync(join(s.config.previewsRoot, 'demo')); writeFileSync(join(s.config.previewsRoot, 'demo', 'index.html'), 'ALT');
|
||||
const r = await wait(s.jobs, s.jobs.submit(job({ env: { K: 'wert1234' } })));
|
||||
assert.equal(r.state, 'done'); assert.equal(r.result, 'build');
|
||||
assert.equal(readFileSync(join(s.config.previewsRoot, 'demo', 'index.html'), 'utf8'), 'NEU');
|
||||
assert.deepEqual(readdirSync(s.config.previewsRoot), ['demo']);
|
||||
assert.equal(s.calls.length, 2);
|
||||
assert.deepEqual(s.calls[0].args.slice(-4), ['corepack', 'pnpm', 'install', '--frozen-lockfile']);
|
||||
assert.deepEqual(s.calls[1].args.slice(-4), ['corepack', 'pnpm', 'run', 'build']);
|
||||
assert.deepEqual(s.calls[0].opts.env ?? {}, {}); assert.deepEqual(s.calls[1].opts.env, { K: 'wert1234' });
|
||||
assert.ok(s.chowned.some((p) => p.endsWith('own__rep')) && s.chowned.some((p) => p.includes('/build/')));
|
||||
assert.ok(s.calls[0].args.includes(join(s.config.cacheRoot, 'own__rep') + ':/cache'));
|
||||
});
|
||||
|
||||
test('Build-Fehler: alter Ordner byte-gleich, failed + deutscher Text', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG() }, dockerImpl: (args) => ({ code: args.includes('build') ? 2 : 0 }) });
|
||||
mkdirSync(join(s.config.previewsRoot, 'demo')); writeFileSync(join(s.config.previewsRoot, 'demo', 'index.html'), 'ALT');
|
||||
const r = await wait(s.jobs, s.jobs.submit(job()));
|
||||
assert.equal(r.state, 'failed'); assert.match(r.error, /Build fehlgeschlagen \(Exit-Code 2\)/);
|
||||
assert.equal(readFileSync(join(s.config.previewsRoot, 'demo', 'index.html'), 'utf8'), 'ALT');
|
||||
assert.deepEqual(readdirSync(s.config.previewsRoot), ['demo']);
|
||||
});
|
||||
|
||||
test('Zeitlimit (124) ⇒ failed mit Zeitlimit-Text', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG() }, dockerImpl: (args) => ({ code: args.includes('build') ? 124 : 0 }) });
|
||||
const r = await wait(s.jobs, s.jobs.submit(job({ mode: 'build' })));
|
||||
assert.match(r.error, /Zeitlimit/);
|
||||
});
|
||||
|
||||
test('auto mit dev-Skript nach Build-Fehler ⇒ result dev, kein Publish', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG({ dev: 'vite' }) }, dockerImpl: (args) => ({ code: args.includes('build') ? 1 : 0 }) });
|
||||
const r = await wait(s.jobs, s.jobs.submit(job()));
|
||||
assert.equal(r.state, 'done'); assert.equal(r.result, 'dev');
|
||||
assert.deepEqual(readdirSync(s.config.previewsRoot), []);
|
||||
const b = await wait(s.jobs, s.jobs.submit(job({ mode: 'build' })));
|
||||
assert.equal(b.state, 'failed');
|
||||
});
|
||||
|
||||
test('erzwungener Modus mit Erkennungsfehler ⇒ failed', async () => {
|
||||
const s = setup({ files: { 'index.html': 'x' } });
|
||||
const r = await wait(s.jobs, s.jobs.submit(job({ mode: 'build' })));
|
||||
assert.equal(r.state, 'failed'); assert.match(r.error, /build-Skript/);
|
||||
});
|
||||
|
||||
test('Ausgabeordner als Symlink nach außen wird ignoriert', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG() }, dockerImpl: (args) => { if (args.includes('build')) symlinkSync('/etc', join(workOf(args), 'dist')); return { code: 0 }; } });
|
||||
const r = await wait(s.jobs, s.jobs.submit(job({ mode: 'build' })));
|
||||
assert.equal(r.state, 'failed'); assert.match(r.error, /Ausgabeordner/);
|
||||
});
|
||||
|
||||
test('Token nie im Log (auch base64), Log auf 300 Zeilen gekürzt', async () => {
|
||||
const s = setup({
|
||||
files: { 'package.json': PKG() },
|
||||
git: async ({ onLine }) => { onLine(`fatal: url https://x-access-token:${TOKEN}@git.x/own/rep`); onLine(`Authorization: Basic ${B64}`); onLine(`raw ${B64}`); throw new Error(`clone ${TOKEN} kaputt`); },
|
||||
});
|
||||
const r = await wait(s.jobs, s.jobs.submit(job()));
|
||||
assert.equal(r.state, 'failed');
|
||||
const all = r.log.join('\n') + r.error;
|
||||
assert.ok(!all.includes(TOKEN) && !all.includes(B64));
|
||||
const s2 = setup({ files: { 'package.json': PKG() }, dockerImpl: (a, o) => { for (let i = 0; i < 1000; i++) o.onLine(`zeile ${i}`, 'out'); return { code: 1 }; } });
|
||||
const r2 = await wait(s2.jobs, s2.jobs.submit(job({ mode: 'build' })));
|
||||
assert.equal(r2.log.length, 300);
|
||||
assert.equal(r2.log.filter((l) => l === 'zeile 999').length >= 1, true);
|
||||
});
|
||||
|
||||
test('zwei Aufträge laufen nacheinander', async () => {
|
||||
let running = 0; let maxRunning = 0;
|
||||
const s = setup({ files: { 'index.html': 'x' }, git: async ({ dir }) => { running++; maxRunning = Math.max(maxRunning, running); await new Promise((r) => setTimeout(r, 20)); mkdirSync(dir, { recursive: true }); writeFileSync(join(dir, 'index.html'), 'x'); running--; } });
|
||||
const a = s.jobs.submit(job({ sub: 'a' })); const b = s.jobs.submit(job({ sub: 'b' }));
|
||||
assert.equal(s.jobs.queueLength(), 2);
|
||||
await wait(s.jobs, a); await wait(s.jobs, b);
|
||||
assert.equal(maxRunning, 1);
|
||||
assert.ok(existsSync(join(s.config.previewsRoot, 'a')) && existsSync(join(s.config.previewsRoot, 'b')));
|
||||
});
|
||||
|
||||
test('wenig Speicher ⇒ bleibt queued, Wiederholung nach 30 s', async () => {
|
||||
let free = 10; // zu wenig: minFreeBytes 1000
|
||||
const s2 = setup({ files: { 'index.html': 'x' }, mem: () => free, cfg: { minFreeBytes: 1000 } });
|
||||
const id = s2.jobs.submit(job());
|
||||
assert.equal(s2.jobs.get(id).state, 'queued');
|
||||
assert.equal(s2.timers.length, 1); assert.equal(s2.timers[0].ms, 30000);
|
||||
free = 5000; s2.timers[0].fn();
|
||||
assert.equal((await wait(s2.jobs, id)).state, 'done');
|
||||
});
|
||||
|
||||
test('prepareWorkspace: Klon + Installation in angegebenem workDir, bleibt bestehen', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG({ dev: 'vite' }) } });
|
||||
const wd = join(s.config.workRoot, 'dev', 'demo');
|
||||
const r = await s.jobs.prepareWorkspace(job(), { workDir: wd });
|
||||
assert.equal(r.workDir, wd); assert.equal(r.detected.kind, 'build');
|
||||
assert.ok(existsSync(join(wd, 'package.json')));
|
||||
assert.equal(s.calls.length, 1);
|
||||
assert.ok(s.calls[0].args.includes('install'));
|
||||
});
|
||||
|
||||
test('prepareWorkspace: Installationsfehler ⇒ reject', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG() }, dockerImpl: () => ({ code: 1 }) });
|
||||
await assert.rejects(s.jobs.prepareWorkspace(job(), { workDir: join(s.config.workRoot, 'dev', 'demo') }), /Installation fehlgeschlagen/);
|
||||
});
|
||||
|
||||
test('erledigte Läufe werden nach 1 h verworfen', async () => {
|
||||
let t = 1000;
|
||||
const root = mkdtempSync(join(tmpdir(), 'jobs-'));
|
||||
const config = { giteaUrl: 'x', giteaToken: 't', previewsRoot: join(root, 'p'), workRoot: join(root, 'w'), cacheRoot: join(root, 'c'), minFreeBytes: 0, buildTimeoutMs: 1000 };
|
||||
const jobs = createJobs({ config, docker: {}, git: async ({ dir }) => { mkdirSync(dir, { recursive: true }); writeFileSync(join(dir, 'index.html'), 'x'); }, mem: () => 1e12, now: () => t, fs: { chownR: async () => {} } });
|
||||
const id = jobs.submit(job());
|
||||
await wait(jobs, id);
|
||||
t += 3600e3 - 1; assert.ok(jobs.get(id));
|
||||
t += 2; assert.equal(jobs.get(id), null);
|
||||
});
|
||||
|
||||
test('pruneCache: > 14 Tage weg, dann älteste bis <= Limit', async () => {
|
||||
const s = setup({ cfg: { cacheMaxBytes: 250 } });
|
||||
const mk = (n, bytes, ageDays) => { const d = join(s.config.cacheRoot, n); mkdirSync(d, { recursive: true }); writeFileSync(join(d, 'f'), Buffer.alloc(bytes)); const t = (Date.now() - ageDays * 864e5) / 1000; utimesSync(d, t, t); };
|
||||
mk('old__r', 10, 20); mk('a__r', 100, 5); mk('b__r', 100, 3); mk('c__r', 100, 1);
|
||||
await s.jobs.pruneCache();
|
||||
assert.deepEqual(readdirSync(s.config.cacheRoot).sort(), ['b__r', 'c__r']);
|
||||
rmSync(s.root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function fakeSpawn() {
|
||||
const calls = [];
|
||||
const fn = (cmd, args, opts) => {
|
||||
const c = new EventEmitter(); c.stdout = new PassThrough(); c.stderr = new PassThrough(); c.killed = null;
|
||||
c.kill = (sig) => { c.killed = sig; };
|
||||
calls.push({ cmd, args, opts, child: c });
|
||||
return c;
|
||||
};
|
||||
return { fn, calls };
|
||||
}
|
||||
|
||||
test('createGit: Token nicht in argv/URL, Umgebung minimal + GIT_CONFIG_*', async () => {
|
||||
const sp = fakeSpawn();
|
||||
const git = createGit({ giteaUrl: 'https://git.x/', giteaToken: TOKEN }, { spawnFn: sp.fn });
|
||||
const p = git({ repo: 'own/rep', branch: 'main', dir: '/w/x' });
|
||||
const { args, opts, child } = sp.calls[0];
|
||||
assert.ok(!args.join(' ').includes(TOKEN) && !args.join(' ').includes(B64));
|
||||
assert.ok(args.includes('https://git.x/own/rep.git'));
|
||||
assert.deepEqual(args.slice(args.indexOf('clone')), ['clone', '--depth', '1', '--branch', 'main', '--', 'https://git.x/own/rep.git', '/w/x']);
|
||||
assert.ok(args.includes('http.lowSpeedLimit=1000') && args.includes('http.lowSpeedTime=60'));
|
||||
assert.deepEqual(Object.keys(opts.env).sort(), ['GIT_CONFIG_COUNT', 'GIT_CONFIG_KEY_0', 'GIT_CONFIG_NOSYSTEM', 'GIT_CONFIG_VALUE_0', 'GIT_TERMINAL_PROMPT', 'HOME', 'PATH']);
|
||||
assert.equal(opts.env.GIT_CONFIG_KEY_0, 'http.extraHeader');
|
||||
assert.equal(opts.env.GIT_CONFIG_VALUE_0, `Authorization: Basic ${B64}`);
|
||||
child.emit('close', 0);
|
||||
await p;
|
||||
});
|
||||
|
||||
test('createGit: Zeitlimit beendet git mit SIGKILL und lehnt ab', async () => {
|
||||
const sp = fakeSpawn();
|
||||
const git = createGit({ giteaUrl: 'https://git.x', giteaToken: TOKEN }, { spawnFn: sp.fn });
|
||||
await assert.rejects(git({ repo: 'own/rep', branch: 'main', dir: '/w/x', timeoutMs: 1000 }), /Zeitlimit/);
|
||||
assert.equal(sp.calls[0].child.killed, 'SIGKILL');
|
||||
});
|
||||
|
||||
test('publish: fehlt <sub> und nur .old existiert, wird .old zurückgeholt (Build-Fehler danach)', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG() }, dockerImpl: (args) => ({ code: args.includes('build') ? 2 : 0 }) });
|
||||
// Zustand nach Absturz: nur demo.old; Erfolg-Pfad über static
|
||||
mkdirSync(join(s.config.previewsRoot, 'demo.old')); writeFileSync(join(s.config.previewsRoot, 'demo.old', 'index.html'), 'GUT');
|
||||
const s2 = setup({ files: { 'index.html': 'NEU' } });
|
||||
mkdirSync(join(s2.config.previewsRoot, 'demo.old')); writeFileSync(join(s2.config.previewsRoot, 'demo.old', 'index.html'), 'GUT');
|
||||
const r = await wait(s2.jobs, s2.jobs.submit(job()));
|
||||
assert.equal(r.state, 'done');
|
||||
assert.equal(readFileSync(join(s2.config.previewsRoot, 'demo', 'index.html'), 'utf8'), 'NEU');
|
||||
});
|
||||
|
||||
test('publish: Aufräumfehler nach dem Tausch lässt den Lauf nicht scheitern', async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'jobs-'));
|
||||
const config = { giteaUrl: 'x', giteaToken: 't', previewsRoot: join(root, 'p'), workRoot: join(root, 'w'), cacheRoot: join(root, 'c'), minFreeBytes: 0, buildTimeoutMs: 1000 };
|
||||
mkdirSync(join(config.previewsRoot, 'demo'), { recursive: true });
|
||||
const jobs = createJobs({ config, docker: {}, git: async ({ dir }) => { mkdirSync(dir, { recursive: true }); writeFileSync(join(dir, 'index.html'), 'N'); }, mem: () => 1e12,
|
||||
fs: { chownR: async () => {}, rm: async (p, o) => { if (p.endsWith('demo.old') && existsSync(p)) throw new Error('boom'); return (await import('node:fs/promises')).rm(p, o); } } });
|
||||
const r = await wait(jobs, jobs.submit(job()));
|
||||
assert.equal(r.state, 'done');
|
||||
assert.equal(readFileSync(join(config.previewsRoot, 'demo', 'index.html'), 'utf8'), 'N');
|
||||
assert.ok(r.log.some((l) => /Aufräumen fehlgeschlagen/.test(l)));
|
||||
});
|
||||
|
||||
test('mem() wirft ⇒ wie wenig Speicher; Cache während des Laufs vor pruneCache geschützt', async () => {
|
||||
let boom = true; let s; let during;
|
||||
s = setup({ files: { 'package.json': PKG() }, mem: () => { if (boom) throw new Error('x'); return 1e12; },
|
||||
dockerImpl: async () => { const d = join(s.config.cacheRoot, 'own__rep'); const t = (Date.now() - 40 * 864e5) / 1000; utimesSync(d, t, t); await s.jobs.pruneCache(); during = existsSync(d); return { code: 1 }; } });
|
||||
const id = s.jobs.submit(job({ mode: 'build' }));
|
||||
assert.equal(s.jobs.get(id).state, 'queued');
|
||||
boom = false; s.timers[0].fn();
|
||||
await wait(s.jobs, id);
|
||||
assert.equal(during, true);
|
||||
});
|
||||
|
||||
test('I8: npm ci scheitert ⇒ Wiederholung mit npm install (Protokoll, Build läuft weiter)', async () => {
|
||||
const s = setup({
|
||||
files: { 'package.json': PKG(), 'package-lock.json': '{}' },
|
||||
dockerImpl: (args) => {
|
||||
if (args.includes('ci')) return { code: 1 };
|
||||
if (args.includes('build')) { mkdirSync(join(workOf(args), 'dist'), { recursive: true }); writeFileSync(join(workOf(args), 'dist', 'index.html'), 'N'); }
|
||||
return { code: 0 };
|
||||
},
|
||||
});
|
||||
const r = await wait(s.jobs, s.jobs.submit(job()));
|
||||
assert.equal(r.state, 'done'); assert.equal(r.result, 'build');
|
||||
assert.equal(s.calls.length, 3);
|
||||
assert.deepEqual(s.calls[1].args.slice(-5), ['npm', 'install', '--include=dev', '--no-audit', '--no-fund']);
|
||||
assert.ok(r.log.some((l) => l.includes('npm install')));
|
||||
});
|
||||
|
||||
test('I8: auch npm install scheitert ⇒ Installationsfehler', async () => {
|
||||
const s = setup({ files: { 'package.json': PKG(), 'package-lock.json': '{}' }, dockerImpl: () => ({ code: 1 }) });
|
||||
const r = await wait(s.jobs, s.jobs.submit(job()));
|
||||
assert.equal(r.state, 'failed'); assert.match(r.error, /Installation fehlgeschlagen/);
|
||||
assert.equal(s.calls.length, 2);
|
||||
});
|
||||
@@ -0,0 +1,27 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { listenAddresses, inSubnet } from '../src/listen.js';
|
||||
|
||||
const ifs = {
|
||||
lo: [{ family: 'IPv4', address: '127.0.0.1', internal: true }],
|
||||
eth0: [{ family: 'IPv4', address: '172.18.0.5', internal: false }, { family: 'IPv6', address: 'fe80::1', internal: false }],
|
||||
eth1: [{ family: 'IPv4', address: '172.30.0.9', internal: false }],
|
||||
};
|
||||
|
||||
test('inSubnet', () => {
|
||||
assert.equal(inSubnet('172.30.0.9', '172.30.0.0/24'), true);
|
||||
assert.equal(inSubnet('172.30.1.9', '172.30.0.0/24'), false);
|
||||
assert.throws(() => inSubnet('1.2.3.4', 'müll'));
|
||||
});
|
||||
|
||||
test('listenAddresses: Sandbox-Adresse, IPv6 und Loopback ausgeschlossen', () => {
|
||||
assert.deepEqual(listenAddresses(ifs, '172.30.0.0/24'), ['172.18.0.5']);
|
||||
});
|
||||
|
||||
test('listenAddresses: Loopback nur mit Option', () => {
|
||||
assert.deepEqual(listenAddresses(ifs, '172.30.0.0/24', { allowLoopback: true }), ['127.0.0.1', '172.18.0.5']);
|
||||
});
|
||||
|
||||
test('listenAddresses: nichts gefunden ⇒ deutscher Fehler', () => {
|
||||
assert.throws(() => listenAddresses({ eth1: ifs.eth1 }, '172.30.0.0/24'), /Keine Listen-Adresse/);
|
||||
});
|
||||
@@ -0,0 +1,234 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import http from 'node:http';
|
||||
import net from 'node:net';
|
||||
import crypto from 'node:crypto';
|
||||
import { createProxy, stripHop } from '../src/proxy.js';
|
||||
|
||||
const config = { baseHost: 'project.webklar.com' };
|
||||
|
||||
// Test-Ziel: HTTP-Echo + WebSocket-Echo per rohem Upgrade
|
||||
function startTarget() {
|
||||
const seen = [];
|
||||
const srv = http.createServer((req, res) => {
|
||||
const chunks = [];
|
||||
req.on('data', (c) => chunks.push(c));
|
||||
req.on('end', () => {
|
||||
seen.push({ method: req.method, url: req.url, headers: req.headers });
|
||||
res.writeHead(201, { 'X-Test': 'ja' });
|
||||
res.end(`${req.method} ${req.url} ${Buffer.concat(chunks).toString()}`);
|
||||
});
|
||||
});
|
||||
srv.on('upgrade', (req, socket) => {
|
||||
const acc = crypto.createHash('sha1').update(`${req.headers['sec-websocket-key']}258EAFA5-E914-47DA-95CA-C5AB0DC85B11`).digest('base64');
|
||||
socket.write(`HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: ${acc}\r\n\r\n`);
|
||||
socket.on('data', (d) => socket.write(d)); // Echo
|
||||
socket.on('end', () => socket.end());
|
||||
socket.on('error', () => {});
|
||||
});
|
||||
return new Promise((r) => srv.listen(0, '127.0.0.1', () => r({ srv, seen, port: srv.address().port })));
|
||||
}
|
||||
|
||||
async function withProxy(devImpl, fn, opts = {}) {
|
||||
const touched = [];
|
||||
const dev = { touch: (s) => touched.push(s), ...devImpl };
|
||||
const proxy = createProxy({ config, dev, ...opts });
|
||||
await new Promise((r) => proxy.listen(0, '127.0.0.1', r));
|
||||
try { await fn({ port: proxy.address().port, touched }); } finally { proxy.closeAllConnections?.(); await new Promise((r) => proxy.close(r)); }
|
||||
}
|
||||
|
||||
const get = (port, host, { method = 'GET', path = '/', body } = {}) => new Promise((resolve, reject) => {
|
||||
const req = http.request({ port, host: '127.0.0.1', method, path, headers: { Host: host } }, (res) => {
|
||||
const c = []; res.on('data', (d) => c.push(d)); res.on('end', () => resolve({ res, text: Buffer.concat(c).toString() }));
|
||||
});
|
||||
req.on('error', reject);
|
||||
req.end(body);
|
||||
});
|
||||
|
||||
test('online: Durchreichen inkl. POST-Body, X-Forwarded-*, Host, touch', async () => {
|
||||
const t = await startTarget();
|
||||
try {
|
||||
await withProxy({ wake: async (s) => ({ state: 'online', target: `http://127.0.0.1:${t.port}` }) }, async ({ port, touched }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com:443', { method: 'POST', path: '/a?b=1', body: 'hallo' });
|
||||
assert.equal(res.statusCode, 201);
|
||||
assert.equal(res.headers['x-test'], 'ja');
|
||||
assert.equal(text, 'POST /a?b=1 hallo');
|
||||
assert.deepEqual(touched, ['demo']);
|
||||
const h = t.seen[0].headers;
|
||||
assert.equal(h.host, 'demo.project.webklar.com:443');
|
||||
assert.equal(h['x-forwarded-host'], 'demo.project.webklar.com:443');
|
||||
assert.equal(h['x-forwarded-proto'], 'https');
|
||||
assert.ok(h['x-forwarded-for']);
|
||||
});
|
||||
} finally { t.srv.closeAllConnections?.(); await new Promise((r) => t.srv.close(r)); }
|
||||
});
|
||||
|
||||
test('WebSocket-Upgrade wird durchgereicht (Echo)', async () => {
|
||||
const t = await startTarget();
|
||||
try {
|
||||
await withProxy({ wake: async () => ({ state: 'online', target: `http://127.0.0.1:${t.port}` }) }, async ({ port, touched }) => {
|
||||
const sock = net.connect(port, '127.0.0.1');
|
||||
sock.on('error', () => {});
|
||||
let buf = '';
|
||||
const got = new Promise((resolve) => sock.on('data', (d) => { buf += d.toString('latin1'); if (buf.includes('\r\n\r\n') && buf.endsWith('ping-pong')) resolve(); }));
|
||||
sock.write('GET /ws HTTP/1.1\r\nHost: demo.project.webklar.com\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n\r\n');
|
||||
await new Promise((r) => { const i = setInterval(() => { if (buf.includes('\r\n\r\n')) { clearInterval(i); r(); } }, 5); });
|
||||
assert.match(buf, /^HTTP\/1\.1 101/);
|
||||
try {
|
||||
sock.write('ping-pong');
|
||||
await got;
|
||||
assert.ok(buf.endsWith('ping-pong'));
|
||||
assert.deepEqual(touched, ['demo']);
|
||||
} finally { sock.destroy(); }
|
||||
});
|
||||
} finally { t.srv.closeAllConnections?.(); await new Promise((r) => t.srv.close(r)); }
|
||||
});
|
||||
|
||||
test('starting ⇒ 503 mit Text, Retry-After und Meta-Refresh', async () => {
|
||||
await withProxy({ wake: async () => ({ state: 'starting' }) }, async ({ port, touched }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 503);
|
||||
assert.equal(res.headers['retry-after'], '3');
|
||||
assert.match(text, /Preview startet …/);
|
||||
assert.match(text, /<meta http-equiv="refresh" content="3">/);
|
||||
assert.deepEqual(touched, []);
|
||||
});
|
||||
});
|
||||
|
||||
test('failed ⇒ 503 mit Fehlerseite', async () => {
|
||||
await withProxy({ wake: async () => ({ state: 'failed' }) }, async ({ port }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 503);
|
||||
assert.match(text, /Preview konnte nicht starten/);
|
||||
assert.match(text, /Protokoll im WOMS/);
|
||||
assert.ok(!text.includes('http-equiv="refresh"'));
|
||||
});
|
||||
});
|
||||
|
||||
test('wake unknown ⇒ 503 „nicht verfügbar“', async () => {
|
||||
await withProxy({ wake: async () => ({ state: 'unknown' }) }, async ({ port }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 503);
|
||||
assert.match(text, /Diese Preview ist gerade nicht verfügbar\./);
|
||||
});
|
||||
});
|
||||
|
||||
test('Speicherschutz (wake wirft 503) ⇒ 503-Seite, kein Absturz', async () => {
|
||||
await withProxy({ wake: async () => { throw Object.assign(new Error('Server ausgelastet'), { status: 503 }); } }, async ({ port }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 503);
|
||||
assert.match(text, /ausgelastet/);
|
||||
});
|
||||
});
|
||||
|
||||
test('fremder Host, Großschreibung, ungültiger sub, mehrstufig ⇒ 404 ohne wake', async () => {
|
||||
let woken = 0;
|
||||
await withProxy({ wake: async () => { woken++; return { state: 'starting' }; } }, async ({ port }) => {
|
||||
for (const h of ['evil.example.com', 'project.webklar.com', 'Demo.project.webklar.com', 'a_b.project.webklar.com', 'a.b.project.webklar.com', '.project.webklar.com', 'demo.project.webklar.com.evil.com', '']) {
|
||||
const { res } = await get(port, h);
|
||||
assert.equal(res.statusCode, 404, h);
|
||||
}
|
||||
assert.equal(woken, 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('Upstream-Fehler ⇒ 502 deutscher Text', async () => {
|
||||
const dead = net.createServer(); await new Promise((r) => dead.listen(0, '127.0.0.1', r));
|
||||
const port0 = dead.address().port; await new Promise((r) => dead.close(r));
|
||||
await withProxy({ wake: async () => ({ state: 'online', target: `http://127.0.0.1:${port0}` }) }, async ({ port }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 502);
|
||||
assert.match(text, /nicht erreichbar/);
|
||||
});
|
||||
});
|
||||
|
||||
test('stripHop: Hop-by-hop und in Connection genannte Header fallen weg; Upgrade-Pfad behält Upgrade', () => {
|
||||
const h = { connection: 'keep-alive, X-Geheim', 'keep-alive': 'timeout=5', 'x-geheim': '1', te: 'trailers', upgrade: 'websocket', 'proxy-authorization': 'x', 'x-ok': '2' };
|
||||
assert.deepEqual(stripHop(h), { 'x-ok': '2' });
|
||||
assert.deepEqual(stripHop(h, { upgrade: true }), { connection: 'Upgrade', upgrade: 'websocket', 'x-ok': '2' });
|
||||
});
|
||||
|
||||
test('Hop-by-hop-Header werden in beide Richtungen nicht durchgereicht', async () => {
|
||||
const srv = http.createServer((req, res) => {
|
||||
res.writeHead(200, { Connection: 'close, X-Weg', 'X-Weg': 'a', 'Keep-Alive': 'timeout=1', 'X-Bleibt': 'b' });
|
||||
res.end(JSON.stringify(req.headers));
|
||||
});
|
||||
await new Promise((r) => srv.listen(0, '127.0.0.1', r));
|
||||
try {
|
||||
await withProxy({ wake: async () => ({ state: 'online', target: `http://127.0.0.1:${srv.address().port}` }) }, async ({ port }) => {
|
||||
const { res, text } = await new Promise((resolve, reject) => {
|
||||
const rq = http.request({ port, host: '127.0.0.1', headers: { Host: 'demo.project.webklar.com', Connection: 'keep-alive, X-Hop', 'X-Hop': '1', 'Proxy-Authorization': 'Basic x', TE: 'trailers', 'X-Normal': 'ja' } }, (rs) => { const c = []; rs.on('data', (d) => c.push(d)); rs.on('end', () => resolve({ res: rs, text: Buffer.concat(c).toString() })); });
|
||||
rq.on('error', reject); rq.end();
|
||||
});
|
||||
const seen = JSON.parse(text);
|
||||
assert.equal(seen['x-hop'], undefined); assert.equal(seen['proxy-authorization'], undefined); assert.equal(seen.te, undefined);
|
||||
assert.equal(seen['x-normal'], 'ja');
|
||||
assert.equal(res.headers['x-weg'], undefined); assert.notEqual(res.headers['keep-alive'], 'timeout=1');
|
||||
assert.equal(res.headers['x-bleibt'], 'b');
|
||||
});
|
||||
} finally { srv.closeAllConnections?.(); await new Promise((r) => srv.close(r)); }
|
||||
});
|
||||
|
||||
test('Upstream antwortet nicht ⇒ 504', async () => {
|
||||
const srv = http.createServer(() => {}); // antwortet nie
|
||||
await new Promise((r) => srv.listen(0, '127.0.0.1', r));
|
||||
try {
|
||||
await withProxy({ wake: async () => ({ state: 'online', target: `http://127.0.0.1:${srv.address().port}` }) }, async ({ port }) => {
|
||||
const { res, text } = await get(port, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 504);
|
||||
assert.match(text, /nicht rechtzeitig/);
|
||||
}, { upstreamTimeoutMs: 100 });
|
||||
} finally { srv.closeAllConnections?.(); await new Promise((r) => srv.close(r)); }
|
||||
});
|
||||
|
||||
test('Upgrade-Race: Client weg während wake ⇒ kein Upstream-Request', async () => {
|
||||
let release; const gate = new Promise((r) => { release = r; });
|
||||
let upstreamCalls = 0;
|
||||
const request = () => { upstreamCalls++; throw new Error('darf nicht aufgerufen werden'); };
|
||||
await withProxy({ wake: async () => { await gate; return { state: 'online', target: 'http://127.0.0.1:1' }; } }, async ({ port, touched }) => {
|
||||
const sock = net.connect(port, '127.0.0.1');
|
||||
sock.on('error', () => {});
|
||||
sock.write('GET /ws HTTP/1.1\r\nHost: demo.project.webklar.com\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n\r\n');
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
sock.destroy();
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
release();
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
assert.equal(upstreamCalls, 0);
|
||||
assert.deepEqual(touched, []);
|
||||
}, { request });
|
||||
});
|
||||
|
||||
test('WebSocket-Daten touchen höchstens einmal pro Minute', async () => {
|
||||
const t = await startTarget();
|
||||
const clock = { t: 1000 };
|
||||
try {
|
||||
await withProxy({ wake: async () => ({ state: 'online', target: `http://127.0.0.1:${t.port}` }) }, async ({ port, touched }) => {
|
||||
const sock = net.connect(port, '127.0.0.1'); sock.on('error', () => {});
|
||||
let buf = '';
|
||||
sock.on('data', (d) => { buf += d.toString('latin1'); });
|
||||
const waitFor = async (f) => { for (let i = 0; i < 200 && !f(); i++) await new Promise((r) => setTimeout(r, 5)); assert.ok(f()); };
|
||||
try {
|
||||
sock.write('GET /ws HTTP/1.1\r\nHost: demo.project.webklar.com\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n\r\n');
|
||||
await waitFor(() => buf.includes('\r\n\r\n'));
|
||||
assert.deepEqual(touched, ['demo']);
|
||||
sock.write('a'); await waitFor(() => buf.endsWith('a'));
|
||||
assert.deepEqual(touched, ['demo']); // innerhalb der Minute: kein weiteres touch
|
||||
clock.t += 61e3;
|
||||
sock.write('b'); await waitFor(() => buf.endsWith('b'));
|
||||
assert.deepEqual(touched, ['demo', 'demo']);
|
||||
} finally { sock.destroy(); }
|
||||
}, { now: () => clock.t });
|
||||
} finally { t.srv.closeAllConnections?.(); await new Promise((r) => t.srv.close(r)); }
|
||||
});
|
||||
|
||||
test('I7: Upstream ECONNREFUSED ⇒ dev.markDead + 502', async () => {
|
||||
const dead = [];
|
||||
const srv = http.createServer(); await new Promise((r) => srv.listen(0, '127.0.0.1', r));
|
||||
const port = srv.address().port; await new Promise((r) => srv.close(r)); // Port ist jetzt zu
|
||||
await withProxy({ wake: async () => ({ state: 'online', target: `http://127.0.0.1:${port}` }), markDead: async (s) => { dead.push(s); } }, async ({ port: p }) => {
|
||||
const { res } = await get(p, 'demo.project.webklar.com');
|
||||
assert.equal(res.statusCode, 502);
|
||||
assert.deepEqual(dead, ['demo']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,153 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { writeFileSync, mkdtempSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { createApiServer, readMemAvailable } from '../src/server.js';
|
||||
import { loadConfig } from '../src/config.js';
|
||||
import { validateJob } from '../src/validate.js';
|
||||
|
||||
|
||||
const config = { ...loadConfig({ PREVIEW_RUNNER_TOKEN: 'geheim' }) };
|
||||
const RID = '123e4567-e89b-42d3-a456-426614174000';
|
||||
const submitted = [];
|
||||
const jobs = {
|
||||
submit: (j) => { submitted.push(j); return { runId: RID }; },
|
||||
get: (id) => (id === RID ? { runId: RID, state: 'building' } : null),
|
||||
queueLength: () => 0,
|
||||
};
|
||||
const stopped = [];
|
||||
const registered = [];
|
||||
const dev = {
|
||||
ensure: async (sub) => ({ sub, state: 'starting' }),
|
||||
register: async (sub, j) => { registered.push([sub, j]); return { state: 'sleeping', extra: 'x' }; },
|
||||
stop: async (sub) => { stopped.push(sub); },
|
||||
list: () => [],
|
||||
};
|
||||
|
||||
async function withServer(cfg, fn) {
|
||||
const srv = createApiServer({ config: cfg, jobs, dev, mem: () => 123 });
|
||||
await new Promise((r) => srv.listen(0, '127.0.0.1', r));
|
||||
const base = `http://127.0.0.1:${srv.address().port}`;
|
||||
try { await fn(base); } finally { await new Promise((r) => srv.close(r)); }
|
||||
}
|
||||
const auth = { Authorization: 'Bearer geheim', 'Content-Type': 'application/json' };
|
||||
const good = { sub: 'demo', repo: 'JUSN/demo', sha: 'abcdef1', branch: 'main' };
|
||||
|
||||
test('loadConfig: Vorgaben', () => {
|
||||
const c = loadConfig({});
|
||||
assert.equal(c.baseHost, 'project.webklar.com');
|
||||
assert.equal(c.workRoot, '/srv/webklar-preview-runner/work');
|
||||
assert.equal(c.cacheRoot, '/srv/webklar-preview-runner/cache');
|
||||
assert.equal(c.previewsRoot, '/var/www/previews');
|
||||
assert.equal(c.apiPort, 8080);
|
||||
assert.equal(c.proxyPort, 8081);
|
||||
assert.equal(c.minFreeBytes, 1.5 * 1024 ** 3);
|
||||
assert.equal(c.buildTimeoutMs, 900000);
|
||||
assert.equal(c.devIdleMs, 1200000);
|
||||
assert.equal(c.devMax, 3);
|
||||
assert.equal(c.devReadyMs, 180000);
|
||||
assert.equal(c.token, '');
|
||||
});
|
||||
|
||||
test('401 ohne/mit falschem Token und bei leerem konfiguriertem Token', async () => {
|
||||
await withServer(config, async (base) => {
|
||||
assert.equal((await fetch(`${base}/status`)).status, 401);
|
||||
assert.equal((await fetch(`${base}/status`, { headers: { Authorization: 'Bearer falsch' } })).status, 401);
|
||||
});
|
||||
await withServer({ ...config, token: '' }, async (base) => {
|
||||
assert.equal((await fetch(`${base}/status`, { headers: { Authorization: 'Bearer ' } })).status, 401);
|
||||
});
|
||||
});
|
||||
|
||||
test('GET /status', async () => {
|
||||
await withServer(config, async (base) => {
|
||||
const r = await fetch(`${base}/status`, { headers: auth });
|
||||
assert.equal(r.status, 200);
|
||||
assert.deepEqual(await r.json(), { dev: [], known: [], queue: 0, memAvailable: 123 });
|
||||
});
|
||||
});
|
||||
|
||||
test('POST /jobs/build: 400 bei ungültiger Eingabe, 200 bei gültiger', async () => {
|
||||
await withServer(config, async (base) => {
|
||||
for (const bad of [{ sub: '../x' }, { repo: 'a/b;rm' }, { sha: 'xyz' }]) {
|
||||
const r = await fetch(`${base}/jobs/build`, { method: 'POST', headers: auth, body: JSON.stringify({ ...good, ...bad }) });
|
||||
assert.equal(r.status, 400);
|
||||
}
|
||||
const ok = await fetch(`${base}/jobs/build`, { method: 'POST', headers: auth, body: JSON.stringify(good) });
|
||||
assert.equal(ok.status, 200);
|
||||
assert.deepEqual(await ok.json(), { runId: RID });
|
||||
assert.equal(submitted[0].sub, 'demo');
|
||||
const nojson = await fetch(`${base}/jobs/build`, { method: 'POST', headers: auth, body: '{kaputt' });
|
||||
assert.equal(nojson.status, 400);
|
||||
});
|
||||
});
|
||||
|
||||
test('GET /jobs/:runId, Dev-Start/Stop', async () => {
|
||||
await withServer(config, async (base) => {
|
||||
assert.equal((await fetch(`${base}/jobs/${RID}`, { headers: auth })).status, 200);
|
||||
assert.equal((await fetch(`${base}/jobs/223e4567-e89b-42d3-a456-426614174000`, { headers: auth })).status, 404);
|
||||
const s = await fetch(`${base}/dev/demo/start`, { method: 'POST', headers: auth, body: JSON.stringify(good) });
|
||||
assert.equal(s.status, 200);
|
||||
assert.equal((await fetch(`${base}/dev/..x/start`, { method: 'POST', headers: auth, body: JSON.stringify(good) })).status, 400);
|
||||
const t = await fetch(`${base}/dev/demo/stop`, { method: 'POST', headers: auth });
|
||||
assert.equal(t.status, 200);
|
||||
assert.deepEqual(stopped, ['demo']);
|
||||
});
|
||||
});
|
||||
|
||||
test('Body über 256 KB ⇒ 413', async () => {
|
||||
await withServer(config, async (base) => {
|
||||
const r = await fetch(`${base}/jobs/build`, { method: 'POST', headers: auth, body: JSON.stringify({ ...good, x: 'a'.repeat(300 * 1024) }) });
|
||||
assert.equal(r.status, 413);
|
||||
});
|
||||
});
|
||||
|
||||
test('validateJob: env, index, mode', () => {
|
||||
assert.equal(validateJob({ ...good, mode: 'dev', index: 'src/a.html', env: { A_B: 'x' } }).ok, true);
|
||||
assert.equal(validateJob({ ...good, mode: 'off' }).ok, false);
|
||||
assert.equal(validateJob({ ...good, index: '../etc' }).ok, false);
|
||||
assert.equal(validateJob({ ...good, index: '/abs' }).ok, false);
|
||||
assert.equal(validateJob({ ...good, env: { 'a-b': 'x' } }).ok, false);
|
||||
assert.equal(validateJob({ ...good, env: { A: 'x'.repeat(4097) } }).ok, false);
|
||||
const many = Object.fromEntries(Array.from({ length: 31 }, (_, i) => [`V${i}`, '1']));
|
||||
assert.equal(validateJob({ ...good, env: many }).ok, false);
|
||||
assert.equal(validateJob({ ...good, branch: 'a b' }).ok, false);
|
||||
assert.equal(validateJob(null).ok, false);
|
||||
});
|
||||
|
||||
test('R8: Repo/Branch-Härtung, Whitelist, Sub, Lauf-ID', async () => {
|
||||
for (const repo of ['../..', './.', '-x/y', 'a/..']) assert.equal(validateJob({ ...good, repo }).ok, false, repo);
|
||||
for (const branch of ['-x', '--upload-pack=x', 'a..b', 'x.lock', 'x/']) assert.equal(validateJob({ ...good, branch }).ok, false, branch);
|
||||
const v = validateJob({ ...good, devCmd: 'rm -rf /', image: 'evil' });
|
||||
assert.equal(v.ok, true);
|
||||
assert.deepEqual(Object.keys(v.job).sort(), ['branch', 'env', 'mode', 'repo', 'sha', 'sub']);
|
||||
assert.equal(validateJob({ ...good, sub: 'demo\n' }).ok, false);
|
||||
await withServer(config, async (base) => {
|
||||
const r = await fetch(`${base}/dev/demo/start`, { method: 'POST', headers: auth, body: JSON.stringify({ ...good, sub: 'other' }) });
|
||||
assert.equal(r.status, 400);
|
||||
assert.equal((await fetch(`${base}/jobs/not-a-uuid`, { headers: auth })).status, 400);
|
||||
});
|
||||
});
|
||||
|
||||
test('readMemAvailable liest MemAvailable in Bytes', () => {
|
||||
const p = join(mkdtempSync(join(tmpdir(), 'mem-')), 'meminfo');
|
||||
writeFileSync(p, 'MemTotal: 8000000 kB\nMemFree: 100000 kB\nMemAvailable: 2048000 kB\n');
|
||||
assert.equal(readMemAvailable(p), 2048000 * 1024);
|
||||
});
|
||||
|
||||
test('PUT /dev/:sub: Auth, 400, 200 ⇒ register', async () => {
|
||||
await withServer(config, async (base) => {
|
||||
const put = (path, headers, body) => fetch(base + path, { method: 'PUT', headers, body: JSON.stringify(body) });
|
||||
assert.equal((await put('/dev/demo', { 'Content-Type': 'application/json' }, good)).status, 401);
|
||||
assert.equal((await put('/dev/demo', auth, { ...good, sub: 'andere' })).status, 400);
|
||||
assert.equal((await put('/dev/demo', auth, { ...good, sha: 'zz' })).status, 400);
|
||||
assert.equal((await put('/dev/DEMO', auth, good)).status, 400);
|
||||
assert.equal(registered.length, 0);
|
||||
const r = await put('/dev/demo', auth, good);
|
||||
assert.equal(r.status, 200);
|
||||
assert.deepEqual(await r.json(), { state: 'sleeping' });
|
||||
assert.equal(registered[0][0], 'demo');
|
||||
assert.equal(registered[0][1].repo, 'JUSN/demo');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Preview 2.0: legt die Preview-Runner-Attribute an der Collection websiteProjects an.
|
||||
* Idempotent: vorhandene Attribute werden uebersprungen.
|
||||
* Usage:
|
||||
* node scripts/preview-runner-setup.mjs --check (nur melden, nichts anlegen)
|
||||
* node scripts/preview-runner-setup.mjs (fehlende Attribute anlegen)
|
||||
*/
|
||||
import { pathToFileURL } from 'node:url'
|
||||
import { config } from '../server/config.js'
|
||||
|
||||
export const PREVIEW_RUNNER_ATTRIBUTES = [
|
||||
{ key: 'previewMode', type: 'string', size: 10, required: false },
|
||||
{ key: 'previewEnv', type: 'string', size: 20000, required: false },
|
||||
{ key: 'previewState', type: 'string', size: 20, required: false },
|
||||
{ key: 'previewDetected', type: 'string', size: 40, required: false },
|
||||
{ key: 'previewBuiltSha', type: 'string', size: 40, required: false },
|
||||
{ key: 'previewBranch', type: 'string', size: 100, required: false },
|
||||
{ key: 'previewBuiltAt', type: 'string', size: 40, required: false },
|
||||
{ key: 'previewError', type: 'string', size: 500, required: false },
|
||||
]
|
||||
|
||||
/** Echter Appwrite-Client (REST) fuer die Collection websiteProjects. */
|
||||
export function createAppwriteClient() {
|
||||
const { endpoint, projectId, databaseId, apiKey } = config.appwrite
|
||||
const collectionId = config.collections.websiteProjects
|
||||
const base = `${endpoint.replace(/\/$/, '')}/databases/${databaseId}/collections/${collectionId}`
|
||||
|
||||
async function aw(path, { method = 'GET', body } = {}) {
|
||||
const response = await fetch(`${base}${path}`, {
|
||||
method,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-Appwrite-Project': projectId,
|
||||
'X-Appwrite-Key': apiKey,
|
||||
},
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
})
|
||||
const data = await response.json().catch(() => ({}))
|
||||
if (!response.ok) throw new Error(data.message || `Appwrite ${response.status}`)
|
||||
return data
|
||||
}
|
||||
|
||||
return {
|
||||
async listAttributes() {
|
||||
// Ohne limit liefert Appwrite nur 25 Attribute - die Sammlung hat mehr
|
||||
const q = encodeURIComponent(JSON.stringify({ method: 'limit', values: [200] }))
|
||||
const data = await aw(`/attributes?queries[]=${q}`)
|
||||
return data.attributes || []
|
||||
},
|
||||
async createAttribute({ type, ...body }) {
|
||||
await aw(`/attributes/${type}`, { method: 'POST', body })
|
||||
},
|
||||
async waitForAttribute(key, maxSec = 30) {
|
||||
for (let i = 0; i < maxSec; i += 1) {
|
||||
const attr = await aw(`/attributes/${key}`).catch(() => null)
|
||||
if (attr?.status === 'available') return
|
||||
await new Promise((r) => setTimeout(r, 1000))
|
||||
}
|
||||
throw new Error(`Attribut ${key} wurde nicht verfuegbar`)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
export async function main({ check = false, client, log = console.log } = {}) {
|
||||
const api = client || createAppwriteClient()
|
||||
const existing = new Set((await api.listAttributes()).map((a) => a.key))
|
||||
const missing = PREVIEW_RUNNER_ATTRIBUTES.filter((a) => !existing.has(a.key))
|
||||
|
||||
for (const attr of PREVIEW_RUNNER_ATTRIBUTES) {
|
||||
if (existing.has(attr.key)) log(`= Attribut ${attr.key} (existiert)`)
|
||||
else if (check) log(`- Attribut ${attr.key} FEHLT`)
|
||||
}
|
||||
|
||||
if (check) {
|
||||
log(missing.length ? `${missing.length} Attribut(e) fehlen.` : 'Alle Attribute vorhanden.')
|
||||
return { missing }
|
||||
}
|
||||
|
||||
for (const attr of missing) {
|
||||
await api.createAttribute(attr)
|
||||
log(`+ Attribut ${attr.key}`)
|
||||
}
|
||||
for (const attr of missing) await api.waitForAttribute(attr.key)
|
||||
log('Fertig.')
|
||||
return { missing }
|
||||
}
|
||||
|
||||
const invokedDirectly = process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href
|
||||
if (invokedDirectly) {
|
||||
const check = process.argv.includes('--check')
|
||||
main({ check })
|
||||
.then(({ missing }) => process.exit(check && missing.length ? 1 : 0))
|
||||
.catch((e) => {
|
||||
console.error(e.message)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
@@ -303,6 +303,13 @@ export const config = {
|
||||
sessionMaxAge: Number(process.env.PREVIEW_SESSION_MAX_AGE || 60 * 60 * 24 * 7),
|
||||
},
|
||||
|
||||
previewRunner: {
|
||||
enabled: process.env.PREVIEW_RUNNER === 'on',
|
||||
url: process.env.PREVIEW_RUNNER_URL || 'http://webklar-preview-runner:8080',
|
||||
token: process.env.PREVIEW_RUNNER_TOKEN || '',
|
||||
logDir: process.env.PREVIEW_LOG_DIR || path.join(process.cwd(), 'upload-data', 'preview-logs'),
|
||||
},
|
||||
|
||||
invoiceNinja: {
|
||||
url: (process.env.INVOICE_NINJA_URL || 'https://invoice.webklar.com').replace(/\/$/, ''),
|
||||
apiToken: process.env.INVOICE_NINJA_API_TOKEN || '',
|
||||
|
||||
@@ -5,6 +5,7 @@ import { fileURLToPath } from 'node:url'
|
||||
import { config, assertServerConfig, WOMS_DATABASE_ID } from './config.js'
|
||||
import { verifyDatabaseAccess } from './services/appwriteAdmin.js'
|
||||
import { syncPreviewTlsOnStartup } from './services/previewDeploy.js'
|
||||
import { runReconcile } from './services/previewReconcile.js'
|
||||
import { requireSession, sessionMiddleware } from './middleware/session.js'
|
||||
import authRoutes from './routes/auth.js'
|
||||
import projectsRoutes from './routes/projects.js'
|
||||
@@ -22,6 +23,7 @@ import { runActivation } from './services/siteFeatures.js'
|
||||
import giteaWebhookRoutes from './routes/webhook/gitea.js'
|
||||
import pushTimeRoutes from './routes/pushTime.js'
|
||||
import adminGiteaRoutes from './routes/admin/gitea.js'
|
||||
import adminPreviewRoutes from './routes/admin/preview.js'
|
||||
import adminWebsiteProjectsRoutes from './routes/admin/websiteProjects.js'
|
||||
import adminServicesRoutes from './routes/admin/services.js'
|
||||
import adminCustomersRoutes from './routes/admin/customers.js'
|
||||
@@ -72,6 +74,7 @@ app.use('/api/abo', aboRoutes)
|
||||
app.use('/api/funktionen', funktionenRoutes)
|
||||
app.use('/api/portal-admin', portalAdminRoutes)
|
||||
app.use('/api/admin/gitea', adminGiteaRoutes)
|
||||
app.use('/api/admin/website-projects', adminPreviewRoutes)
|
||||
app.use('/api/admin/website-projects', adminWebsiteProjectsRoutes)
|
||||
app.use('/api/admin/services', adminServicesRoutes)
|
||||
app.use('/api/admin/customers', adminCustomersRoutes)
|
||||
@@ -170,6 +173,12 @@ const server = app.listen(config.port, '0.0.0.0', () => {
|
||||
)
|
||||
setTimeout(repoAssign, 60 * 1000).unref()
|
||||
setInterval(repoAssign, 10 * 60 * 1000).unref()
|
||||
|
||||
// Preview-Runner: Zustaende mit dem Runner abgleichen (alle 2 Minuten, nur wenn eingeschaltet)
|
||||
if (config.previewRunner.enabled) {
|
||||
setTimeout(runReconcile, 30 * 1000).unref()
|
||||
setInterval(runReconcile, 2 * 60 * 1000).unref()
|
||||
}
|
||||
})
|
||||
|
||||
server.on('error', (err) => {
|
||||
|
||||
@@ -0,0 +1,191 @@
|
||||
// Admin-API fuer den Preview-Block je Projekt (WOMS). Nie Variablenwerte ausgeben.
|
||||
import { Router } from 'express'
|
||||
import { config } from '../../config.js'
|
||||
import { getDocument, updateDocument } from '../../services/appwriteAdmin.js'
|
||||
import { requireAppwriteStaff } from '../../middleware/appwriteStaff.js'
|
||||
import { envKey, encryptEnv, decryptEnv, envNames, validateEnvChange } from '../../services/previewEnv.js'
|
||||
import { listRunLogs, readRunLog, isRunFile } from '../../services/previewLogs.js'
|
||||
import * as runnerClient from '../../services/previewRunnerClient.js'
|
||||
import { runDeploy, fetchPreviewConfig } from '../../services/previewDeploy.js'
|
||||
import { withLocks } from '../../services/keyedLock.js'
|
||||
import { getRepoInfo } from '../../services/giteaAdmin.js'
|
||||
|
||||
const router = Router()
|
||||
router.use(requireAppwriteStaff)
|
||||
|
||||
const BUSY = 'Server ausgelastet, versuche es gleich noch mal'
|
||||
const MODES = ['auto', 'build', 'dev', 'off']
|
||||
const SHA_RE = /^[0-9a-f]{7,40}$/
|
||||
|
||||
function fail(status, message) {
|
||||
const err = new Error(message)
|
||||
err.status = status
|
||||
err.expected = true
|
||||
return err
|
||||
}
|
||||
|
||||
const col = () => config.collections.websiteProjects
|
||||
|
||||
async function loadProject(id) {
|
||||
let doc
|
||||
try {
|
||||
doc = await getDocument(col(), id)
|
||||
} catch (err) {
|
||||
if (err.status === 404) throw fail(404, 'Projekt nicht gefunden')
|
||||
throw err
|
||||
}
|
||||
if (!doc.subdomain) throw fail(400, 'Projekt hat keine Preview-Subdomain')
|
||||
return doc
|
||||
}
|
||||
|
||||
function requireRunner() {
|
||||
if (!config.previewRunner.enabled) throw fail(409, 'Preview-Runner ist nicht aktiv')
|
||||
}
|
||||
|
||||
async function view(doc) {
|
||||
const key = envKey()
|
||||
let names = []
|
||||
if (doc.previewEnv && key) {
|
||||
try { names = envNames(doc.previewEnv, key) } catch { names = [] }
|
||||
}
|
||||
const mode = doc.previewMode || 'auto'
|
||||
return {
|
||||
mode,
|
||||
state: doc.previewState || (mode === 'off' ? 'off' : ''),
|
||||
detected: doc.previewDetected || '',
|
||||
builtSha: doc.previewBuiltSha || '',
|
||||
builtAt: doc.previewBuiltAt || '',
|
||||
error: doc.previewError || '',
|
||||
branch: doc.previewBranch || '',
|
||||
envNames: names,
|
||||
logs: await listRunLogs(doc.$id),
|
||||
}
|
||||
}
|
||||
|
||||
// Neubau anstossen (wie beim Bearbeiten/Reaktivieren eines Projekts)
|
||||
async function triggerRebuild(doc) {
|
||||
const repo = doc.repoFullName
|
||||
const branch = doc.previewBranch || (await getRepoInfo(repo).catch(() => null))?.defaultBranch || 'main'
|
||||
const found = await fetchPreviewConfig(repo, branch).catch(() => null)
|
||||
const cfg = {
|
||||
enabled: true,
|
||||
type: 'static',
|
||||
displayName: doc.projectName,
|
||||
...(found || {}),
|
||||
branch,
|
||||
subdomain: doc.subdomain,
|
||||
}
|
||||
const result = await runDeploy(repo, branch, cfg, { project: doc })
|
||||
// runDeploy meldet Runner-Fehler im Ergebnis; Schutzschalter (503) an WOMS durchreichen
|
||||
if (result?.error && /Server ausgelastet/.test(result.error)) throw fail(503, BUSY)
|
||||
return result
|
||||
}
|
||||
|
||||
const wrap = (fn) => async (req, res) => {
|
||||
try {
|
||||
await fn(req, res)
|
||||
} catch (err) {
|
||||
if (!err.expected && err.status === 503) return res.status(503).json({ error: BUSY })
|
||||
const status = err.expected ? err.status : 500
|
||||
if (!err.expected) console.error('[admin/preview]', err.message)
|
||||
res.status(status).json({ error: !err.expected ? 'Preview-Aktion fehlgeschlagen' : err.message })
|
||||
}
|
||||
}
|
||||
|
||||
router.get('/:id/preview', wrap(async (req, res) => {
|
||||
res.json(await view(await loadProject(req.params.id)))
|
||||
}))
|
||||
|
||||
router.patch('/:id/preview', wrap(async (req, res) => {
|
||||
requireRunner()
|
||||
const mode = req.body?.mode
|
||||
if (!MODES.includes(mode)) throw fail(400, 'Ungültiger Modus (auto, build, dev oder off)')
|
||||
const doc = await loadProject(req.params.id)
|
||||
if (mode === 'off') {
|
||||
const saved = await updateDocument(col(), doc.$id, { previewMode: mode })
|
||||
// runDeploy (Modus off) belegt den Platz des Repos: laufende Verfolger schreiben danach nichts mehr
|
||||
await triggerRebuild(saved)
|
||||
try { await runnerClient.stopDev(doc.subdomain) } catch (err) {
|
||||
console.error('[admin/preview] Dev stoppen fehlgeschlagen:', err.message)
|
||||
}
|
||||
return res.json(await view(await getDocument(col(), doc.$id)))
|
||||
}
|
||||
const updated = await updateDocument(col(), doc.$id, { previewMode: mode })
|
||||
await triggerRebuild(updated)
|
||||
res.json(await view(await getDocument(col(), doc.$id)))
|
||||
}))
|
||||
|
||||
router.put('/:id/preview/env', wrap(async (req, res) => {
|
||||
requireRunner()
|
||||
const key = envKey()
|
||||
if (!key) throw fail(503, 'Preview-Variablen sind nicht eingerichtet')
|
||||
const { set = {}, remove = [] } = req.body || {}
|
||||
if (!set || typeof set !== 'object' || Array.isArray(set)) throw fail(400, '„set“ muss ein Objekt sein.')
|
||||
if (!Array.isArray(remove)) throw fail(400, '„remove“ muss eine Liste sein.')
|
||||
const updated = await withLocks(['preview-env:' + req.params.id], async () => {
|
||||
const doc = await loadProject(req.params.id)
|
||||
let current
|
||||
try { current = decryptEnv(doc.previewEnv, key) } catch {
|
||||
throw fail(409, 'Vorhandene Preview-Variablen sind nicht entschlüsselbar')
|
||||
}
|
||||
const result = validateEnvChange(current, { set, remove })
|
||||
if (!result.ok) throw fail(400, result.error)
|
||||
const stored = Object.keys(result.env).length ? encryptEnv(result.env, key) : ''
|
||||
if (stored.length > 20000) throw fail(400, 'Zu viele oder zu lange Variablen')
|
||||
return updateDocument(col(), doc.$id, { previewEnv: stored })
|
||||
})
|
||||
if ((updated.previewMode || 'auto') !== 'off') await triggerRebuild(updated)
|
||||
res.json(await view(await getDocument(col(), updated.$id)))
|
||||
}))
|
||||
|
||||
router.post('/:id/preview/rebuild', wrap(async (req, res) => {
|
||||
requireRunner()
|
||||
const doc = await loadProject(req.params.id)
|
||||
if ((doc.previewMode || 'auto') === 'off') throw fail(409, 'Preview ist ausgeschaltet')
|
||||
await triggerRebuild(doc)
|
||||
res.json(await view(await getDocument(col(), doc.$id)))
|
||||
}))
|
||||
|
||||
router.post('/:id/preview/dev/start', wrap(async (req, res) => {
|
||||
requireRunner()
|
||||
const doc = await loadProject(req.params.id)
|
||||
if ((doc.previewMode || 'auto') === 'off') throw fail(409, 'Preview ist ausgeschaltet')
|
||||
// Gespeicherten Stand verwenden (R17), nie raten
|
||||
if (!doc.repoFullName || !doc.previewBranch || !SHA_RE.test(String(doc.previewBuiltSha || ''))) {
|
||||
throw fail(409, 'Noch kein Stand gespeichert – bitte zuerst neu bauen')
|
||||
}
|
||||
let env = {}
|
||||
if (doc.previewEnv) {
|
||||
const key = envKey()
|
||||
try { env = decryptEnv(doc.previewEnv, key) } catch {
|
||||
throw fail(409, 'Preview-Variablen nicht entschlüsselbar (PREVIEW_ENV_KEY fehlt)')
|
||||
}
|
||||
}
|
||||
const out = await runnerClient.startDevNow(doc.subdomain, {
|
||||
sub: doc.subdomain, repo: doc.repoFullName, sha: doc.previewBuiltSha, branch: doc.previewBranch, mode: 'dev', env,
|
||||
})
|
||||
const state = ['starting', 'online'].includes(out?.state) ? out.state : 'starting'
|
||||
const updated = await updateDocument(col(), doc.$id, { previewState: state })
|
||||
res.json(await view(updated))
|
||||
}))
|
||||
|
||||
router.post('/:id/preview/dev/stop', wrap(async (req, res) => {
|
||||
requireRunner()
|
||||
const doc = await loadProject(req.params.id)
|
||||
await runnerClient.stopDev(doc.subdomain)
|
||||
const fresh = await getDocument(col(), doc.$id)
|
||||
const updated = ['online', 'starting', 'sleeping'].includes(fresh.previewState)
|
||||
? await updateDocument(col(), doc.$id, { previewState: 'sleeping' })
|
||||
: fresh
|
||||
res.json(await view(updated))
|
||||
}))
|
||||
|
||||
router.get('/:id/preview/logs/:runFile', wrap(async (req, res) => {
|
||||
const doc = await loadProject(req.params.id)
|
||||
if (!isRunFile(req.params.runFile)) throw fail(400, 'Ungültiger Protokollname')
|
||||
const text = await readRunLog(doc.$id, req.params.runFile)
|
||||
if (text === null) throw fail(404, 'Protokoll nicht gefunden')
|
||||
res.json({ lines: text.replace(/\n$/, '').split('\n') })
|
||||
}))
|
||||
|
||||
export default router
|
||||
@@ -51,6 +51,7 @@ router.get('/', async (_req, res) => {
|
||||
ticketId: p.ticketId || '',
|
||||
woid: tickets.get(p.ticketId)?.woid || '',
|
||||
previewUrl: previewUrlFor(p),
|
||||
previewState: p.previewState || '',
|
||||
features: {},
|
||||
})
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
fetchPreviewConfig,
|
||||
defaultPreviewConfig,
|
||||
runDeploy,
|
||||
deployKeepsStatus,
|
||||
buildPreviewUrl,
|
||||
} from '../../services/previewDeploy.js'
|
||||
import { ensureRepoPreviewConfig } from '../../services/giteaAdmin.js'
|
||||
@@ -108,7 +109,7 @@ router.post('/gitea', async (req, res) => {
|
||||
|
||||
if (hasPreview && previewConfig.enabled !== false && branch === configBranch) {
|
||||
try {
|
||||
deployResult = await runDeploy(name, branch, previewConfig)
|
||||
deployResult = await runDeploy(name, branch, previewConfig, { sha: commitSha })
|
||||
} catch (deployErr) {
|
||||
console.error('[webhook] deploy failed:', deployErr.message)
|
||||
deployResult = { deployed: false, error: deployErr.message }
|
||||
@@ -143,7 +144,9 @@ router.post('/gitea', async (req, res) => {
|
||||
if (subdomain) projectPayload.subdomain = subdomain
|
||||
if (previewUrl) projectPayload.previewUrl = previewUrl
|
||||
|
||||
if (deployResult.deployed) {
|
||||
if (deployKeepsStatus(deployResult) && hasPreview) {
|
||||
// Preview-Runner (C1): Status/Bereitschaft bleiben unveraendert, der Verfolger setzt sie
|
||||
} else if (deployResult.deployed) {
|
||||
projectPayload.status = 'deployed'
|
||||
projectPayload.provisioningStatus = 'ready'
|
||||
} else if (hasPreview) {
|
||||
|
||||
@@ -263,6 +263,8 @@ export async function upsertWebsiteProjectByRepo(repoFullName, data) {
|
||||
...payload,
|
||||
customerId: payload.customerId ?? '',
|
||||
ticketId: payload.ticketId ?? '',
|
||||
status: payload.status ?? 'pending',
|
||||
provisioningStatus: payload.provisioningStatus ?? 'pending',
|
||||
projectName: payload.projectName || repoFullName.split('/').pop() || repoFullName,
|
||||
createdAt: now,
|
||||
documentId: ID.unique(),
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
} from './appwriteAdmin.js'
|
||||
import {
|
||||
runDeploy,
|
||||
deployKeepsStatus,
|
||||
buildPreviewUrl,
|
||||
removeDeploy,
|
||||
syncPreviewTls,
|
||||
@@ -334,10 +335,14 @@ export async function createProjectFromTemplate({
|
||||
subdomain: safeSubdomain,
|
||||
previewUrl,
|
||||
projectType: hasWebsite ? 'website' : 'repo',
|
||||
...(hasWebsite && deployKeepsStatus(deployResult)
|
||||
? {} // Preview-Runner (C1): Status setzt der Verfolger
|
||||
: {
|
||||
status: hasWebsite ? (deployResult.deployed ? 'deployed' : 'pending') : 'created',
|
||||
provisioningStatus: hasWebsite
|
||||
? (deployResult.deployed ? 'ready' : 'deploy_failed')
|
||||
: 'ready',
|
||||
}),
|
||||
})
|
||||
|
||||
return {
|
||||
@@ -475,7 +480,7 @@ export async function updateWebsiteProject(projectId, { projectName, subdomain,
|
||||
}
|
||||
}
|
||||
try {
|
||||
deployResult = await runDeploy(repoFullName, branch, previewConfig)
|
||||
deployResult = await runDeploy(repoFullName, branch, previewConfig, { project: doc })
|
||||
} catch (err) {
|
||||
console.error('[updateWebsiteProject] deploy fehlgeschlagen:', err.message)
|
||||
deployResult = { deployed: false, error: err.message }
|
||||
@@ -494,10 +499,14 @@ export async function updateWebsiteProject(projectId, { projectName, subdomain,
|
||||
projectType: nextType,
|
||||
isPublic,
|
||||
isTemplate,
|
||||
...(hasWebsite && deployKeepsStatus(deployResult)
|
||||
? {} // Preview-Runner (C1): Status setzt der Verfolger
|
||||
: {
|
||||
status: hasWebsite ? (deployResult.deployed ? 'deployed' : 'pending') : 'created',
|
||||
provisioningStatus: hasWebsite
|
||||
? (deployResult.deployed ? 'ready' : (deployResult.error ? 'deploy_failed' : 'pending'))
|
||||
: 'ready',
|
||||
}),
|
||||
updatedAt: new Date().toISOString(),
|
||||
})
|
||||
|
||||
@@ -566,7 +575,7 @@ export async function archiveRepo(projectId, archived = true) {
|
||||
subdomain: doc.subdomain,
|
||||
displayName: doc.projectName,
|
||||
}
|
||||
await runDeploy(repoFullName, branch, cfg)
|
||||
await runDeploy(repoFullName, branch, cfg, { project: doc })
|
||||
} catch (err) {
|
||||
console.error('[archiveRepo] Redeploy bei Reaktivierung fehlgeschlagen:', err.message)
|
||||
}
|
||||
|
||||
@@ -3,8 +3,12 @@ import { promisify } from 'node:util'
|
||||
import fs from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { config } from '../config.js'
|
||||
import { listDocuments, Query } from './appwriteAdmin.js'
|
||||
import { listDocuments, updateDocument, Query } from './appwriteAdmin.js'
|
||||
import { withLocks } from './keyedLock.js'
|
||||
import { giteaFetch, splitRepo } from './giteaClient.js'
|
||||
import { envKey, decryptEnv, maskLog } from './previewEnv.js'
|
||||
import { saveRunLog } from './previewLogs.js'
|
||||
import * as runnerClient from './previewRunnerClient.js'
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
|
||||
@@ -55,6 +59,43 @@ export function publicSubdomainsFromHostsFile(content) {
|
||||
return result
|
||||
}
|
||||
|
||||
/**
|
||||
* Dev-Subdomains aus einer von syncPreviewTls geschriebenen preview-hosts.yml:
|
||||
* alle HTTPS-Router, deren Service preview-runner@docker ist.
|
||||
*/
|
||||
export function devSubdomainsFromHostsFile(content) {
|
||||
const result = new Set()
|
||||
let current = null
|
||||
for (const line of String(content || '').split('\n')) {
|
||||
if (/^ {4}\S/.test(line)) {
|
||||
const match = line.match(/^ {4}preview-([a-z0-9-]+)-https:\s*$/)
|
||||
current = match ? match[1] : null
|
||||
} else if (current && /^ {6}service:\s*preview-runner@docker\s*$/.test(line)) {
|
||||
result.add(current)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
/**
|
||||
* Projekte mit laufendem/schlafendem Dev-Server (nur mit Preview-Runner).
|
||||
* Liefert null, wenn Appwrite nicht antwortet.
|
||||
*/
|
||||
async function getDevProjects() {
|
||||
try {
|
||||
const docs = await listDocuments(config.collections.websiteProjects, [
|
||||
Query.equal('previewState', ['sleeping', 'starting', 'online', 'building', 'failed']),
|
||||
Query.limit(500),
|
||||
])
|
||||
return docs.filter(
|
||||
(d) => d.subdomain && /^[a-z0-9-]+$/.test(d.subdomain) && String(d.previewDetected || '').startsWith('dev')
|
||||
)
|
||||
} catch (err) {
|
||||
console.error('[previewDeploy] getDevProjects fehlgeschlagen:', err.message)
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchPreviewConfig(repoFullName, branch) {
|
||||
const [owner, repo] = repoFullName.split('/')
|
||||
if (!owner || !repo || !config.gitea.apiToken) {
|
||||
@@ -337,12 +378,25 @@ export async function syncPreviewTls() {
|
||||
entries = []
|
||||
}
|
||||
|
||||
const slugs = entries
|
||||
const folderSlugs = entries
|
||||
.filter((entry) => entry.isDirectory() && /^[a-z0-9-]+$/.test(entry.name))
|
||||
.map((entry) => entry.name)
|
||||
.sort()
|
||||
|
||||
const hostsFile = path.join(outDir, 'preview-hosts.yml')
|
||||
// Dev-Server (Preview-Runner): Hosts auch ohne Ordner, Service preview-runner
|
||||
let devSet = new Set()
|
||||
let devProjects = []
|
||||
if (config.previewRunner.enabled) {
|
||||
const found = await getDevProjects()
|
||||
if (found) {
|
||||
devProjects = found
|
||||
devSet = new Set(found.map((d) => d.subdomain))
|
||||
} else {
|
||||
// Appwrite antwortet nicht: Dev-Hosts aus der letzten Datei uebernehmen
|
||||
devSet = await fs.readFile(hostsFile, 'utf8').then(devSubdomainsFromHostsFile, () => new Set())
|
||||
}
|
||||
}
|
||||
const slugs = [...new Set([...folderSlugs, ...devSet])].sort()
|
||||
let publicSet = await getPublicSubdomains()
|
||||
let publicSource = 'appwrite'
|
||||
if (!publicSet) {
|
||||
@@ -357,6 +411,7 @@ export async function syncPreviewTls() {
|
||||
const host = `${name}.${baseHost}`
|
||||
const id = `preview-${name}`
|
||||
const isPublic = publicSet.has(name)
|
||||
const service = devSet.has(name) ? 'preview-runner@docker' : 'preview-nginx@docker'
|
||||
// Oeffentliche Website: HTTPS-Router OHNE preview-auth -> frei zugaenglich.
|
||||
// Preview (Standard): mit preview-chain (ForwardAuth) -> login-geschuetzt.
|
||||
const httpsMiddlewareLines = isPublic
|
||||
@@ -367,7 +422,7 @@ export async function syncPreviewTls() {
|
||||
` rule: "Host(\`${host}\`)"`,
|
||||
' entryPoints:',
|
||||
' - websecure',
|
||||
' service: preview-nginx@docker',
|
||||
` service: ${service}`,
|
||||
...httpsMiddlewareLines,
|
||||
' priority: 100',
|
||||
' tls:',
|
||||
@@ -377,7 +432,7 @@ export async function syncPreviewTls() {
|
||||
` rule: "Host(\`${host}\`)"`,
|
||||
' entryPoints:',
|
||||
' - web',
|
||||
' service: preview-nginx@docker',
|
||||
` service: ${service}`,
|
||||
' middlewares:',
|
||||
' - preview-sites-redirect@docker',
|
||||
' priority: 100',
|
||||
@@ -394,9 +449,61 @@ export async function syncPreviewTls() {
|
||||
|
||||
await fs.mkdir(outDir, { recursive: true })
|
||||
await fs.writeFile(hostsFile, content, 'utf8')
|
||||
if (devProjects.length) registerDevSpecs(devProjects)
|
||||
return { synced: true, hosts: slugs.length, publicSource }
|
||||
}
|
||||
|
||||
// R12: Dev-Auftraege beim Runner hinterlegen (Hintergrund, nie blockierend/werfend)
|
||||
let registering = null
|
||||
|
||||
/** Wartet auf eine laufende Registrierungsrunde (Tests). */
|
||||
export async function waitForDevRegistration() {
|
||||
while (registering) await registering
|
||||
}
|
||||
|
||||
let pendingProjects = null
|
||||
|
||||
export async function registerDevRound(projects) {
|
||||
const key = envKey()
|
||||
await Promise.allSettled(
|
||||
projects.map(async (project) => {
|
||||
const sub = project.subdomain
|
||||
try {
|
||||
const repo = project.repoFullName
|
||||
const { previewBranch: branch, previewBuiltSha: sha } = project
|
||||
// Kein Raten (R17): ohne gespeicherten Branch/Stand wird uebersprungen
|
||||
if (!repo || !branch || !SHA_RE.test(String(sha || ''))) {
|
||||
console.error(`[previewDeploy] Dev-Registrierung ${sub} uebersprungen: Branch oder Stand unbekannt`)
|
||||
return
|
||||
}
|
||||
let env = {}
|
||||
if (project.previewEnv) {
|
||||
try { env = decryptEnv(project.previewEnv, key) } catch {
|
||||
console.error(`[previewDeploy] Dev-Registrierung ${sub} uebersprungen: Variablen nicht entschluesselbar`)
|
||||
return
|
||||
}
|
||||
}
|
||||
await runnerClient.startDev(sub, { sub, repo, sha, branch, mode: 'dev', env })
|
||||
} catch (err) {
|
||||
console.error(`[previewDeploy] Dev-Registrierung ${sub} fehlgeschlagen:`, err.message)
|
||||
}
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
function registerDevSpecs(projects) {
|
||||
// Laeuft schon eine Runde: genau eine Wiederholung danach (mit dem neuesten Stand)
|
||||
if (registering) { pendingProjects = projects; return }
|
||||
registering = (async () => {
|
||||
let next = projects
|
||||
while (next) {
|
||||
pendingProjects = null
|
||||
await registerDevRound(next)
|
||||
next = pendingProjects
|
||||
}
|
||||
})().finally(() => { registering = null })
|
||||
}
|
||||
|
||||
/**
|
||||
* Nach einem Server-Neustart startet der Kundenbereich oft vor Appwrite. Dann
|
||||
* kennt syncPreviewTls die oeffentlichen Seiten nur aus der letzten Datei; der
|
||||
@@ -423,7 +530,219 @@ export async function syncPreviewTlsOnStartup({
|
||||
return null
|
||||
}
|
||||
|
||||
export async function runDeploy(repoFullName, branch, previewConfig) {
|
||||
// ---- Preview-Runner (PREVIEW_RUNNER=on) ----
|
||||
|
||||
// Zeiten des Hintergrund-Verfolgers (fuer Tests ueberschreibbar)
|
||||
export const runnerTiming = { pollMs: 2000, maxMs: 60 * 60 * 1000, unreachableMs: 5 * 60 * 1000 }
|
||||
// Neuester Lauf je Repo: aeltere Verfolger schreiben nichts mehr (R14)
|
||||
const latestRun = new Map()
|
||||
const followers = new Set()
|
||||
const SHA_RE = /^[0-9a-f]{7,40}$/
|
||||
const KEY_MISSING = 'Preview-Variablen nicht entschlüsselbar (PREVIEW_ENV_KEY fehlt)'
|
||||
const KEY_BROKEN = 'Preview-Variablen nicht entschlüsselbar'
|
||||
|
||||
/** Gibt es fuer das Repo einen laufenden Verfolger in diesem Prozess? (Abgleich, R18) */
|
||||
export const hasLiveFollower = (repoFullName) => latestRun.has(repoFullName)
|
||||
|
||||
/** Wartet auf alle laufenden Hintergrund-Verfolger (Tests, geordnetes Beenden). */
|
||||
export async function waitForRunFollowers() {
|
||||
while (followers.size) await Promise.allSettled([...followers])
|
||||
}
|
||||
|
||||
/**
|
||||
* C1: Mit aktivem Runner liefert runDeploy nur "queued"/"skipped"/Fehler des Auftrags - das ist
|
||||
* kein Deploy-Ergebnis. Aufrufer duerfen dann status/provisioningStatus NICHT anfassen
|
||||
* (sonst sperrt auth.js "not_ready" ein bereits bereites Projekt aus). Der Verfolger setzt
|
||||
* bei Erfolg 'deployed'/'ready'.
|
||||
*/
|
||||
export function deployKeepsStatus(deployResult) {
|
||||
return Boolean(config.previewRunner.enabled && deployResult && !deployResult.deployed)
|
||||
}
|
||||
|
||||
// previewDetected: static | build:<pm> | dev:<pm> (der Runner liefert kein Framework)
|
||||
export function detectedLabel(detected, result) {
|
||||
const kind = result || detected?.kind
|
||||
if (kind === 'static') return 'static'
|
||||
const pm = detected?.pm || 'npm'
|
||||
return kind === 'dev' ? `dev:${pm}` : `build:${pm}`
|
||||
}
|
||||
|
||||
async function findProject(repoFullName) {
|
||||
try {
|
||||
const docs = await listDocuments(config.collections.websiteProjects, [
|
||||
Query.equal('repoFullName', repoFullName),
|
||||
Query.limit(1),
|
||||
])
|
||||
return docs[0] || null
|
||||
} catch (err) {
|
||||
console.error('[previewDeploy] Projekt-Lookup fehlgeschlagen:', err.message)
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
// Attributgroessen (Appwrite lehnt laengere Werte ab und der Patch ginge verloren)
|
||||
const ATTR_MAX = { previewError: 500, previewDetected: 40, previewBranch: 100 }
|
||||
|
||||
function clampAttrs(data) {
|
||||
const out = { ...data }
|
||||
for (const [key, max] of Object.entries(ATTR_MAX)) {
|
||||
if (typeof out[key] === 'string' && out[key].length > max) out[key] = out[key].slice(0, max)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function patchProject(repoFullName, project, data) {
|
||||
data = clampAttrs(data)
|
||||
const doc = project?.$id ? project : await findProject(repoFullName)
|
||||
if (!doc?.$id) return null
|
||||
try {
|
||||
return await updateDocument(config.collections.websiteProjects, doc.$id, data)
|
||||
} catch (err) {
|
||||
console.error('[previewDeploy] Projekt-Update fehlgeschlagen:', err.message)
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveSha(repoFullName, branch, sha) {
|
||||
if (SHA_RE.test(String(sha || ''))) return String(sha)
|
||||
const [owner, repo] = splitRepo(repoFullName)
|
||||
const data = await giteaFetch(`/repos/${owner}/${repo}/branches/${encodeURIComponent(branch)}`)
|
||||
const head = data?.commit?.id || data?.commit?.sha
|
||||
if (!SHA_RE.test(String(head || ''))) throw new Error('Commit des Branches nicht ermittelbar')
|
||||
return head
|
||||
}
|
||||
|
||||
async function followRun({ repoFullName, project, token, runId, subdomain, job, env, warning }) {
|
||||
const deadline = Date.now() + runnerTiming.maxMs
|
||||
const current = () => latestRun.get(repoFullName) === token
|
||||
let run = null
|
||||
let failure = null
|
||||
let unreachableSince = 0
|
||||
try {
|
||||
while (current()) {
|
||||
await new Promise((r) => setTimeout(r, runnerTiming.pollMs))
|
||||
if (!current()) return
|
||||
try {
|
||||
run = await runnerClient.getRun(runId)
|
||||
unreachableSince = 0
|
||||
} catch (err) {
|
||||
run = null
|
||||
if (err.status === 404) { failure = 'Der Preview-Lauf ist beim Runner nicht mehr bekannt'; break }
|
||||
console.error('[previewDeploy] Runner-Abfrage fehlgeschlagen:', err.message)
|
||||
unreachableSince ||= Date.now()
|
||||
if (Date.now() - unreachableSince > runnerTiming.unreachableMs) {
|
||||
failure = 'Preview-Runner nicht erreichbar'
|
||||
break
|
||||
}
|
||||
}
|
||||
if (run?.state === 'done' || run?.state === 'failed') break
|
||||
if (Date.now() > deadline) { failure = 'Status unbekannt – bitte neu bauen'; break }
|
||||
}
|
||||
if (!current()) return
|
||||
|
||||
if (run?.log?.length) {
|
||||
try { await saveRunLog((project?.$id || (await findProject(repoFullName))?.$id), run.log, env) } catch (err) {
|
||||
console.error('[previewDeploy] Protokoll speichern fehlgeschlagen:', err.message)
|
||||
}
|
||||
}
|
||||
if (!current()) return
|
||||
|
||||
const now = new Date().toISOString()
|
||||
if (!failure && run?.state === 'done') {
|
||||
const detected = detectedLabel(run.detected, run.result)
|
||||
const base = {
|
||||
previewDetected: detected,
|
||||
previewError: warning || '',
|
||||
previewBranch: job.branch,
|
||||
previewBuiltSha: run.sha || job.sha,
|
||||
// Wie der Legacy-Erfolg: Projekt ist bereit (C1). Ein Fehlschlag senkt nie herab.
|
||||
status: 'deployed',
|
||||
provisioningStatus: 'ready',
|
||||
}
|
||||
if (run.result === 'dev') {
|
||||
await patchProject(repoFullName, project, { ...base, previewState: 'sleeping' })
|
||||
try { await runnerClient.startDev(subdomain, { ...job, mode: 'dev' }) } catch (err) {
|
||||
console.error('[previewDeploy] Dev-Registrierung fehlgeschlagen:', err.message)
|
||||
}
|
||||
} else {
|
||||
await patchProject(repoFullName, project, {
|
||||
...base, previewState: 'online', previewBuiltAt: now,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
await patchProject(repoFullName, project, {
|
||||
previewState: 'failed',
|
||||
previewError: maskLog(failure || run?.error || 'Preview-Build fehlgeschlagen', env),
|
||||
})
|
||||
}
|
||||
if (!current()) return
|
||||
try { await syncPreviewTls() } catch (err) {
|
||||
console.error('[previewDeploy] TLS-Sync fehlgeschlagen:', err.message)
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`[previewDeploy] Verfolger ${runId} abgebrochen:`, err.message)
|
||||
} finally {
|
||||
if (latestRun.get(repoFullName) === token) latestRun.delete(repoFullName)
|
||||
}
|
||||
}
|
||||
|
||||
async function runDeployViaRunner(repoFullName, branch, previewConfig, { project, sha } = {}) {
|
||||
const subdomain = previewConfig.subdomain || repoFullName.split('/').pop()
|
||||
// Platz fuer diesen Aufruf sofort belegen (vor jedem await): aeltere Verfolger scheiden aus
|
||||
const token = Symbol('run')
|
||||
latestRun.set(repoFullName, token)
|
||||
const current = () => latestRun.get(repoFullName) === token
|
||||
const release = () => { if (current()) latestRun.delete(repoFullName) }
|
||||
|
||||
const doc = project || (await findProject(repoFullName))
|
||||
const mode = doc?.previewMode || 'auto'
|
||||
|
||||
if (mode === 'off') {
|
||||
try {
|
||||
if (current()) await patchProject(repoFullName, doc, { previewState: 'off' })
|
||||
} finally { release() }
|
||||
return { deployed: false, skipped: true, reason: 'Preview ist ausgeschaltet', subdomain }
|
||||
}
|
||||
|
||||
let env = {}
|
||||
let warning = ''
|
||||
if (doc?.previewEnv) {
|
||||
const key = envKey()
|
||||
if (!key) warning = KEY_MISSING
|
||||
else {
|
||||
try { env = decryptEnv(doc.previewEnv, key) } catch { warning = KEY_BROKEN }
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const job = {
|
||||
sub: subdomain,
|
||||
repo: repoFullName,
|
||||
sha: await resolveSha(repoFullName, branch, sha),
|
||||
branch,
|
||||
mode: ['build', 'dev'].includes(mode) ? mode : 'auto',
|
||||
...(previewConfig.index ? { index: previewConfig.index } : {}),
|
||||
env,
|
||||
}
|
||||
const { runId } = await runnerClient.submitBuild(job)
|
||||
if (current()) await patchProject(repoFullName, doc, { previewState: 'building', previewError: warning })
|
||||
const follower = followRun({ repoFullName, project: doc, token, runId, subdomain, job, env, warning })
|
||||
followers.add(follower)
|
||||
follower.finally(() => followers.delete(follower))
|
||||
return { deployed: false, queued: true, runId, subdomain }
|
||||
} catch (err) {
|
||||
console.error(`[previewDeploy] Runner-Auftrag fuer ${repoFullName} fehlgeschlagen:`, err.message)
|
||||
try {
|
||||
if (current()) await patchProject(repoFullName, doc, { previewState: 'failed', previewError: maskLog(err.message, env) })
|
||||
} finally { release() }
|
||||
return { deployed: false, error: err.message, subdomain }
|
||||
}
|
||||
}
|
||||
|
||||
export async function runDeploy(repoFullName, branch, previewConfig, options = {}) {
|
||||
if (config.previewRunner.enabled) {
|
||||
return runDeployViaRunner(repoFullName, branch, previewConfig, options)
|
||||
}
|
||||
if (!config.preview.deployRoot) {
|
||||
return { deployed: false, reason: 'PREVIEW_DEPLOY_ROOT nicht gesetzt' }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
// Preview-Variablen: Verschluesselung (AES-256-GCM), Validierung, Log-Maskierung
|
||||
import crypto from 'node:crypto'
|
||||
|
||||
const NAME_RE = /^[A-Z_][A-Z0-9_]*$/
|
||||
const MAX_VARS = 30
|
||||
const MAX_BYTES = 4096
|
||||
|
||||
// Schluessel aus PREVIEW_ENV_KEY (base64, genau 32 Byte) oder null
|
||||
export function envKey() {
|
||||
const raw = process.env.PREVIEW_ENV_KEY
|
||||
if (!raw) return null
|
||||
const buf = Buffer.from(raw, 'base64')
|
||||
return buf.length === 32 ? buf : null
|
||||
}
|
||||
|
||||
export function encryptEnv(obj, key) {
|
||||
const iv = crypto.randomBytes(12)
|
||||
const cipher = crypto.createCipheriv('aes-256-gcm', key, iv, { authTagLength: 16 })
|
||||
const ct = Buffer.concat([cipher.update(JSON.stringify(obj || {}), 'utf8'), cipher.final()])
|
||||
const tag = cipher.getAuthTag()
|
||||
return ['v1', iv, tag, ct].map((p) => (typeof p === 'string' ? p : p.toString('base64'))).join(':')
|
||||
}
|
||||
|
||||
export function decryptEnv(str, key) {
|
||||
if (!str) return {}
|
||||
const parts = String(str).split(':')
|
||||
if (parts.length !== 4 || parts[0] !== 'v1') throw new Error('Ungueltiges Variablenformat')
|
||||
const [, iv, tag, ct] = parts.map((p, i) => (i === 0 ? p : Buffer.from(p, 'base64')))
|
||||
if (iv.length !== 12 || tag.length !== 16) throw new Error('Ungueltiges Variablenformat')
|
||||
const decipher = crypto.createDecipheriv('aes-256-gcm', key, iv, { authTagLength: 16 })
|
||||
decipher.setAuthTag(tag)
|
||||
const plain = Buffer.concat([decipher.update(ct), decipher.final()]).toString('utf8')
|
||||
const obj = JSON.parse(plain)
|
||||
if (!obj || typeof obj !== 'object' || Array.isArray(obj)
|
||||
|| !Object.values(obj).every((v) => typeof v === 'string')) {
|
||||
throw new Error('Ungueltiges Variablenformat')
|
||||
}
|
||||
return obj
|
||||
}
|
||||
|
||||
export function envNames(str, key) {
|
||||
return Object.keys(decryptEnv(str, key)).sort()
|
||||
}
|
||||
|
||||
export function validateEnvChange(current, { set = {}, remove = [] } = {}) {
|
||||
const env = { ...(current || {}) }
|
||||
for (const name of remove || []) delete env[name]
|
||||
for (const [name, value] of Object.entries(set || {})) {
|
||||
if (!NAME_RE.test(name)) {
|
||||
return { ok: false, error: `Ungültiger Variablenname „${name}“ (erlaubt: Großbuchstaben, Ziffern, Unterstrich; nicht mit Ziffer beginnen).` }
|
||||
}
|
||||
if (typeof value !== 'string') {
|
||||
return { ok: false, error: `Der Wert von ${name} muss Text sein.` }
|
||||
}
|
||||
if (Buffer.byteLength(value, 'utf8') > MAX_BYTES) {
|
||||
return { ok: false, error: `Der Wert von ${name} ist zu lang (maximal ${MAX_BYTES} Byte).` }
|
||||
}
|
||||
if (/[\0\r\n]/.test(value)) {
|
||||
return { ok: false, error: `Der Wert von ${name} darf keine Zeilenumbrüche oder Nullzeichen enthalten.` }
|
||||
}
|
||||
env[name] = value
|
||||
}
|
||||
if (Object.keys(env).length > MAX_VARS) {
|
||||
return { ok: false, error: `Zu viele Variablen (maximal ${MAX_VARS}).` }
|
||||
}
|
||||
return { ok: true, env }
|
||||
}
|
||||
|
||||
// Werte (>= 4 Zeichen) im Protokoll durch ••• ersetzen, laengste zuerst
|
||||
export function maskLog(text, env) {
|
||||
let out = String(text ?? '')
|
||||
const values = Object.values(env || {})
|
||||
.filter((v) => typeof v === 'string' && v.length >= 4)
|
||||
.sort((a, b) => b.length - a.length)
|
||||
for (const v of values) out = out.split(v).join('•••')
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
// Build-Protokolle je Projekt: upload-data/preview-logs/<projectId>/<runFile>, maskiert, letzte 5
|
||||
import crypto from 'node:crypto'
|
||||
import fs from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { config } from '../config.js'
|
||||
import { maskLog } from './previewEnv.js'
|
||||
|
||||
const KEEP = 5
|
||||
const MAX_LINES = 300
|
||||
const PROJECT_RE = /^[A-Za-z0-9][A-Za-z0-9._-]{0,35}$/
|
||||
const FILE_RE = /^\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}-\d{3}Z-[0-9a-f]{6}\.log$/
|
||||
|
||||
const projectDir = (projectId) => {
|
||||
if (!PROJECT_RE.test(String(projectId || '')) || String(projectId).includes('..')) {
|
||||
throw new Error('Ungültige Projekt-ID')
|
||||
}
|
||||
return path.join(config.previewRunner.logDir, projectId)
|
||||
}
|
||||
|
||||
export async function saveRunLog(projectId, lines, env) {
|
||||
const dir = projectDir(projectId)
|
||||
await fs.mkdir(dir, { recursive: true })
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, '-')
|
||||
const runFile = `${stamp}-${crypto.randomBytes(3).toString('hex')}.log`
|
||||
const text = maskLog((Array.isArray(lines) ? lines : []).slice(-MAX_LINES).join('\n'), env)
|
||||
await fs.writeFile(path.join(dir, runFile), text + '\n', 'utf8')
|
||||
const all = (await fs.readdir(dir)).filter((f) => FILE_RE.test(f)).sort()
|
||||
for (const old of all.slice(0, Math.max(0, all.length - KEEP))) {
|
||||
await fs.rm(path.join(dir, old), { force: true })
|
||||
}
|
||||
return runFile
|
||||
}
|
||||
|
||||
export const isRunFile = (name) => FILE_RE.test(String(name || ''))
|
||||
|
||||
// Neueste zuerst
|
||||
export async function listRunLogs(projectId) {
|
||||
let files
|
||||
try { files = await fs.readdir(projectDir(projectId)) } catch { return [] }
|
||||
return files
|
||||
.filter((f) => FILE_RE.test(f))
|
||||
.sort()
|
||||
.reverse()
|
||||
.map((runFile) => ({
|
||||
runFile,
|
||||
at: runFile.slice(0, 24).replace(/^(.{13})-(.{2})-(.{2})-/, '$1:$2:$3.'),
|
||||
}))
|
||||
}
|
||||
|
||||
export async function readRunLog(projectId, runFile) {
|
||||
if (!FILE_RE.test(String(runFile || ''))) return null
|
||||
try {
|
||||
return await fs.readFile(path.join(projectDir(projectId), runFile), 'utf8')
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
// R18: Abgleich der Preview-Zustaende mit dem Runner (alle 2 Minuten, nur mit PREVIEW_RUNNER=on).
|
||||
// (a) 'building' ohne laufenden Verfolger und aelter als 70 Minuten => 'failed'
|
||||
// (b) Dev-Projekte: Zustand laut Runner (laeuft => online/starting, sonst sleeping); Modus 'off' bleibt unberuehrt
|
||||
// (c) Dev-Spezifikationen, die der Runner nicht kennt (z. B. nach Neustart), erneut hinterlegen
|
||||
// Wirft nie: Fehler werden protokolliert und im Ergebnis gemeldet.
|
||||
import { config } from '../config.js'
|
||||
import { listDocuments, updateDocument, Query } from './appwriteAdmin.js'
|
||||
import * as runnerClient from './previewRunnerClient.js'
|
||||
import { hasLiveFollower, registerDevRound } from './previewDeploy.js'
|
||||
|
||||
export const STALE_BUILD_MS = 70 * 60 * 1000
|
||||
const STALE_TEXT = 'Status unbekannt – bitte neu bauen'
|
||||
|
||||
const isDev = (d) => String(d.previewDetected || '').startsWith('dev')
|
||||
|
||||
export async function reconcilePreviews({ now = Date.now() } = {}) {
|
||||
if (!config.previewRunner.enabled) return { skipped: true }
|
||||
const result = { staleFailed: 0, stateUpdated: 0, reregistered: 0 }
|
||||
let docs
|
||||
try {
|
||||
docs = await listDocuments(config.collections.websiteProjects, [
|
||||
Query.equal('previewState', ['building', 'sleeping', 'starting', 'online']),
|
||||
Query.limit(500),
|
||||
])
|
||||
} catch (err) {
|
||||
console.error('[previewReconcile] Projekte nicht lesbar:', err.message)
|
||||
return { ...result, error: err.message }
|
||||
}
|
||||
|
||||
const patch = async (doc, data) => {
|
||||
try { await updateDocument(config.collections.websiteProjects, doc.$id, data); return true } catch (err) {
|
||||
console.error('[previewReconcile] Update fehlgeschlagen:', err.message)
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// (a)
|
||||
for (const doc of docs) {
|
||||
if (doc.previewState !== 'building' || !doc.repoFullName) continue
|
||||
if (hasLiveFollower(doc.repoFullName)) continue
|
||||
const since = Date.parse(doc.$updatedAt || doc.$createdAt || '')
|
||||
if (!Number.isFinite(since) || now - since <= STALE_BUILD_MS) continue
|
||||
if (await patch(doc, { previewState: 'failed', previewError: STALE_TEXT })) result.staleFailed++
|
||||
}
|
||||
|
||||
// (b) + (c)
|
||||
const devDocs = docs.filter((d) => isDev(d) && d.previewMode !== 'off' && d.previewState !== 'building' && d.subdomain)
|
||||
if (!devDocs.length) return result
|
||||
let st
|
||||
try { st = await runnerClient.status() } catch (err) {
|
||||
console.error('[previewReconcile] Runner-Status nicht abrufbar:', err.message)
|
||||
return { ...result, error: err.message }
|
||||
}
|
||||
const entries = new Map((st?.dev || []).map((e) => [e.sub, e]))
|
||||
const known = new Set([...(st?.known || []), ...entries.keys()])
|
||||
const unknown = []
|
||||
for (const doc of devDocs) {
|
||||
const entry = entries.get(doc.subdomain)
|
||||
const want = entry?.state === 'online' ? 'online' : entry?.state === 'starting' ? 'starting' : 'sleeping'
|
||||
if (doc.previewState !== want && (await patch(doc, { previewState: want }))) result.stateUpdated++
|
||||
if (!known.has(doc.subdomain)) unknown.push(doc)
|
||||
}
|
||||
if (unknown.length) {
|
||||
try { await registerDevRound(unknown); result.reregistered = unknown.length } catch (err) {
|
||||
console.error('[previewReconcile] Registrierung fehlgeschlagen:', err.message)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
/** Fuer setInterval: nie werfen. */
|
||||
export function runReconcile() {
|
||||
return reconcilePreviews().catch((err) => {
|
||||
console.error('[previewReconcile] Lauf fehlgeschlagen:', err.message)
|
||||
return { error: err.message }
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
// HTTP-Client fuer den Preview-Runner (Bearer-Token, JSON)
|
||||
import { config } from '../config.js'
|
||||
|
||||
const TIMEOUT_MS = 15_000
|
||||
|
||||
async function call(method, path, body) {
|
||||
const { url, token } = config.previewRunner
|
||||
let response
|
||||
try {
|
||||
response = await fetch(`${String(url).replace(/\/$/, '')}${path}`, {
|
||||
method,
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
...(body !== undefined ? { 'Content-Type': 'application/json' } : {}),
|
||||
},
|
||||
body: body !== undefined ? JSON.stringify(body) : undefined,
|
||||
signal: AbortSignal.timeout(TIMEOUT_MS),
|
||||
})
|
||||
} catch (err) {
|
||||
throw new Error(`Preview-Runner nicht erreichbar: ${err.message}`)
|
||||
}
|
||||
const text = await response.text()
|
||||
let data = null
|
||||
try { data = text ? JSON.parse(text) : null } catch { data = { error: text } }
|
||||
if (!response.ok) {
|
||||
const error = new Error(data?.error || `Preview-Runner ${response.status}`)
|
||||
error.status = response.status
|
||||
throw error
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
export const submitBuild = (job) => call('POST', '/jobs/build', job)
|
||||
export const getRun = (runId) => call('GET', `/jobs/${encodeURIComponent(runId)}`)
|
||||
// Dev-Auftrag beim Runner hinterlegen (startet nicht, R12)
|
||||
export const startDev = (sub, job) => call('PUT', `/dev/${encodeURIComponent(sub)}`, job)
|
||||
// Dev-Server jetzt starten (mit gespeichertem Auftrag)
|
||||
export const startDevNow = (sub, job) => call('POST', `/dev/${encodeURIComponent(sub)}/start`, job)
|
||||
export const stopDev = (sub) => call('POST', `/dev/${encodeURIComponent(sub)}/stop`, {})
|
||||
export const status = () => call('GET', '/status')
|
||||
@@ -95,6 +95,8 @@ export async function completeAssignment(
|
||||
forCustomer: customer,
|
||||
kindSetBy: staffName || '',
|
||||
kindSetAt: now.toISOString(),
|
||||
// Preview-Modus nur vorbelegen, wenn noch keiner gesetzt ist
|
||||
...(project.previewMode ? {} : { previewMode: kind === 'website' || kind === 'webapp' ? 'auto' : 'off' }),
|
||||
})
|
||||
|
||||
const result = { ticketId: finalTicketId }
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
import { test, before, after, beforeEach } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import http from 'node:http'
|
||||
import fs from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import crypto from 'node:crypto'
|
||||
import express from 'express'
|
||||
import { startFakeAppwrite } from './helpers/fakeAppwrite.js'
|
||||
|
||||
let aw, runner, gitea, app, base, tmp, cfg, envLib, logs
|
||||
const key = crypto.randomBytes(32)
|
||||
const SHA = 'c'.repeat(40)
|
||||
const rs = { calls: [], busy: false, slow: false, result: 'static' }
|
||||
|
||||
before(async () => {
|
||||
const seed = [
|
||||
{ $id: 'pw', projectName: 'Web', repoFullName: 'JUSN/web', subdomain: 'web', previewBranch: 'main', previewBuiltSha: SHA, previewState: 'online', previewDetected: 'static' },
|
||||
{ $id: 'pr', projectName: 'R', repoFullName: 'JUSN/race', subdomain: 'race', previewBranch: 'main', previewBuiltSha: SHA },
|
||||
{ $id: 'pe', projectName: 'E', repoFullName: 'JUSN/envp', subdomain: 'envp', previewBranch: 'main', previewBuiltSha: SHA },
|
||||
{ $id: 'pbad', projectName: 'B', repoFullName: 'JUSN/bad', subdomain: 'bad', previewEnv: 'v1:x:y:z' },
|
||||
{ $id: 'pns', projectName: 'N', repoFullName: 'JUSN/nosha', subdomain: 'nosha' },
|
||||
{ $id: 'pnosub', projectName: 'Ohne', repoFullName: 'JUSN/ohne' },
|
||||
{ $id: 'pa', projectName: 'App', repoFullName: 'JUSN/app', customerId: '', ticketId: '' },
|
||||
{ $id: 'pb', projectName: 'Web2', repoFullName: 'JUSN/web2', customerId: '', ticketId: '' },
|
||||
{ $id: 'pc', projectName: 'Web3', repoFullName: 'JUSN/web3', customerId: '', ticketId: '', previewMode: 'dev' },
|
||||
]
|
||||
aw = await startFakeAppwrite({ seed: { websiteProjects: seed, services: [] } })
|
||||
runner = http.createServer((req, res) => {
|
||||
let raw = ''
|
||||
req.on('data', (c) => (raw += c))
|
||||
req.on('end', () => {
|
||||
if (rs.busy && (req.url === '/jobs/build' || req.url.endsWith('/start'))) { res.statusCode = 503; res.setHeader('Content-Type', 'application/json'); return res.end(JSON.stringify({ error: 'Server ausgelastet' })) }
|
||||
rs.calls.push({ method: req.method, url: req.url, body: raw ? JSON.parse(raw) : null })
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
if (req.method === 'POST' && req.url === '/jobs/build') return res.end(JSON.stringify({ runId: 'run-x' }))
|
||||
if (req.method === 'GET' && req.url.startsWith('/jobs/')) return res.end(JSON.stringify(rs.slow ? { state: 'running' } : { state: 'done', detected: { kind: 'static', pm: 'npm' }, sha: 'b'.repeat(40), log: ['ok'], error: null, result: rs.result }))
|
||||
if (req.url.endsWith('/start')) return res.end(JSON.stringify({ state: 'starting' }))
|
||||
res.end(JSON.stringify({ state: 'sleeping', ok: true }))
|
||||
})
|
||||
})
|
||||
await new Promise((r) => runner.listen(0, '127.0.0.1', r))
|
||||
gitea = http.createServer((req, res) => {
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
res.end(JSON.stringify({ commit: { id: 'b'.repeat(40) }, default_branch: 'main' }))
|
||||
})
|
||||
await new Promise((r) => gitea.listen(0, '127.0.0.1', r))
|
||||
tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'preview-admin-'))
|
||||
Object.assign(process.env, {
|
||||
APPWRITE_ENDPOINT: aw.endpoint,
|
||||
APPWRITE_API_KEY: 'test-key',
|
||||
GITEA_BASE_URL: `http://127.0.0.1:${gitea.address().port}`,
|
||||
GITEA_API_TOKEN: 'tok',
|
||||
PREVIEW_RUNNER_URL: `http://127.0.0.1:${runner.address().port}`,
|
||||
PREVIEW_RUNNER_TOKEN: 'sekret',
|
||||
PREVIEW_DEPLOY_ROOT: path.join(tmp, 'previews'),
|
||||
TRAEFIK_DYNAMIC_DIR: path.join(tmp, 'dynamic'),
|
||||
PREVIEW_ENV_KEY: key.toString('base64'),
|
||||
})
|
||||
;({ config: cfg } = await import('../server/config.js'))
|
||||
cfg.previewRunner.enabled = true
|
||||
cfg.previewRunner.logDir = path.join(tmp, 'logs')
|
||||
envLib = await import('../server/services/previewEnv.js')
|
||||
logs = await import('../server/services/previewLogs.js')
|
||||
const pd = await import('../server/services/previewDeploy.js')
|
||||
pd.runnerTiming.pollMs = 10
|
||||
const { default: preview } = await import('../server/routes/admin/preview.js')
|
||||
const { default: wp } = await import('../server/routes/admin/websiteProjects.js')
|
||||
app = express()
|
||||
app.use(express.json({ limit: '2mb' }))
|
||||
app.use('/api/admin/website-projects', preview)
|
||||
app.use('/api/admin/website-projects', wp)
|
||||
const server = await new Promise((r) => { const s = app.listen(0, '127.0.0.1', () => r(s)) })
|
||||
app.server = server
|
||||
base = `http://127.0.0.1:${server.address().port}/api/admin/website-projects`
|
||||
})
|
||||
|
||||
after(async () => {
|
||||
// Hintergrund-Verfolger ausklingen lassen, bevor der Fake-Runner schliesst
|
||||
for (let i = 0; i < 100; i++) {
|
||||
const building = ['pw', 'pa', 'pb', 'pc', 'pr', 'pe'].some((id) => aw.get('websiteProjects', id)?.previewState === 'building')
|
||||
if (!building) break
|
||||
await new Promise((r) => setTimeout(r, 20))
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 100))
|
||||
for (const srv of [app.server, runner, gitea]) { srv.closeAllConnections(); await new Promise((r) => srv.close(r)) }
|
||||
await aw.close()
|
||||
await fs.rm(tmp, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
beforeEach(() => { rs.busy = false; rs.slow = false; rs.result = 'static'; rs.calls = []; cfg.previewRunner.enabled = true; process.env.PREVIEW_ENV_KEY = key.toString('base64') })
|
||||
|
||||
const call = (method, p, body, jwt = 'jwt-admin') =>
|
||||
fetch(`${base}${p}`, {
|
||||
method,
|
||||
headers: { 'Content-Type': 'application/json', ...(jwt ? { Authorization: `Bearer ${jwt}` } : {}) },
|
||||
body: body === undefined || method === 'GET' ? undefined : JSON.stringify(body),
|
||||
})
|
||||
|
||||
test('ohne JWT => 401 auf allen Routen', async () => {
|
||||
for (const [m, p] of [['GET', '/pw/preview'], ['PATCH', '/pw/preview'], ['PUT', '/pw/preview/env'], ['POST', '/pw/preview/rebuild'],
|
||||
['POST', '/pw/preview/dev/start'], ['POST', '/pw/preview/dev/stop'], ['GET', '/pw/preview/logs/x.log']]) {
|
||||
assert.equal((await call(m, p, {}, null)).status, 401, `${m} ${p}`)
|
||||
}
|
||||
})
|
||||
|
||||
test('GET: Form, leerer Modus = auto, 404 und keine Subdomain', async () => {
|
||||
const r = await call('GET', '/pw/preview')
|
||||
assert.equal(r.status, 200)
|
||||
assert.deepEqual(await r.json(), {
|
||||
mode: 'auto', state: 'online', detected: 'static', builtSha: SHA, builtAt: '', error: '', branch: 'main', envNames: [], logs: [],
|
||||
})
|
||||
assert.equal((await call('GET', '/gibtsnicht/preview')).status, 404)
|
||||
const r2 = await call('GET', '/pnosub/preview')
|
||||
assert.equal(r2.status, 400)
|
||||
assert.equal((await r2.json()).error, 'Projekt hat keine Preview-Subdomain')
|
||||
})
|
||||
|
||||
test('PATCH: Modus ungueltig => 400; off stoppt Dev; sonst Neubau', async () => {
|
||||
assert.equal((await call('PATCH', '/pw/preview', { mode: 'foo' })).status, 400)
|
||||
assert.equal((await call('PATCH', '/pw/preview', {})).status, 400)
|
||||
const off = await call('PATCH', '/pw/preview', { mode: 'off' })
|
||||
assert.equal(off.status, 200)
|
||||
const b = await off.json()
|
||||
assert.equal(b.mode, 'off')
|
||||
assert.equal(b.state, 'off')
|
||||
assert.ok(rs.calls.some((c) => c.url === '/dev/web/stop'))
|
||||
assert.equal(rs.calls.filter((c) => c.url === '/jobs/build').length, 0)
|
||||
rs.calls = []
|
||||
const on = await call('PATCH', '/pw/preview', { mode: 'build' })
|
||||
assert.equal(on.status, 200)
|
||||
assert.equal((await on.json()).mode, 'build')
|
||||
const sub = rs.calls.find((c) => c.url === '/jobs/build')
|
||||
assert.equal(sub.body.mode, 'build')
|
||||
assert.equal(sub.body.sub, 'web')
|
||||
assert.equal(aw.get('websiteProjects', 'pw').previewState, 'building')
|
||||
})
|
||||
|
||||
test('Route-Vorrang: PATCH /:id bleibt bei der Projektbearbeitung', async () => {
|
||||
const r = await call('PATCH', '/pw', { projectName: 'X' })
|
||||
const b = await r.json()
|
||||
assert.equal(b.mode, undefined) // kein Preview-Handler
|
||||
assert.equal(b.ok !== undefined || b.error !== undefined, true)
|
||||
})
|
||||
|
||||
test('env: ohne Schluessel 503, ungueltig 400, Werte nie in Antworten', async () => {
|
||||
delete process.env.PREVIEW_ENV_KEY
|
||||
const r503 = await call('PUT', '/pw/preview/env', { set: { A: 'b' } })
|
||||
assert.equal(r503.status, 503)
|
||||
assert.deepEqual(await r503.json(), { error: 'Preview-Variablen sind nicht eingerichtet' })
|
||||
process.env.PREVIEW_ENV_KEY = key.toString('base64')
|
||||
const bad = await call('PUT', '/pw/preview/env', { set: { 'kleinbuchstabe': 'x' } })
|
||||
assert.equal(bad.status, 400)
|
||||
assert.match((await bad.json()).error, /Ungültiger Variablenname/)
|
||||
const ok = await call('PUT', '/pw/preview/env', { set: { API_URL: 'geheim-wert-123', OTHER: 'zweiter-wert-456' } })
|
||||
const text = await ok.text()
|
||||
assert.equal(ok.status, 200)
|
||||
assert.ok(!text.includes('geheim-wert-123') && !text.includes('zweiter-wert-456'))
|
||||
assert.deepEqual(JSON.parse(text).envNames, ['API_URL', 'OTHER'])
|
||||
const stored = aw.get('websiteProjects', 'pw').previewEnv
|
||||
assert.ok(stored.startsWith('v1:') && !stored.includes('geheim'))
|
||||
assert.equal(rs.calls.filter((c) => c.url === '/jobs/build').length, 1)
|
||||
const get = await (await call('GET', '/pw/preview')).text()
|
||||
assert.ok(!get.includes('geheim-wert-123'))
|
||||
const rem = await call('PUT', '/pw/preview/env', { remove: ['API_URL'] })
|
||||
assert.deepEqual((await rem.json()).envNames, ['OTHER'])
|
||||
})
|
||||
|
||||
test('Protokolle: Liste, lesen, ../x => 400', async () => {
|
||||
const file = await logs.saveRunLog('pw', ['a', 'b'], {})
|
||||
const list = await (await call('GET', '/pw/preview')).json()
|
||||
assert.ok(list.logs.some((l) => l.runFile === file))
|
||||
const r = await call('GET', `/pw/preview/logs/${file}`)
|
||||
assert.deepEqual(await r.json(), { lines: ['a', 'b'] })
|
||||
assert.equal((await call('GET', '/pw/preview/logs/..%2Fx')).status, 400)
|
||||
assert.equal((await call('GET', '/pw/preview/logs/x.log')).status, 400)
|
||||
})
|
||||
|
||||
test('Rebuild und Dev start/stop', async () => {
|
||||
const rb = await call('POST', '/pw/preview/rebuild')
|
||||
assert.equal(rb.status, 200)
|
||||
assert.equal(rs.calls.filter((c) => c.url === '/jobs/build').length, 1)
|
||||
rs.calls = []
|
||||
const st = await call('POST', '/pw/preview/dev/start')
|
||||
assert.equal(st.status, 200)
|
||||
const start = rs.calls.find((c) => c.url === '/dev/web/start')
|
||||
assert.ok([SHA, 'b'.repeat(40)].includes(start.body.sha)) // gespeicherter Stand (je nach Verfolger)
|
||||
assert.deepEqual(start.body, { sub: 'web', repo: 'JUSN/web', sha: start.body.sha, branch: 'main', mode: 'dev', env: { OTHER: 'zweiter-wert-456' } })
|
||||
assert.equal((await st.json()).state, 'starting')
|
||||
const stop = await call('POST', '/pw/preview/dev/stop')
|
||||
assert.equal((await stop.json()).state, 'sleeping')
|
||||
assert.ok(rs.calls.some((c) => c.url === '/dev/web/stop'))
|
||||
})
|
||||
|
||||
test('Schalter aus: Schreib-Routen 409, GET geht', async () => {
|
||||
cfg.previewRunner.enabled = false
|
||||
for (const [m, p, b] of [['PATCH', '/pw/preview', { mode: 'auto' }], ['PUT', '/pw/preview/env', { set: {} }],
|
||||
['POST', '/pw/preview/rebuild'], ['POST', '/pw/preview/dev/start'], ['POST', '/pw/preview/dev/stop']]) {
|
||||
const r = await call(m, p, b)
|
||||
assert.equal(r.status, 409, `${m} ${p}`)
|
||||
assert.deepEqual(await r.json(), { error: 'Preview-Runner ist nicht aktiv' })
|
||||
}
|
||||
assert.equal((await call('GET', '/pw/preview')).status, 200)
|
||||
assert.equal(rs.calls.filter((c) => c.method !== 'GET').length, 0)
|
||||
})
|
||||
|
||||
test('Zuordnen: Vorgabe website => auto, andere => off, vorhandener Wert bleibt', async () => {
|
||||
const { completeAssignment } = await import('../server/services/projectKindAssign.js')
|
||||
const deps = { assignProject: async () => ({}), updateWebsiteProject: async () => ({}) }
|
||||
await completeAssignment('pb', { kind: 'website', forCustomer: false }, { deps })
|
||||
await completeAssignment('pa', { kind: 'other', forCustomer: false }, { deps })
|
||||
await completeAssignment('pc', { kind: 'website', forCustomer: false }, { deps })
|
||||
assert.equal(aw.get('websiteProjects', 'pb').previewMode, 'auto')
|
||||
assert.equal(aw.get('websiteProjects', 'pa').previewMode, 'off')
|
||||
assert.equal(aw.get('websiteProjects', 'pc').previewMode, 'dev')
|
||||
})
|
||||
|
||||
test('Services: Eintraege tragen previewState', async () => {
|
||||
const { default: services } = await import('../server/routes/admin/services.js')
|
||||
const a2 = express()
|
||||
a2.use('/s', services)
|
||||
const s = await new Promise((r) => { const x = a2.listen(0, '127.0.0.1', () => r(x)) })
|
||||
try {
|
||||
const { updateDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await updateDocument('websiteProjects', 'pw', { previewState: 'online' })
|
||||
const r = await fetch(`http://127.0.0.1:${s.address().port}/s`, { headers: { Authorization: 'Bearer jwt-admin' } })
|
||||
const all = Object.values((await r.json()).groups).flat()
|
||||
assert.equal(all.find((p) => p.id === 'pw').previewState, 'online')
|
||||
assert.equal(all.find((p) => p.id === 'pnosub').previewState, '')
|
||||
} finally { await new Promise((r) => s.close(r)) }
|
||||
})
|
||||
|
||||
void envLib
|
||||
|
||||
test('PATCH off während eines laufenden Baus: Zustand bleibt off, keine Dev-Registrierung', async () => {
|
||||
rs.slow = true
|
||||
rs.result = 'dev'
|
||||
assert.equal((await call('POST', '/pr/preview/rebuild')).status, 200)
|
||||
assert.equal(aw.get('websiteProjects', 'pr').previewState, 'building')
|
||||
const off = await call('PATCH', '/pr/preview', { mode: 'off' })
|
||||
assert.equal(off.status, 200)
|
||||
rs.slow = false
|
||||
await new Promise((r) => setTimeout(r, 300))
|
||||
assert.equal(aw.get('websiteProjects', 'pr').previewState, 'off')
|
||||
assert.equal(rs.calls.filter((c) => c.method === 'PUT').length, 0)
|
||||
})
|
||||
|
||||
test('parallele PUT env: beide Variablen bleiben erhalten', async () => {
|
||||
const [a, b] = await Promise.all([
|
||||
call('PUT', '/pe/preview/env', { set: { AAA: 'wert-eins' } }),
|
||||
call('PUT', '/pe/preview/env', { set: { BBB: 'wert-zwei' } }),
|
||||
])
|
||||
assert.equal(a.status, 200)
|
||||
assert.equal(b.status, 200)
|
||||
const names = (await (await call('GET', '/pe/preview')).json()).envNames
|
||||
assert.deepEqual(names, ['AAA', 'BBB'])
|
||||
})
|
||||
|
||||
test('env: set/remove falsch typisiert 400, zu lang 400, unentschlüsselbar 409', async () => {
|
||||
assert.equal((await call('PUT', '/pe/preview/env', { set: [1] })).status, 400)
|
||||
assert.equal((await call('PUT', '/pe/preview/env', { set: 'x' })).status, 400)
|
||||
assert.equal((await call('PUT', '/pe/preview/env', { remove: 'AAA' })).status, 400)
|
||||
const big = Object.fromEntries(Array.from({ length: 10 }, (_, i) => [`V${i}`, 'x'.repeat(4000)]))
|
||||
const r = await call('PUT', '/pe/preview/env', { set: big })
|
||||
assert.equal(r.status, 400)
|
||||
assert.equal((await r.json()).error, 'Zu viele oder zu lange Variablen')
|
||||
assert.equal((await call('PUT', '/pbad/preview/env', { set: { A: 'b' } })).status, 409)
|
||||
})
|
||||
|
||||
test('dev/start ohne gespeicherten Stand: 409; dev/stop ändert failed nicht', async () => {
|
||||
assert.equal((await call('POST', '/pns/preview/dev/start')).status, 409)
|
||||
const { updateDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await updateDocument('websiteProjects', 'pns', { previewState: 'failed' })
|
||||
const r = await call('POST', '/pns/preview/dev/stop')
|
||||
assert.equal((await r.json()).state, 'failed')
|
||||
})
|
||||
|
||||
test('Runner-Schutzschalter 503: rebuild und dev/start melden Server ausgelastet', async () => {
|
||||
rs.busy = true
|
||||
for (const p of ['/pe/preview/rebuild', '/pw/preview/dev/start']) {
|
||||
const r = await call('POST', p)
|
||||
assert.equal(r.status, 503, p)
|
||||
assert.deepEqual(await r.json(), { error: 'Server ausgelastet, versuche es gleich noch mal' })
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,270 @@
|
||||
import { test, before, after, beforeEach } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import http from 'node:http'
|
||||
import fs from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import crypto from 'node:crypto'
|
||||
import { startFakeAppwrite } from './helpers/fakeAppwrite.js'
|
||||
|
||||
let aw, runner, gitea, tmp, pd, config, hostsFile, envLib
|
||||
const key = crypto.randomBytes(32)
|
||||
const BUILT = 'c'.repeat(40)
|
||||
const HEAD = 'd'.repeat(40)
|
||||
const puts = []
|
||||
let giteaDown = false
|
||||
let runnerFails = false
|
||||
|
||||
before(async () => {
|
||||
aw = await startFakeAppwrite({ seed: { websiteProjects: [] } })
|
||||
runner = http.createServer((req, res) => {
|
||||
let raw = ''
|
||||
req.on('data', (c) => (raw += c))
|
||||
req.on('end', () => {
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
if (req.method === 'PUT' && req.url.startsWith('/dev/')) {
|
||||
puts.push({ url: req.url, body: JSON.parse(raw) })
|
||||
res.statusCode = runnerFails ? 500 : 200
|
||||
return res.end(JSON.stringify(runnerFails ? { error: 'kaputt' } : { state: 'sleeping' }))
|
||||
}
|
||||
res.statusCode = 404
|
||||
res.end('{}')
|
||||
})
|
||||
})
|
||||
await new Promise((r) => runner.listen(0, '127.0.0.1', r))
|
||||
gitea = http.createServer((req, res) => {
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
giteaHits++
|
||||
if (giteaDown) { res.statusCode = 500; return res.end('{}') }
|
||||
res.end(JSON.stringify({ commit: { id: HEAD } }))
|
||||
})
|
||||
await new Promise((r) => gitea.listen(0, '127.0.0.1', r))
|
||||
tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'preview-dev-'))
|
||||
await fs.mkdir(path.join(tmp, 'previews', 'statisch'), { recursive: true })
|
||||
await fs.mkdir(path.join(tmp, 'previews', 'offen'), { recursive: true })
|
||||
hostsFile = path.join(tmp, 'dynamic', 'preview-hosts.yml')
|
||||
Object.assign(process.env, {
|
||||
APPWRITE_ENDPOINT: aw.endpoint,
|
||||
APPWRITE_API_KEY: 'test-key',
|
||||
PREVIEW_DEPLOY_ROOT: path.join(tmp, 'previews'),
|
||||
TRAEFIK_DYNAMIC_DIR: path.join(tmp, 'dynamic'),
|
||||
GITEA_BASE_URL: `http://127.0.0.1:${gitea.address().port}`,
|
||||
GITEA_API_TOKEN: 'tok',
|
||||
PREVIEW_RUNNER_URL: `http://127.0.0.1:${runner.address().port}`,
|
||||
PREVIEW_RUNNER_TOKEN: 'sekret',
|
||||
PREVIEW_ENV_KEY: key.toString('base64'),
|
||||
})
|
||||
;({ config } = await import('../server/config.js'))
|
||||
pd = await import('../server/services/previewDeploy.js')
|
||||
envLib = await import('../server/services/previewEnv.js')
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const base = { previewMode: 'auto' }
|
||||
await createDocument('websiteProjects', { ...base, subdomain: 'offen', isPublic: true, previewState: 'online', previewDetected: 'static' }, 'p1')
|
||||
await createDocument('websiteProjects', { ...base, subdomain: 'statisch', previewState: 'online', previewDetected: 'build:npm' }, 'p2')
|
||||
await createDocument('websiteProjects', {
|
||||
...base, subdomain: 'devapp', repoFullName: 'JUSN/devapp', previewState: 'sleeping', previewDetected: 'dev:npm',
|
||||
previewBuiltSha: BUILT, previewBranch: 'feature-x', previewEnv: envLib.encryptEnv({ API_KEY: 'geheim-wert' }, key),
|
||||
}, 'p3')
|
||||
await createDocument('websiteProjects', {
|
||||
...base, subdomain: 'devoff', repoFullName: 'JUSN/devoff', previewState: 'failed', previewDetected: 'dev:npm',
|
||||
}, 'p4')
|
||||
})
|
||||
|
||||
after(async () => {
|
||||
await new Promise((r) => runner.close(r))
|
||||
await new Promise((r) => gitea.close(r))
|
||||
await aw.close()
|
||||
await fs.rm(tmp, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
beforeEach(() => {
|
||||
puts.length = 0
|
||||
giteaDown = false
|
||||
runnerFails = false
|
||||
config.previewRunner.enabled = true
|
||||
})
|
||||
|
||||
const read = () => fs.readFile(hostsFile, 'utf8')
|
||||
|
||||
// Erwartete Datei von VOR Task 10 (Schalter aus): zwei Ordner, 'offen' oeffentlich
|
||||
const block = (name, isPublic) => [
|
||||
` preview-${name}-https:`,
|
||||
` rule: "Host(\`${name}.project.webklar.com\`)"`,
|
||||
' entryPoints:',
|
||||
' - websecure',
|
||||
' service: preview-nginx@docker',
|
||||
...(isPublic ? [] : [' middlewares:', ' - preview-chain@docker']),
|
||||
' priority: 100',
|
||||
' tls:',
|
||||
' certResolver: letsencrypt',
|
||||
'',
|
||||
` preview-${name}-http:`,
|
||||
` rule: "Host(\`${name}.project.webklar.com\`)"`,
|
||||
' entryPoints:',
|
||||
' - web',
|
||||
' service: preview-nginx@docker',
|
||||
' middlewares:',
|
||||
' - preview-sites-redirect@docker',
|
||||
' priority: 100',
|
||||
].join('\n')
|
||||
const LEGACY = [
|
||||
'# Auto-generated by previewDeploy.syncPreviewTls - do not edit manually',
|
||||
'http:',
|
||||
' routers:',
|
||||
block('offen', true),
|
||||
block('statisch', false),
|
||||
'',
|
||||
].join('\n')
|
||||
|
||||
const routerBlock = (content, name) => {
|
||||
const start = content.indexOf(` preview-${name}-https:`)
|
||||
assert.ok(start >= 0, `Router ${name} fehlt`)
|
||||
const rest = content.slice(start + 1)
|
||||
const next = rest.search(/\n {4}preview-[a-z0-9-]+-https:/)
|
||||
return next < 0 ? content.slice(start) : content.slice(start, start + 1 + next)
|
||||
}
|
||||
|
||||
test('Schalter aus: Datei byte-identisch wie bisher, keine Dev-Hosts, keine Registrierung', async () => {
|
||||
config.previewRunner.enabled = false
|
||||
const result = await pd.syncPreviewTls()
|
||||
assert.deepEqual(result, { synced: true, hosts: 2, publicSource: 'appwrite' })
|
||||
assert.equal(await read(), LEGACY)
|
||||
await pd.waitForDevRegistration()
|
||||
assert.equal(puts.length, 0)
|
||||
})
|
||||
|
||||
test('Dev-Projekt bekommt Runner-Service (auch ohne Ordner), static bleibt nginx', async () => {
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
const content = await read()
|
||||
const dev = routerBlock(content, 'devapp')
|
||||
assert.equal((dev.match(/service: preview-runner@docker/g) || []).length, 2) // HTTPS + HTTP
|
||||
assert.match(dev, /preview-chain@docker/) // nicht oeffentlich => geschuetzt
|
||||
assert.match(dev, /preview-sites-redirect@docker/)
|
||||
assert.match(routerBlock(content, 'statisch'), /preview-nginx@docker/)
|
||||
assert.doesNotMatch(routerBlock(content, 'statisch'), /preview-runner/)
|
||||
// I5: auch ein fehlgeschlagener/neu bauender Dev-Host bleibt beim Runner (kein Umschalten auf nginx)
|
||||
assert.match(routerBlock(content, 'devoff'), /preview-runner@docker/)
|
||||
assert.deepEqual([...pd.devSubdomainsFromHostsFile(content)].sort(), ['devapp', 'devoff'])
|
||||
})
|
||||
|
||||
test('Appwrite nicht erreichbar: Dev-Hosts aus der letzten Datei bleiben', async () => {
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
aw.failNext({ method: 'GET', pathIncludes: 'documents', status: 500 }) // getPublicSubdomains
|
||||
aw.failNext({ method: 'GET', pathIncludes: 'documents', status: 500 }) // getDevProjects
|
||||
const errors = console.error
|
||||
console.error = () => {}
|
||||
try {
|
||||
const result = await pd.syncPreviewTls()
|
||||
assert.equal(result.publicSource, 'previous-file')
|
||||
} finally { console.error = errors }
|
||||
await pd.waitForDevRegistration()
|
||||
assert.deepEqual([...pd.devSubdomainsFromHostsFile(await read())].sort(), ['devapp', 'devoff'])
|
||||
assert.match(routerBlock(await read(), 'devapp'), /preview-runner@docker/)
|
||||
assert.equal(puts.length, 1) // nur die Runde mit Appwrite-Daten
|
||||
})
|
||||
|
||||
test('devSubdomainsFromHostsFile: nur HTTPS-Router mit Runner-Service', () => {
|
||||
const content = [
|
||||
'http:', ' routers:',
|
||||
' preview-a-https:', ' service: preview-runner@docker',
|
||||
' preview-a-http:', ' service: preview-runner@docker',
|
||||
' preview-b-https:', ' service: preview-nginx@docker',
|
||||
' preview-c-d-https:', ' rule: x', ' service: preview-runner@docker',
|
||||
].join('\n')
|
||||
assert.deepEqual([...pd.devSubdomainsFromHostsFile(content)].sort(), ['a', 'c-d'])
|
||||
assert.equal(pd.devSubdomainsFromHostsFile('').size, 0)
|
||||
assert.equal(pd.devSubdomainsFromHostsFile(undefined).size, 0)
|
||||
})
|
||||
|
||||
test('Registrierung: Runner bekommt Spec mit entschluesselten Variablen', async () => {
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
assert.equal(puts.length, 1)
|
||||
assert.equal(puts[0].url, '/dev/devapp')
|
||||
assert.deepEqual(puts[0].body, {
|
||||
sub: 'devapp', repo: 'JUSN/devapp', sha: BUILT, branch: 'feature-x', mode: 'dev', env: { API_KEY: 'geheim-wert' },
|
||||
})
|
||||
})
|
||||
|
||||
test('Registrierung: nicht entschluesselbare Variablen (Schluessel fehlt) => uebersprungen', async () => {
|
||||
const saved = process.env.PREVIEW_ENV_KEY
|
||||
delete process.env.PREVIEW_ENV_KEY
|
||||
const errors = console.error
|
||||
console.error = () => {}
|
||||
try {
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
} finally { process.env.PREVIEW_ENV_KEY = saved; console.error = errors }
|
||||
assert.equal(puts.length, 0)
|
||||
assert.match(await read(), /preview-runner@docker/)
|
||||
})
|
||||
|
||||
test('Registrierung: Runner-Fehler bricht den Abgleich nicht, Datei ist geschrieben', async () => {
|
||||
runnerFails = true
|
||||
await fs.rm(hostsFile, { force: true })
|
||||
const errors = console.error
|
||||
console.error = () => {}
|
||||
try {
|
||||
const result = await pd.syncPreviewTls()
|
||||
assert.equal(result.synced, true)
|
||||
await pd.waitForDevRegistration()
|
||||
} finally { console.error = errors }
|
||||
assert.equal(puts.length, 1)
|
||||
assert.match(await read(), /preview-runner@docker/)
|
||||
})
|
||||
|
||||
let giteaHits = 0
|
||||
|
||||
test('Registrierung: fehlender Stand oder Branch => uebersprungen, kein Gitea-Aufruf', async () => {
|
||||
const { updateDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const errors = console.error
|
||||
console.error = () => {}
|
||||
try {
|
||||
for (const patch of [{ previewBuiltSha: '' }, { previewBranch: '' }]) {
|
||||
await updateDocument('websiteProjects', 'p3', patch)
|
||||
giteaHits = 0
|
||||
puts.length = 0
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
assert.equal(puts.length, 0)
|
||||
assert.equal(giteaHits, 0)
|
||||
assert.match(await read(), /preview-runner@docker/)
|
||||
await updateDocument('websiteProjects', 'p3', { previewBuiltSha: BUILT, previewBranch: 'feature-x' })
|
||||
}
|
||||
} finally { console.error = errors }
|
||||
})
|
||||
|
||||
test('Registrierung: kaputte verschluesselte Variablen => uebersprungen', async () => {
|
||||
const { updateDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const good = envLib.encryptEnv({ API_KEY: 'geheim-wert' }, key)
|
||||
await updateDocument('websiteProjects', 'p3', { previewEnv: 'v1:xx:yy:zz' })
|
||||
const errors = console.error
|
||||
console.error = () => {}
|
||||
try {
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
} finally {
|
||||
console.error = errors
|
||||
await updateDocument('websiteProjects', 'p3', { previewEnv: good })
|
||||
}
|
||||
assert.equal(puts.length, 0)
|
||||
})
|
||||
|
||||
test('Registrierung: laeuft eine Runde, folgt genau eine Wiederholung', async () => {
|
||||
await Promise.all([pd.syncPreviewTls(), pd.syncPreviewTls(), pd.syncPreviewTls()])
|
||||
await pd.waitForDevRegistration()
|
||||
assert.equal(puts.length, 2)
|
||||
})
|
||||
|
||||
test('I5: Dev-Projekt im Zustand building bleibt beim Runner, static im Zustand building nicht', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await createDocument('websiteProjects', { previewMode: 'auto', subdomain: 'baut', repoFullName: 'JUSN/baut', previewState: 'building', previewDetected: 'dev:npm' }, 'p5')
|
||||
await createDocument('websiteProjects', { previewMode: 'auto', subdomain: 'baustatic', repoFullName: 'JUSN/baustatic', previewState: 'building', previewDetected: 'static' }, 'p6')
|
||||
await pd.syncPreviewTls()
|
||||
await pd.waitForDevRegistration()
|
||||
const content = await read()
|
||||
assert.match(routerBlock(content, 'baut'), /preview-runner@docker/)
|
||||
assert.ok(!pd.devSubdomainsFromHostsFile(content).has('baustatic'))
|
||||
})
|
||||
@@ -0,0 +1,103 @@
|
||||
import { test } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import crypto from 'node:crypto'
|
||||
import {
|
||||
encryptEnv, decryptEnv, validateEnvChange, envNames, maskLog, envKey,
|
||||
} from '../server/services/previewEnv.js'
|
||||
|
||||
const key = crypto.randomBytes(32)
|
||||
|
||||
test('Round-Trip und Format', () => {
|
||||
const s = encryptEnv({ A: 'x', B: 'ü' }, key)
|
||||
assert.match(s, /^v1:[^:]+:[^:]+:[^:]+$/)
|
||||
assert.deepEqual(decryptEnv(s, key), { A: 'x', B: 'ü' })
|
||||
assert.notEqual(encryptEnv({ A: 'x' }, key), encryptEnv({ A: 'x' }, key))
|
||||
})
|
||||
|
||||
test('leer', () => {
|
||||
assert.deepEqual(decryptEnv(encryptEnv({}, key), key), {})
|
||||
assert.deepEqual(decryptEnv('', key), {})
|
||||
assert.deepEqual(decryptEnv(null, key), {})
|
||||
})
|
||||
|
||||
test('falscher Schluessel und manipulierter Tag => Fehler', () => {
|
||||
const s = encryptEnv({ A: 'geheim' }, key)
|
||||
assert.throws(() => decryptEnv(s, crypto.randomBytes(32)))
|
||||
const p = s.split(':')
|
||||
const tag = Buffer.from(p[2], 'base64')
|
||||
tag[0] ^= 1
|
||||
p[2] = tag.toString('base64')
|
||||
assert.throws(() => decryptEnv(p.join(':'), key))
|
||||
assert.throws(() => decryptEnv('v2:a:b:c', key))
|
||||
})
|
||||
|
||||
test('envNames sortiert', () => {
|
||||
assert.deepEqual(envNames(encryptEnv({ B: '1', A: '2' }, key), key), ['A', 'B'])
|
||||
assert.deepEqual(envNames('', key), [])
|
||||
})
|
||||
|
||||
test('envKey', () => {
|
||||
const old = process.env.PREVIEW_ENV_KEY
|
||||
try {
|
||||
delete process.env.PREVIEW_ENV_KEY
|
||||
assert.equal(envKey(), null)
|
||||
process.env.PREVIEW_ENV_KEY = Buffer.alloc(16).toString('base64')
|
||||
assert.equal(envKey(), null)
|
||||
process.env.PREVIEW_ENV_KEY = key.toString('base64')
|
||||
assert.ok(envKey().equals(key))
|
||||
} finally {
|
||||
if (old === undefined) delete process.env.PREVIEW_ENV_KEY
|
||||
else process.env.PREVIEW_ENV_KEY = old
|
||||
}
|
||||
})
|
||||
|
||||
test('validateEnvChange', () => {
|
||||
let r = validateEnvChange({ A: '1' }, { set: { B: '2' }, remove: ['A', 'NIX'] })
|
||||
assert.deepEqual(r, { ok: true, env: { B: '2' } })
|
||||
assert.equal(validateEnvChange({}, { set: { abc: '1' } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, { set: { '1A': '1' } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, { set: { A: 'x'.repeat(4097) } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, { set: { A: 'x'.repeat(4096) } }).ok, true)
|
||||
assert.equal(validateEnvChange({}, { set: { A: 'ü'.repeat(2049) } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, { set: { A: 1 } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, { set: { A: 'a\nb' } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, { set: { A: 'a\0b' } }).ok, false)
|
||||
const many = Object.fromEntries(Array.from({ length: 31 }, (_, i) => [`V${i}`, '1']))
|
||||
r = validateEnvChange({}, { set: many })
|
||||
assert.equal(r.ok, false)
|
||||
assert.match(r.error, /30/)
|
||||
const thirty = Object.fromEntries(Array.from({ length: 30 }, (_, i) => [`V${i}`, '1']))
|
||||
assert.equal(validateEnvChange(thirty, { set: { V0: '2' } }).ok, true)
|
||||
assert.equal(validateEnvChange(thirty, { set: { NEU: '2' }, remove: ['V1'] }).ok, true)
|
||||
assert.equal(validateEnvChange(thirty, { set: { NEU: '2' } }).ok, false)
|
||||
assert.equal(validateEnvChange({}, {}).ok, true)
|
||||
})
|
||||
|
||||
test('maskLog', () => {
|
||||
const env = { A: 'geheim', B: 'abc', C: 'geheimer' , D: 'a.b+'}
|
||||
assert.equal(maskLog('x geheimer y geheim z abc a.b+ axbb', env), 'x ••• y ••• z abc ••• axbb')
|
||||
assert.equal(maskLog('', env), '')
|
||||
assert.equal(maskLog('text', {}), 'text')
|
||||
})
|
||||
|
||||
test('gekuerzter Tag, falsche IV-Laenge, manipulierter Ciphertext => Fehler', () => {
|
||||
const p = encryptEnv({ A: 'geheim' }, key).split(':')
|
||||
const b = (i) => Buffer.from(p[i], 'base64')
|
||||
const join = (x) => x.join(':')
|
||||
const short = [...p]; short[2] = b(2).subarray(0, 8).toString('base64')
|
||||
assert.throws(() => decryptEnv(join(short), key))
|
||||
const iv = [...p]; iv[1] = b(1).subarray(0, 8).toString('base64')
|
||||
assert.throws(() => decryptEnv(join(iv), key))
|
||||
const ct = [...p]; const c = b(3); c[0] ^= 1; ct[3] = c.toString('base64')
|
||||
assert.throws(() => decryptEnv(join(ct), key))
|
||||
})
|
||||
|
||||
test('entschluesselter Inhalt muss Objekt mit String-Werten sein', () => {
|
||||
for (const bad of [['x'], 'text', { A: 1 }, null]) {
|
||||
const iv = crypto.randomBytes(12)
|
||||
const c = crypto.createCipheriv('aes-256-gcm', key, iv, { authTagLength: 16 })
|
||||
const ct = Buffer.concat([c.update(JSON.stringify(bad), 'utf8'), c.final()])
|
||||
const s = ['v1', iv, c.getAuthTag(), ct].map((x) => (typeof x === 'string' ? x : x.toString('base64'))).join(':')
|
||||
assert.throws(() => decryptEnv(s, key))
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,263 @@
|
||||
// Abschluss-Fixes Preview-Runner: C1 (Status bleibt), I1 (Label), I3/I4/I6 (Abgleich), Laengen
|
||||
import { test, before, after, beforeEach } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import http from 'node:http'
|
||||
import fs from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import crypto from 'node:crypto'
|
||||
import { createHmac } from 'node:crypto'
|
||||
import express from 'express'
|
||||
import { startFakeAppwrite } from './helpers/fakeAppwrite.js'
|
||||
import { startFakeGitea } from './helpers/fakeGitea.js'
|
||||
|
||||
let aw, runner, gitea, tmp, pd, rec, config, baseUrl, server
|
||||
const key = crypto.randomBytes(32)
|
||||
const SHA = 'a'.repeat(40)
|
||||
const SECRET = 's3cret'
|
||||
const rs = { calls: [], runs: new Map(), next: 0, script: null, status: { dev: [], known: [], queue: 0 }, statusDown: false }
|
||||
|
||||
const done = (over = {}) => (job) => ({ state: 'done', detected: { kind: 'static', pm: 'npm' }, sha: job.sha, log: ['ok'], error: null, result: 'static', ...over })
|
||||
|
||||
before(async () => {
|
||||
aw = await startFakeAppwrite({ seed: { websiteProjects: [] } })
|
||||
runner = http.createServer((req, res) => {
|
||||
let raw = ''
|
||||
req.on('data', (c) => (raw += c))
|
||||
req.on('end', () => {
|
||||
const body = raw ? JSON.parse(raw) : null
|
||||
rs.calls.push({ method: req.method, url: req.url, body })
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
const send = (code, obj) => { res.statusCode = code; res.end(JSON.stringify(obj)) }
|
||||
if (req.method === 'GET' && req.url === '/status') return rs.statusDown ? send(500, { error: 'kaputt' }) : send(200, rs.status)
|
||||
if (req.method === 'POST' && req.url === '/jobs/build') {
|
||||
const runId = `run-${++rs.next}`
|
||||
rs.runs.set(runId, rs.script(body))
|
||||
return send(200, { runId })
|
||||
}
|
||||
const m = req.url.match(/^\/jobs\/(.+)$/)
|
||||
if (m && req.method === 'GET') return rs.runs.has(m[1]) ? send(200, rs.runs.get(m[1])) : send(404, { error: 'x' })
|
||||
if (req.method === 'PUT' && req.url.startsWith('/dev/')) return send(200, { state: 'sleeping' })
|
||||
send(404, {})
|
||||
})
|
||||
})
|
||||
await new Promise((r) => runner.listen(0, '127.0.0.1', r))
|
||||
gitea = await startFakeGitea({
|
||||
repos: [{ full_name: 'WEBklar/site-x' }, { full_name: 'WEBklar/site-y' }],
|
||||
contents: {
|
||||
'WEBklar/site-x:.webklar-preview.json': JSON.stringify({ enabled: true, subdomain: 'site-x', type: 'static', branch: 'main' }),
|
||||
'WEBklar/site-y:.webklar-preview.json': JSON.stringify({ enabled: true, subdomain: 'site-y', type: 'static', branch: 'main' }),
|
||||
},
|
||||
})
|
||||
tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'preview-final-'))
|
||||
Object.assign(process.env, {
|
||||
APPWRITE_ENDPOINT: aw.endpoint,
|
||||
APPWRITE_API_KEY: 'test-key',
|
||||
PREVIEW_DEPLOY_ROOT: path.join(tmp, 'previews'),
|
||||
TRAEFIK_DYNAMIC_DIR: path.join(tmp, 'dynamic'),
|
||||
GITEA_BASE_URL: gitea.baseUrl,
|
||||
GITEA_API_TOKEN: 'tok',
|
||||
GITEA_WEBHOOK_TOKEN: SECRET,
|
||||
PREVIEW_RUNNER_URL: `http://127.0.0.1:${runner.address().port}`,
|
||||
PREVIEW_RUNNER_TOKEN: 'sekret',
|
||||
PREVIEW_ENV_KEY: key.toString('base64'),
|
||||
})
|
||||
;({ config } = await import('../server/config.js'))
|
||||
config.previewRunner.enabled = true
|
||||
config.previewRunner.logDir = path.join(tmp, 'logs')
|
||||
pd = await import('../server/services/previewDeploy.js')
|
||||
pd.runnerTiming.pollMs = 10
|
||||
pd.runnerTiming.maxMs = 2000
|
||||
rec = await import('../server/services/previewReconcile.js')
|
||||
const { default: routes } = await import('../server/routes/webhook/gitea.js')
|
||||
const app = express()
|
||||
app.use(express.json({ limit: '2mb', verify: (req, _res, buf) => { req.rawBody = buf } }))
|
||||
app.use('/webhook', routes)
|
||||
server = await new Promise((r) => { const s = app.listen(0, '127.0.0.1', () => r(s)) })
|
||||
baseUrl = `http://127.0.0.1:${server.address().port}`
|
||||
})
|
||||
|
||||
after(async () => {
|
||||
await new Promise((r) => server.close(r))
|
||||
await new Promise((r) => runner.close(r))
|
||||
await gitea.close()
|
||||
await aw.close()
|
||||
await fs.rm(tmp, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
beforeEach(() => {
|
||||
rs.calls = []; rs.runs = new Map(); rs.script = done(); rs.status = { dev: [], known: [], queue: 0 }; rs.statusDown = false
|
||||
config.previewRunner.enabled = true
|
||||
})
|
||||
|
||||
async function push(repo, sha = SHA) {
|
||||
const payload = {
|
||||
ref: 'refs/heads/main',
|
||||
repository: { full_name: repo, name: repo.split('/')[1], html_url: `${gitea.baseUrl}/${repo}` },
|
||||
commits: [{ id: sha, message: 'x', author: { name: 'nico' } }],
|
||||
pusher: { username: 'nico' },
|
||||
after: sha,
|
||||
}
|
||||
const text = JSON.stringify(payload)
|
||||
const sig = createHmac('sha256', SECRET).update(text).digest('hex')
|
||||
const res = await fetch(`${baseUrl}/webhook/gitea`, { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Gitea-Signature': sig }, body: text })
|
||||
return res.status
|
||||
}
|
||||
const byRepo = (repo) => aw.all('websiteProjects').find((d) => d.repoFullName === repo)
|
||||
|
||||
// ---- C1 ----
|
||||
test('C1: Push mit Runner an behält status/provisioningStatus eines bereiten Projekts (auch bei Build-Fehler)', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await createDocument('websiteProjects', { repoFullName: 'WEBklar/site-x', projectName: 'X', subdomain: 'site-x', status: 'deployed', provisioningStatus: 'ready', previewState: 'online' }, 'cx')
|
||||
rs.script = done({ state: 'failed', result: null, error: 'Build kaputt' })
|
||||
assert.equal(await push('WEBklar/site-x'), 200)
|
||||
let d = byRepo('WEBklar/site-x')
|
||||
assert.equal(d.status, 'deployed')
|
||||
assert.equal(d.provisioningStatus, 'ready')
|
||||
await pd.waitForRunFollowers()
|
||||
d = byRepo('WEBklar/site-x')
|
||||
assert.equal(d.previewState, 'failed')
|
||||
assert.equal(d.status, 'deployed') // alte Preview bleibt online: nie herabstufen
|
||||
assert.equal(d.provisioningStatus, 'ready')
|
||||
})
|
||||
|
||||
test('C1: neues Projekt per Push wird nach Verfolger-Erfolg bereit', async () => {
|
||||
assert.equal(await push('WEBklar/site-y'), 200)
|
||||
let d = byRepo('WEBklar/site-y')
|
||||
assert.equal(d.provisioningStatus, 'pending')
|
||||
await pd.waitForRunFollowers()
|
||||
d = byRepo('WEBklar/site-y')
|
||||
assert.equal(d.previewState, 'online')
|
||||
assert.equal(d.status, 'deployed')
|
||||
assert.equal(d.provisioningStatus, 'ready')
|
||||
})
|
||||
|
||||
test('C1: deployKeepsStatus nur mit Runner an', () => {
|
||||
assert.equal(pd.deployKeepsStatus({ deployed: false, queued: true }), true)
|
||||
assert.equal(pd.deployKeepsStatus({ deployed: false, skipped: true }), true)
|
||||
assert.equal(pd.deployKeepsStatus({ deployed: true }), false)
|
||||
config.previewRunner.enabled = false
|
||||
assert.equal(pd.deployKeepsStatus({ deployed: false, queued: true }), false)
|
||||
})
|
||||
|
||||
// ---- I1 ----
|
||||
test('I1: Build fehlgeschlagen -> dev-Rückfall wird als dev:<pm> gespeichert', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const doc = await createDocument('websiteProjects', { repoFullName: 'WEBklar/auto-dev', projectName: 'A' }, 'i1')
|
||||
rs.script = done({ result: 'dev', detected: { kind: 'build', pm: 'pnpm' } })
|
||||
await pd.runDeploy('WEBklar/auto-dev', 'main', { subdomain: 'auto-dev', type: 'static', branch: 'main' }, { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(aw.get('websiteProjects', 'i1').previewDetected, 'dev:pnpm')
|
||||
assert.equal(aw.get('websiteProjects', 'i1').previewState, 'sleeping')
|
||||
assert.equal(pd.detectedLabel({ kind: 'build', pm: 'npm' }, 'dev'), 'dev:npm')
|
||||
assert.equal(pd.detectedLabel({ kind: 'build', pm: 'npm' }, 'build'), 'build:npm')
|
||||
assert.equal(pd.detectedLabel({ kind: 'static' }, undefined), 'static')
|
||||
})
|
||||
|
||||
// ---- Laengen ----
|
||||
test('previewError wird auf 500 Zeichen gekürzt (Patch geht nicht verloren)', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const doc = await createDocument('websiteProjects', { repoFullName: 'WEBklar/lang', projectName: 'L' }, 'lang')
|
||||
rs.script = done({ state: 'failed', result: null, error: 'x'.repeat(2000) })
|
||||
await pd.runDeploy('WEBklar/lang', 'main', { subdomain: 'lang', type: 'static', branch: 'main' }, { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
const d = aw.get('websiteProjects', 'lang')
|
||||
assert.equal(d.previewState, 'failed')
|
||||
assert.equal(d.previewError.length, 500)
|
||||
})
|
||||
|
||||
// ---- R18 Abgleich ----
|
||||
test('I3: building ohne Verfolger und > 70 min => failed; jung oder mit Verfolger bleibt', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await createDocument('websiteProjects', { repoFullName: 'WEBklar/r-alt', subdomain: 'r-alt', previewState: 'building' }, 'r1')
|
||||
await createDocument('websiteProjects', { repoFullName: 'WEBklar/r-jung', subdomain: 'r-jung', previewState: 'building' }, 'r2')
|
||||
const res = await rec.reconcilePreviews({ now: Date.now() + 80 * 60 * 1000 }) // "jetzt" in 80 min: beide dann aelter als 70 min
|
||||
assert.equal(res.staleFailed >= 2, true)
|
||||
assert.equal(aw.get('websiteProjects', 'r1').previewState, 'failed')
|
||||
assert.match(aw.get('websiteProjects', 'r1').previewError, /Status unbekannt/)
|
||||
})
|
||||
|
||||
test('I3: junges building bleibt, building mit laufendem Verfolger bleibt', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await createDocument('websiteProjects', { repoFullName: 'WEBklar/j1', subdomain: 'j1', previewState: 'building' }, 'j1')
|
||||
await rec.reconcilePreviews()
|
||||
assert.equal(aw.get('websiteProjects', 'j1').previewState, 'building')
|
||||
// Verfolger laeuft (Lauf haengt in 'running')
|
||||
const { createDocument: cd } = await import('../server/services/appwriteAdmin.js')
|
||||
const doc = await cd('websiteProjects', { repoFullName: 'WEBklar/j2', subdomain: 'j2', projectName: 'J' }, 'j2')
|
||||
rs.script = () => ({ state: 'running', log: [] })
|
||||
await pd.runDeploy('WEBklar/j2', 'main', { subdomain: 'j2', type: 'static', branch: 'main' }, { project: doc, sha: SHA })
|
||||
assert.equal(pd.hasLiveFollower('WEBklar/j2'), true)
|
||||
const res = await rec.reconcilePreviews({ now: Date.now() + 80 * 60 * 1000 })
|
||||
assert.equal(aw.get('websiteProjects', 'j2').previewState, 'building')
|
||||
assert.ok(res.staleFailed >= 0)
|
||||
// aufraeumen: Lauf beenden
|
||||
rs.runs.set('run-' + rs.next, { state: 'done', detected: { kind: 'static' }, sha: SHA, log: [], result: 'static' })
|
||||
await pd.waitForRunFollowers()
|
||||
})
|
||||
|
||||
test('I6: Dev-Zustand folgt dem Runner; Modus off bleibt unberührt', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const base = { previewDetected: 'dev:npm', previewBranch: 'main', previewBuiltSha: SHA, previewMode: 'auto' }
|
||||
await createDocument('websiteProjects', { ...base, repoFullName: 'WEBklar/d-on', subdomain: 'd-on', previewState: 'starting' }, 'd1')
|
||||
await createDocument('websiteProjects', { ...base, repoFullName: 'WEBklar/d-gone', subdomain: 'd-gone', previewState: 'online' }, 'd2')
|
||||
await createDocument('websiteProjects', { ...base, repoFullName: 'WEBklar/d-off', subdomain: 'd-off', previewState: 'sleeping', previewMode: 'off' }, 'd3')
|
||||
await createDocument('websiteProjects', { ...base, repoFullName: 'WEBklar/d-fail', subdomain: 'd-fail', previewState: 'online' }, 'd4')
|
||||
rs.status = { dev: [{ sub: 'd-on', state: 'online' }, { sub: 'd-fail', state: 'failed' }], known: ['d-on', 'd-gone', 'd-off', 'd-fail'], queue: 0 }
|
||||
const res = await rec.reconcilePreviews()
|
||||
assert.equal(aw.get('websiteProjects', 'd1').previewState, 'online')
|
||||
assert.equal(aw.get('websiteProjects', 'd2').previewState, 'sleeping')
|
||||
assert.equal(aw.get('websiteProjects', 'd3').previewState, 'sleeping')
|
||||
assert.equal(aw.get('websiteProjects', 'd4').previewState, 'sleeping')
|
||||
assert.equal(res.reregistered, 0)
|
||||
})
|
||||
|
||||
test('I4: dem Runner unbekannte Dev-Spezifikation wird erneut hinterlegt', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
await createDocument('websiteProjects', {
|
||||
previewDetected: 'dev:npm', previewBranch: 'main', previewBuiltSha: SHA, previewMode: 'dev',
|
||||
repoFullName: 'WEBklar/d-neu', subdomain: 'd-neu', previewState: 'sleeping',
|
||||
previewEnv: (await import('../server/services/previewEnv.js')).encryptEnv({ K: 'wert1234' }, key),
|
||||
}, 'd5')
|
||||
rs.status = { dev: [], known: [], queue: 0 }
|
||||
const res = await rec.reconcilePreviews()
|
||||
const put = rs.calls.filter((c) => c.method === 'PUT' && c.url === '/dev/d-neu')
|
||||
assert.equal(put.length, 1)
|
||||
assert.deepEqual(put[0].body, { sub: 'd-neu', repo: 'WEBklar/d-neu', sha: SHA, branch: 'main', mode: 'dev', env: { K: 'wert1234' } })
|
||||
assert.ok(res.reregistered >= 1)
|
||||
assert.equal(aw.get('websiteProjects', 'd5').previewState, 'sleeping')
|
||||
})
|
||||
|
||||
test('Abgleich: Runner nicht erreichbar / Schalter aus wirft nie und ändert nichts', async () => {
|
||||
const errors = console.error
|
||||
console.error = () => {}
|
||||
try {
|
||||
rs.statusDown = true
|
||||
const r1 = await rec.reconcilePreviews()
|
||||
assert.ok(r1.error)
|
||||
assert.equal(aw.get('websiteProjects', 'd2').previewState, 'sleeping')
|
||||
assert.equal(await rec.runReconcile().then((r) => typeof r), 'object')
|
||||
} finally { console.error = errors }
|
||||
config.previewRunner.enabled = false
|
||||
assert.deepEqual(await rec.reconcilePreviews(), { skipped: true })
|
||||
})
|
||||
|
||||
// ---- R19 ----
|
||||
test('R19: Verfolger meldet dev immer mit mode dev; registerDevRound liefert identischen Auftrag', async () => {
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const envLib = await import('../server/services/previewEnv.js')
|
||||
const doc = await createDocument('websiteProjects', {
|
||||
repoFullName: 'WEBklar/r19', projectName: 'R', subdomain: 'r19', previewMode: 'auto',
|
||||
previewEnv: envLib.encryptEnv({ K: 'wert1234' }, key),
|
||||
}, 'r19')
|
||||
rs.script = done({ result: 'dev', detected: { kind: 'build', pm: 'npm' } })
|
||||
await pd.runDeploy('WEBklar/r19', 'main', { subdomain: 'r19', type: 'static', branch: 'main' }, { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(rs.calls.find((c) => c.method === 'POST' && c.url === '/jobs/build').body.mode, 'auto')
|
||||
const first = rs.calls.find((c) => c.method === 'PUT' && c.url === '/dev/r19').body
|
||||
assert.equal(first.mode, 'dev')
|
||||
rs.calls = []
|
||||
await pd.registerDevRound([aw.get('websiteProjects', 'r19')])
|
||||
const second = rs.calls.find((c) => c.method === 'PUT' && c.url === '/dev/r19').body
|
||||
assert.deepEqual(second, first)
|
||||
})
|
||||
@@ -0,0 +1,357 @@
|
||||
import { test, before, after, beforeEach } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import http from 'node:http'
|
||||
import fs from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import crypto from 'node:crypto'
|
||||
import { startFakeAppwrite } from './helpers/fakeAppwrite.js'
|
||||
|
||||
let aw, runner, gitea, tmp, pd, logs, envLib
|
||||
const key = crypto.randomBytes(32)
|
||||
const SHA = 'a'.repeat(40)
|
||||
|
||||
// Fake-Runner: zaehlt Aufrufe, Laeufe per Skript steuerbar
|
||||
const rs = { calls: [], runs: new Map(), next: 0, script: null, submitError: null }
|
||||
function startRunner() {
|
||||
return http.createServer((req, res) => {
|
||||
let raw = ''
|
||||
req.on('data', (c) => (raw += c))
|
||||
req.on('end', () => {
|
||||
const body = raw ? JSON.parse(raw) : null
|
||||
rs.calls.push({ method: req.method, url: req.url, auth: req.headers.authorization, body })
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
const send = (code, obj) => { res.statusCode = code; res.end(JSON.stringify(obj)) }
|
||||
if (req.method === 'POST' && req.url === '/jobs/build') {
|
||||
if (rs.submitError) return send(rs.submitError, { error: 'Server ausgelastet' })
|
||||
const runId = `run-${++rs.next}`
|
||||
rs.runs.set(runId, rs.script(body))
|
||||
return send(200, { runId })
|
||||
}
|
||||
const m = req.url.match(/^\/jobs\/(.+)$/)
|
||||
if (m && req.method === 'GET') return rs.runs.has(m[1]) ? send(200, rs.runs.get(m[1])) : send(404, { error: 'unbekannt' })
|
||||
if (req.method === 'PUT' && req.url.startsWith('/dev/')) return send(200, { state: 'sleeping' })
|
||||
send(404, { error: 'x' })
|
||||
})
|
||||
})
|
||||
}
|
||||
const done = (over = {}) => (job) => ({ state: 'done', detected: { kind: 'static', pm: 'npm' }, sha: job.sha, log: ['ok'], error: null, result: 'static', ...over })
|
||||
const submits = () => rs.calls.filter((c) => c.url === '/jobs/build')
|
||||
|
||||
before(async () => {
|
||||
aw = await startFakeAppwrite({ seed: { websiteProjects: [] } })
|
||||
runner = startRunner()
|
||||
await new Promise((r) => runner.listen(0, '127.0.0.1', r))
|
||||
gitea = http.createServer((req, res) => {
|
||||
res.setHeader('Content-Type', 'application/json')
|
||||
res.end(JSON.stringify({ commit: { id: 'b'.repeat(40) } }))
|
||||
})
|
||||
await new Promise((r) => gitea.listen(0, '127.0.0.1', r))
|
||||
tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'preview-runner-'))
|
||||
Object.assign(process.env, {
|
||||
APPWRITE_ENDPOINT: aw.endpoint,
|
||||
APPWRITE_API_KEY: 'test-key',
|
||||
PREVIEW_DEPLOY_ROOT: path.join(tmp, 'previews'),
|
||||
TRAEFIK_DYNAMIC_DIR: path.join(tmp, 'dynamic'),
|
||||
GITEA_BASE_URL: `http://127.0.0.1:${gitea.address().port}`,
|
||||
GITEA_API_TOKEN: 'tok',
|
||||
PREVIEW_RUNNER_URL: `http://127.0.0.1:${runner.address().port}`,
|
||||
PREVIEW_RUNNER_TOKEN: 'sekret',
|
||||
PREVIEW_ENV_KEY: key.toString('base64'),
|
||||
})
|
||||
const { config } = await import('../server/config.js')
|
||||
config.previewRunner.enabled = true
|
||||
config.previewRunner.logDir = path.join(tmp, 'logs')
|
||||
pd = await import('../server/services/previewDeploy.js')
|
||||
pd.runnerTiming.pollMs = 10
|
||||
pd.runnerTiming.maxMs = 2000
|
||||
logs = await import('../server/services/previewLogs.js')
|
||||
envLib = await import('../server/services/previewEnv.js')
|
||||
globalThis.__cfg = config
|
||||
})
|
||||
|
||||
after(async () => {
|
||||
await new Promise((r) => runner.close(r))
|
||||
await new Promise((r) => gitea.close(r))
|
||||
await aw.close()
|
||||
await fs.rm(tmp, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
let n = 0
|
||||
async function project(extra = {}) {
|
||||
const id = `proj${++n}`
|
||||
const repo = `JUSN/site-${n}`
|
||||
const { createDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const doc = await createDocument('websiteProjects', { repoFullName: repo, projectName: `S${n}`, ...extra }, id)
|
||||
return { id, repo, doc }
|
||||
}
|
||||
const cfg = (repo) => ({ subdomain: repo.split('/')[1], type: 'static', branch: 'main' })
|
||||
|
||||
beforeEach(() => {
|
||||
rs.calls = []; rs.runs = new Map(); rs.script = done(); rs.submitError = null
|
||||
globalThis.__cfg.previewRunner.enabled = true
|
||||
})
|
||||
|
||||
test('Schalter aus: kein Runner-Aufruf, alter Pfad', async () => {
|
||||
const { repo } = await project()
|
||||
globalThis.__cfg.previewRunner.enabled = false
|
||||
globalThis.__cfg.preview.deployRoot = ''
|
||||
const r = await pd.runDeploy(repo, 'main', cfg(repo))
|
||||
globalThis.__cfg.preview.deployRoot = path.join(tmp, 'previews')
|
||||
assert.equal(r.deployed, false)
|
||||
assert.match(r.reason, /PREVIEW_DEPLOY_ROOT/)
|
||||
assert.equal(rs.calls.length, 0)
|
||||
})
|
||||
|
||||
test('an: Auftrag mit entschlüsselten Variablen, sofortige Rückgabe, Ergebnis online', async () => {
|
||||
const { id, repo, doc } = await project({ previewEnv: envLib.encryptEnv({ API_URL: 'https://x.test' }, key) })
|
||||
const r = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
assert.equal(r.queued, true)
|
||||
assert.equal(r.deployed, false)
|
||||
assert.equal(r.runId, 'run-' + rs.next)
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'building')
|
||||
const [sub] = submits()
|
||||
assert.equal(sub.auth, 'Bearer sekret')
|
||||
assert.deepEqual(sub.body, { sub: repo.split('/')[1], repo, sha: SHA, branch: 'main', mode: 'auto', env: { API_URL: 'https://x.test' } })
|
||||
await pd.waitForRunFollowers()
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'online')
|
||||
assert.equal(d.previewDetected, 'static')
|
||||
assert.equal(d.previewBuiltSha, SHA)
|
||||
assert.ok(d.previewBuiltAt)
|
||||
assert.equal(d.previewError, '')
|
||||
assert.equal((await logs.listRunLogs(id)).length, 1)
|
||||
})
|
||||
|
||||
test('Sha fehlt: Branch-Kopf per Gitea-API', async () => {
|
||||
const { repo, doc } = await project()
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(submits()[0].body.sha, 'b'.repeat(40))
|
||||
})
|
||||
|
||||
test('ohne project-Option: Dokument per Repo gefunden', async () => {
|
||||
const { id, repo } = await project({ previewMode: 'build' })
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(submits()[0].body.mode, 'build')
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'online')
|
||||
})
|
||||
|
||||
test('failed: Zustand failed, Fehler gespeichert, Folge-Stand nicht online', async () => {
|
||||
rs.script = done({ state: 'failed', result: null, error: 'Build fehlgeschlagen', detected: { kind: 'build', pm: 'pnpm' }, log: ['boom'] })
|
||||
const { id, repo, doc } = await project({ previewBuiltSha: 'c'.repeat(40), previewState: 'online' })
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'failed')
|
||||
assert.equal(d.previewError, 'Build fehlgeschlagen')
|
||||
assert.equal(d.previewBuiltSha, 'c'.repeat(40))
|
||||
assert.equal(await fs.stat(path.join(tmp, 'previews', repo.split('/')[1])).catch(() => null), null)
|
||||
})
|
||||
|
||||
test('dev: sleeping + Registrierung per PUT /dev/<sub>', async () => {
|
||||
rs.script = done({ result: 'dev', detected: { kind: 'dev', pm: 'pnpm' } })
|
||||
const { id, repo, doc } = await project()
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'sleeping')
|
||||
assert.equal(d.previewBranch, 'main')
|
||||
assert.equal(d.previewBuiltSha, SHA)
|
||||
assert.equal(d.previewDetected, 'dev:pnpm')
|
||||
const put = rs.calls.find((c) => c.method === 'PUT')
|
||||
assert.equal(put.url, `/dev/${repo.split('/')[1]}`)
|
||||
assert.equal(put.body.sha, SHA)
|
||||
})
|
||||
|
||||
test('build: previewDetected build:<pm>', async () => {
|
||||
rs.script = done({ result: 'build', detected: { kind: 'build', pm: 'yarn' } })
|
||||
const { id, repo, doc } = await project()
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(aw.get('websiteProjects', id).previewDetected, 'build:yarn')
|
||||
})
|
||||
|
||||
test('Modus off: kein Auftrag, previewState off', async () => {
|
||||
const { id, repo, doc } = await project({ previewMode: 'off' })
|
||||
const r = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
assert.equal(r.skipped, true)
|
||||
assert.equal(rs.calls.length, 0)
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'off')
|
||||
})
|
||||
|
||||
test('Protokoll maskiert und nur 5 behalten', async () => {
|
||||
const env = { TOKEN: 'supergeheim' }
|
||||
const id = 'logproj'
|
||||
const files = []
|
||||
for (let i = 0; i < 7; i++) files.push(await logs.saveRunLog(id, [`Zeile ${i}`, 'nutzt supergeheim hier'], env))
|
||||
const list = await logs.listRunLogs(id)
|
||||
assert.equal(list.length, 5)
|
||||
assert.equal(list[0].runFile, files[6].toString())
|
||||
const text = await logs.readRunLog(id, list[0].runFile)
|
||||
assert.ok(!text.includes('supergeheim'))
|
||||
assert.ok(text.includes('•••'))
|
||||
assert.equal(await logs.readRunLog(id, files[0]), null)
|
||||
})
|
||||
|
||||
test('Protokoll: nur die letzten 300 Zeilen, keine Pfadtricks', async () => {
|
||||
const lines = Array.from({ length: 400 }, (_, i) => `z${i}`)
|
||||
const f = await logs.saveRunLog('logproj2', lines, {})
|
||||
const text = (await logs.readRunLog('logproj2', f)).trim().split('\n')
|
||||
assert.equal(text.length, 300)
|
||||
assert.equal(text[0], 'z100')
|
||||
assert.equal(await logs.readRunLog('logproj2', '../../x.log'), null)
|
||||
assert.equal(await logs.readRunLog('logproj2', '../logproj/' + f), null)
|
||||
assert.equal(await logs.readRunLog('..', f), null)
|
||||
assert.deepEqual(await logs.listRunLogs('..'), [])
|
||||
})
|
||||
|
||||
test('Variablen im Lauf-Protokoll maskiert (Ende-zu-Ende)', async () => {
|
||||
rs.script = done({ log: ['Schluessel=supergeheim123'] })
|
||||
const { id, repo, doc } = await project({ previewEnv: envLib.encryptEnv({ K: 'supergeheim123' }, key) })
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
const [{ runFile }] = await logs.listRunLogs(id)
|
||||
assert.equal((await logs.readRunLog(id, runFile)).trim(), 'Schluessel=•••')
|
||||
})
|
||||
|
||||
test('neuerer Lauf verdrängt den älteren Verfolger', async () => {
|
||||
// Lauf 1 bleibt lange "running", Lauf 2 ist sofort fertig
|
||||
const { id, repo, doc } = await project()
|
||||
rs.script = () => ({ state: 'running', detected: null, sha: SHA, log: [], error: null, result: null })
|
||||
const r1 = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
rs.script = done({ log: ['zwei'] })
|
||||
const sha2 = 'd'.repeat(40)
|
||||
const r2 = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: sha2 })
|
||||
assert.notEqual(r1.runId, r2.runId)
|
||||
// Lauf 1 meldet jetzt "failed" - darf nichts mehr schreiben
|
||||
rs.runs.set(r1.runId, { state: 'failed', error: 'alt', log: ['eins'], detected: null, sha: SHA, result: null })
|
||||
await pd.waitForRunFollowers()
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'online')
|
||||
assert.equal(d.previewBuiltSha, sha2)
|
||||
assert.equal((await logs.listRunLogs(id)).length, 1)
|
||||
})
|
||||
|
||||
test('PREVIEW_ENV_KEY fehlt bei gespeicherten Variablen: Build ohne Variablen, Fehlertext', async () => {
|
||||
const { id, repo, doc } = await project({ previewEnv: envLib.encryptEnv({ A: 'xxxxxx' }, key) })
|
||||
const saved = process.env.PREVIEW_ENV_KEY
|
||||
delete process.env.PREVIEW_ENV_KEY
|
||||
try {
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
} finally {
|
||||
process.env.PREVIEW_ENV_KEY = saved
|
||||
}
|
||||
assert.deepEqual(submits()[0].body.env, {})
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'online')
|
||||
assert.equal(d.previewError, 'Preview-Variablen nicht entschlüsselbar (PREVIEW_ENV_KEY fehlt)')
|
||||
})
|
||||
|
||||
test('Runner ausgelastet (503): failed mit Meldung, kein Wurf', async () => {
|
||||
rs.submitError = 503
|
||||
const { id, repo, doc } = await project()
|
||||
const r = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
assert.equal(r.deployed, false)
|
||||
assert.equal(r.error, 'Server ausgelastet')
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'failed')
|
||||
})
|
||||
|
||||
test('Modus off während ein Lauf läuft: Zustand bleibt off, kein TLS-Sync vom alten Verfolger', async () => {
|
||||
const { id, repo, doc } = await project()
|
||||
rs.script = () => ({ state: 'running', detected: null, sha: SHA, log: [], error: null, result: null })
|
||||
const r1 = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
const { updateDocument } = await import('../server/services/appwriteAdmin.js')
|
||||
const off = await updateDocument('websiteProjects', id, { previewMode: 'off' })
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: off, sha: SHA })
|
||||
rs.runs.set(r1.runId, { state: 'done', detected: { kind: 'static', pm: 'npm' }, sha: SHA, log: ['alt'], error: null, result: 'static' })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'off')
|
||||
assert.equal((await logs.listRunLogs(id)).length, 0)
|
||||
})
|
||||
|
||||
test('gleichzeitige Aufrufe: der neueste gewinnt', async () => {
|
||||
const { id, repo, doc } = await project()
|
||||
rs.script = (job) => ({ state: 'done', detected: { kind: 'static', pm: 'npm' }, sha: job.sha, log: [job.sha], error: null, result: 'static' })
|
||||
const sha2 = 'e'.repeat(40)
|
||||
const [, r2] = await Promise.all([
|
||||
pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA }),
|
||||
pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: sha2 }),
|
||||
])
|
||||
await pd.waitForRunFollowers()
|
||||
assert.ok(r2.queued)
|
||||
assert.equal(aw.get('websiteProjects', id).previewBuiltSha, sha2)
|
||||
})
|
||||
|
||||
test('fehlgeschlagener neuer Auftrag verdrängt den alten Verfolger', async () => {
|
||||
const { id, repo, doc } = await project()
|
||||
rs.script = () => ({ state: 'running', detected: null, sha: SHA, log: [], error: null, result: null })
|
||||
const r1 = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
rs.submitError = 503
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
rs.runs.set(r1.runId, { state: 'done', detected: { kind: 'static' }, sha: SHA, log: [], error: null, result: 'static' })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'failed')
|
||||
})
|
||||
|
||||
test('R15: folgt queued/running über die alte 16-Minuten-Grenze, Runner-Ausfall lange toleriert', async () => {
|
||||
const { id, repo, doc } = await project()
|
||||
let polls = 0
|
||||
rs.script = () => ({ state: 'queued', detected: null, sha: SHA, log: [], error: null, result: null })
|
||||
const r = await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
const orig = rs.runs.get(r.runId)
|
||||
Object.defineProperty(orig, 'state', { get() { polls++; return polls < 15 ? 'queued' : 'done' }, enumerable: true })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.ok(polls >= 15)
|
||||
assert.equal(aw.get('websiteProjects', id).previewState, 'online')
|
||||
})
|
||||
|
||||
test('R15: Gesamtzeit überschritten => failed mit "Status unbekannt"', async () => {
|
||||
const { id, repo, doc } = await project()
|
||||
rs.script = () => ({ state: 'running', detected: null, sha: SHA, log: [], error: null, result: null })
|
||||
const saved = pd.runnerTiming.maxMs
|
||||
pd.runnerTiming.maxMs = 60
|
||||
try {
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
} finally { pd.runnerTiming.maxMs = saved }
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'failed')
|
||||
assert.equal(d.previewError, 'Status unbekannt – bitte neu bauen')
|
||||
})
|
||||
|
||||
test('R15: Runner dauerhaft unerreichbar (> unreachableMs) => failed', async () => {
|
||||
const { id, repo, doc } = await project()
|
||||
rs.script = () => ({ state: 'running', detected: null, sha: SHA, log: [], error: null, result: null })
|
||||
const { config } = await import('../server/config.js')
|
||||
const savedUrl = config.previewRunner.url
|
||||
const saved = pd.runnerTiming.unreachableMs
|
||||
pd.runnerTiming.unreachableMs = 50
|
||||
try {
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
config.previewRunner.url = 'http://127.0.0.1:1'
|
||||
await pd.waitForRunFollowers()
|
||||
} finally { config.previewRunner.url = savedUrl; pd.runnerTiming.unreachableMs = saved }
|
||||
const d = aw.get('websiteProjects', id)
|
||||
assert.equal(d.previewState, 'failed')
|
||||
assert.match(d.previewError, /nicht erreichbar/)
|
||||
})
|
||||
|
||||
test('Runner-Fehlertext wird maskiert', async () => {
|
||||
rs.script = done({ state: 'failed', result: null, error: 'Fehler mit supergeheim99', log: [] })
|
||||
const { id, repo, doc } = await project({ previewEnv: envLib.encryptEnv({ K: 'supergeheim99' }, key) })
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
assert.equal(aw.get('websiteProjects', id).previewError, 'Fehler mit •••')
|
||||
})
|
||||
|
||||
test('dev-Ergebnis: PUT /dev/<sub> mit vollständigem Auftrag inkl. Variablen', async () => {
|
||||
rs.script = done({ result: 'dev', detected: { kind: 'dev', pm: 'npm' } })
|
||||
const { repo, doc } = await project({ previewEnv: envLib.encryptEnv({ A: 'wert1234' }, key), previewMode: 'dev' })
|
||||
await pd.runDeploy(repo, 'main', cfg(repo), { project: doc, sha: SHA })
|
||||
await pd.waitForRunFollowers()
|
||||
const put = rs.calls.find((c) => c.method === 'PUT')
|
||||
assert.deepEqual(put.body, { sub: repo.split('/')[1], repo, sha: SHA, branch: 'main', mode: 'dev', env: { A: 'wert1234' } })
|
||||
})
|
||||
@@ -0,0 +1,58 @@
|
||||
import { test } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { PREVIEW_RUNNER_ATTRIBUTES, main } from '../scripts/preview-runner-setup.mjs'
|
||||
import { config } from '../server/config.js'
|
||||
|
||||
function mockClient(existing = []) {
|
||||
const created = []
|
||||
return {
|
||||
created,
|
||||
async listAttributes() {
|
||||
return existing.map((key) => ({ key }))
|
||||
},
|
||||
async createAttribute(attr) {
|
||||
created.push(attr.key)
|
||||
},
|
||||
async waitForAttribute() {},
|
||||
}
|
||||
}
|
||||
|
||||
test('Schema: genau die 8 Attribute mit Groessen, alle optional', () => {
|
||||
assert.deepEqual(
|
||||
PREVIEW_RUNNER_ATTRIBUTES.map((a) => a.key),
|
||||
['previewMode', 'previewEnv', 'previewState', 'previewDetected', 'previewBuiltSha', 'previewBranch', 'previewBuiltAt', 'previewError']
|
||||
)
|
||||
assert.ok(PREVIEW_RUNNER_ATTRIBUTES.every((a) => a.required === false && a.type === 'string'))
|
||||
const sizes = Object.fromEntries(PREVIEW_RUNNER_ATTRIBUTES.map((a) => [a.key, a.size]))
|
||||
assert.deepEqual(sizes, {
|
||||
previewMode: 10,
|
||||
previewEnv: 20000,
|
||||
previewState: 20,
|
||||
previewDetected: 40,
|
||||
previewBuiltSha: 40,
|
||||
previewBranch: 100,
|
||||
previewBuiltAt: 40,
|
||||
previewError: 500,
|
||||
})
|
||||
})
|
||||
|
||||
test('main legt nur fehlende Attribute an', async () => {
|
||||
const client = mockClient(['previewMode'])
|
||||
const result = await main({ check: false, client, log: () => {} })
|
||||
assert.equal(client.created.length, 7)
|
||||
assert.ok(!client.created.includes('previewMode'))
|
||||
assert.equal(result.missing.length, 7)
|
||||
})
|
||||
|
||||
test('main --check legt nichts an und meldet Fehlende', async () => {
|
||||
const client = mockClient(['previewMode'])
|
||||
const result = await main({ check: true, client, log: () => {} })
|
||||
assert.equal(client.created.length, 0)
|
||||
assert.equal(result.missing.length, 7)
|
||||
})
|
||||
|
||||
test('Konfiguration: previewRunner ist standardmaessig aus', () => {
|
||||
assert.equal(config.previewRunner.enabled, false)
|
||||
assert.equal(config.previewRunner.url, 'http://webklar-preview-runner:8080')
|
||||
assert.equal(config.previewRunner.token, '')
|
||||
})
|
||||
@@ -53,7 +53,7 @@ test('Katalog + Gruppen, Archiv/Vorlage ausgeschlossen, Namen aufgelöst', async
|
||||
assert.deepEqual(body.groups.website[0], {
|
||||
id: 'p1', projectName: 'Doener', repoFullName: 'JUSN/doener', customerId: 'c1',
|
||||
customerName: 'Döner Nation', ticketId: 't1', woid: '100115',
|
||||
previewUrl: 'https://doener.project.webklar.com', features: {},
|
||||
previewUrl: 'https://doener.project.webklar.com', previewState: '', features: {},
|
||||
})
|
||||
assert.equal(body.groups.missing[0].previewUrl, 'https://ohne.de')
|
||||
assert.equal(body.groups.missing[0].customerName, '')
|
||||
|
||||
Reference in new issue
Block a user