ct-public/isparkclaw-webui
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(tests): restore conftest default model in test_default_model_updates_hermes_config — fixes CI ordering failure

Browse Source 
The test was restoring original_model from /api/models, but after prior runs
the config.yaml model.default field could be stale, causing the restore to
bake in the wrong value. Fix: always restore to TEST_DEFAULT_MODEL (the
conftest-injected env value) for deterministic ordering-independent cleanup.

Also exposes TEST_DEFAULT_MODEL from _pytest_port.py so other tests that
mutate the default model can use it for clean teardown.

TESTING.md: update automated test count from 1353 to 1578.
This commit is contained in:
nesquena-hermes
2026-04-21 02:25:14 +00:00
parent 28b4777b5a
commit 629d4290ed
3 changed files with 9 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
TESTING.md
View File

@@ -8,7 +8,7 @@
> Prerequisites: SSH tunnel is active on port 8787. Open http://localhost:8787 in browser.
> Server health check: curl http://127.0.0.1:8787/health should return {"status":"ok"}.
>
> Automated coverage: 1353 tests collected via `pytest tests/ --collect-only -q`. Includes onboarding coverage for bootstrap/static wizard presence, real provider config persistence (`config.yaml` + `.env`), the `/api/onboarding/*` backend, the onboarding skip/existing-config guard, and CSS regression coverage for smooth thinking/tool card disclosure animation.
> Automated coverage: 1578 tests collected via `pytest tests/ --collect-only -q`. Includes onboarding coverage for bootstrap/static wizard presence, real provider config persistence (`config.yaml` + `.env`), the `/api/onboarding/*` backend, the onboarding skip/existing-config guard, and CSS regression coverage for smooth thinking/tool card disclosure animation.
> Run: `pytest tests/ -v --timeout=60`
>
> Local regression focus: verify that a previously closed workspace panel stays visually closed from first paint through boot completion on desktop refresh; there should be no brief open-then-close flash.

4
tests/_pytest_port.py
View File

@@ -40,3 +40,7 @@ TEST_STATE_DIR = pathlib.Path(os.environ.get(
'HERMES_WEBUI_TEST_STATE_DIR',
str(_HERMES_HOME / _auto_state_dir_name(_REPO_ROOT))
))
# Default model injected by conftest — tests that mutate the default model
# must restore to this value so later tests see a consistent baseline.
TEST_DEFAULT_MODEL = os.environ.get('HERMES_WEBUI_DEFAULT_MODEL', 'openai/gpt-5.4-mini')

10
tests/test_sprint12.py
View File

@@ -3,7 +3,7 @@ Sprint 12 Tests: settings panel, session pinning, session import, SSE reconnect.
"""
import json, pathlib, urllib.error, urllib.request, urllib.parse
from tests._pytest_port import BASE
from tests._pytest_port import BASE, TEST_DEFAULT_MODEL
def get(path):
@@ -40,8 +40,6 @@ def test_settings_get_returns_defaults():
def test_default_model_updates_hermes_config():
"""POST /api/default-model updates the effective Hermes default model."""
original, _ = get("/api/models")
original_model = original.get("default_model") or ""
try:
d, status = post("/api/default-model", {"model": "anthropic/claude-sonnet-4.6"})
assert status == 200
@@ -50,9 +48,9 @@ def test_default_model_updates_hermes_config():
# Both should resolve to the same model (may differ in prefix normalization)
assert 'claude-sonnet-4.6' in d2['default_model']
finally:
# Always restore — regardless of test ordering or failures
if original_model:
post("/api/default-model", {"model": original_model})
# Always restore to the conftest-injected default so later tests see
# a consistent baseline regardless of test ordering.
post("/api/default-model", {"model": TEST_DEFAULT_MODEL})
def test_settings_does_not_persist_default_model():