fix(tests): pin _cfg_mtime in _models_with_cfg to prevent ordering-dependent failure — v0.50.109 (fixes #754) (#756)

## Summary

Fixes the ordering-dependent test failure in `test_custom_provider_display_name.py` (issue #754).

**Root cause:** `_models_with_cfg()` patches `config.cfg` then calls `get_available_models()`. That function checks `config.yaml`'s mtime on every call — if it has changed since the last `reload_config()`, it calls `reload_config()` again, which reads from disk and silently overwrites the patch. Any test that writes `config.yaml` (e.g. via `save_settings()`) before this test runs changes the mtime and triggers the reload.

**Fix:** Pin `config._cfg_mtime` to the current `config.yaml` mtime before calling `get_available_models()`, then restore it in the `finally` block. This is the same pattern already used in `test_model_resolver.py` (lines 249, 393).

**Also restores `_cfg_mtime`** in the `finally` block so the patch leaves no side effects on subsequent tests.

## Tests

1510 passed — the previously-flaky test now passes regardless of which tests ran before it.

Closes #754
This commit is contained in:
nesquena-hermes
2026-04-20 00:39:24 -07:00
committed by GitHub
parent 69570ca77c
commit a1c5c395e5
2 changed files with 21 additions and 1 deletions

View File

@@ -1,5 +1,10 @@
# Hermes Web UI -- Changelog
## [v0.50.109] — 2026-04-20
### Fixed
- **Named custom provider test isolation** — `_models_with_cfg()` in `tests/test_custom_provider_display_name.py` now pins `_cfg_mtime` before calling `get_available_models()`, preventing the mtime-guard inside that function from firing `reload_config()` and silently discarding the patched `config.cfg`. This fixes an ordering-dependent test failure where any test that wrote `config.yaml` before this test ran would cause `get_available_models()` to return the real OpenRouter model list instead of the patched Agent37 group. (Fixes #754)
## [v0.50.108] — 2026-04-20
### Fixed

View File

@@ -9,18 +9,33 @@ import api.config as config
def _models_with_cfg(model_cfg=None, custom_providers=None, active_provider=None):
"""Temporarily patch config.cfg, call get_available_models(), restore."""
"""Temporarily patch config.cfg, call get_available_models(), restore.
Also pins _cfg_mtime to the current config.yaml mtime before calling
get_available_models(). Without this, if a prior test wrote config.yaml
(changing its mtime), the mtime-guard inside get_available_models() fires
reload_config() which overwrites config.cfg with the real on-disk values,
silently discarding the patch and causing ordering-dependent failures.
This matches the pattern used in test_model_resolver.py.
"""
old_cfg = dict(config.cfg)
old_mtime = config._cfg_mtime
config.cfg.clear()
if model_cfg:
config.cfg["model"] = model_cfg
if custom_providers is not None:
config.cfg["custom_providers"] = custom_providers
# Pin mtime so get_available_models() skips its reload_config() guard.
try:
config._cfg_mtime = config.Path(config._get_config_path()).stat().st_mtime
except Exception:
config._cfg_mtime = 0.0 # no config.yaml present; reload guard is a no-op
try:
return config.get_available_models()
finally:
config.cfg.clear()
config.cfg.update(old_cfg)
config._cfg_mtime = old_mtime
# ── Named provider shows its name in the dropdown ─────────────────────────────