test: run pytest in CI — real unit job + Qdrant-backed integration job #52

Merged

jcabillot merged 10 commits from tests/ci-pytest into main 2026-10-09 15:53:53 -04:00

Conversation 1 Commits 10 Files Changed 11

+530 -34

opencodecabilloteu commented 2026-10-08 20:20:58 -04:00

Owner

Copy link

Copy source

Problem

pytest never runs in CI: the 'test' job in all 4 workflows is only a docker build. A regression in server.py or indexer.py goes green.

Changes

• pytest.ini: new integration marker, deselected by default (addopts); tests/test_integration.py requires a live Qdrant.
• tests/test_integration.py: pytestmark = pytest.mark.integration so marker selection works.
• src/indexer.py: env validation moved from import time to _validate_config() called by main() — module importable without env vars (enables unit tests, no behavior change).
• tests/test_indexer.py (new, 27 tests, no network/model): decode_mime_words (encoded-words), extract_text_from_html, parse_email_message with real EmailMessage objects (plain body, attachments excluded from body, HTML alternative, unknown charset fallback, encoded attachment filename), normalize_email_address, get_recent_keys against a real tmp_path Maildir (mtime fresh/old, colon-suffix keys).
• tests/test_server.py: +10 tests covering build_filter (should-clause sender/receiver, normalization, date gte/lte, combined).
• Workflows (pr, main, cron, tag): real pytest unit job (integration deselected) and integration-test job starting Qdrant with --network container:$(hostname) (bypasses the Gitea runner service-container bug), health-check wait, pytest -m integration, teardown. On main/cron/tag, build/build-push jobs now needs: [test, integration-test].

Validation

• Local: 43 passed, 6 deselected (unit) in ~2.6s.
• Integration suite: previously unrunnable; now runs in CI with a real Qdrant (nightly + main + tag + this PR).

## Problem pytest never runs in CI: the 'test' job in all 4 workflows is only a docker build. A regression in server.py or indexer.py goes green. ## Changes - **pytest.ini**: new `integration` marker, deselected by default (`addopts`); tests/test_integration.py requires a live Qdrant. - **tests/test_integration.py**: `pytestmark = pytest.mark.integration` so marker selection works. - **src/indexer.py**: env validation moved from import time to `_validate_config()` called by `main()` — module importable without env vars (enables unit tests, no behavior change). - **tests/test_indexer.py** (new, 27 tests, no network/model): decode_mime_words (encoded-words), extract_text_from_html, parse_email_message with real EmailMessage objects (plain body, attachments excluded from body, HTML alternative, unknown charset fallback, encoded attachment filename), normalize_email_address, get_recent_keys against a real tmp_path Maildir (mtime fresh/old, colon-suffix keys). - **tests/test_server.py**: +10 tests covering `build_filter` (should-clause sender/receiver, normalization, date gte/lte, combined). - **Workflows** (pr, main, cron, tag): real `pytest` unit job (integration deselected) and `integration-test` job starting Qdrant with `--network container:$(hostname)` (bypasses the Gitea runner service-container bug), health-check wait, `pytest -m integration`, teardown. On main/cron/tag, build/build-push jobs now `needs: [test, integration-test]`. ## Validation - Local: 43 passed, 6 deselected (unit) in ~2.6s. - Integration suite: previously unrunnable; now runs in CI with a real Qdrant (nightly + main + tag + this PR).

opencodecabilloteu added 4 commits 2026-10-08 20:20:59 -04:00

test: add indexer unit tests + integration marker; make indexer importable without env ... e32806aab9

- pytest.ini: 'integration' marker, deselected by default; the integration
  suite requires a live Qdrant and will only run in the nightly CI job.
- src/indexer.py: move env validation from import time into _validate_config()
  called by main() — the module can now be imported without MAILDIR_PATH /
  QDRANT_URL / COLLECTION_NAME set, enabling unit tests.
- tests/test_indexer.py: 27 new unit tests, no network and no model:
  decode_mime_words (encoded-words utf-8/iso-8859-1, mixed), extract_text_from_html
  (tags, scripts), normalize_email_address, parse_email_message with real
  EmailMessage objects (plain bodies, attachments detected and excluded from
  body, HTML alternative, unknown charset fallback via raw bytes message,
  encoded attachment filename), get_recent_keys against a real tmp_path
  Maildir (fresh mtime selected, old mtime excluded, colon-suffix keys).
  mailbox.Maildir(create=True) does not create cur/new/tmp on Python 3.11 —
  the test helper pre-creates them.

test: cover build_filter (participant should-clause, date ranges, combination) b9e22766c0

ci: run pytest in all workflows — real unit job + Qdrant-backed integration job ... de27820cc7

- pr.yaml: new pytest job (unit tests only, integration deselected by marker)
- main.yaml / cron.yaml / tag.yaml: replace the build-only 'test' job with a
  real pytest unit job + an integration-test job that starts a Qdrant
  container sharing the job container's network namespace
  (--network container:$(hostname) — bypasses the Gitea runner bug with
  service containers), waits for /healthz, runs 'pytest -m integration',
  then tears down.
- build/build-push jobs now require [test, integration-test]; previously
  'test' was only a docker build and pytest never ran in CI.
- integration-test first cleans up any stale qdrant container (rerun-safe).
- setup-python SHA aligned with docker-build.yaml (v7, python 3.13/3.14).

test: mark integration suite with pytest.mark.integration ... 9417e577a9

Required for -m 'not integration' to actually deselect them (pytest.ini
addopts) and for 'pytest -m integration' in the nightly CI job.

opencodecabilloteu requested review from jcabillot 2026-10-08 20:21:28 -04:00

opencodecabilloteu added 1 commit 2026-10-08 21:07:43 -04:00

ci: use pytest -m integration in docker-build.yaml (marker replaces --ignore file check) 55841310bb

opencodecabilloteu added 1 commit 2026-10-08 21:29:15 -04:00

ci: unique qdrant container name per run — parallel workflows on one host clash ... aa30111a8e

Two concurrent jobs (docker-build + PR Checks on the same push) both
used container name 'qdrant'; the second job's 'docker rm -f qdrant'
destroyed the first job's Qdrant mid-run, so the test collection 404'd.
Name is now qdrant-${{ github.run_id }} everywhere (cleanup included).

opencodecabilloteu added 1 commit 2026-10-09 08:21:50 -04:00

test: instrument qdrant_setup fixture (log url + collections before/after create) a597ecf6a2

opencodecabilloteu added 1 commit 2026-10-09 08:27:55 -04:00

test: fix fixture debug attribute (QdrantClient has no rest_uri) 6997326b81

opencodecabilloteu added 1 commit 2026-10-09 08:35:58 -04:00

test: introspect http client base_url safely ef8fe35da7

opencodecabilloteu added 1 commit 2026-10-09 08:42:28 -04:00

test: fix collection name mismatch — env wiring centralized in conftest ... 679d72288f

Root cause of the CI 404s: test modules mutated os.environ at import time
(test_server.py set COLLECTION_NAME='test', test_integration.py
'test_mcp_maildir'). With pytest's alphabetical import order,
server.COLLECTION_NAME froze at whichever value was set when server was
first imported, while the fixture read os.environ at runtime — often a
different name (log proof: fixture created 'test', server queried
'test_mcp_maildir' → 404 on all 6 tests).

conftest.py now sets QDRANT_URL/COLLECTION_NAME/EMBEDDING_MODEL_NAME once,
before any test module import; env mutation removed from test files.
Unit suite still green (43 passed, 6 deselected).

opencodecabilloteu commented 2026-10-09 08:47:38 -04:00

Author

Owner

Copy link

Copy source

CI fix history (investigation record)

Initial runs failed with 404 Collection 'test_mcp_maildir' doesn't exist on all 6 integration tests, despite the module-scoped fixture yielding.

Root cause: test modules mutated os.environ at import time (test_server.py → COLLECTION_NAME='test', test_integration.py → 'test_mcp_maildir'). Pytest imports modules alphabetically, so server.COLLECTION_NAME froze at one value while the fixture read os.environ at runtime — it created one collection, server functions queried another. Log proof: collections AFTER create: ['test'] then Not found: Collection 'test_mcp_maildir'.

Fix (679d722): tests/conftest.py sets QDRANT_URL/COLLECTION_NAME/EMBEDDING_MODEL_NAME once, before any test module import; env mutation removed from test files. Import order no longer matters.

Also fixed along the way: the qdrant container name is now per-run (qdrant-${{ github.run_id }}) — concurrent workflows on one Docker host used to kill each other's Qdrant; docker-build.yaml now selects tests via -m integration instead of a file-path ignore.

## CI fix history (investigation record) Initial runs failed with `404 Collection 'test_mcp_maildir' doesn't exist` on all 6 integration tests, despite the module-scoped fixture yielding. **Root cause**: test modules mutated `os.environ` at import time (`test_server.py` → `COLLECTION_NAME='test'`, `test_integration.py` → `'test_mcp_maildir'`). Pytest imports modules alphabetically, so `server.COLLECTION_NAME` froze at one value while the fixture read `os.environ` at runtime — it created one collection, server functions queried another. Log proof: `collections AFTER create: ['test']` then `Not found: Collection 'test_mcp_maildir'`. **Fix (679d722)**: `tests/conftest.py` sets QDRANT_URL/COLLECTION_NAME/EMBEDDING_MODEL_NAME once, before any test module import; env mutation removed from test files. Import order no longer matters. Also fixed along the way: the `qdrant` container name is now per-run (`qdrant-${{ github.run_id }}`) — concurrent workflows on one Docker host used to kill each other's Qdrant; `docker-build.yaml` now selects tests via `-m integration` instead of a file-path ignore.

jcabillot merged commit 916850b692 into main 2026-10-09 15:53:53 -04:00

jcabillot deleted branch tests/ci-pytest 2026-10-09 15:53:55 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

jcabillot

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

cloudix_mcp_server jcabillot opencodecabilloteu renovate

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: perso/mcp-maildir#52