Metadata-Version: 2.4
Name: DevTestKit
Version: 0.1.0
Summary: Pytest plugin for dev_test integration suites: calls a module's handler in-process against real Postgres, Valkey and LocalStack containers
Author: João Edinaldo
Author-email: joao.gomes@e-deploy.com.br
Requires-Python: >=3.12,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Description-Content-Type: text/markdown

# DevTestKit

The shared harness of the `dev_test` suites: integration tests that call a module's Lambda handler
in-process against real Postgres/Valkey/LocalStack containers. The mechanism lives here; what is data of
the module lives in `<module>/dev_test/devtest.toml`; domain behavior stays in fixtures of the module's
own `dev_test/conftest.py`.

## Installation

The distribution is `DevTestKit` (import name `devtestkit`), meant for the E-Deploy pip index. Add it to the
module's `devtest` dependency group:

```toml
[dependency-groups]
devtest = ["devtestkit==0.1.0"]
```

Python `>=3.12,<4.0`. The package declares no dependencies: the module provides the ones the kit imports at run
time, in the same group.

| Dependency | Provided by the module for |
|---|---|
| `pytest`, `pytest-xdist` | the plugin and up to 16 workers |
| `filelock` | the state shared between xdist workers |
| `DockerTestContainers==1.0.0` | the Postgres, Valkey and LocalStack containers |
| `boto3` | LocalStack (S3 and SQS) |
| `psycopg2` or `psycopg2-binary` | the database |
| `alembic`, `SQLAlchemy` | the schema build and `devtestkit.alembic_env` |

## Adopting it in a module

1. `<module>/pyproject.toml`: add `"--ignore=dev_test"` to `addopts` (the unit CI has no Docker) and the
   dependencies above to the `devtest` group. Nothing about the kit goes in `pythonpath`.
2. `<module>/dev_test/conftest.py`: `pytest_plugins = ["devtestkit.plugin"]`. pytest only accepts it
   because `dev_test` is always an initial path of the invocation (`pytest dev_test ...`).
3. `<module>/dev_test/devtest.toml`: the schema below. The plugin reads `<rootdir>/dev_test/devtest.toml`.

## `devtest.toml`

Relative paths resolve against the module root. An unknown key is an error that names it.

| Section | Key | Meaning |
|---|---|---|
| `[app]` | `config_file`, `context_file` | the module's real config files; the test config is built from them |
| | `context_layout` | how `context_file` declares its contexts: `list` (default; a `contexts` array whose entries carry a `key`) or `map` (a `context` object keyed by context name). Either way the first context is kept and renamed to `context_key`; a file without a non-empty array or object of that shape is refused naming the layout |
| | `log_config` | becomes `LOG_PATH` |
| | `context_key` | key of the single test context, and the default `Host` of the Kong helpers |
| | `drop_feature_flags` | flags removed from that context (`EnableAuthorization` skips Cedar) |
| | `config_delivery` | `inline`: `CONFIG_PATH` carries the JSON; `file`: a per-worker temp file path |
| | `handler` | `module:attribute` of the Lambda handler under test |
| | `handler_reset_globals` | module globals set to `None` at session teardown |
| | `service_address_base_port` | optional; sets `SERVICE_ADDRESS` (base on master, base+1+N on worker N) |
| | `entrypoint` | optional; `lambda_kong` (default) or `flask`. With `flask` the `handler` is the Flask application itself (`main_flask:application`), `[kong]` is refused if present, and `kong_helpers_from` refuses the TOML |
| | `database_key` | optional; the context key that receives the test database, default `database` (a Flask module such as `delivery/partner` uses `"db"`). The context must already have that key, default included: a context without it is refused naming the key and `context_file` |
| | `environment` | optional `{NAME = "value"}` exported for the session and restored after it, for switches the module reads from the environment (e.g. `EDP_AUTHORIZATION_SERVICE`); a variable the kit sets itself is refused |
| `[containers]` | `localstack_services` | e.g. `["s3", "sqs"]`; empty or absent = no LocalStack and no AWS rewrite |
| | `postgres_extensions` | e.g. `["uuid-ossp"]`; created before the first chain, for extensions the chains or the app use but no migration creates |
| | `valkey_image` | e.g. `"valkey/valkey-bundle:9.0.3-alpine"`; absent = the `dockertestcontainers` default image. For an app that needs a Valkey module (JSON) |
| `[[alembic]]` | `dir`, `ini`, `pythonpath`, `log_config` | chains run in order, each in a subprocess with cwd at `dir`, on the template DB |
| `[aws]` | `bucket_config_paths` | config paths naming every bucket the app writes to |
| | `queue_url_parent_and_key`, `queue_url_parents` | config keys holding SQS queue URLs |
| `[[upstream_stubs]]` | `context_key`, `path_prefix`, `routes` | repoints `context[context_key].url` at a loopback stub; `routes` are `{method, path_suffix, body}`; anything else answers 404 `{}` |
| `[reset]` | `schemas`, `backup_schema` | schemas truncated before every test |
| | `preserve` | `[[schema, table], ...]` kept out of the TRUNCATE and restored only if the CASCADE emptied them |
| | `restore_seeded` | `true`: every table with rows at baseline is back to those rows before every test |
| | `queues` | `true`: every message left in this worker's SQS queues is deleted before every test; needs `sqs` in `localstack_services` |
| | `valkey` | `true`: this process's Valkey is emptied before every test (`FLUSHDB` of the worker's logical database under xdist, `FLUSHALL` otherwise), so a cache filled by one test cannot answer the next with rows the database reset already removed. Default `false` |
| | `strategy` | `dirty` (default): only the tables the previous test touched are truncated; `full`: every table in `schemas`, before every test |
| | `verify` | `true`: after every reset, the test setup fails if a table, a restored row or an owned sequence differs from the baseline; for validation runs (~0.5 s per test), not CI |
| `[kong]` | `event_format` | `v2` (konglambdamessagehandler 2.x: dict body) or `v1` (1.x: JSON string body, key omitted when bodyless) |
| | `default_user_id` | optional `X-Authenticated-Userid` default |
| `[xdist]` | `target_chunk_seconds`, `file_seconds` | measured seconds per test file for the weighted scheduler; absent = plain loadfile |

`preserve` and `restore_seeded` are two reset modes; the loader refuses both at once. `restore_seeded`,
`queues`, `verify` and `valkey` must be TOML booleans. The three delivery keys (`entrypoint`, `database_key`,
`valkey`) are opt-in: a TOML without them behaves as before, except that its context must have the `database` key
(the fudaca copy created a missing one, leaving the app on its configured database).

`strategy = "dirty"` puts a statement-level trigger on every table in `schemas`; it records the table's oid in
`<backup_schema>.devtest_written_tables` on any INSERT, UPDATE, DELETE or TRUNCATE. A table is truncated when it was
written, when it has no trigger yet (created or cloned since the last reset), or when one of its owned sequences
moved (a rolled-back insert, a `nextval` from `clone_schema`). The first reset of each worker truncates everything.
The truncate and the restore run with `session_replication_role = replica`, so neither the tracking trigger nor the
app's triggers fire, and a `preserve`/`restore_seeded` table is restored only when it ends up empty. A write that
bypasses triggers (`session_replication_role = replica`, `DISABLE TRIGGER`) goes unseen: switch that module to
`strategy = "full"`.

## Fixtures

`handler`, `postgres`, `valkey`, `localstack` (`None` without services), `queues`, `_stack`, `_set_env`
(autouse; yields `queue_urls`), `_upstream_stub`, `_reset_db`, `_purge_queues` and `_flush_valkey` (autouse), `devtest_config`,
and the fallback `worker_id` when pytest-xdist is not installed. Up to 16 xdist workers share one container set
and one schema build; each worker clones the template database and gets its own Valkey database, SQS queues and
S3 buckets.

`queues` (`WorkerQueues`, `None` without LocalStack) is what a module's `run_job` drains: `receive(wait_seconds)`
receives and deletes what every worker queue holds and returns the raw `receive_message` messages, the same
envelope the Lambda SQS path gets; `purge()` empties them. How each message is handled (which handler, whether
an exception propagates) is domain and stays in the module's `conftest.py`.

Override by defining a fixture with the same name in the module's `conftest.py`:

| Fixture | Default | Override when |
|---|---|---|
| `devtest_baseline_ready` (session) | nothing | the baseline snapshot must wait for some state (e.g. an opened menu) |
| `devtest_before_reset` (session) | returns a no-op | something must run on the reset cursor before the tables are discovered; return `Callable[[cur], None]` |

Test code imports from the package (`from devtestkit import kong_helpers_from, parse_response, ...`);
the barrel has no third-party imports, so the kit's own tests run with pytest alone.

## `testutil`

The Kong helpers of a module come from its own toml, so `testutil` holds no configuration:

```python
_KONG = kong_helpers_from(devtest_toml=Path(__file__).resolve().parent.parent / "devtest.toml")
kong_get, kong_post, kong_put, kong_patch, kong_delete = _KONG.get, _KONG.post, _KONG.put, _KONG.patch, _KONG.delete
```

`_KONG.default_user_id` is the `[kong] default_user_id` the helpers send.

## Dev_test-only Alembic chains

A chain that exists only for the suite (tables the module references but another chain owns, seeds a fresh
database lacks) is a directory with an `alembic.ini` (`[alembic] script_location = .`), a `versions/` and this
`env.py`:

```python
from devtestkit.alembic_env import run_migrations_online

run_migrations_online(version_table="dev_test_<chain>_version")
```

It connects with the `DB_*` variables the kit exports and keeps its revisions in its own version table, so it
coexists with the module's chain in one database. When the kit runs from a source tree, it puts its own `src` on every
chain's `PYTHONPATH` (after the chain's `pythonpath`), which is what makes the import resolve; an installed kit is
already importable, so its site-packages is not added and the inherited `PYTHONPATH` keeps its place. `alembic_env` is not in the barrel:
it imports alembic and SQLAlchemy.

## Kit tests

They run against the installed package, not against `src/` (there is no `pythonpath = src`):

```bash
python -m build
python -m venv .venv
.venv/bin/python -m pip install dist/devtestkit-0.1.0-py3-none-any.whl pytest   # .venv\Scripts\python on Windows
.venv/bin/python -m pytest
```

