Testing Strategy

Overview

The testing strategy employs a layered approach to ensure correctness at different levels of the system:

Unit tests for individual primitives and utilities (e.g., functions in @os/* packages).
Integration tests for API routes and interactions between components (e.g., testing the server API with a test database).
End-to-end (E2E) tests for critical user journeys that span the full stack (e.g., sign-up, tenant onboarding, capability invocation).
Property-based testing for complex logic where invariants can be expressed as properties (e.g., using a library like fast-check).

Current State

As of now, the repository includes:

A quality gate script (bun run check) that runs linting, formatting, and type checking (see Quality Checks).
An end-to-end test suite using @e2e-dev/web (see End-to-End Tests) with one sample test for admin sign-up.
Unit and integration tests are not yet implemented but are planned for the @os/core package and other workspace packages.

Unit tests will focus on testing individual functions, classes, and modules in isolation. They will be colocated with the source code or placed in a __tests__ directory within each package. The testing framework will be Vitest (or Jest) as chosen by the package, but the repository currently does not enforce a specific framework.

Each unit test should aim for high coverage of critical logic and edge cases.
Mocking of external dependencies (e.g., database, network) will be used to isolate the unit under test.

Integration Testing

Integration tests will verify the interactions between multiple components, such as:

API route handlers (in apps/server or @os/api) with the database and authentication middleware.
Interactions between the OS primitives (Principal, Capability, Task, Receipt, Policy).
These tests will use a test database (PostgreSQL) and may run against a real instance or a mock, depending on the speed and fidelity required.

End-to-End Testing

The E2E test suite is already configured and runs against a full stack (admin UI, server API, database) using a real browser (Chromium by default).

See End-to-End Tests for details on configuration and extending the suite.
Critical user journeys to cover include:
- Tenant (admin) sign-up and onboarding.
- User login and access to the dashboard.
- Invocation of a capability (e.g., creating a task) and verifying the receipt.
- Cross-tenant isolation (ensuring that a user from one tenant cannot access another tenant’s data).

Property-Based Testing

For complex algorithms or data transformations where exhaustive testing is infeasible, property-based testing will be employed.

The library fast-check is recommended for generating random inputs and asserting invariants.
Examples of properties to test:
- The Policy evaluation function should always return a boolean for any valid input.
- The Receipt generation should be deterministic given the same input and tenant context.

Running Tests

Quality checks (linting, formatting, type checking): bun run check
End-to-end tests: bun run test:e2e (requires Postgres to be running via bun run db:start)
Unit and integration tests: To be defined per package (e.g., bun run test in each workspace).

CI Integration

All test types are intended to be run in the continuous integration pipeline:

The quality checks run on every push (via a pre-push hook, planned) and in CI.
The E2E tests run in CI against a preview environment.
Unit and integration tests run in CI on every push.

Relationships with Other Documentation

Quality Checks: Describes the bun run check command that runs linting, formatting, and type checking.
End-to-End Tests: Details the E2E test setup and configuration.
Request Flow: Understanding the server-side request handling aids in writing integration and E2E tests.

Extension Points

Adding a new unit test framework: Update the package’s package.json test script and ensure it is recognized by the CI.
Adding a new E2E test scenario: Create a new *.e2e.ts file in the /tests directory.
Adding property-based tests: Install fast-check and write test files following the chosen unit test framework.