Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 15 additions & 20 deletions content/en/docs/next/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,40 +348,35 @@ Feel free to look inside each Makefile to better understand the logic.

### Testing

The platform includes an [`e2e.sh`](https://github.com/cozystack/cozystack/blob/main/hack/e2e.sh) script that performs the following tasks:
End-to-end tests run against a cluster that lives in a **sandbox container**. The sandbox is a privileged Docker container on the machine where you run `make`, and it starts three QEMU virtual machines with KVM. A run has three stages: preparing the cluster (VMs and Talos Linux), installing Cozystack, and running the tests. The suite has two parts:

- Runs three QEMU virtual machines
- Configures Talos Linux
- Installs Cozystack
- Waits for all HelmReleases to be installed
- Performs additional checks to ensure that components are up and running
- **Application tests** — [Kyverno Chainsaw](https://kyverno.github.io/chainsaw/) suites under [`hack/e2e-chainsaw/`](https://github.com/cozystack/cozystack/tree/main/hack/e2e-chainsaw), one directory per application, each with a `chainsaw-test.yaml`.
- **Bootstrap and API tests** — BATS scripts: `hack/e2e-prepare-cluster.bats` boots the VMs and configures Talos, `hack/e2e-install-cozystack.bats` installs Cozystack on that cluster, and `hack/e2e-test-openapi.bats` checks the aggregated API.

You can run e2e.sh either locally or directly within a Kubernetes container.
Conventions for writing and stabilizing these tests live in the [E2E testing guide](https://github.com/cozystack/cozystack/blob/main/docs/agents/e2e-testing.md).

To run tests in a Kubernetes cluster, navigate to the `packages/core/testing` directory and execute the following commands:
To run the tests, execute the following commands from the root of the repository, in this order:

```shell
make apply # Create testing sandbox in Kubernetes cluster
make test # Run the end-to-end tests in existing sandbox
make delete # Remove testing sandbox from Kubernetes cluster
make assets-talos # Build the Talos disk image the VMs boot from (_out/assets/nocloud-amd64.raw.xz)
make prepare-env # Start the sandbox container, boot the VMs and configure Talos
make -C packages/core/testing install-cozystack # Install Cozystack in the cluster
make -C packages/core/testing e2e # Run the end-to-end tests (OpenAPI + Chainsaw)
make -C packages/core/testing delete # Remove the sandbox container
```

The root `make test` is `apply` followed by `e2e`. It starts an empty sandbox and never creates the cluster, so it is not a shortcut for the commands above.

{{% alert color="warning" %}}
:warning: To run e2e tests in a Kubernetes cluster, your nodes must have sufficient free resources to create 3 VMs and store the data for the deployed applications.
:warning: To run e2e tests, the machine that runs Docker must support KVM and have enough free resources for 3 VMs and the data of the deployed applications. By default each VM gets 8 vCPUs and 24 GiB of memory.

It is recommended to use bare-metal nodes of the parent Cozystack cluster.
A bare-metal machine is recommended.
{{% /alert %}}

### Dynamic Development Environment

If you prefer to develop Cozystack in virtual machines instead of modifying the existing cluster, you can utilize the same sandbox from testing environment. The Makefile in the `packages/core/testing` includes additional options:
If you prefer to develop Cozystack against the sandbox instead of modifying the existing cluster, open an interactive shell inside the running sandbox container from the `packages/core/testing` directory:

```shell
make exec # Opens an interactive shell in the sandbox container.
make login # Downloads the kubeconfig into a temporary directory and runs a shell with the sandbox environment; mirrord must be installed.
make proxy # Enable a SOCKS5 proxy server; mirrord and gost must be installed.
```

Socks5 proxy can be configured in a browser to access services of a cluster running in sandbox. Firefox has a handy extension for toogling proxy on/off:

- [Proxy Toggle](https://addons.mozilla.org/en-US/firefox/addon/proxy-toggle/)
Loading