Add monitor and defaults for running the agents together

README.md

@@ -3,13 +3,18 @@
 Agents built with [Go Micro](https://github.com/micro/go-micro).
 
 - [DevOps](devops): investigate local systemd service failures using status and journal logs.
+- [Monitor](monitor): watch the agents' services and report failures and recoveries.
 - [Release](release): release and deploy through a configured GitHub Actions workflow.
 - [Support](support): investigate GitHub issues and draft responses with supporting evidence.
 
 Each agent is a Go program. Service integrations live in `services/` and can also
 be called directly over RPC. Run agents in your own environment with your own
 model configuration.
 
+By default, Support covers this repository and Go Micro. DevOps and Monitor
+inspect the units in [deploy](deploy). Follow that guide to run the agents together
+on one host, or override their targets for your own environment.
+
 ## Development
 
 Requires Go 1.26 or newer.

deploy/README.md

@@ -0,0 +1,83 @@
+# Run the agents together
+
+These systemd units run the repository's four binaries on one Linux host.
+There is no additional runtime. Support reads this repo and Go Micro; DevOps
+inspects the units; Monitor records their process state. Release uses a GitHub
+Actions workflow you configure.
+
+## Install
+
+From a checkout, build the binaries with Go 1.26 or newer:
+
+```sh
+mkdir -p bin
+for agent in support devops release monitor; do
+  go build -o "bin/$agent" "./$agent"
+done
+sudo useradd --system --home-dir /var/lib/micro-agents --shell /usr/sbin/nologin micro-agents
+sudo install -d /opt/micro-agents /etc/micro-agents
+sudo install -m 755 bin/support bin/devops bin/release bin/monitor /opt/micro-agents/
+sudo install -m 644 deploy/systemd/*.service /etc/systemd/system/
+sudo install -m 600 /dev/null /etc/micro-agents/common.env
+sudoedit /etc/micro-agents/common.env
+```
+
+Create the user only if it does not already exist. On upgrades, preserve existing
+environment files. Put your model settings in `common.env`:
+
+```ini
+MICRO_AI_PROVIDER=openai
+MICRO_AI_API_KEY=your-model-key
+```
+
+Other providers work too. The common file is required by all units. Optional
+`support.env`, `devops.env`, `release.env` and `monitor.env` in the same directory
+override it for each agent. Keep credential files root-owned with mode 600;
+systemd loads them before switching to the service user.
+
+Support needs no GitHub token for public reads. Add `GITHUB_TOKEN` to
+`support.env` for higher rate limits or private repository access. Do not share
+Release's write token through `common.env`.
+
+Release still needs an existing `workflow_dispatch` workflow. There is no
+deployment workflow in this repository yet. Configure `release.env` using the
+[release instructions](../release), including `GITHUB_REPOSITORY`,
+`RELEASE_WORKFLOW` and a suitably scoped `GITHUB_TOKEN`. Do not start Release
+until that is configured; Monitor will report it as inactive or missing.
+
+```sh
+sudo systemctl daemon-reload
+sudo systemctl enable --now micro-support micro-devops micro-monitor
+# After configuring release.env:
+sudo systemctl enable --now micro-release
+```
+
+The DevOps unit joins `systemd-journal` to read logs. That account-level permission
+can read more than the agent's configured allowlist. Adjust the group for your
+distribution. No agent runs as root or has permission to restart these units.
+
+## Use
+
+Use a current Go Micro CLI on the same host, with its default local discovery:
+
+```sh
+micro chat support
+micro chat devops
+micro chat monitor
+micro chat release
+```
+
+The servers bind to loopback. Keep the registry and host trusted; these examples
+do not add RPC authentication or isolate untrusted local users.
+
+| Setting | Default | Override |
+| --- | --- | --- |
+| Support repositories | micro/go-micro, micro/agents | `SUPPORT_REPOSITORIES`; legacy `GITHUB_REPOSITORY` selects one |
+| DevOps units | All four micro-*.service units above | `DEVOPS_SERVICES`, comma-separated exact names |
+| Monitor units | Same as DevOps | `MONITOR_SERVICES` |
+| Release target | Explicit configuration required | `GITHUB_REPOSITORY`, `RELEASE_WORKFLOW`, `RELEASE_REF` |
+
+To inspect only installed units, set `DEVOPS_SERVICES` in `common.env` to that
+subset. Restart the affected units after changing configuration. Running an agent
+with `go run` does not install a systemd unit, so these checks will report missing
+units until the deployment above is installed.

deploy/systemd/micro-devops.service

@@ -0,0 +1,24 @@
+[Unit]
+Description=Micro devops agent
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+User=micro-agents
+Group=micro-agents
+SupplementaryGroups=systemd-journal
+WorkingDirectory=/opt/micro-agents
+ExecStart=/opt/micro-agents/devops
+Environment=HOME=/var/lib/micro-agents/devops
+EnvironmentFile=/etc/micro-agents/common.env
+EnvironmentFile=-/etc/micro-agents/devops.env
+StateDirectory=micro-agents/devops
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectHome=true
+ProtectSystem=strict
+
+[Install]
+WantedBy=multi-user.target

deploy/systemd/micro-monitor.service

@@ -0,0 +1,23 @@
+[Unit]
+Description=Micro monitor agent
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+User=micro-agents
+Group=micro-agents
+WorkingDirectory=/opt/micro-agents
+ExecStart=/opt/micro-agents/monitor
+Environment=HOME=/var/lib/micro-agents/monitor
+EnvironmentFile=/etc/micro-agents/common.env
+EnvironmentFile=-/etc/micro-agents/monitor.env
+StateDirectory=micro-agents/monitor
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectHome=true
+ProtectSystem=strict
+
+[Install]
+WantedBy=multi-user.target

deploy/systemd/micro-release.service

@@ -0,0 +1,23 @@
+[Unit]
+Description=Micro release agent
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+User=micro-agents
+Group=micro-agents
+WorkingDirectory=/opt/micro-agents
+ExecStart=/opt/micro-agents/release
+Environment=HOME=/var/lib/micro-agents/release
+EnvironmentFile=/etc/micro-agents/common.env
+EnvironmentFile=-/etc/micro-agents/release.env
+StateDirectory=micro-agents/release
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectHome=true
+ProtectSystem=strict
+
+[Install]
+WantedBy=multi-user.target

deploy/systemd/micro-support.service

@@ -0,0 +1,23 @@
+[Unit]
+Description=Micro support agent
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+User=micro-agents
+Group=micro-agents
+WorkingDirectory=/opt/micro-agents
+ExecStart=/opt/micro-agents/support
+Environment=HOME=/var/lib/micro-agents/support
+EnvironmentFile=/etc/micro-agents/common.env
+EnvironmentFile=-/etc/micro-agents/support.env
+StateDirectory=micro-agents/support
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectHome=true
+ProtectSystem=strict
+
+[Install]
+WantedBy=multi-user.target

devops/README.md

@@ -10,13 +10,15 @@ Run on the Linux machine you want to inspect, with `systemctl` and `journalctl`
 installed. From the repository root:
 
 ```sh
-export DEVOPS_SERVICES=api.service,worker.service
 export MICRO_AI_PROVIDER=openai
 export MICRO_AI_API_KEY=your-model-key
 go run ./devops
 ```
 
-Use exact service names, including `.service`. Only configured units can be
+By default it inspects `micro-support.service`, `micro-devops.service`,
+`micro-release.service` and `micro-monitor.service`, matching [the deployment
+units](../deploy). Set `DEVOPS_SERVICES=api.service,worker.service` to replace
+this set. Use exact service names, including `.service`. Only configured units can be
 inspected (up to 16). Provider-specific keys and `MICRO_AI_MODEL` also work.
 Run as an unprivileged account with permission to read the relevant journal.
 The agent does not invoke sudo or change permissions. Journal access groups can
@@ -30,12 +32,12 @@ go install go-micro.dev/v6/cmd/micro@master
 micro chat devops
 ```
 
-Ask **“What is running?”** or **“Why is api.service failing?”** The model key
+Ask **“What is running?”** or **“Why is micro-support.service failing?”** The model key
 belongs to the running agent; the chat client does not need it.
 
 The agent runs on the host being inspected. The CLI must share its registry;
 this example uses default local discovery. Like support, this starter does not
-add RPC authentication or tenant isolation. Keep it on a trusted local network.
+add RPC authentication or tenant isolation. It binds to loopback by default; use the CLI on the same machine.
 Retrieved status and logs are sent to your configured model provider; use units
 whose logs are appropriate to share.
 
@@ -45,8 +47,8 @@ These are ordinary Go Micro service endpoints, usable without a model:
 
 ```sh
 micro call devops Systemd.List '{}'
-micro call devops Systemd.Status '{"service":"api.service"}'
-micro call devops Systemd.Logs '{"service":"api.service","minutes":60,"lines":100}'
+micro call devops Systemd.Status '{"service":"micro-support.service"}'
+micro call devops Systemd.Logs '{"service":"micro-support.service","minutes":60,"lines":100}'
 ```
 
 Logs default to the last hour and 100 entries, with a maximum of 24 hours and

devops/main.go

@@ -4,26 +4,22 @@ package main
 import (
 	_ "embed"
 	"log"
-	"os"
-	"strings"
 
+	"github.com/micro/agents/internal/config"
 	"github.com/micro/agents/services/systemd"
 	micro "go-micro.dev/v6"
+	"go-micro.dev/v6/service"
 )
 
 //go:embed instructions.md
 var instructions string
 
 func main() {
-	units := os.Getenv("DEVOPS_SERVICES")
-	if units == "" {
-		log.Fatal("set DEVOPS_SERVICES to comma-separated service names, e.g. api.service,worker.service")
-	}
-	host, err := systemd.New(strings.Split(units, ","))
+	host, err := systemd.New(config.Services())
 	if err != nil {
 		log.Fatal(err)
 	}
-	service := micro.NewService("devops")
+	service := micro.NewService("devops", service.Address("127.0.0.1:0"))
 	service.Init()
 	if err := service.Handle(host); err != nil {
 		log.Fatal(err)

internal/config/config.go

@@ -0,0 +1,25 @@
+// Package config holds defaults shared by the agents in this repository.
+package config
+
+import (
+	"os"
+	"strings"
+)
+
+func SupportRepositories() []string {
+	if value := os.Getenv("SUPPORT_REPOSITORIES"); value != "" {
+		return strings.Split(value, ",")
+	}
+	if value := os.Getenv("GITHUB_REPOSITORY"); value != "" {
+		return []string{value}
+	}
+	return []string{"micro/go-micro", "micro/agents"}
+}
+
+// Services uses the same set for DevOps and monitoring unless explicitly overridden.
+func Services() []string {
+	if value := os.Getenv("DEVOPS_SERVICES"); value != "" {
+		return strings.Split(value, ",")
+	}
+	return []string{"micro-support.service", "micro-devops.service", "micro-release.service", "micro-monitor.service"}
+}

internal/config/config_test.go

@@ -0,0 +1,40 @@
+package config
+
+import (
+	"os"
+	"path/filepath"
+	"reflect"
+	"strings"
+	"testing"
+)
+
+func TestDefaultsAndOverrides(t *testing.T) {
+	for _, name := range []string{"SUPPORT_REPOSITORIES", "GITHUB_REPOSITORY", "DEVOPS_SERVICES"} {
+		t.Setenv(name, "")
+	}
+	if got := SupportRepositories(); !reflect.DeepEqual(got, []string{"micro/go-micro", "micro/agents"}) {
+		t.Fatal(got)
+	}
+	for _, unit := range Services() {
+		data, err := os.ReadFile(filepath.Join("..", "..", "deploy", "systemd", unit))
+		if err != nil {
+			t.Fatal(err)
+		}
+		name := strings.TrimSuffix(strings.TrimPrefix(unit, "micro-"), ".service")
+		if !strings.Contains(string(data), "ExecStart=/opt/micro-agents/"+name+"\n") {
+			t.Fatalf("%s does not run %s", unit, name)
+		}
+	}
+	t.Setenv("GITHUB_REPOSITORY", "owner/legacy")
+	if got := SupportRepositories(); !reflect.DeepEqual(got, []string{"owner/legacy"}) {
+		t.Fatal(got)
+	}
+	t.Setenv("SUPPORT_REPOSITORIES", "owner/one,owner/two")
+	if got := SupportRepositories(); !reflect.DeepEqual(got, []string{"owner/one", "owner/two"}) {
+		t.Fatal(got)
+	}
+	t.Setenv("DEVOPS_SERVICES", "api.service,worker.service")
+	if got := Services(); !reflect.DeepEqual(got, []string{"api.service", "worker.service"}) {
+		t.Fatal(got)
+	}
+}

monitor/README.md

@@ -0,0 +1,42 @@
+# Monitor
+
+Watch the agents' systemd services. Checks run immediately and once a minute
+without model calls. State changes go to the journal and can be queried in chat.
+
+```sh
+export MICRO_AI_PROVIDER=openai
+export MICRO_AI_API_KEY=your-model-key
+go run ./monitor
+```
+
+In another terminal on the same machine:
+
+```sh
+micro chat monitor
+```
+
+Ask **“What is down?”** or **“What changed recently?”**
+
+The default set is `micro-support.service`, `micro-devops.service`,
+`micro-release.service` and `micro-monitor.service`, matching [deploy](../deploy).
+`DEVOPS_SERVICES` overrides the shared set; `MONITOR_SERVICES` overrides it for
+monitoring only. Uninstalled units are reported as missing.
+
+These are process checks. An active unit does not prove that an agent can answer,
+its model is available, or an application is healthy. Permission and read errors
+are reported as unknown. History holds the last 100 changes in memory and resets
+on restart. The journal retains logged changes according to the host's policy.
+There are no external alerts or automatic repairs. Use DevOps to investigate logs.
+
+## Service
+
+The read-only service is usable without a model:
+
+```sh
+micro call monitor Monitor.Status '{}'
+micro call monitor Monitor.History '{}'
+```
+
+The agent binds to loopback and uses local discovery. It has no RPC authentication.
+The polling service lives in [services/monitor](../services/monitor); its `Run`
+method takes a context for shutdown. Only explaining observations requires a model.