Skip to content

fix: change websocket session ended log level from Error to Warn in Client.Run - #179

Open
nudgebee-dev[bot] wants to merge 1 commit into
mainfrom
fix/fix-change-websocket-session-ended-log-l-1793347294
Open

nudgebee-dev[bot] wants to merge 1 commit into
mainfrom
fix/fix-change-websocket-session-ended-log-l-1793347294

Conversation

@nudgebee-dev

@nudgebee-dev nudgebee-dev Bot commented Oct 30, 2026

Copy link
Copy Markdown

Description

In pkg/ws/client.go, Client.Run() logs any non-nil error returned by connectAndServe using c.logger.Error("websocket session ended", "err", err). Because the client automatically attempts reconnection using exponential backoff, transient connection resets and normal disconnects are recoverable events rather than unrecoverable fatal application errors. Logging at Error level triggers alerts based on container_log_messages_total{level="error"}. Changing the log level to Warn prevents transient disconnects from triggering false-positive alerts while preserving observability during reconnections.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Enhancement (non-breaking change which improves existing functionality)
  • Refactor (non-breaking change which improves code structure)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation
  • CI/CD

Original Error/Logs

Click to expand the original error that triggered this fix
Change the log level in Client.Run() from c.logger.Error("websocket session ended", "err", err) to c.logger.Warn("websocket session ended", "err", err) or differentiate normal connection drops and timeouts from internal fatal errors. The incident evidence shows that transient socket resets trigger high-priority alerts via container_log_messages_total despite the client successfully executing exponential backoff reconnection. Regression check: verify that disconnect events log at WARN level and do not increment container_log_messages_total{level="error"}.

Files located during the investigation: pkg/ws/client.go

The investigation already read this code:
**Code examined**

- `pkg/ws/client.go`:120-123 — Error log triggered on any non-nil error from connectAndServe

err := c.connectAndServe(ctx)
if err != nil {
c.logger.Error("websocket session ended", "err", err)
}


- `pkg/ws/client.go`:266-275 — readLoop handles CloseNormalClosure and CloseGoingAway as nil but wraps all other read failures as errors

func (c *Client) readLoop(ctx context.Context, conn *websocket.Conn) error {
for {
conn.SetReadDeadline(time.Now().Add(pongWait)) // nolint:errcheck
_, msg, err := conn.ReadMessage()
if err != nil {
if websocket.IsCloseError(err, websocket.CloseNormalClosure, websocket.CloseGoingAway) {
return nil
}
return fmt.Errorf("read error: %w", err)
}

How Has This Been Tested?

  • Unit tests
  • Manual testing

Checklist

  • CLA signed (the CLA bot will prompt on your first PR)
  • make validate passes (fmt + lint + test)
  • Docs updated if the wire shape, config surface, or proxy module behavior changed

🤖 This PR was automatically generated by NudgeBee AI coding agent

Powered by AI-driven code analysis and automated fix generation

🔍 View Original Investigation

Click the link above to see the full investigation and analysis that led to this fix

View Detailed Nubi Conversation


✅ Verification

Build verification passed for the affected modules.

Commands run
PASSED (cd . && go build ./...) exit=0
go: downloading github.com/spf13/viper v1.21.0
go: downloading go.mongodb.org/mongo-driver/v2 v2.8.0
go: downloading github.com/redis/go-redis/v9 v9.22.0
go: downloading cloud.google.com/go/auth v0.23.1
go: downloading cloud.google.com/go/secretmanager v1.21.0
go: downloading github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.0
go: downloading github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.0
go: downloading github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets v1.5.0
go: downloading github.com/aws/aws-sdk-go-v2/config v1.32.37
go: downloading github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.44.6
go: downloading github.com/aws/aws-sdk-go-v2 v1.43.6
go: downloading google.golang.org/api v0.293.0
go: downloading github.com/ClickHouse/ch-go v0.74.0
go: downloading github.com/andybalholm/brotli v1.2.2
go: downloading go.opentelemetry.io/otel/trace v1.44.0
go: downloading go.opentelemetry.io/otel v1.44.0
go: downloading filippo.io/edwards25519 v1.2.0
go: downloading github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9
go: downloading github.com/golang-sql/sqlexp v0.1.0
go: downloading github.com/shopspring/decimal v1.4.0
go: downloading golang.org/x/text v0.42.0
go: downloading github.com/fsnotify/fsnotify v1.9.0
go: downloading github.com/go-viper/mapstructure/v2 v2.4.0
go: downloading github.com/sagikazarmark/locafero v0.11.0
go: downloading github.com/spf13/afero v1.15.0
go: downloading github.com/spf13/c…

…lient.Run

# Description

In `pkg/ws/client.go`, `Client.Run()` logs any non-nil error returned by `connectAndServe` using `c.logger.Error("websocket session ended", "err", err)`. Because the client automatically attempts reconnection using exponential backoff, transient connection resets and normal disconnects are recoverable events rather than unrecoverable fatal application errors. Logging at `Error` level triggers alerts based on `container_log_messages_total{level="error"}`. Changing the log level to `Warn` prevents transient disconnects from triggering false-positive alerts while preserving observability during reconnections.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Enhancement (non-breaking change which improves existing functionality)
- [ ] Refactor (non-breaking change which improves code structure)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation
- [ ] CI/CD

## Original Error/Logs

<details>
<summary>Click to expand the original error that triggered this fix</summary>

```
Change the log level in Client.Run() from c.logger.Error("websocket session ended", "err", err) to c.logger.Warn("websocket session ended", "err", err) or differentiate normal connection drops and timeouts from internal fatal errors. The incident evidence shows that transient socket resets trigger high-priority alerts via container_log_messages_total despite the client successfully executing exponential backoff reconnection. Regression check: verify that disconnect events log at WARN level and do not increment container_log_messages_total{level="error"}.

Files located during the investigation: pkg/ws/client.go

The investigation already read this code:
**Code examined**

- `pkg/ws/client.go`:120-123 — Error log triggered on any non-nil error from connectAndServe

```
err := c.connectAndServe(ctx)
		if err != nil {
			c.logger.Error("websocket session ended", "err", err)
		}
```

- `pkg/ws/client.go`:266-275 — readLoop handles CloseNormalClosure and CloseGoingAway as nil but wraps all other read failures as errors

```
func (c *Client) readLoop(ctx context.Context, conn *websocket.Conn) error {
	for {
		conn.SetReadDeadline(time.Now().Add(pongWait)) // nolint:errcheck
		_, msg, err := conn.ReadMessage()
		if err != nil {
			if websocket.IsCloseError(err, websocket.CloseNormalClosure, websocket.CloseGoingAway) {
				return nil
			}
			return fmt.Errorf("read error: %w", err)
		}
```
```

</details>

# How Has This Been Tested?

- [x] Unit tests
- [ ] Manual testing

# Checklist

- [ ] CLA signed (the CLA bot will prompt on your first PR)
- [x] `make validate` passes (fmt + lint + test)
- [ ] Docs updated if the wire shape, config surface, or proxy module behavior changed

---

🤖 **This PR was automatically generated by [NudgeBee](https://nudgebee.com) AI coding agent**

*Powered by AI-driven code analysis and automated fix generation*

🔍 **[View Original Investigation](https://dev.nudgebee.pollux.in/investigate?id=442b4d95-d882-41cd-a26f-c17cf9084ffc)**

*Click the link above to see the full investigation and analysis that led to this fix*

View Detailed **[Nubi Conversation](https://dev.nudgebee.pollux.in/ask-nudgebee?accountId=f13ddc45-07f3-42b4-bd0e-19606c852d18&session_id=event-f48d9d60d6537ce9)**

---

### ✅ Verification

Build verification passed for the affected modules.

<details><summary>Commands run</summary>

```
PASSED (cd . && go build ./...) exit=0
go: downloading github.com/spf13/viper v1.21.0
go: downloading go.mongodb.org/mongo-driver/v2 v2.8.0
go: downloading github.com/redis/go-redis/v9 v9.22.0
go: downloading cloud.google.com/go/auth v0.23.1
go: downloading cloud.google.com/go/secretmanager v1.21.0
go: downloading github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.0
go: downloading github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.0
go: downloading github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets v1.5.0
go: downloading github.com/aws/aws-sdk-go-v2/config v1.32.37
go: downloading github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.44.6
go: downloading github.com/aws/aws-sdk-go-v2 v1.43.6
go: downloading google.golang.org/api v0.293.0
go: downloading github.com/ClickHouse/ch-go v0.74.0
go: downloading github.com/andybalholm/brotli v1.2.2
go: downloading go.opentelemetry.io/otel/trace v1.44.0
go: downloading go.opentelemetry.io/otel v1.44.0
go: downloading filippo.io/edwards25519 v1.2.0
go: downloading github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9
go: downloading github.com/golang-sql/sqlexp v0.1.0
go: downloading github.com/shopspring/decimal v1.4.0
go: downloading golang.org/x/text v0.42.0
go: downloading github.com/fsnotify/fsnotify v1.9.0
go: downloading github.com/go-viper/mapstructure/v2 v2.4.0
go: downloading github.com/sagikazarmark/locafero v0.11.0
go: downloading github.com/spf13/afero v1.15.0
go: downloading github.com/spf13/c…
```

</details>
@nudgebee-dev
nudgebee-dev Bot force-pushed the fix/fix-change-websocket-session-ended-log-l-1793347294 branch from c49206e to 4a8e290 Compare October 30, 2026 08:17
@nudgebee-dev

nudgebee-dev Bot commented Oct 30, 2026

Copy link
Copy Markdown
Author

Nudgebee Automated Followup

The CI workflow failed on the 'vuln' job running govulncheck against Go 1.26.9 due to standard library vulnerabilities discovered in Go 1.26.9. Updating the Go directive in go.mod to 1.26.10 ensures setup-go installs Go 1.26.10 in CI, eliminating stdlib vulnerabilities reported by govulncheck.

Commit: 4a8e2906ba9b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant