feat: add /metrics endpoint and request timeout handling - #32
Open
tamish560 wants to merge 1 commit into
Open
Conversation
Metrics endpoint (issue Gilfeather#4): - GET /metrics returns JSON with uptime, total_requests, total_errors, total_inferences, average_inference_time_ms, memory_usage_bytes - Request and error counters tracked via atomic counters in ServerState - Test coverage for the new endpoint Request timeout handling (issue Gilfeather#10): - Configurable HTTP request timeout via ServerConfig.request_timeout_secs - Default 30 seconds, configurable via CLI - Inference runs in spawn_blocking with tokio::time::timeout wrapper - Returns HTTP 408 Request Timeout on timeout - Returns HTTP 500 on task join errors or inference errors Also removes duplicate tower dependency in Cargo.toml.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses issues #4 and #10.
/metrics endpoint (issue #4)
GET /metricsreturns JSON with server metrics:uptime_seconds,total_requests,total_errors,total_inferences,average_inference_time_ms,last_inference_time_ms,memory_usage_bytes,timestampAtomicU64inServerStatemodel_infostats but in a dedicated monitoring-friendly formatRequest timeout handling (issue #10)
request_timeout_secstoServerConfig(default: 30s)tokio::task::spawn_blockingwrapped withtokio::time::timeout408 Request Timeouton timeout500on join errors or inference failuresOther changes
towerdependency inCargo.tomltimeoutfeature totowercrateTest plan
cargo test --no-default-features --lib- 15/15 passtest_metrics_endpoint- verifies /metrics returns 200 with correct fieldstest_predict_endpoint_success- still passes with timeout wrappertest_predict_endpoint_invalid_input- validation errors still return 400test_predict_endpoint_empty_input- empty input still returns 400