Skip to content

Error during response serialization discards stack trace #3091

Description

@sybereal

Problem description

When an error occurs during response serialization, i.e., the responseSerialize callback of the MethodDefinition throws, this section of code extracts the error message and discards the rest of the error:

try {
response = this.serializeMessage(message);
} catch (e) {
this.sendStatus({
code: Status.INTERNAL,
details: `Error serializing response: ${getErrorMessage(e)}`,
metadata: null,
});
return;
}

Since this only ever surfaces as an INTERNAL error on the client side, the server operator has no indication in their logs that anything might be wrong until a client complains.
Additionally, even if reported by a client, the error message may be of limited use without a stack trace. In our case, I ended up needing to attach a debugger to the server process and put a breakpoint in the above catch block to log the error object incl. stack trace to the console.

I propose logging such errors or exposing some other mechanism, e.g., a callback, for library users to react to them. For example, we have integrated Sentry reporting for exactly scenarios like this, and were very confused why nothing showed up there.

Reproduction steps

Apply the following patch to the helloworld example in this repo:

diff --git a/examples/helloworld/static_codegen/greeter_server.js b/examples/helloworld/static_codegen/greeter_server.js
index ae2ab949..66df1e6a 100644
--- a/examples/helloworld/static_codegen/greeter_server.js
+++ b/examples/helloworld/static_codegen/greeter_server.js
@@ -36,6 +36,11 @@ function sayHello(call, callback) {
  */
 function main() {
   var server = new grpc.Server();
+
+  services.GreeterService.sayHello.responseSerialize = () => {
+    throw new Error("oops");
+  };
+
   server.addService(services.GreeterService, {sayHello: sayHello});
   server.bindAsync('0.0.0.0:50051', grpc.ServerCredentials.createInsecure(), (err, port) => {
     if (err != null) {

Then invoke the sayHello method any way you want.

Environment

  • OS name, version and architecture: Debian Bookworm AArch64
  • Node version: 24.18.1
  • Node installation method: Docker Hub library/node
  • Package name and version: @grpc/grpc-js@1.14.3

Additional context

No logs, which is the problem.

As for what led to this error: In our case, what happened was that bad data had ended up in a database row—a null in a PostgreSQL text[] column. This lead to generated serialization code attempting to call string(null) on a Writer object from protobufjs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions