Integrate Structured Outputs with Scala Classes/Functions

[pstutz]

Last year, I worked on a project called Syncodia, where I explored using Scala classes/functions to specify JSON schemas and allow invoking functions seamlessly. Although it was an early proof of concept, I believe that adapting schema-inference macros from JSON serialization libraries could allow for a cleaner implementation.

With the recent support for Structured Outputs, integrating with Scala classes has become even more useful. This feature could provide type safety and simplify schema definitions when interacting with the OpenAI API.

Would this be within the scope of the project? If so, I’d be interested in contributing designs and code. I’d appreciate your thoughts on this proposal.

[adamw]

This does sound interesting, though I'd need to understand a bit more in detail as to what's the scope of the feature that you are proposing.

So first there's a component that generates a JSON Schema from a case class? This does sound familiar to a part of what Tapir does. Although originally developed for an HTTP library, maybe could be useful here as well?

Secondly, there would be some code needed to integrate with structured outputs and the protocol that OpenAI expects. I think this would definitely be in the scope of a project such as sttp-openai.

[pstutz]

The scope of this would be two features that I think would allow for innovative and seamless ways to combine LLMs with traditional programming/computation. Much of this is only possible thanks to the static type system and the advanced meta-programming features of Scala.

Automatic Function Registration and Invocation
This feature allows passing a Scala function to the sttp-openai client, which automatically registers the function as a tool, along with its signature converted to a JSON schema. When the LLM invokes the function, it passes the parameters as JSON, which are then converted back to Scala values. The function is invoked with these values, and the result is serialized to JSON and returned as the tool’s output. This concept has already been demonstrated as a proof of concept in Syncodia.

Have an LLM Simulate a Function Invocation
This feature allows defining unimplemented functions within a trait, which are then “implemented” by sttp-openai. The function’s signature and comments are extracted and sent to the LLM, along with any invocation parameters and an instruction to "compute" the result of that function. The LLM generates a response that matches the expected return type, enforced by structured outputs. The result, in JSON format, is converted back to a Scala value that is an instance of the function’s return type.

[adamw]

Thanks for the in-depth explanation. Both features look exciting :) Would be great if this could be part of sttp-openai. As I wrote before, maybe the process of generating the JSON Schema can be simplified by just using tapir-core to derive the schemas.