feat(agents): generator-as-protocol-adapter and tool coercion

[Hartorn]

Known limitation

The serialize_tools, serialize_messages, and deserialize_response methods on BaseGenerator are currently "advisory" — they exist as override points but are called from inside LiteLLMGenerator._complete_once, not orchestrated by the base class. A new provider subclass could skip them entirely with no enforcement.

Next step: Restructure so BaseGenerator owns the serialize → call → deserialize pipeline, with the subclass implementing only a _send(wire_messages, wire_tools, params) method. This also ties into replacing the mixin-based retry/rate-limiting with a composable middleware pipeline. Tracked as a follow-up.

Add serialize_tools, serialize_messages, deserialize_response to BaseGenerator so provider subclasses own all wire-format translation. LiteLLMGenerator now delegates through these methods instead of calling to_litellm/from_litellm directly.

Tool.run() gains input coercion (TypeAdapter + _params_model) and output serialization (_return_adapter), returning str to eliminate double-quoting in workflow. Workflow _run_tools simplified accordingly.

Description

Related Issue

Type of Change

• 📚 Examples / docs / tutorials / dependencies update
• 🔧 Bug fix (non-breaking change which fixes an issue)
• 🥂 Improvement (non-breaking change which improves an existing feature)
• 🚀 New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to change)
• 🔐 Security fix

Checklist

• I've read the CODE_OF_CONDUCT.md document.
• I've read the CONTRIBUTING.md guide.
• I've written tests for all new methods and classes that I created.
• I've written the docstring in Google format for all the methods and classes that I used.
• I've updated the pdm.lock running pdm update-lock (only applicable when pyproject.toml has been
  modified)

[gemini-code-assist]

Summary of Changes

Hello @Hartorn, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant architectural improvement by formalizing the role of generators as protocol adapters, centralizing the logic for translating between internal data structures and external LLM provider formats. It also enhances the Tool class with automatic input coercion and output serialization, making tool execution more robust and predictable. These changes streamline the integration of new LLM providers and simplify the development of agent workflows by abstracting away low-level data transformations.

Highlights

• Generator as Protocol Adapter: The BaseGenerator class has been refactored to act as a protocol adapter, centralizing the responsibility for converting internal Message and Tool objects to and from provider-specific wire formats. This is achieved through new serialize_tools, serialize_messages, and deserialize_response methods, which subclasses can override.
• Tool Input Coercion and Output Serialization: The Tool.run() method now includes robust input coercion, validating arguments against a Pydantic model derived from the tool's signature. It also performs output serialization, ensuring that the result is always a JSON-safe string, handling complex types like BaseModel, datetime, and UUID automatically.
• LiteLLM Integration Update: The LiteLLMGenerator has been updated to delegate its serialization and deserialization tasks to the new methods on BaseGenerator, removing direct calls to Message.to_litellm() and Message.from_litellm().
• Simplified Workflow: The _run_tools method in the workflow has been simplified, as Tool.run() now directly returns the serialized string content, eliminating the need for manual json.dumps calls within the workflow.
• Architectural Rules Document: A new markdown document, generator-adapter.mdc, has been added to formally define the architectural principles and rules for generator-based protocol adaptation and tool handling.
• Comprehensive Testing: Extensive new test cases have been added to validate the new generator translation methods and the input coercion and output serialization capabilities of the Tool.run() method.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• libs/giskard-agents/.cursor/rules/generator-adapter.mdc
  • Added new architectural rules document for generator protocol adaptation.
• libs/giskard-agents/src/giskard/agents/generators/base.py
  • Updated BaseGenerator docstring to describe its role as a protocol adapter.
  • Added default serialize_tools, serialize_messages, and deserialize_response methods to BaseGenerator.
• libs/giskard-agents/src/giskard/agents/generators/litellm_generator.py
  • Removed unused LiteLLMMessage import.
  • Updated LiteLLMGenerator docstring.
  • Modified _complete_once to use BaseGenerator's new serialization and deserialization methods.
• libs/giskard-agents/src/giskard/agents/tools/tool.py
  • Imported json, PrivateAttr, and TypeAdapter.
  • Added _params_model and _return_adapter private attributes to Tool for coercion and serialization.
  • Modified from_callable to capture return annotations and parameter models for TypeAdapter.
  • Changed Tool.run() return type to str.
  • Updated Tool.run() docstring to reflect input coercion and output serialization.
  • Implemented input coercion using Pydantic's model_validate in Tool.run().
  • Implemented output serialization using TypeAdapter.dump_python or json.dumps in Tool.run().
• libs/giskard-agents/src/giskard/agents/workflow.py
  • Modified _run_tools to directly use the serialized string output from tool.run().
• libs/giskard-agents/tests/test_generator.py
  • Added imports for json, Any, BaseGenerator, Tool, tool, Field, PrivateAttr.
  • Added SpyGenerator for testing generator translation method calls.
  • Added tests for test_translation_methods_called_during_workflow, test_custom_serialize_messages_override, and test_custom_serialize_tools_override.
• libs/giskard-agents/tests/test_tools.py
  • Added imports for json, datetime, timezone, UUID, BaseModel.
  • Added CityWeather, Address, Person, TimestampedRecord Pydantic models for testing.
  • Added tests for Tool.run() output serialization for various types, including BaseModel.
  • Added tests for Tool.run() input coercion for BaseModel, optional BaseModel, and lists of BaseModel.
  • Added tests for primitive type handling in Tool.run().

Activity

• The pull request is currently marked as ON-HOLD, indicating it's a foundational feature that might be awaiting further architectural decisions or related changes.
• The author has clearly outlined known limitations and proposed next steps for future restructuring, suggesting a phased approach to these architectural improvements.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a significant and well-executed architectural refactoring by establishing the BaseGenerator as a protocol adapter. This change, along with the introduction of input coercion and output serialization in Tool.run(), greatly improves the separation of concerns and simplifies the workflow logic. The new architectural rules documented in generator-adapter.mdc are a fantastic addition for long-term maintainability.

I've found one issue related to error handling during argument validation in tools, which I've detailed in a specific comment. Once that is addressed, this will be an excellent contribution to the codebase.

[Hartorn]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a significant and well-executed architectural refactoring, turning the BaseGenerator into a protocol adapter. This decouples the core agent logic from provider-specific data formats. The introduction of serialize_tools, serialize_messages, and deserialize_response provides clear extension points for new providers. Additionally, the Tool.run method is enhanced with input coercion and output serialization, ensuring that tools always return a string, which simplifies the workflow logic. The changes are supported by a comprehensive set of new and updated tests. My review found one potential issue with error handling during tool argument validation. Overall, this is a high-quality contribution that greatly improves the extensibility and robustness of the agent framework.

[Hartorn]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a significant and valuable architectural refactoring by establishing the generator as a protocol adapter. The introduction of serialize_tools, serialize_messages, and deserialize_response methods in BaseGenerator effectively decouples the core agent logic from provider-specific data formats. The changes to Tool.run() to include input coercion and automatic output serialization are also excellent improvements, simplifying tool implementation and workflow logic. The updates are consistently applied across the codebase, including extensive and well-written tests for the new functionality. I have one suggestion to simplify the argument preparation logic in Tool.run() for better clarity and to remove redundancy. Overall, this is a great step forward for the agent's architecture.

[Hartorn]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a significant and beneficial architectural refactoring, positioning the BaseGenerator as a protocol adapter. This change centralizes provider-specific translation logic, improving modularity and maintainability. The addition of input coercion and output serialization for tools is also a great enhancement, making tool execution more robust and predictable. The implementation is solid and well-tested for the most part. My feedback focuses on a few areas: ensuring type safety for FinishReason, a minor refactoring opportunity in tool argument handling, and, most importantly, updating several test mocks in the giskard-checks library to align with the new generator architecture, which is crucial for maintaining test effectiveness.

[mattbit]

Great improvement on tools and removing the to_litellm methods which don't belong to the tools and chats. This stuff should be handled internally by the Generator implementation.

The "protocol adapter" for the serialization is not necessary, this serialize/deserialize clean up is specific to litellm and internal to the final BaseGenerator implementation.
We shouldn't impose a structure on how these transformations should be implemented (e.g. by message/by tools), as it will be different for every provider.
We should move what's needed in the LiteLLMGenerator (as internal helpers) and leave the BaseGenerator untouched.

I agree that in some cases having standardized serialization for chat completions could help, but right now it's not needed. It's also not necessarily compatible with different APIs. Especially we see providers APIs diverge from the chat completion standard, so it could be counterproductive to impose such abstraction.

[mattbit]

All good!