Skip to content

Enable Ollama integration - #150

Closed
navd wants to merge 1 commit into
google:mainfrom
navd:main
Closed

navd wants to merge 1 commit into
google:mainfrom
navd:main

Conversation

@navd

@navd navd commented Apr 13, 2025

Copy link
Copy Markdown

feat: Add Ollama LLM support to ADK models

This commit introduces support for using Ollama-served language models
within the Agent Development Kit (ADK). This allows developers to
easily integrate locally running LLMs (including various open-source
models) into their ADK agents, facilitating local development,
testing, and experimentation without reliance on external cloud APIs.

Rationale:

  • Enable use of local and open-source LLMs with ADK.
  • Provide flexibility for developers during prototyping and testing.
  • Reduce dependency on cloud provider APIs for certain use cases.

Changes:

  • Introduced the OllamaLlm class in src/google/adk/models/ollama_llm.py,
    inheriting from BaseLlm.
  • Implements generate_content_async to communicate with the Ollama
    /api/chat endpoint using httpx.
  • Supports both streaming (stream=True) and non-streaming (stream=False)
    generation.
  • Includes helper functions (_convert_content_to_ollama_message,
    _build_ollama_request_payload) to convert between ADK LlmRequest/Content
    and Ollama's message format.
  • Added OllamaLlm to src/google/adk/models/__init__.py and the model
    registry (src/google/adk/models/registry.py) for automatic discovery.
  • Added comprehensive unit tests in tests/unittests/models/test_ollama_llm.py
    covering configuration, payload generation, streaming/non-streaming
    responses, and error handling using pytest and unittest.mock.

Dependencies:

  • Introduces a dependency on the httpx library (pip install httpx)
    for asynchronous HTTP requests.

Usage:

Ensure Ollama server is running (ollama serve) and the desired model
is pulled (ollama pull <model_name>).

import asyncio
from google.adk.models import OllamaLlm, LlmRequest
from google.genai.types import Content, Part

async def run_ollama_example():
    # Model name must match a model pulled in Ollama
    MODEL_NAME = "gemma3:12b" # Adjust as needed

    try:
        # 1. Instantiate the Ollama LLM
        ollama_llm = OllamaLlm(model=MODEL_NAME, api_base="http://localhost:11434")

        # 2. Create a request
        request = LlmRequest(
            model=MODEL_NAME, # Provide model name as string
            contents=[Content(role="user", parts=[Part(text="Why is the sky blue?")])]
        )

        # 3. Generate content (non-streaming)
        print(f"Sending request to Ollama model: {MODEL_NAME}...")
        async for response in ollama_llm.generate_content_async(request, stream=False):
            if response.content and response.content.parts:
                print(f"Response: {response.content.parts[0].text}")
            else:
                print("Received empty response.")

    except Exception as e:
        print(f"Error: {e}")
        print("Ensure Ollama is running and the model is pulled.")

if __name__ == "__main__":
    # Prerequisites: ollama serve & ollama pull gemma3:12b (or your MODEL_NAME)
    asyncio.run(run_ollama_example())

See examples/multi_agent_debate.py for a more detailed example involving
multiple agents and tools.

Testing:

  • The implementation is covered by unit tests located in
    tests/unittests/models/test_ollama_llm.py. These can be run using pytest.
  • Manual testing was also performed using the script above and the
    multi-agent example against a running Ollama instance serving models
    like gemma3:12b.

@navd navd closed this Apr 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant