Skip to content

aimlapi.mdx

Latest commit

 

History

History
104 lines (73 loc) · 3.22 KB

aimlapi.mdx

File metadata and controls

104 lines (73 loc) · 3.22 KB
Raw
title ChatAIMLAPI integration
description Integrate with the ChatAIMLAPI chat model using LangChain Python.

This guide helps you get started with AI/ML API chat models.

AI/ML API provides unified access to hundreds of hosted foundation models with high availability and throughput.

Overview

Integration details

Class Package Serializable JS support Downloads Version
ChatAIMLAPI langchain-aimlapi beta ❌ PyPI - Downloads PyPI - Version

Model features

Tool calling Structured output Image input Audio input Video input Token-level streaming Native async Token usage Logprobs
✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅

Setup

To access AI/ML API models you'll need to create an account, get an API key, and install the langchain-aimlapi integration package.

Credentials

Head to aimlapi.com to sign up and generate an API key. Once you've done this set the AIMLAPI_API_KEY environment variable:

import getpass
import os

if not os.getenv("AIMLAPI_API_KEY"):
    os.environ["AIMLAPI_API_KEY"] = getpass.getpass("Enter your AI/ML API key: ")

To enable automated tracing of your model calls, set your LangSmith API key:

os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
os.environ["LANGSMITH_TRACING"] = "true"

Installation

The LangChain AI/ML API integration lives in the langchain-aimlapi package:

pip install -qU langchain-aimlapi

Instantiation

Now we can instantiate our model object and generate chat completions:

from langchain_aimlapi import ChatAIMLAPI

llm = ChatAIMLAPI(
    model="meta-llama/Llama-3-70b-chat-hf",
    temperature=0.7,
    max_tokens=512,
    timeout=30,
    max_retries=3,
)

Invocation

messages = [
    ("system", "You are a helpful assistant that translates English to French."),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg
AIMessage(content="J'adore la programmation.", response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 23, 'total_tokens': 32}, 'model_name': 'meta-llama/Llama-3-70b-chat-hf'}, id='run-...')

print(ai_msg.content)
J'adore la programmation.

Streaming invocation

You can also stream responses token-by-token:

stream = llm.stream_events("List top 5 programming languages in 2025 with reasons.", version="v3")
for token in stream.text:
    print(token, end="", flush=True)