Skip to main content

Large Language Models (LLM)

Overview​

LLM services act as the brain of your agent. They process the transcribed text and conversation history to generate intelligent responses.

Supported Providers​

OpenAI​

The industry standard for reasoning and complex task handling.

from piopiy.services.openai.llm import OpenAILLMService

llm = OpenAILLMService(
    api_key="YOUR_OPENAI_KEY",
    model="gpt-4o-mini"
)

Anthropic​

Excellent for long-context and highly nuanced conversations.

from piopiy.services.anthropic.llm import AnthropicLLMService

llm = AnthropicLLMService(
    api_key="YOUR_ANTHROPIC_KEY",
    model="claude-3-5-sonnet-latest"
)

Groq​

Ultra-low latency inference for incredibly fast response times.

from piopiy.services.groq.llm import GroqLLMService

llm = GroqLLMService(
    api_key="YOUR_GROQ_KEY",
    model="llama-3.1-70b-versatile"
)

Key Capabilities​

  • Streaming Responses: Tokens are streamed to the TTS service as they are generated, minimizing wait time.
  • Function Calling: Define tools that the LLM can use to interact with your backend.
  • System Prompts: Control the agent's personality and rules via instructions.

Optimization​

  • Prompt Engineering: Keep instructions clear and concise to reduce token count and latency.
  • Model Choice: Use smaller, faster models (e.g., gpt-4o-mini) unless complex reasoning is required.