SambaNova

Overview

The SambaNovaLLMService leverages SambaNova Systems' specialized AI hardware for high-speed inference. It is fully OpenAI-compatible and optimized for large-scale production workloads.

Installation

pip install piopiy-ai

Prerequisites

• A SambaNova API key (Get yours here).
• Set your API key in your environment:

  export SAMBANOVA_API_KEY="your_api_key_here"

Configuration

SambaNovaLLMService Parameters

Parameter	Type	Default	Description
api_key	str	Required	Your SambaNova API key.
model	str	"Llama-3.1-8B-Instruct"	Model identifier.
base_url	str	"https://api.sambanova.ai/v1"	API endpoint.

Usage

Basic Setup

import os
from piopiy.services.sambanova.llm import SambaNovaLLMService

llm = SambaNovaLLMService(
    api_key=os.getenv("SAMBANOVA_API_KEY"),
    model="Llama-3.1-70B-Instruct"
)

Notes

• Hardware Accelerated: Optimized to run on SambaNova's SN40L reusable AI data centers.
• Scalability: Designed to handle high concurrency with stable latency.