The CLI agent now supports multiple LLM providers with real-time streaming. Connect your Anthropic API key, Claude Code, or any OpenAI-compatible endpoint, and stream completions through SessionCast.

Supported Providers

• anthropic — Direct Anthropic API (Claude 3.5 Sonnet, Opus, etc.)
• claude-code — Claude Code CLI (uses your local Claude Code installation)
• openai — OpenAI API (GPT-4, etc.)

Streaming

LLM responses now stream chunk by chunk over WebSocket. The web viewer and SDK clients receive partial responses as they're generated — no more waiting for the full completion.

Client → relay → agent → LLM provider
                           ↓ (streaming)
Client ← relay ← agent ← chunk 1
Client ← relay ← agent ← chunk 2
Client ← relay ← agent ← chunk 3
Client ← relay ← agent ← [done]

Using via Go SDK

stream, err := client.LlmChatStream(ctx, &sessioncast.LlmChatRequest{
    Model: "anthropic",
    Messages: []sessioncast.ChatMessage{
        {Role: "user", Content: "Explain WebSockets in 3 sentences."},
    },
})

for event := range stream {
    if event.Chunk != nil {
        fmt.Print(event.Chunk.Content)
    }
}

Using via Web Viewer

The web viewer's command bar supports LLM chat. Type a question and get a streaming response directly in the terminal interface.

Provider Configuration

Providers are auto-detected based on environment variables:

• ANTHROPIC_API_KEY — enables the anthropic provider
• Claude Code installed at ~/.claude — enables the claude-code provider
• OPENAI_API_KEY — enables the openai provider

The agent negotiates capabilities with connecting clients, so they only see providers that are actually available.

Requirements

• CLI Agent v2.2.2 or later
• At least one LLM provider configured on the agent machine