Cerebras Provider

Cerebras provides ultra-fast AI inference using wafer-scale engines — ideal for latency-critical and high-throughput applications.

Setup

Installation

Cerebras has its own dedicated package (not the generic openai package with a custom BaseURL):

import (
    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/cerebras"
)

Configuration

provider := cerebras.New(cerebras.Config{
    APIKey: os.Getenv("CEREBRAS_API_KEY"),
})

model, err := provider.LanguageModel(cerebras.ModelGPTOSS120B)

cerebras.Config also accepts BaseURL (default https://api.cerebras.ai/v1) and HTTPClient.

Get API Key

export CEREBRAS_API_KEY=...

Available Models

Model ID	Go constant	Best For
gpt-oss-120b	cerebras.ModelGPTOSS120B	General-purpose, high throughput
gemma-4-31b	cerebras.ModelGemma4_31B	Fast, cost-effective

Removed this cycle: llama3.1-8b, qwen-3-235b-a22b-instruct-2507, qwen-3-235b-a22b-thinking-2507, zai-glm-4.6, and zai-glm-4.7 were retired from Cerebras and no longer have Go constants. Cerebras accepts any model ID string, so passing a retired ID compiles but fails against the live API — use provider.LanguageModel(cerebras.ModelGPTOSS120B) or check inference-docs.cerebras.ai/models for the current catalog.

Provider-Specific Features

Measuring Throughput

start := time.Now()
result, err := ai.GenerateText(ctx, ai.GenerateTextOptions{
    Model:  model,
    Prompt: "Write a detailed 500-word essay about AI",
})
if err != nil {
    log.Fatal(err)
}

elapsed := time.Since(start)
tokensPerSec := float64(result.Usage.GetOutputTokens()) / elapsed.Seconds()

fmt.Printf("Generated %d tokens in %v (%.0f tokens/sec)\n",
    result.Usage.GetOutputTokens(), elapsed, tokensPerSec)

Streaming

stream, err := ai.StreamText(ctx, ai.StreamTextOptions{Model: model, Prompt: "Write a story"})
if err != nil {
    log.Fatal(err)
}
defer stream.Close()

for chunk := range stream.Chunks() {
    fmt.Print(chunk.Text)
}

Video Parts

Chat requests sent through this provider set AllowVideo: true internally, so video/* file parts are sent as video_url content parts — see OpenAI-compatible providers: Video Parts.

High Throughput

Handle massive concurrent requests:

var wg sync.WaitGroup
requests := 100

start := time.Now()
for i := 0; i < requests; i++ {
    wg.Add(1)
    go func(idx int) {
        defer wg.Done()
        _, err := ai.GenerateText(ctx, ai.GenerateTextOptions{
            Model:  model,
            Prompt: fmt.Sprintf("Query %d", idx),
        })
        if err != nil {
            log.Printf("Request %d failed: %v", idx, err)
        }
    }(i)
}
wg.Wait()

elapsed := time.Since(start)
fmt.Printf("Processed %d requests in %v\n", requests, elapsed)

Examples

Basic Text Generation

package main

import (
    "context"
    "fmt"
    "log"
    "os"
    "time"

    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/cerebras"
)

func main() {
    provider := cerebras.New(cerebras.Config{
        APIKey: os.Getenv("CEREBRAS_API_KEY"),
    })

    model, err := provider.LanguageModel(cerebras.ModelGPTOSS120B)
    if err != nil {
        log.Fatal(err)
    }

    start := time.Now()
    result, err := ai.GenerateText(context.Background(), ai.GenerateTextOptions{
        Model:  model,
        Prompt: "Explain wafer-scale computing in one paragraph",
    })
    if err != nil {
        log.Fatal(err)
    }

    elapsed := time.Since(start)
    tokensPerSec := float64(result.Usage.GetOutputTokens()) / elapsed.Seconds()

    fmt.Println(result.Text)
    fmt.Printf("\nGenerated %d tokens in %v (%.0f tokens/sec)\n",
        result.Usage.GetOutputTokens(), elapsed, tokensPerSec)
}

Interactive Chat Application

func interactiveChat(ctx context.Context, model provider.LanguageModel) {
    scanner := bufio.NewScanner(os.Stdin)
    messages := []types.Message{}

    for {
        fmt.Print("You: ")
        scanner.Scan()
        userInput := scanner.Text()

        if userInput == "exit" {
            break
        }

        messages = append(messages, types.Message{
            Role: types.RoleUser,
            Content: []types.ContentPart{
                types.TextContent{Text: userInput},
            },
        })

        stream, err := ai.StreamText(ctx, ai.StreamTextOptions{
            Model:    model,
            Messages: messages,
        })
        if err != nil {
            log.Printf("Error: %v", err)
            continue
        }

        fmt.Print("AI: ")
        var response string
        for chunk := range stream.Chunks() {
            fmt.Print(chunk.Text)
            response += chunk.Text
        }
        fmt.Println()

        messages = append(messages, types.Message{
            Role: types.RoleAssistant,
            Content: []types.ContentPart{
                types.TextContent{Text: response},
            },
        })
    }
}

High-Volume Processing

func processHighVolume(ctx context.Context, model provider.LanguageModel, prompts []string) {
    results := make(chan string, len(prompts))
    semaphore := make(chan struct{}, 50) // Limit concurrency

    start := time.Now()

    var wg sync.WaitGroup
    for i, prompt := range prompts {
        wg.Add(1)
        semaphore <- struct{}{} // Acquire

        go func(idx int, p string) {
            defer wg.Done()
            defer func() { <-semaphore }() // Release

            result, err := ai.GenerateText(ctx, ai.GenerateTextOptions{Model: model, Prompt: p})
            if err != nil {
                log.Printf("Failed %d: %v", idx, err)
                return
            }

            results <- result.Text
        }(i, prompt)
    }

    wg.Wait()
    close(results)

    elapsed := time.Since(start)
    fmt.Printf("Processed %d prompts in %v\n", len(prompts), elapsed)
    fmt.Printf("Average: %.2fs per prompt\n",
        elapsed.Seconds()/float64(len(prompts)))
}

Best Practices

1. Leverage Speed

   • Build real-time interactive applications
   • Enable instant user feedback
   • Process high volumes efficiently

2. Model Selection

   • Check inference-docs.cerebras.ai/models for current production models
   • Use cerebras.ModelGPTOSS120B for general-purpose tasks
   • Use cerebras.ModelGemma4_31B for cost-sensitive, high-throughput tasks

3. Architecture

   • Wafer-scale engine eliminates GPU bottlenecks
   • No memory bandwidth constraints
   • Consistent low latency

4. Use Cases

   • Real-time chat applications
   • Live content generation
   • High-volume batch processing
   • Interactive AI assistants

Rate Limits & Pricing

Check the Cerebras dashboard for current rate limits and per-model pricing — both vary by model and plan.

Error Handling

result, err := ai.GenerateText(ctx, ai.GenerateTextOptions{Model: model, Prompt: prompt})
if err != nil {
    if strings.Contains(err.Error(), "rate_limit") {
        log.Println("Rate limited")
    }
    log.Fatal(err)
}

Workflow Serialization

Cerebras language models can cross a workflow boundary with providerutils.SerializeModel / DeserializeModel. See Provider Serialization for the mechanism.

See Also

• API Reference: GenerateText
• Cerebras Documentation
• Groq Provider - Alternative fast inference