Speech Generation

The Go AI SDK provides the ai.GenerateSpeech() function to generate speech from text using a speech model.

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/openai"
)

func main() {
    ctx := context.Background()

    provider := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
    speechModel, _ := provider.SpeechModel("tts-1")

    result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model: speechModel,
        Text:  "Hello, world!",
        Voice: "alloy",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Save audio to file
    err = os.WriteFile("output.mp3", result.Audio.Data, 0644)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Audio saved to output.mp3")
}

Accessing Audio Data

To access the generated audio:

speed := 1.25

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})

// Audio bytes
audioData := result.Audio.Data

// MIME type (e.g., "audio/mpeg")
mimeType := result.Audio.MediaType

fmt.Printf("Generated %d bytes as %s\n", len(audioData), mimeType)

Settings

Voice Selection

Different models support different voices. Check your provider's documentation for available voices:

// OpenAI voices: alloy, echo, fable, onyx, nova, shimmer
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "nova",
})

Speech Speed

Control the speed of speech generation (typically 0.25 to 4.0):

speed := 1.5 // 1.5x speed

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
    Speed: &speed,
})

Language Setting

You can specify the language for speech generation (provider support varies):

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model:    speechModel,
    Text:     "Hola, mundo!",
    Voice:    "alloy",
    Language: "es", // Spanish
})

Audio Format

Some providers allow you to specify the output audio format:

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model:  speechModel,
    Text:   "Hello, world!",
    Voice:  "alloy",
    OutputFormat: "mp3", // or "wav", "opus", "flac", etc.
})

Provider-Specific Settings

You can set model-specific settings with the ProviderOptions parameter:

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model:        speechModel,
    Text:         "Hello, world!",
    Voice:        "alloy",
    OutputFormat: "opus",
    Speed:        &speed,
})

Timeouts with Context

Use Go's context for timeouts and cancellation:

import "time"

// Timeout after 30 seconds
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})
if err != nil {
    if ctx.Err() == context.DeadlineExceeded {
        fmt.Println("Speech generation timed out")
    }
    log.Fatal(err)
}

Custom Headers

Add custom headers to the speech generation request:

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
    Headers: map[string]string{
        "X-Custom-Header": "custom-value",
    },
})

Warnings

If the model returns warnings, they will be available in the Warnings field:

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})
if err != nil {
    log.Fatal(err)
}

for _, warning := range result.Warnings {
    fmt.Printf("Warning: %s\n", warning.Message)
}

Error Handling

When ai.GenerateSpeech() cannot generate valid audio, it returns an error:

import "errors"

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})
if err != nil {
    if ai.IsNoSpeechGeneratedError(err) {
        fmt.Println("No speech was generated")
        var noSpeech *ai.NoSpeechGeneratedError
        if errors.As(err, &noSpeech) {
            fmt.Printf("Responses: %d\n", len(noSpeech.Responses))
        }
    } else {
        fmt.Printf("Speech generation error: %v\n", err)
    }
    return
}

Common speech generation errors:

• NoSpeechGeneratedError: The model returned no generated audio
• ProviderError: Provider-specific HTTP or API failure
• InvalidArgumentError: Invalid input such as negative MaxRetries
• context.DeadlineExceeded: Operation timed out
• context.Canceled: Operation was canceled

Practical Examples

Generate and Save Audio

func generateAndSaveAudio(ctx context.Context, model provider.SpeechModel, text, voice, outputPath string) error {
    result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model: model,
        Text:  text,
        Voice: voice,
    })
    if err != nil {
        return fmt.Errorf("speech generation failed: %w", err)
    }

    err = os.WriteFile(outputPath, result.Audio.Data, 0644)
    if err != nil {
        return fmt.Errorf("failed to save audio: %w", err)
    }

    fmt.Printf("Audio saved to %s (%d bytes)\n", outputPath, len(result.Audio.Data))
    return nil
}

Generate Multiple Voices

func generateWithMultipleVoices(ctx context.Context, model provider.SpeechModel, text string, voices []string) error {
    for i, voice := range voices {
        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  text,
            Voice: voice,
        })
        if err != nil {
            return fmt.Errorf("failed to generate with voice %s: %w", voice, err)
        }

        filename := fmt.Sprintf("output-%s.mp3", voice)
        err = os.WriteFile(filename, result.Audio.Data, 0644)
        if err != nil {
            return fmt.Errorf("failed to save %s: %w", filename, err)
        }

        fmt.Printf("%d. Generated %s\n", i+1, filename)
    }

    return nil
}

// Usage
voices := []string{"alloy", "echo", "fable", "onyx", "nova", "shimmer"}
generateWithMultipleVoices(ctx, speechModel, "Hello, world!", voices)

Text-to-Speech with Retry Logic

func generateSpeechWithRetry(ctx context.Context, model provider.SpeechModel, text, voice string, maxRetries int) (*ai.GenerateSpeechResult, error) {
    return ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model:      model,
        Text:       text,
        Voice:      voice,
        MaxRetries: &maxRetries,
    })
}

Batch Text-to-Speech

func batchGenerateSpeech(ctx context.Context, model provider.SpeechModel, texts []string, voice string) ([][]byte, error) {
    audioFiles := make([][]byte, len(texts))

    for i, text := range texts {
        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  text,
            Voice: voice,
        })
        if err != nil {
            return nil, fmt.Errorf("failed to generate audio for text %d: %w", i, err)
        }

        audioFiles[i] = result.Audio.Data
        fmt.Printf("Generated audio %d/%d (%d bytes)\n", i+1, len(texts), len(result.Audio.Data))
    }

    return audioFiles, nil
}

Parallel Speech Generation with Goroutines

func generateSpeechParallel(ctx context.Context, model provider.SpeechModel, texts []string, voice string) ([][]byte, error) {
    type result struct {
        index int
        audio []byte
        err   error
    }

    resultChan := make(chan result, len(texts))
    audioFiles := make([][]byte, len(texts))

    // Start goroutines
    for i, text := range texts {
        go func(index int, textContent string) {
            speech, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
                Model: model,
                Text:  textContent,
                Voice: voice,
            })
            if err != nil {
                resultChan <- result{index: index, err: err}
                return
            }

            resultChan <- result{index: index, audio: speech.Audio.Data}
        }(i, text)
    }

    // Collect results
    for i := 0; i < len(texts); i++ {
        res := <-resultChan
        if res.err != nil {
            return nil, fmt.Errorf("speech generation failed for text %d: %w", res.index, res.err)
        }
        audioFiles[res.index] = res.audio
    }

    return audioFiles, nil
}

Generate Speech with Different Speeds

func generateAtDifferentSpeeds(ctx context.Context, model provider.SpeechModel, text, voice string) error {
    speeds := []float64{0.5, 0.75, 1.0, 1.25, 1.5, 2.0}

    for _, speed := range speeds {
        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  text,
            Voice: voice,
            Speed: &speed,
        })
        if err != nil {
            return fmt.Errorf("failed to generate at speed %.2fx: %w", speed, err)
        }

        filename := fmt.Sprintf("output-%.2fx.mp3", speed)
        err = os.WriteFile(filename, result.Audio.Data, 0644)
        if err != nil {
            return fmt.Errorf("failed to save %s: %w", filename, err)
        }

        fmt.Printf("Generated %s at %.2fx speed\n", filename, speed)
    }

    return nil
}

Streaming to HTTP Response

import "net/http"

func speechHandler(w http.ResponseWriter, r *http.Request) {
    text := r.URL.Query().Get("text")
    voice := r.URL.Query().Get("voice")

    if text == "" || voice == "" {
        http.Error(w, "Missing text or voice parameter", http.StatusBadRequest)
        return
    }

    provider := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
    speechModel, _ := provider.SpeechModel("tts-1")

    result, err := ai.GenerateSpeech(r.Context(), ai.GenerateSpeechOptions{
        Model: speechModel,
        Text:  text,
        Voice: voice,
    })
    if err != nil {
        http.Error(w, fmt.Sprintf("Speech generation failed: %v", err), http.StatusInternalServerError)
        return
    }

    // Set appropriate headers
    w.Header().Set("Content-Type", result.Audio.MediaType)
    w.Header().Set("Content-Length", fmt.Sprintf("%d", len(result.Audio.Data)))
    w.Header().Set("Content-Disposition", "inline; filename=\"speech.mp3\"")

    // Write audio data
    w.Write(result.Audio.Data)
}

Long Text with Chunking

import "strings"

func generateLongText(ctx context.Context, model provider.SpeechModel, longText, voice string, maxChunkSize int) ([]byte, error) {
    // Split text into chunks
    sentences := strings.Split(longText, ". ")
    var chunks []string
    var currentChunk strings.Builder

    for _, sentence := range sentences {
        if currentChunk.Len()+len(sentence) > maxChunkSize {
            chunks = append(chunks, currentChunk.String())
            currentChunk.Reset()
        }
        currentChunk.WriteString(sentence)
        currentChunk.WriteString(". ")
    }

    if currentChunk.Len() > 0 {
        chunks = append(chunks, currentChunk.String())
    }

    // Generate audio for each chunk
    var allAudio []byte
    for i, chunk := range chunks {
        fmt.Printf("Generating chunk %d/%d...\n", i+1, len(chunks))

        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  chunk,
            Voice: voice,
        })
        if err != nil {
            return nil, fmt.Errorf("failed to generate chunk %d: %w", i+1, err)
        }

        allAudio = append(allAudio, result.Audio.Data...)
    }

    return allAudio, nil
}

Speech Models

Several providers offer speech synthesis models:

Provider	Model	Voices	Languages
OpenAI	tts-1	alloy, echo, fable, onyx, nova, shimmer	Multiple
OpenAI	tts-1-hd	alloy, echo, fable, onyx, nova, shimmer	Multiple
OpenAI	gpt-4o-mini-tts	alloy, echo, fable, onyx, nova, shimmer	Multiple
ElevenLabs	eleven_v3	Custom voices	30+ languages
ElevenLabs	eleven_multilingual_v2	Custom voices	Multilingual
ElevenLabs	eleven_flash_v2_5	Custom voices	Fast, low latency
ElevenLabs	eleven_flash_v2	Custom voices	Fast
ElevenLabs	eleven_turbo_v2_5	Custom voices	Ultra-fast
ElevenLabs	eleven_turbo_v2	Custom voices	Ultra-fast
LMNT	aurora	Various	English
LMNT	blizzard	Various	English
Hume	default	Expressive	English

Example with Different Providers

// OpenAI
openaiProvider := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
openaiModel, _ := openaiProvider.SpeechModel("tts-1-hd")

// ElevenLabs
elevenProvider := elevenlabs.New(elevenlabs.Config{APIKey: os.Getenv("ELEVENLABS_API_KEY")})
elevenModel, _ := elevenProvider.SpeechModel("eleven_multilingual_v2")

// LMNT
lmntProvider := lmnt.New(lmnt.Config{APIKey: os.Getenv("LMNT_API_KEY")})
lmntModel, _ := lmntProvider.SpeechModel("aurora")

// Use any model with the same API
result, _ := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: openaiModel, // or elevenModel, or lmntModel
    Text:  "Hello, world!",
    Voice: "alloy", // voice name depends on provider
})

Supported Audio Formats

Common audio formats supported by providers:

• MP3 (audio/mpeg) - Most common, good compression
• WAV (audio/wav) - Uncompressed, high quality
• Opus (audio/opus) - Efficient for speech, low latency
• AAC (audio/aac) - Good quality, streaming-friendly
• FLAC (audio/flac) - Lossless compression
• PCM (audio/pcm) - Raw audio data

Format availability varies by provider. Check provider documentation for specific support.

Best Practices

1. Choose the Right Model: Balance quality, speed, and cost
2. Set Timeouts: Speech generation can be slow; always use context with timeout
3. Handle Errors: Implement proper error handling and retry logic
4. Select Appropriate Voice: Test different voices for your use case
5. Monitor Costs: Track character usage and generation costs
6. Respect Rate Limits: Implement rate limiting for bulk generation
7. Use Parallel Processing: For multiple texts, use goroutines
8. Chunk Long Texts: Split long texts into manageable chunks
9. Cache Results: Store generated audio when appropriate
10. Test Quality: Validate audio quality before production use

Performance Tips

Rate Limiting

import "golang.org/x/time/rate"

// Create rate limiter (e.g., 10 requests per minute)
limiter := rate.NewLimiter(rate.Every(6*time.Second), 1)

func generateSpeechWithRateLimit(ctx context.Context, model provider.SpeechModel, text, voice string) (*ai.GenerateSpeechResult, error) {
    // Wait for rate limiter
    if err := limiter.Wait(ctx); err != nil {
        return nil, err
    }

    return ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model: model,
        Text:  text,
        Voice: voice,
    })
}

Character Count Validation

func validateTextLength(text string, maxChars int) error {
    if len(text) > maxChars {
        return fmt.Errorf("text too long: %d characters (max: %d)", len(text), maxChars)
    }
    return nil
}

// Before generating speech
if err := validateTextLength(text, 4096); err != nil {
    log.Fatal(err)
}

Speech Translation (Experimental)

Speech-to-speech translation streams source-language audio in and receives translated audio (and text) out, over a WebSocket connection. It is separate from text-to-speech (GenerateSpeech) and speech-to-text (Transcribe).

Google (SpeechTranslationModel, Live Translation over the BidiGenerateContent WebSocket) and OpenAI (SpeechTranslationModel, the /realtime/translations WebSocket) implement provider.SpeechTranslationModel, accessed through Provider.SpeechTranslationModel(modelID) or the TS-compatible alias Provider.Translation(modelID):

prov := google.New(google.Config{APIKey: os.Getenv("GOOGLE_GENERATIVE_AI_API_KEY")})
translationModel, err := prov.SpeechTranslationModel("gemini-2.0-flash-live-001")
if err != nil {
    log.Fatal(err)
}

result, err := ai.ExperimentalStreamTranslate(ctx, ai.StreamTranslateOptions{
    Model:            translationModel,
    Audio:            audioStream, // provider.AudioStream of raw input chunks
    InputAudioFormat: provider.AudioFormat{Type: "audio/pcm", Rate: intPtr(16000)},
    TargetLanguage:   "es",
})
if err != nil {
    log.Fatal(err)
}
defer result.Close()

This feature is experimental and its API may change. See the package docs under pkg/providers/google and pkg/providers/openai for the current provider.AudioStream and streamed-event shapes.

Next Steps

• Learn about Transcription
• Explore Settings
• See Provider Documentation

See Also

• Providers
• Settings
• Error Handling
• API Reference: GenerateSpeech