# Speech Generation

The Go AI SDK provides the [`ai.GenerateSpeech()`](https://goaisdk.com/docs/reference/ai/generate-speech.md) function to generate speech from text using a speech model.

```go
package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/openai"
)

func main() {
    ctx := context.Background()

    provider := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
    speechModel, _ := provider.SpeechModel("tts-1")

    result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model: speechModel,
        Text:  "Hello, world!",
        Voice: "alloy",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Save audio to file
    err = os.WriteFile("output.mp3", result.Audio.Data, 0644)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Audio saved to output.mp3")
}
```

## Accessing Audio Data

To access the generated audio:

```go
speed := 1.25

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})

// Audio bytes
audioData := result.Audio.Data

// MIME type (e.g., "audio/mpeg")
mimeType := result.Audio.MediaType

fmt.Printf("Generated %d bytes as %s\n", len(audioData), mimeType)
```

## Settings

### Voice Selection

Different models support different voices. Check your provider's documentation for available voices:

```go
// OpenAI voices: alloy, echo, fable, onyx, nova, shimmer
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "nova",
})
```

### Speech Speed

Control the speed of speech generation (typically 0.25 to 4.0):

```go
speed := 1.5 // 1.5x speed

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
    Speed: &speed,
})
```

### Language Setting

You can specify the language for speech generation (provider support varies):

```go
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model:    speechModel,
    Text:     "Hola, mundo!",
    Voice:    "alloy",
    Language: "es", // Spanish
})
```

### Audio Format

Some providers allow you to specify the output audio format:

```go
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model:  speechModel,
    Text:   "Hello, world!",
    Voice:  "alloy",
    OutputFormat: "mp3", // or "wav", "opus", "flac", etc.
})
```

### Provider-Specific Settings

You can set model-specific settings with the `ProviderOptions` parameter:

```go
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model:        speechModel,
    Text:         "Hello, world!",
    Voice:        "alloy",
    OutputFormat: "opus",
    Speed:        &speed,
})
```

### Timeouts with Context

Use Go's context for timeouts and cancellation:

```go
import "time"

// Timeout after 30 seconds
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})
if err != nil {
    if ctx.Err() == context.DeadlineExceeded {
        fmt.Println("Speech generation timed out")
    }
    log.Fatal(err)
}
```

### Custom Headers

Add custom headers to the speech generation request:

```go
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
    Headers: map[string]string{
        "X-Custom-Header": "custom-value",
    },
})
```

### Warnings

If the model returns warnings, they will be available in the `Warnings` field:

```go
result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})
if err != nil {
    log.Fatal(err)
}

for _, warning := range result.Warnings {
    fmt.Printf("Warning: %s\n", warning.Message)
}
```

## Error Handling

When `ai.GenerateSpeech()` cannot generate valid audio, it returns an error:

```go
import "errors"

result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: speechModel,
    Text:  "Hello, world!",
    Voice: "alloy",
})
if err != nil {
    if ai.IsNoSpeechGeneratedError(err) {
        fmt.Println("No speech was generated")
        var noSpeech *ai.NoSpeechGeneratedError
        if errors.As(err, &noSpeech) {
            fmt.Printf("Responses: %d\n", len(noSpeech.Responses))
        }
    } else {
        fmt.Printf("Speech generation error: %v\n", err)
    }
    return
}
```

Common speech generation errors:

- **NoSpeechGeneratedError**: The model returned no generated audio
- **ProviderError**: Provider-specific HTTP or API failure
- **InvalidArgumentError**: Invalid input such as negative `MaxRetries`
- **context.DeadlineExceeded**: Operation timed out
- **context.Canceled**: Operation was canceled

## Practical Examples

### Generate and Save Audio

```go
func generateAndSaveAudio(ctx context.Context, model provider.SpeechModel, text, voice, outputPath string) error {
    result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model: model,
        Text:  text,
        Voice: voice,
    })
    if err != nil {
        return fmt.Errorf("speech generation failed: %w", err)
    }

    err = os.WriteFile(outputPath, result.Audio.Data, 0644)
    if err != nil {
        return fmt.Errorf("failed to save audio: %w", err)
    }

    fmt.Printf("Audio saved to %s (%d bytes)\n", outputPath, len(result.Audio.Data))
    return nil
}
```

### Generate Multiple Voices

```go
func generateWithMultipleVoices(ctx context.Context, model provider.SpeechModel, text string, voices []string) error {
    for i, voice := range voices {
        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  text,
            Voice: voice,
        })
        if err != nil {
            return fmt.Errorf("failed to generate with voice %s: %w", voice, err)
        }

        filename := fmt.Sprintf("output-%s.mp3", voice)
        err = os.WriteFile(filename, result.Audio.Data, 0644)
        if err != nil {
            return fmt.Errorf("failed to save %s: %w", filename, err)
        }

        fmt.Printf("%d. Generated %s\n", i+1, filename)
    }

    return nil
}

// Usage
voices := []string{"alloy", "echo", "fable", "onyx", "nova", "shimmer"}
generateWithMultipleVoices(ctx, speechModel, "Hello, world!", voices)
```

### Text-to-Speech with Retry Logic

```go
func generateSpeechWithRetry(ctx context.Context, model provider.SpeechModel, text, voice string, maxRetries int) (*ai.GenerateSpeechResult, error) {
    return ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model:      model,
        Text:       text,
        Voice:      voice,
        MaxRetries: &maxRetries,
    })
}
```

### Batch Text-to-Speech

```go
func batchGenerateSpeech(ctx context.Context, model provider.SpeechModel, texts []string, voice string) ([][]byte, error) {
    audioFiles := make([][]byte, len(texts))

    for i, text := range texts {
        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  text,
            Voice: voice,
        })
        if err != nil {
            return nil, fmt.Errorf("failed to generate audio for text %d: %w", i, err)
        }

        audioFiles[i] = result.Audio.Data
        fmt.Printf("Generated audio %d/%d (%d bytes)\n", i+1, len(texts), len(result.Audio.Data))
    }

    return audioFiles, nil
}
```

### Parallel Speech Generation with Goroutines

```go
func generateSpeechParallel(ctx context.Context, model provider.SpeechModel, texts []string, voice string) ([][]byte, error) {
    type result struct {
        index int
        audio []byte
        err   error
    }

    resultChan := make(chan result, len(texts))
    audioFiles := make([][]byte, len(texts))

    // Start goroutines
    for i, text := range texts {
        go func(index int, textContent string) {
            speech, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
                Model: model,
                Text:  textContent,
                Voice: voice,
            })
            if err != nil {
                resultChan <- result{index: index, err: err}
                return
            }

            resultChan <- result{index: index, audio: speech.Audio.Data}
        }(i, text)
    }

    // Collect results
    for i := 0; i < len(texts); i++ {
        res := <-resultChan
        if res.err != nil {
            return nil, fmt.Errorf("speech generation failed for text %d: %w", res.index, res.err)
        }
        audioFiles[res.index] = res.audio
    }

    return audioFiles, nil
}
```

### Generate Speech with Different Speeds

```go
func generateAtDifferentSpeeds(ctx context.Context, model provider.SpeechModel, text, voice string) error {
    speeds := []float64{0.5, 0.75, 1.0, 1.25, 1.5, 2.0}

    for _, speed := range speeds {
        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  text,
            Voice: voice,
            Speed: &speed,
        })
        if err != nil {
            return fmt.Errorf("failed to generate at speed %.2fx: %w", speed, err)
        }

        filename := fmt.Sprintf("output-%.2fx.mp3", speed)
        err = os.WriteFile(filename, result.Audio.Data, 0644)
        if err != nil {
            return fmt.Errorf("failed to save %s: %w", filename, err)
        }

        fmt.Printf("Generated %s at %.2fx speed\n", filename, speed)
    }

    return nil
}
```

### Streaming to HTTP Response

```go
import "net/http"

func speechHandler(w http.ResponseWriter, r *http.Request) {
    text := r.URL.Query().Get("text")
    voice := r.URL.Query().Get("voice")

    if text == "" || voice == "" {
        http.Error(w, "Missing text or voice parameter", http.StatusBadRequest)
        return
    }

    provider := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
    speechModel, _ := provider.SpeechModel("tts-1")

    result, err := ai.GenerateSpeech(r.Context(), ai.GenerateSpeechOptions{
        Model: speechModel,
        Text:  text,
        Voice: voice,
    })
    if err != nil {
        http.Error(w, fmt.Sprintf("Speech generation failed: %v", err), http.StatusInternalServerError)
        return
    }

    // Set appropriate headers
    w.Header().Set("Content-Type", result.Audio.MediaType)
    w.Header().Set("Content-Length", fmt.Sprintf("%d", len(result.Audio.Data)))
    w.Header().Set("Content-Disposition", "inline; filename=\"speech.mp3\"")

    // Write audio data
    w.Write(result.Audio.Data)
}
```

### Long Text with Chunking

```go
import "strings"

func generateLongText(ctx context.Context, model provider.SpeechModel, longText, voice string, maxChunkSize int) ([]byte, error) {
    // Split text into chunks
    sentences := strings.Split(longText, ". ")
    var chunks []string
    var currentChunk strings.Builder

    for _, sentence := range sentences {
        if currentChunk.Len()+len(sentence) > maxChunkSize {
            chunks = append(chunks, currentChunk.String())
            currentChunk.Reset()
        }
        currentChunk.WriteString(sentence)
        currentChunk.WriteString(". ")
    }

    if currentChunk.Len() > 0 {
        chunks = append(chunks, currentChunk.String())
    }

    // Generate audio for each chunk
    var allAudio []byte
    for i, chunk := range chunks {
        fmt.Printf("Generating chunk %d/%d...\n", i+1, len(chunks))

        result, err := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
            Model: model,
            Text:  chunk,
            Voice: voice,
        })
        if err != nil {
            return nil, fmt.Errorf("failed to generate chunk %d: %w", i+1, err)
        }

        allAudio = append(allAudio, result.Audio.Data...)
    }

    return allAudio, nil
}
```

## Speech Models

Several providers offer speech synthesis models:

| Provider | Model | Voices | Languages |
|----------|-------|--------|-----------|
| **OpenAI** | `tts-1` | alloy, echo, fable, onyx, nova, shimmer | Multiple |
| **OpenAI** | `tts-1-hd` | alloy, echo, fable, onyx, nova, shimmer | Multiple |
| **OpenAI** | `gpt-4o-mini-tts` | alloy, echo, fable, onyx, nova, shimmer | Multiple |
| **ElevenLabs** | `eleven_v3` | Custom voices | 30+ languages |
| **ElevenLabs** | `eleven_multilingual_v2` | Custom voices | Multilingual |
| **ElevenLabs** | `eleven_flash_v2_5` | Custom voices | Fast, low latency |
| **ElevenLabs** | `eleven_flash_v2` | Custom voices | Fast |
| **ElevenLabs** | `eleven_turbo_v2_5` | Custom voices | Ultra-fast |
| **ElevenLabs** | `eleven_turbo_v2` | Custom voices | Ultra-fast |
| **LMNT** | `aurora` | Various | English |
| **LMNT** | `blizzard` | Various | English |
| **Hume** | `default` | Expressive | English |

### Example with Different Providers

```go
// OpenAI
openaiProvider := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
openaiModel, _ := openaiProvider.SpeechModel("tts-1-hd")

// ElevenLabs
elevenProvider := elevenlabs.New(elevenlabs.Config{APIKey: os.Getenv("ELEVENLABS_API_KEY")})
elevenModel, _ := elevenProvider.SpeechModel("eleven_multilingual_v2")

// LMNT
lmntProvider := lmnt.New(lmnt.Config{APIKey: os.Getenv("LMNT_API_KEY")})
lmntModel, _ := lmntProvider.SpeechModel("aurora")

// Use any model with the same API
result, _ := ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
    Model: openaiModel, // or elevenModel, or lmntModel
    Text:  "Hello, world!",
    Voice: "alloy", // voice name depends on provider
})
```

## Supported Audio Formats

Common audio formats supported by providers:

- **MP3** (audio/mpeg) - Most common, good compression
- **WAV** (audio/wav) - Uncompressed, high quality
- **Opus** (audio/opus) - Efficient for speech, low latency
- **AAC** (audio/aac) - Good quality, streaming-friendly
- **FLAC** (audio/flac) - Lossless compression
- **PCM** (audio/pcm) - Raw audio data

Format availability varies by provider. Check provider documentation for specific support.

## Best Practices

1. **Choose the Right Model**: Balance quality, speed, and cost
2. **Set Timeouts**: Speech generation can be slow; always use context with timeout
3. **Handle Errors**: Implement proper error handling and retry logic
4. **Select Appropriate Voice**: Test different voices for your use case
5. **Monitor Costs**: Track character usage and generation costs
6. **Respect Rate Limits**: Implement rate limiting for bulk generation
7. **Use Parallel Processing**: For multiple texts, use goroutines
8. **Chunk Long Texts**: Split long texts into manageable chunks
9. **Cache Results**: Store generated audio when appropriate
10. **Test Quality**: Validate audio quality before production use

## Performance Tips

### Rate Limiting

```go
import "golang.org/x/time/rate"

// Create rate limiter (e.g., 10 requests per minute)
limiter := rate.NewLimiter(rate.Every(6*time.Second), 1)

func generateSpeechWithRateLimit(ctx context.Context, model provider.SpeechModel, text, voice string) (*ai.GenerateSpeechResult, error) {
    // Wait for rate limiter
    if err := limiter.Wait(ctx); err != nil {
        return nil, err
    }

    return ai.GenerateSpeech(ctx, ai.GenerateSpeechOptions{
        Model: model,
        Text:  text,
        Voice: voice,
    })
}
```

### Character Count Validation

```go
func validateTextLength(text string, maxChars int) error {
    if len(text) > maxChars {
        return fmt.Errorf("text too long: %d characters (max: %d)", len(text), maxChars)
    }
    return nil
}

// Before generating speech
if err := validateTextLength(text, 4096); err != nil {
    log.Fatal(err)
}
```

## Speech Translation (Experimental)

Speech-to-speech translation streams source-language audio in and receives
translated audio (and text) out, over a WebSocket connection. It is separate
from text-to-speech (`GenerateSpeech`) and speech-to-text (`Transcribe`).

Google (`SpeechTranslationModel`, Live Translation over the
BidiGenerateContent WebSocket) and OpenAI (`SpeechTranslationModel`, the
`/realtime/translations` WebSocket) implement `provider.SpeechTranslationModel`,
accessed through `Provider.SpeechTranslationModel(modelID)` or the
TS-compatible alias `Provider.Translation(modelID)`:

```go
prov := google.New(google.Config{APIKey: os.Getenv("GOOGLE_GENERATIVE_AI_API_KEY")})
translationModel, err := prov.SpeechTranslationModel("gemini-2.0-flash-live-001")
if err != nil {
    log.Fatal(err)
}

result, err := ai.ExperimentalStreamTranslate(ctx, ai.StreamTranslateOptions{
    Model:            translationModel,
    Audio:            audioStream, // provider.AudioStream of raw input chunks
    InputAudioFormat: provider.AudioFormat{Type: "audio/pcm", Rate: intPtr(16000)},
    TargetLanguage:   "es",
})
if err != nil {
    log.Fatal(err)
}
defer result.Close()
```

This feature is experimental and its API may change. See the package docs
under `pkg/providers/google` and `pkg/providers/openai` for the current
`provider.AudioStream` and streamed-event shapes.

## Next Steps

- Learn about [Transcription](https://goaisdk.com/docs/ai-sdk-core/transcription.md)
- Explore [Settings](https://goaisdk.com/docs/ai-sdk-core/settings.md)
- See [Provider Documentation](https://goaisdk.com/docs/providers/overview.md)

## See Also

- [Providers](https://goaisdk.com/docs/providers/overview.md)
- [Settings](https://goaisdk.com/docs/ai-sdk-core/settings.md)
- [Error Handling](https://goaisdk.com/docs/ai-sdk-core/error-handling.md)
- [API Reference: GenerateSpeech](https://goaisdk.com/docs/reference/ai/generate-speech.md)