Skip to main content
View as MarkdownOpen in ChatGPTOpen in Claude

Together AI Provider

Together AI provides fast inference for open-source models including Llama, Mixtral, Qwen, and more. Offers competitive pricing and high-quality model hosting.

Setup​

Installation​

Together has its own dedicated package — it is not built on top of the generic openai package:

import (
    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/together"
)

Configuration​

provider := together.New(together.Config{
    APIKey: os.Getenv("TOGETHER_API_KEY"),
})

model, err := provider.LanguageModel("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo")
if err != nil {
    log.Fatal(err)
}

together.Config also accepts BaseURL (default https://api.together.xyz) and Headers.

Get API Key​

export TOGETHER_API_KEY=...

TOGETHER_AI_API_KEY is accepted as a deprecated fallback and logs a deprecation warning; prefer TOGETHER_API_KEY.

Available Models​

Language Models​

Model IDContextInput PriceOutput PriceBest For
meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo131K$0.88/1M$0.88/1MGeneral purpose
meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo131K$0.18/1M$0.18/1MFast, cheap
mistralai/Mixtral-8x7B-Instruct-v0.132K$0.60/1M$0.60/1MBalanced
Qwen/Qwen2.5-72B-Instruct-Turbo32K$0.88/1M$0.88/1MMultilingual

Image Generation​

Model IDQualityPriceBest For
stabilityai/stable-diffusion-xl-base-1.0High$0.025/imageImages
black-forest-labs/FLUX.1-schnellHigh$0.025/imageFast generation

Embeddings​

embeddingModel, err := provider.EmbeddingModel("togethercomputer/m2-bert-80M-8k-retrieval")

Reranking​

provider.RerankingModel(modelID) calls POST /rerank:

reranker, err := provider.RerankingModel("Salesforce/Llama-Rank-v1")
if err != nil {
    log.Fatal(err)
}

topN := 3
result, err := reranker.DoRerank(ctx, &provider.RerankOptions{
    Query:     "What is the capital of France?",
    Documents: []string{"Paris is the capital of France.", "Berlin is the capital of Germany."},
    TopN:      &topN,
    ProviderOptions: map[string]interface{}{
        "togetherai": map[string]interface{}{
            // RankFields selects which keys of a JSON-object document to
            // rank on; defaults to every supplied key.
            "rankFields": []string{"text"},
        },
    },
})

Provider Options​

Together-specific chat options go under the "togetherai" key (the TS provider options name); the Go package's own name, "together", is also accepted as a fallback:

result, err := ai.GenerateText(ctx, ai.GenerateTextOptions{
    Model:  model,
    Prompt: "Explain open-source AI",
    ProviderOptions: map[string]interface{}{
        "togetherai": map[string]interface{}{
            "topK": 40,
        },
    },
})

Together does not support speech synthesis or transcription — those model factories return an error.

Workflow Serialization​

Together image models can cross a workflow boundary with providerutils.SerializeModel / DeserializeModel (language models could already be serialized). See Provider Serialization for the mechanism; embedding and reranking models are not yet serializable.

Examples​

Basic Text Generation​

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/together"
)

func main() {
    ctx := context.Background()
    provider := together.New(together.Config{
        APIKey: os.Getenv("TOGETHER_API_KEY"),
    })

    model, err := provider.LanguageModel("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo")
    if err != nil {
        log.Fatal(err)
    }

    result, err := ai.GenerateText(ctx, ai.GenerateTextOptions{Model: model, Prompt: "Explain open-source AI"})
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(result.Text)
}

Best Practices​

  1. Model Selection

    • Use Llama 3.1 70B for best quality
    • Use Llama 3.1 8B for cost efficiency
    • Use Mixtral for balanced performance

  2. Cost Optimization

    • Open-source models are cost-effective
    • Monitor usage and optimize prompts

Rate Limits​

Varies by plan - check dashboard for limits.

See Also​