Skip to main content
View as MarkdownOpen in ChatGPTOpen in Claude

Baseten Provider

Baseten hosts an OpenAI-compatible Model APIs endpoint alongside custom model deployments. pkg/providers/baseten builds on the shared openai package (like the TypeScript provider) — there is no deployment orchestration API in this SDK; use the Baseten dashboard or CLI to deploy models, and this provider to call them.

Setup​

Installation​

import (
    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/baseten"
)

Configuration​

provider := baseten.New(baseten.Config{
    APIKey: os.Getenv("BASETEN_API_KEY"),
})

model, err := provider.LanguageModel("") // "" defaults to "chat" against the Model APIs

Get API Key​

export BASETEN_API_KEY=...

Config.APIKey falls back to BASETEN_API_KEY when left empty.

Config​

type Config struct {
    APIKey     string            // falls back to BASETEN_API_KEY
    BaseURL    string            // default: https://inference.baseten.co/v1
    ModelURL   string            // custom deployment URL (see below)
    Headers    map[string]string
    HTTPClient *http.Client
}

Model APIs vs. Custom Deployments​

  • Model APIs (default): leave ModelURL empty. provider.LanguageModel(modelID) calls Baseten's shared OpenAI-compatible Model APIs at https://inference.baseten.co/v1 with ChatProviderName: "baseten.chat". An empty modelID defaults to "chat".
  • Custom deployment: set Config.ModelURL to your deployment's /sync/v1 endpoint. An empty modelID then defaults to "placeholder" instead of "chat", since dedicated single-model endpoints ignore the wire-level model field. A /predict URL is rejected — chat requires a /sync/v1 (OpenAI-compatible Chat Completions) endpoint.
provider := baseten.New(baseten.Config{
    APIKey:   os.Getenv("BASETEN_API_KEY"),
    ModelURL: "https://model-xxxxxxx.api.baseten.co/environments/production/sync/v1",
})

model, err := provider.LanguageModel("")
if err != nil {
    log.Fatal(err)
}

Breaking change (this cycle): the default base URL changed from the old bridge.baseten.co host to https://inference.baseten.co/v1, and the chat model's provider name is now "baseten.chat". With a custom /sync/v1 ModelURL, an empty model ID now defaults to "placeholder" (was "chat").

Embeddings​

Baseten has no default embeddings endpoint on the Model APIs. Config.ModelURL must point at a /sync or /sync/v1 deployment URL, or EmbeddingModel returns an error:

provider := baseten.New(baseten.Config{
    APIKey:   os.Getenv("BASETEN_API_KEY"),
    ModelURL: "https://model-xxxxxxx.api.baseten.co/environments/production/sync",
})

embeddingModel, err := provider.EmbeddingModel("")
if err != nil {
    log.Fatal(err)
}

Baseten embeddings batch at most 128 inputs per call.

Video Parts​

Chat requests sent through this provider set AllowVideo: true internally, so video/* file parts are sent as video_url content parts — see OpenAI-compatible providers: Video Parts.

Examples​

Basic Text Generation​

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/digitallysavvy/go-ai/pkg/ai"
    "github.com/digitallysavvy/go-ai/pkg/providers/baseten"
)

func main() {
    provider := baseten.New(baseten.Config{
        APIKey: os.Getenv("BASETEN_API_KEY"),
    })

    model, err := provider.LanguageModel("")
    if err != nil {
        log.Fatal(err)
    }

    result, err := ai.GenerateText(context.Background(), ai.GenerateTextOptions{Model: model, Prompt: "Explain Baseten"})
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(result.Text)
}

Workflow Serialization​

Baseten embedding models can cross a workflow boundary with provider.SerializeEmbeddingModel / DeserializeEmbeddingModel (language models could already be serialized with providerutils.SerializeModel / DeserializeModel). See Provider Serialization for the mechanism.

See Also​