Skip to main content
View as MarkdownOpen in ChatGPTOpen in Claude

Built-in Middleware

Pre-built middleware functions available in the Go AI SDK's pkg/middleware package. All of them return *middleware.LanguageModelMiddleware for use with WrapLanguageModel or WrapProvider.

DefaultSettingsMiddleware​

Applies default settings to generation requests when the caller didn't set them.

func DefaultSettingsMiddleware(settings *provider.GenerateOptions) *LanguageModelMiddleware

Example​

temperature := 0.7
maxTokens := 500

defaultsMw := middleware.DefaultSettingsMiddleware(&provider.GenerateOptions{
    Temperature: &temperature,
    MaxTokens:   &maxTokens,
})

wrapped := middleware.WrapLanguageModel(baseModel, []*middleware.LanguageModelMiddleware{defaultsMw}, nil, nil)

DefaultInstructionsMiddleware​

Applies default system instructions when the call doesn't provide any.

func DefaultInstructionsMiddleware(opts DefaultInstructionsOptions) *LanguageModelMiddleware

type DefaultInstructionsOptions struct {
    // Instructions are plain-string default instructions, applied as
    // Prompt.System when the call has none.
    Instructions string

    // InstructionMessages supplies the default instructions as system
    // messages instead, preserving each message's ProviderOptions. Takes
    // precedence over Instructions when non-empty.
    InstructionMessages []types.Message
}

Example​

instructionsMw := middleware.DefaultInstructionsMiddleware(middleware.DefaultInstructionsOptions{
    Instructions: "You are a concise, helpful assistant.",
})

wrapped := middleware.WrapLanguageModel(baseModel, []*middleware.LanguageModelMiddleware{instructionsMw}, nil, nil)

ExtractJSONMiddleware​

Extracts JSON from model output, stripping markdown code fences by default.

func ExtractJSONMiddleware(options *ExtractJSONOptions) *LanguageModelMiddleware

type ExtractJSONOptions struct {
    // Transform is a custom function to extract JSON from text.
    // If nil, the default transform strips markdown code fences.
    Transform func(text string) string
}

Example​

jsonMw := middleware.ExtractJSONMiddleware(nil) // use the default fence-stripping transform
wrapped := middleware.WrapLanguageModel(baseModel, []*middleware.LanguageModelMiddleware{jsonMw}, nil, nil)

ExtractReasoningMiddleware​

Extracts reasoning content wrapped in an XML-style tag (for example <think>) out of the model's text output.

func ExtractReasoningMiddleware(options *ExtractReasoningOptions) *LanguageModelMiddleware

type ExtractReasoningOptions struct {
    // TagName is the XML tag name to extract reasoning from
    // (e.g., "think" for Anthropic, "reasoning" for OpenAI)
    TagName string

    // Separator is the separator to use between reasoning and text sections
    // Default: "\n"
    Separator string

    // StartWithReasoning indicates whether reasoning tokens appear at the beginning
    // Default: false
    StartWithReasoning bool
}

Example​

reasoningMw := middleware.ExtractReasoningMiddleware(&middleware.ExtractReasoningOptions{
    TagName: "think",
})
wrapped := middleware.WrapLanguageModel(baseModel, []*middleware.LanguageModelMiddleware{reasoningMw}, nil, nil)

SimulateStreamingMiddleware​

Simulates a streaming response from a model that only supports non-streaming generation, by generating the full result and then replaying it as stream chunks.

func SimulateStreamingMiddleware() *LanguageModelMiddleware

Example​

simulateMw := middleware.SimulateStreamingMiddleware()
wrapped := middleware.WrapLanguageModel(baseModel, []*middleware.LanguageModelMiddleware{simulateMw}, nil, nil)

AddToolInputExamplesMiddleware​

Appends each tool's InputExamples to its description so the model sees example inputs, then (by default) removes InputExamples from the tool definition sent to the provider.

func AddToolInputExamplesMiddleware(options *AddToolInputExamplesOptions) *LanguageModelMiddleware

type AddToolInputExamplesOptions struct {
    // Prefix is the text to prepend before examples
    // Default: "Input Examples:"
    Prefix string

    // Format is a custom formatter for each example
    // If nil, uses JSON.stringify on the example input
    Format func(example types.ToolInputExample, index int) string

    // Remove indicates whether to remove the InputExamples property
    // after adding them to the description. Default: true.
    Remove *bool
}

Example​

examplesMw := middleware.AddToolInputExamplesMiddleware(&middleware.AddToolInputExamplesOptions{
    Prefix: "Examples:",
})
wrapped := middleware.WrapLanguageModel(baseModel, []*middleware.LanguageModelMiddleware{examplesMw}, nil, nil)

Combining Middleware​

temperature := 0.7

wrapped := middleware.WrapLanguageModel(
    baseModel,
    []*middleware.LanguageModelMiddleware{
        middleware.DefaultSettingsMiddleware(&provider.GenerateOptions{
            Temperature: &temperature,
        }),
        middleware.ExtractReasoningMiddleware(&middleware.ExtractReasoningOptions{
            TagName: "think",
        }),
        middleware.ExtractJSONMiddleware(nil),
    },
    nil,
    nil,
)

Image and embedding model wrappers​

There are no built-in *ImageModelMiddleware or *EmbeddingModelMiddleware constructors analogous to the ones above, but the same wrapping mechanism is available for image and embedding models if you write your own middleware struct (see Middleware Interface):

func WrapImageModel(model provider.ImageModel, middleware []*ImageModelMiddleware, modelID, providerID *string) provider.ImageModel
func WrapEmbeddingModel(model provider.EmbeddingModel, middleware []*EmbeddingModelMiddleware, modelID, providerID *string) provider.EmbeddingModel

WrapProvider​

Applies language model and embedding model middleware to every model returned by a provider, instead of wrapping one model at a time.

func WrapProvider(
    p provider.Provider,
    languageModelMiddleware []*LanguageModelMiddleware,
    embeddingModelMiddleware []*EmbeddingModelMiddleware,
    opts ...ProviderMiddlewareOption,
) provider.Provider

Pass middleware.WithImageModelMiddleware([]*middleware.ImageModelMiddleware{...}) as a trailing option to also apply image model middleware.

Example​

temperature := 0.7

p := openai.New(openai.Config{APIKey: "your-api-key"})

wrappedProvider := middleware.WrapProvider(
    p,
    []*middleware.LanguageModelMiddleware{
        middleware.DefaultSettingsMiddleware(&provider.GenerateOptions{
            Temperature: &temperature,
        }),
    },
    nil,
)

model, _ := wrappedProvider.LanguageModel("gpt-4")

See Also​