Download API Reference

Overview

The Download API provides secure file downloading with built-in size limits to prevent memory exhaustion attacks.

Default Size Limit

The default maximum download size is 2 GiB (2,147,483,648 bytes), used by ai.CreateDownload(nil), ai.CreateURLDownload(nil), and ai.CreateURLDownloadWithMetadata(nil) when options is nil or MaxBytes is 0.

This limit prevents memory exhaustion from unbounded downloads while allowing legitimate large files. All download operations use this limit by default. There is no exported constant for this value in pkg/ai; pass an explicit MaxBytes on ai.DownloadOptions to override it.

Types

URLDownloadFunction

type URLDownloadFunction func(ctx context.Context, url string) ([]byte, error)

A single-URL download function that returns downloaded bytes.

Parameters:

• ctx: Context for cancellation and timeout
• url: URL to download from

Returns:

• []byte: Downloaded data
• error: Error if download fails or exceeds size limit

URLDownloadWithMetadataFunction

type URLDownloadWithMetadataFunction func(ctx context.Context, url string) (*DownloadResult, error)

A single-URL download function that returns downloaded bytes plus an optional media type. GenerateVideoOptions.DownloadWithMetadata uses this shape to match TypeScript generateVideo custom downloads.

DownloadFunction

type DownloadFunction func(ctx context.Context, requests []DownloadRequest) ([]*DownloadResult, error)

A batch download function for prompt file URL normalization. A nil result leaves the original URL in place when the model can consume it directly.

DownloadResult

type DownloadResult struct {
    Data      []byte
    MediaType string
}

Downloaded bytes plus an optional media type.

DownloadOptions

type DownloadOptions struct {
    // MaxBytes is the maximum allowed download size in bytes.
    // Default: 2 GiB (DefaultMaxDownloadSize)
    MaxBytes int64

    // Headers are additional HTTP headers to include in download requests.
    Headers map[string]string
}

Configuration options for creating custom download functions.

Fields:

• MaxBytes: Maximum file size in bytes (default: 2 GiB)
• Headers: Custom HTTP headers to send with requests

DownloadError

type DownloadError struct {
    // URL that was being downloaded
    URL string

    // HTTP status code (if applicable)
    StatusCode int

    // HTTP status text
    StatusText string

    // Error message
    Message string

    // Underlying cause
    Cause error
}

Error type returned when downloads fail due to size limits or HTTP errors.

Methods:

Error

func (e *DownloadError) Error() string

Returns a formatted error message.

Unwrap

func (e *DownloadError) Unwrap() error

Returns the underlying cause of the error.

Functions

CreateDownload

func CreateDownload(options *DownloadOptions) DownloadFunction

Creates a download function with configurable options.

The returned function enforces size limits to prevent memory exhaustion. Pass nil for default options (2 GiB limit).

Parameters:

• options: Download configuration (nil for defaults)

Returns:

• DownloadFunction: Configured download function

CreateURLDownload

func CreateURLDownload(options *DownloadOptions) URLDownloadFunction

Creates a bytes-only single-URL download function.

CreateURLDownloadWithMetadata

func CreateURLDownloadWithMetadata(options *DownloadOptions) URLDownloadWithMetadataFunction

Creates a single-URL download function that preserves response media type metadata for APIs such as GenerateVideo.

Example:

// Create a batch prompt download function with default 2 GiB limit
defaultDownload := ai.CreateDownload(nil)

// Create a single-URL download function with custom 100 MB limit
customDownload := ai.CreateURLDownload(&ai.DownloadOptions{
    MaxBytes: 100 * 1024 * 1024,
})

// Create a single-URL metadata download function with custom headers
authDownload := ai.CreateURLDownloadWithMetadata(&ai.DownloadOptions{
    MaxBytes: 500 * 1024 * 1024,
    Headers: map[string]string{
        "Authorization": "Bearer token",
    },
})

DefaultDownload

var DefaultDownload = CreateDownload(nil)

The default batch prompt download function with 2 GiB limit.

Use CreateURLDownload(nil) when you need a single-URL downloader:

download := ai.CreateURLDownload(nil)
data, err := download(ctx, url)

Internal Implementation

The actual HTTP download logic (Content-Length checking, incremental reads with an abort once the limit is exceeded, timeouts) lives in pkg/internal/fileutil. That package is under an internal/ path, so it is not importable from outside this module — attempting to import github.com/digitallysavvy/go-ai/pkg/internal/fileutil from your own code fails to build with use of internal package ... not allowed. Use the public ai.CreateDownload, ai.CreateURLDownload, and ai.CreateURLDownloadWithMetadata constructors above instead; they wrap fileutil's behavior (Content-Length pre-check, incremental read limit, *providererrors.DownloadError on failure) behind the public API.

Error Handling

Checking for DownloadError

import (
    "errors"
    providererrors "github.com/digitallysavvy/go-ai/pkg/provider/errors"
)

download := ai.CreateURLDownload(nil)
data, err := download(ctx, url)
if err != nil {
    var downloadErr *providererrors.DownloadError
    if errors.As(err, &downloadErr) {
        // Handle download-specific error
        fmt.Printf("Failed to download %s\n", downloadErr.URL)

        if downloadErr.StatusCode > 0 {
            fmt.Printf("HTTP %d: %s\n",
                downloadErr.StatusCode,
                downloadErr.StatusText)
        }

        if downloadErr.Message != "" {
            fmt.Printf("Details: %s\n", downloadErr.Message)
        }
    }
}

IsDownloadError

func IsDownloadError(err error) bool

Helper function to check if an error is a DownloadError:

if providererrors.IsDownloadError(err) {
    // Handle download error
}

Common Error Messages

Error Message	Cause	Solution
download of {url} exceeded maximum size of {limit} bytes (Content-Length: {size})	File size in Content-Length header exceeds limit	Increase limit or validate file size before downloading
download of {url} exceeded maximum size of {limit} bytes	File exceeded limit during streaming download	Increase limit or use chunked processing
failed to download {url}: {status} {text}	HTTP error response	Check URL validity and accessibility
failed to download {url}: {cause}	Network or other error	Check network connectivity and URL

Usage in Generation Functions

Video Generation

customDownload := ai.CreateURLDownloadWithMetadata(&ai.DownloadOptions{
    MaxBytes: 200 * 1024 * 1024, // 200 MB
})

result, err := ai.GenerateVideo(ctx, ai.GenerateVideoOptions{
    Model: model,
    Prompt: ai.VideoPrompt{
        Text: "create video from this image",
        Image: &ai.VideoPromptImage{
            URL: "https://example.com/input.jpg",
        },
    },
    DownloadWithMetadata: customDownload,
})

Image Generation

Image downloads happen automatically within providers and use the default 2 GiB limit. Provider implementations use fileutil.Download internally.

Size Limit Guidelines

Use Case	Recommended Limit	Rationale
Thumbnails	10 MB	Small images
Standard images	100 MB	Most images
High-res images	500 MB	Professional photography
Short videos	500 MB	< 1 minute
Long videos	2 GiB (default)	Several minutes

Security Best Practices

1. Always use size limits - Never disable or set to unlimited
2. Validate URLs - Check domain, scheme, and format
3. Set timeouts - Use context with timeout for all downloads
4. Rate limit - Prevent abuse in public-facing applications
5. Log failures - Monitor for attack attempts

Performance Considerations

Memory Usage

Downloads are read into memory entirely. For large files:

• Peak memory = file size + overhead
• Consider streaming approaches for very large files
• Monitor memory usage in production

Timeout Guidelines

// Quick images (< 10 MB)
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)

// Standard images/videos (< 500 MB)
ctx, cancel := context.WithTimeout(ctx, 60*time.Second)

// Large files (> 500 MB)
ctx, cancel := context.WithTimeout(ctx, 5*time.Minute)

See Also

• Download Security Guide
• Security Advisory
• Error Handling