Streaming Events

Discover how to efficiently handle and process streaming events using Latitude’s API.

When executing a prompt in streaming mode, Latitude will return a stream of Server-Sent Events (SSE) that contain real-time updates from the AI provider. This guide explains how to handle and process streaming events using Latitude’s API.

Overview

There are two main types of events that you will receive when streaming events:

• latitude-event: Contains information about the chain progress and results.
• provider-event: Contains real-time updates from your AI provider.

Latitude Events

Latitude Events originate from Latitude’s AI engine and provide detailed updates on the processing chain, from initiation to completion. Every request is processed as a chain of steps, even if it consists of a single step.

General structure

All Latitude events follow this structure:

{
  "type": "latitude-event",
  "data": {
    "type": "event-type"
    "uuid": "conversation-uuid",
    "messages": [...],
    ... // Additional event-specific data
  }
}

Event Flow

Every chain execution follows the same flow:

1. Chain Starts: Every stream begins with a chain-started event.
2. Processing Steps: Multiple steps can be executed within a chain. All steps start with a step-started event and end with a step-completed event. Within a step, you may receive additional events:
   • Provider Interaction: The LLM processing includes provider-started and provider-completed events.
   • Tool Execution: If Latitude built-in tools are involved, tools-started and tool-completed events occur, indicating the execution status of the requested tool.
     Check out Latitude Tools for more information about built-in tools.

3. Chain Completion: The chain concludes with a chain-completed.

However, the chain execution can be interrupted by any of the following events:

• chain-error: Indicates an error occurred during the processing chain.
• tools-requested: Indicates the AI response requested additional tools to be executed by the client, and they are required to continue processing the chain.

Event Types

Here’s a complete list of all Latitude Event types and their attributes:

ChainStarted

The chain has started processing.

{
  "type": "latitude-event",
  "data": {
    "type": "chain-started",
    "uuid": "conversation-uuid"
    "messages": [...],
  }
}

StepStarted

A new step in the chain has started processing.

{
  "type": "latitude-event",
  "data": {
    "type": "step-started",
    "uuid": "conversation-uuid",
    "messages": [...],
  }
}

ProviderStarted

Your LLM Provider is being requested to generate a new response.

{
  "type": "latitude-event",
  "data": {
    "type": "provider-started",
    "uuid": "conversation-uuid",
    "messages": [...],
    "config": {
      "provider": "provider-name",
      "model": "model-name"
      ... // Rest of the prompt's step configuration
    }
  }
}

ProviderCompleted

Your LLM Provider has completed the response generation.

{
  "type": "latitude-event",
  "data": {
    "type": "provider-completed",
    "uuid": "conversation-uuid",
    "messages": [...], // The provider response is included at the end of the messages array
    "providerLogUuid": "provider-log-uuid", // Identifier of the specific provider log within Latitude
    "finishReason": 'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown',
    "tokenUsage": {
      "promptTokens": 0,
      "completionTokens": 0,
      "totalTokens": 0
    },
    "response": {
      "text": "response-text",
      "toolCalls": [...],
    }
  }
}

ToolsStarted

Latitude has started running built-in tools requested by the LLM response.

Check out Latitude Tools for more information about built-in tools.

{
  "type": "latitude-event",
  "data": {
    "type": "tools-started",
    "uuid": "conversation-uuid",
    "messages": [...],
    "tools": [
      {
        "id": "tool-id",
        "name": "tool-name",
        "arguments": {
          "argument-name": "argument-value"
        }
      }
    ]
  }
}

ToolCompleted

A built-in tool has completed its execution.

Check out Latitude Tools for more information about built-in tools.

{
  "type": "latitude-event",
  "data": {
    "type": "tool-completed",
    "uuid": "conversation-uuid",
    "messages": [...]
  }
}

ToolsRequested

The AI provider has requested additional tools to be executed, and they are required to continue processing the chain.

This event will terminate the SSE stream. To continue it, you will need to use the Chat API Endpoint to send the requested tools response, and continue the chain processing.

{
  "type": "latitude-event",
  "data": {
    "type": "tools-requested",
    "uuid": "conversation-uuid",
    "messages": [...],
    "tools": [
      {
        "id": "tool-id",
        "name": "tool-name",
        "arguments": {
          "argument-name": "argument-value"
        }
      }
    ]
  }
}

ChainError

An error has occurred during the processing of the chain.

This event will terminate the SSE stream.

{
  "type": "latitude-event",
  "data": {
    "type": "chain-error",
    "uuid": "conversation-uuid",
    "messages": [...],
    "error": {
      "message": "error-message",
      "code": "error-code"
    }
  }
}

ChainCompleted

The chain processing has completed successfully.

This event will terminate the SSE stream.

{
  "type": "latitude-event",
  "data": {
    "type": "chain-completed",
    "messages": [...],
    "uuid": "conversation-uuid",
    "tokenUsage": {
      "promptTokens": 0,
      "completionTokens": 0,
      "totalTokens": 0
    },
    "finishReason": 'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown',
  }
}

Provider Events

Provider Events are events that are generated by your AI provider. These events contain real-time updates from your AI provider, providing insights into the ongoing tasks. This is specially useful to render your LLM’s responses in real-time as they are being generated.

These events will always take place between a provider-started and a provider-completed event. You can expect updates on each stage of the processing, providing insights into the ongoing tasks.

Here’s an example of a Provider Event with the response text delta:

{
  "type": "provider-event",
  "data": {
    "type": "text-delta",
    "textDelta": "response-text-delta"
  }
}

For more information about these events, visit Vercel AI SDK’s Documentation

Handling SSE Events

The API uses SSE for real-time updates. Here’s how to handle SSE responses:

1. Set up an EventSource or use a library that supports SSE.
2. Listen for events and parse the JSON data in each event.
3. Handle different event types.