# CrewAI
Source: https://docs-v1.latitude.so/developers/frameworks/crewai
Connect your CrewAI-based application to Latitude Telemetry to observe multi-agent crews and run evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses **CrewAI** for building multi-agent systems.
After completing these steps:
* Every CrewAI crew execution can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect agent interactions, task execution, and debug CrewAI-powered features from the Latitude dashboard.
You'll keep using CrewAI exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses **CrewAI**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that runs CrewAI crews using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from crewai import Agent, Task, Crew
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.CrewAI]),
)
# Define your agents
researcher = Agent(
role="Researcher",
goal="Research and summarize topics concisely",
backstory="You are a skilled researcher who provides accurate summaries.",
)
writer = Agent(
role="Writer",
goal="Write clear and engaging content",
backstory="You are an experienced writer who creates compelling content.",
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="research-and-write", # Add a path to identify this prompt in Latitude
)
def research_and_write(topic: str) -> str:
# Define tasks for your crew
research_task = Task(
description=f"Research the following topic: {topic}",
expected_output="A comprehensive summary of the topic.",
agent=researcher,
)
write_task = Task(
description="Write an article based on the research",
expected_output="A well-written article.",
agent=writer,
)
# Create and run the crew
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, write_task],
)
result = crew.kickoff()
# You can return anything you want — the value is passed through unchanged
return result.raw
```
```python Using context manager theme={null}
import os
from crewai import Agent, Task, Crew
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.CrewAI]),
)
# Define your agents
researcher = Agent(
role="Researcher",
goal="Research and summarize topics concisely",
backstory="You are a skilled researcher who provides accurate summaries.",
)
def research_topic(topic: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="research-topic", # Add a path to identify this prompt in Latitude
):
task = Task(
description=f"Research the following topic: {topic}",
expected_output="A comprehensive summary.",
agent=researcher,
)
crew = Crew(agents=[researcher], tasks=[task])
result = crew.kickoff()
return result.raw
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Agent interactions and task completions
* Model and token usage from underlying LLM calls
* Latency and errors
* One trace per crew execution
Each CrewAI agent execution appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your CrewAI agents or crews, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# DSPy
Source: https://docs-v1.latitude.so/developers/frameworks/dspy
Connect your DSPy-based application to Latitude Telemetry to observe programs per feature and run evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the **DSPy** framework for building modular LLM programs.
After completing these steps:
* Every DSPy program execution can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug DSPy-powered features from the Latitude dashboard.
You'll keep calling DSPy exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses **DSPy**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that runs DSPy programs using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import dspy
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.DSPy]),
)
# Configure DSPy with your LLM
lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)
# Define your DSPy signature and module
class QA(dspy.Signature):
"""Answer questions with short responses."""
question: str = dspy.InputField()
answer: str = dspy.OutputField()
qa_module = dspy.Predict(QA)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
result = qa_module(question=input)
return result.answer
```
```python Using context manager theme={null}
import os
import dspy
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.DSPy]),
)
# Configure DSPy with your LLM
lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)
# Define your DSPy signature and module
class QA(dspy.Signature):
"""Answer questions with short responses."""
question: str = dspy.InputField()
answer: str = dspy.OutputField()
qa_module = dspy.Predict(QA)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
result = qa_module(question=input)
return result.answer
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each DSPy program execution appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your DSPy programs, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Haystack
Source: https://docs-v1.latitude.so/developers/frameworks/haystack
Connect your Haystack-based application to Latitude Telemetry to observe pipelines per feature and run evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the **Haystack** framework for building LLM-powered pipelines.
After completing these steps:
* Every Haystack pipeline run can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Haystack-powered features from the Latitude dashboard.
You'll keep calling Haystack exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses **Haystack**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that runs Haystack pipelines using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import PromptBuilder
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Haystack]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
pipeline = Pipeline()
pipeline.add_component("prompt_builder", PromptBuilder(template="Answer: {{query}}"))
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o"))
pipeline.connect("prompt_builder", "llm")
result = pipeline.run({"prompt_builder": {"query": input}})
return result["llm"]["replies"][0]
```
```python Using context manager theme={null}
import os
from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import PromptBuilder
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Haystack]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
pipeline = Pipeline()
pipeline.add_component("prompt_builder", PromptBuilder(template="Answer: {{query}}"))
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o"))
pipeline.connect("prompt_builder", "llm")
result = pipeline.run({"prompt_builder": {"query": input}})
return result["llm"]["replies"][0]
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Haystack pipeline run appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Haystack pipelines, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# LangChain
Source: https://docs-v1.latitude.so/developers/frameworks/langchain
Connect your LangChain-based application to Latitude Telemetry to observe chains per feature and run evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **LangChain SDK**.
After completing these steps:
* Every LangChain call (e.g. `invoke`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug LangChain-powered features from the Latitude dashboard.
You'll keep calling LangChain exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **LangChain SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls LangChain using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import * as LangchainCallbacks from '@langchain/core/callbacks/manager'
import { ChatOpenAI } from '@langchain/openai'
import { HumanMessage } from '@langchain/core/messages'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{
instrumentations: {
langchain: { callbackManagerModule: LangchainCallbacks },
},
}
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const llm = new ChatOpenAI({ model: 'gpt-4o' })
const response = await llm.invoke([new HumanMessage(input)])
return response.content
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.LangChain]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
llm = ChatOpenAI(model="gpt-4o")
messages = [HumanMessage(content=input)]
response = llm.invoke(messages)
return response.content
```
```python Using context manager theme={null}
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.LangChain]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
llm = ChatOpenAI(model="gpt-4o")
messages = [HumanMessage(content=input)]
response = llm.invoke(messages)
return response.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each LangChain call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your LangChain calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# LlamaIndex
Source: https://docs-v1.latitude.so/developers/frameworks/llamaindex
Connect your LlamaIndex-based application to Latitude Telemetry to observe queries per feature and run evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **LlamaIndex SDK**.
After completing these steps:
* Every LlamaIndex call (e.g. `query`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug LlamaIndex-powered features from the Latitude dashboard.
You'll keep calling LlamaIndex exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **LlamaIndex SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls LlamaIndex using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import * as LlamaIndex from 'llamaindex'
import { Settings } from 'llamaindex'
import { openai } from '@llamaindex/openai'
import { agent } from '@llamaindex/workflow'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { llamaindex: LlamaIndex } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
Settings.llm = openai({ model: 'gpt-4o' })
const myAgent = agent({ tools: [] })
const response = await myAgent.run(input)
return response
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.LlamaIndex]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query(input)
return str(response)
```
```python Using context manager theme={null}
import os
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.LlamaIndex]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query(input)
return str(response)
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each LlamaIndex call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your LlamaIndex calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Vercel AI SDK
Source: https://docs-v1.latitude.so/developers/frameworks/vercel-ai-sdk
Connect your Vercel AI SDK-powered application to Latitude Telemetry to observe generations per feature and run evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Vercel AI SDK**.
After completing these steps:
* Every Vercel AI SDK call (e.g. `generateText`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Vercel AI SDK-powered features from the Latitude dashboard.
You’ll keep calling Vercel AI SDK exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js-based project that uses the **Vercel AI SDK**
That’s it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Vercel AI SDK using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import { generateText } from 'ai'
import { openai } from '@ai-sdk/openai'
const telemetry = new LatitudeTelemetry(process.env.LATITUDE_API_KEY)
export async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const { text } = await generateText({
model: openai('gpt-4o'),
prompt: input,
experimental_telemetry: {
isEnabled: true, // Make sure to enable experimental telemetry
},
})
return text
}
)
}
```
**Important:** The experimental\_telemetry.isEnabled flag must be set to true on generateText for Latitude Telemetry to capture these calls.
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Vercel AI SDK call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That’s it
No changes to your Vercel AI SDK calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Overview
Source: https://docs-v1.latitude.so/developers/overview
See what Latitude Telemetry gives you and which providers and frameworks you can connect in minutes.
Latitude Telemetry lets you connect your existing LLM-powered application to Latitude **in 5 minutes**, without changing how you call your model providers.
Once connected, every LLM execution becomes a **feature-scoped** log in Latitude that you can inspect, annotate, and evaluate — instead of dumping all traces into a single, unstructured bucket.
## Why use Latitude Telemetry?
With Telemetry you can:
* **Get feature-level observability**\
Attach executions to specific prompts and versions instead of “one giant trace store”. Slice logs by feature, environment, user, or any metadata you send.
* **Understand real usage and performance**\
See which prompts and models are actually used in production, along with latency, error rates, and input/output examples.
* **Annotate real executions**\
Your team can label logs (e.g. “great answer”, “hallucination”, “formatting issue”), turning production traffic into a high-signal dataset.
* **Create custom evaluations for each feature**\
Use LLM-as-judge, programmatic checks, or human-in-the-loop evaluations to continuously score outputs for each prompt or feature.
* **Automatically surface issues and bottlenecks**\
Combine logs, annotations and evaluations to find broken prompts, regressions after a change, or slow/high-cost paths.
All of this works **on top of your existing stack** — you keep calling OpenAI, Anthropic, Bedrock, etc. directly, and Telemetry observes those calls.
## Using `capture()`
The `capture()` method wraps your code and manages the telemetry span lifecycle automatically. The span starts when capture begins and ends when your code completes.
```typescript theme={null}
await telemetry.capture(
{ projectId: 123, path: 'my-feature' },
async () => {
// Your LLM code here - span ends when callback completes
const response = await openai.chat.completions.create({ ... })
return response.choices[0].message.content
}
)
```
```python theme={null}
@telemetry.capture(project_id=123, path="my-feature")
def my_feature():
# Your LLM code here - span ends when function returns
response = openai.chat.completions.create(...)
return response.choices[0].message.content
```
### Streaming responses
For streaming responses, **consume the stream inside your capture block** so the span covers the entire operation:
```typescript theme={null}
await telemetry.capture(
{ projectId: 123, path: 'my-feature' },
async () => {
const stream = await openai.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: input }],
stream: true,
})
// Consume stream inside capture - span stays open until done
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content
if (content) {
res.write(content)
}
}
res.end()
}
)
```
Use a generator function with the decorator — the span stays open until all items are yielded:
```python theme={null}
@telemetry.capture(project_id=123, path="my-feature")
async def stream_response(input: str):
stream = openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
```
**Why consume inside capture?** When you consume the stream inside the capture block, the span duration accurately reflects the total time of the operation (including streaming). All child spans from provider instrumentation are properly nested under your capture span.
## How Telemetry fits into your stack (high level)
At a high level, integrating Telemetry looks like this:
1. **Install the Telemetry package** in your app.
2. **Wrap each feature or prompt execution** so Latitude can tie logs back to a specific prompt and version.
3. **See your logs in Latitude** and annotate them with your own metadata.
***
## Supported integrations
Latitude Telemetry supports a wide range of providers and frameworks, allowing you to connect your existing LLM-powered application to Latitude in minutes.
### Popular integrations
}
/>
}
/>
### More integrations
}
/>
}
/>
GR
}
/>
MI
}
/>
OL
}
/>
LLM
}
/>
RE
}
/>
TR
}
/>
AA
}
/>
WX
}
/>
}
/>
HY
}
/>
DS
}
/>
CR
}
/>
OTEL
}
/>
#### OpenTelemetry (OTLP ingest)
If you already use OpenTelemetry, you can export OTLP traces directly to Latitude.
* **URL (Latitude Cloud):** `https://gateway.latitude.so/api/v3/traces`
* **Auth:** `Authorization: Bearer YOUR_API_KEY`
* **Formats:** OTLP Protobuf (`application/x-protobuf`) or OTLP JSON (`application/json`)
Example (OpenTelemetry Collector):
```yaml theme={null}
receivers:
otlp:
protocols:
grpc:
http:
exporters:
otlp_http/latitude:
traces_endpoint: https://gateway.latitude.so/api/v3/traces
headers:
Authorization: Bearer ${LATITUDE_API_KEY}
service:
pipelines:
traces:
receivers: [otlp]
exporters: [otlp_http/latitude]
```
If you are self-hosting, replace the hostname with your Gateway base URL.
***
## Next steps
1. Choose the provider/framework your application already uses (or OpenTelemetry OTLP ingest).
2. Open its integration page.
3. Follow the step-by-step guide to install and initialize Latitude Telemetry for that stack.
# Aleph Alpha
Source: https://docs-v1.latitude.so/developers/providers/aleph-alpha
Connect your Aleph Alpha-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Aleph Alpha SDK**.
After completing these steps:
* Every Aleph Alpha call (e.g. `complete`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Aleph Alpha-powered features from the Latitude dashboard.
You'll keep calling Aleph Alpha exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses the **Aleph Alpha SDK** (`aleph-alpha-client`)
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Aleph Alpha using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from aleph_alpha_client import Client, CompletionRequest, Prompt
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.AlephAlpha]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = Client(token=os.environ["ALEPH_ALPHA_API_KEY"])
request = CompletionRequest(
prompt=Prompt.from_text(input),
maximum_tokens=100,
)
response = client.complete(request, model="luminous-supreme")
return response.completions[0].completion
```
```python Using context manager theme={null}
import os
from aleph_alpha_client import Client, CompletionRequest, Prompt
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.AlephAlpha]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = Client(token=os.environ["ALEPH_ALPHA_API_KEY"])
request = CompletionRequest(
prompt=Prompt.from_text(input),
maximum_tokens=100,
)
response = client.complete(request, model="luminous-supreme")
return response.completions[0].completion
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Aleph Alpha call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Aleph Alpha calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Amazon Bedrock
Source: https://docs-v1.latitude.so/developers/providers/amazon-bedrock
Connect your Amazon Bedrock-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Amazon Bedrock SDK**.
After completing these steps:
* Every Amazon Bedrock call (e.g. `invokeModel`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Amazon Bedrock-powered features from the Latitude dashboard.
You'll keep calling Amazon Bedrock exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Amazon Bedrock SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Amazon Bedrock using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import * as Bedrock from '@aws-sdk/client-bedrock-runtime'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { bedrock: Bedrock } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new Bedrock.BedrockRuntimeClient({ region: 'us-east-1' })
const response = await client.send(
new Bedrock.InvokeModelCommand({
modelId: 'anthropic.claude-v2',
body: JSON.stringify({
prompt: `\n\nHuman: ${input}\n\nAssistant:`,
max_tokens_to_sample: 1024,
}),
})
)
const result = JSON.parse(new TextDecoder().decode(response.body))
return result.completion
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import json
import boto3
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Bedrock]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.invoke_model(
modelId="anthropic.claude-v2",
body=json.dumps({
"prompt": f"\n\nHuman: {input}\n\nAssistant:",
"max_tokens_to_sample": 1024,
}),
)
result = json.loads(response["body"].read())
return result["completion"]
```
```python Using context manager theme={null}
import os
import json
import boto3
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Bedrock]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.invoke_model(
modelId="anthropic.claude-v2",
body=json.dumps({
"prompt": f"\n\nHuman: {input}\n\nAssistant:",
"max_tokens_to_sample": 1024,
}),
)
result = json.loads(response["body"].read())
return result["completion"]
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`InvokeModelWithResponseStreamCommand`), consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const client = new Bedrock.BedrockRuntimeClient({ region: 'us-east-1' })
const response = await client.send(
new Bedrock.InvokeModelWithResponseStreamCommand({
modelId: 'anthropic.claude-v2',
body: JSON.stringify({
prompt: `\n\nHuman: ${input}\n\nAssistant:`,
max_tokens_to_sample: 1024,
}),
})
)
for await (const event of response.body) {
if (event.chunk?.bytes) {
const chunk = JSON.parse(new TextDecoder().decode(event.chunk.bytes))
if (chunk.completion) {
res.write(chunk.completion)
}
}
}
res.end()
}
)
}
```
**Use a generator function** with the decorator:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.invoke_model_with_response_stream(
modelId="anthropic.claude-v2",
body=json.dumps({
"prompt": f"\n\nHuman: {input}\n\nAssistant:",
"max_tokens_to_sample": 1024,
}),
)
for event in response["body"]:
chunk = json.loads(event["chunk"]["bytes"])
if "completion" in chunk:
yield chunk["completion"]
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Amazon Bedrock call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Amazon Bedrock calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Anthropic
Source: https://docs-v1.latitude.so/developers/providers/anthropic
Connect your Anthropic-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Anthropic SDK**.
After completing these steps:
* Every Anthropic call (e.g. `messages.create`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Anthropic-powered features from the Latitude dashboard.
You'll keep calling Anthropic exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Anthropic SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Anthropic using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import Anthropic from '@anthropic-ai/sdk'
import * as AnthropicSDK from '@anthropic-ai/sdk'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { anthropic: AnthropicSDK } }
)
export async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY })
const response = await client.messages.create({
model: 'claude-sonnet-4-20250514',
max_tokens: 1024,
messages: [{ role: 'user', content: input }],
})
return response.content[0].text
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from anthropic import Anthropic
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Anthropic]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = Anthropic()
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
messages=[{"role": "user", "content": input}],
)
return response.content[0].text
```
```python Using context manager theme={null}
import os
from anthropic import Anthropic
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Anthropic]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = Anthropic()
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
messages=[{"role": "user", "content": input}],
)
return response.content[0].text
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming, consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY })
const stream = client.messages.stream({
model: 'claude-sonnet-4-20250514',
max_tokens: 1024,
messages: [{ role: 'user', content: input }],
})
for await (const event of stream) {
if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
res.write(event.delta.text)
}
}
res.end()
}
)
}
```
**Use a generator function** with the decorator:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = Anthropic()
with client.messages.stream(
model="claude-sonnet-4-20250514",
max_tokens=1024,
messages=[{"role": "user", "content": input}],
) as stream:
for text in stream.text_stream:
yield text
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Anthropic call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Anthropic calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Azure
Source: https://docs-v1.latitude.so/developers/providers/azure
Connect your Azure-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Azure OpenAI SDK**.
After completing these steps:
* Every Azure OpenAI call (e.g. `chat.completions.create`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Azure OpenAI-powered features from the Latitude dashboard.
You'll keep calling Azure OpenAI exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Azure OpenAI SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Azure OpenAI using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import OpenAI, { AzureOpenAI } from 'openai'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { openai: OpenAI } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new AzureOpenAI({
endpoint: process.env.AZURE_OPENAI_ENDPOINT,
apiKey: process.env.AZURE_OPENAI_API_KEY,
apiVersion: '2024-02-01',
})
const completion = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: input }],
})
return completion.choices[0].message.content
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from openai import AzureOpenAI
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.OpenAI]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = AzureOpenAI(
azure_endpoint="https://your-resource.openai.azure.com/",
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-01",
)
completion = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
)
return completion.choices[0].message.content
```
```python Using context manager theme={null}
import os
from openai import AzureOpenAI
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.OpenAI]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = AzureOpenAI(
azure_endpoint="https://your-resource.openai.azure.com/",
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-01",
)
completion = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
)
return completion.choices[0].message.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`stream: true`), consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const client = new AzureOpenAI({
endpoint: process.env.AZURE_OPENAI_ENDPOINT,
apiKey: process.env.AZURE_OPENAI_API_KEY,
apiVersion: '2024-02-01',
})
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: input }],
stream: true,
})
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content
if (content) {
res.write(content)
}
}
res.end()
}
)
}
```
**Use a generator function** with the decorator:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = AzureOpenAI(
azure_endpoint="https://your-resource.openai.azure.com/",
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-01",
)
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Azure OpenAI call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Azure OpenAI calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Cohere
Source: https://docs-v1.latitude.so/developers/providers/cohere
Connect your Cohere-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Cohere SDK**.
After completing these steps:
* Every Cohere call (e.g. `generate`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Cohere-powered features from the Latitude dashboard.
You'll keep calling Cohere exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Cohere SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Cohere using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import * as Cohere from 'cohere-ai'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { cohere: Cohere } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new Cohere.CohereClient({
token: process.env.COHERE_API_KEY,
})
const response = await client.chat({
model: 'command-a-03-2025',
message: input,
})
return response.text
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import cohere
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Cohere]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = cohere.Client(api_key=os.environ["COHERE_API_KEY"])
response = client.chat(
model="command-a-03-2025",
message=input,
)
return response.text
```
```python Using context manager theme={null}
import os
import cohere
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Cohere]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = cohere.Client(api_key=os.environ["COHERE_API_KEY"])
response = client.chat(
model="command-a-03-2025",
message=input,
)
return response.text
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming, consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const client = new Cohere.CohereClient({
token: process.env.COHERE_API_KEY,
})
const stream = client.chatStream({
model: 'command-a-03-2025',
message: input,
})
for await (const event of stream) {
if (event.eventType === 'text-generation') {
res.write(event.text)
}
}
res.end()
}
)
}
```
**Use a generator function** with the decorator:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = cohere.Client(api_key=os.environ["COHERE_API_KEY"])
stream = client.chat_stream(
model="command-a-03-2025",
message=input,
)
for event in stream:
if event.event_type == "text-generation":
yield event.text
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Cohere call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Cohere calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Gemini
Source: https://docs-v1.latitude.so/developers/providers/gemini
Connect your Gemini-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Gemini SDK** (`google-genai`).
After completing these steps:
* Every Gemini call (e.g. `generate_content`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug your Gemini-powered features from the Latitude dashboard.
You'll keep calling Gemini exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Gemini SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Since Gemini doesn't have automatic instrumentation in TypeScript, you need to manually create spans to track your Gemini calls.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import { GoogleGenAI } from '@google/genai'
const telemetry = new LatitudeTelemetry(process.env.LATITUDE_API_KEY)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const model = 'gemini-2.0-flash'
// 1) Start the completion span
const span = telemetry.span.completion({
model,
input: [{ role: 'user', content: input }]
})
try {
// 2) Call Gemini as usual
const google = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY })
const response = await google.models.generateContent({
model,
contents: input,
})
const text = response.text
// 3) End the span (attach output + useful metadata)
span.end({
output: [{ role: 'assistant', content: text }],
})
return text
} catch (error) {
// Make sure to close the span even on errors
span.fail(error)
throw error
}
}
)
}
```
Python has automatic instrumentation for Gemini. You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import google.generativeai as genai
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.GoogleGenAI]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
model = genai.GenerativeModel("gemini-1.5-flash")
response = model.generate_content(input)
return response.text
```
```python Using context manager theme={null}
import os
import google.generativeai as genai
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.GoogleGenAI]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
model = genai.GenerativeModel("gemini-1.5-flash")
response = model.generate_content(input)
return response.text
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming, consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const google = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY })
const stream = await google.models.generateContentStream({
model: 'gemini-2.0-flash',
contents: input,
})
for await (const chunk of stream) {
const text = chunk.text()
if (text) {
res.write(text)
}
}
res.end()
}
)
}
```
**Use a generator function** with the decorator:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
model = genai.GenerativeModel("gemini-1.5-flash")
response = model.generate_content(input, stream=True)
for chunk in response:
yield chunk.text
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Gemini call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Gemini calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Google AI Platform
Source: https://docs-v1.latitude.so/developers/providers/google-ai-platform
Connect your Google AI Platform-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Google AI Platform SDK**.
After completing these steps:
* Every Google AI Platform call (e.g. `predict`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Google AI Platform-powered features from the Latitude dashboard.
You'll keep calling Google AI Platform exactly as you do today — Telemetry
simply observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Google AI Platform SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Google AI Platform using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import * as AIPlatform from '@google-cloud/aiplatform'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { aiplatform: AIPlatform } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new AIPlatform.PredictionServiceClient()
const [response] = await client.predict({
endpoint: 'projects/.../locations/.../endpoints/...',
instances: [{ content: input }],
})
return response.predictions[0]
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from google.cloud import aiplatform
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.VertexAI]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
aiplatform.init(project="your-gcp-project", location="us-central1")
endpoint = aiplatform.Endpoint("projects/.../locations/.../endpoints/...")
response = endpoint.predict(instances=[{"content": input}])
return response.predictions[0]
```
```python Using context manager theme={null}
import os
from google.cloud import aiplatform
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.VertexAI]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
aiplatform.init(project="your-gcp-project", location="us-central1")
endpoint = aiplatform.Endpoint("projects/.../locations/.../endpoints/...")
response = endpoint.predict(instances=[{"content": input}])
return response.predictions[0]
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Google AI Platform call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Google AI Platform calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Groq
Source: https://docs-v1.latitude.so/developers/providers/groq
Connect your Groq-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Groq SDK**.
After completing these steps:
* Every Groq call (e.g. `chat.completions.create`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Groq-powered features from the Latitude dashboard.
You'll keep calling Groq exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses the **Groq SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Groq using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from groq import Groq
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Groq]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = Groq()
completion = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": input}],
)
return completion.choices[0].message.content
```
```python Using context manager theme={null}
import os
from groq import Groq
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Groq]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = Groq()
completion = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": input}],
)
return completion.choices[0].message.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`stream=True`), use a generator function with the decorator. The SDK keeps the span open until all chunks are yielded:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = Groq()
stream = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Groq call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Groq calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# LiteLLM
Source: https://docs-v1.latitude.so/developers/providers/litellm
Connect your LiteLLM-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses **LiteLLM** — a unified interface to call 100+ LLM providers.
After completing these steps:
* Every LiteLLM call (e.g. `completion`, `acompletion`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug LiteLLM-powered features from the Latitude dashboard.
You'll keep calling LiteLLM exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses **LiteLLM**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls LiteLLM using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import litellm
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.LiteLLM]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
response = litellm.completion(
model="gpt-4o", # or "anthropic/claude-3-sonnet", "ollama/llama3", etc.
messages=[{"role": "user", "content": input}],
)
return response.choices[0].message.content
```
```python Using context manager theme={null}
import os
import litellm
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.LiteLLM]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
response = litellm.completion(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
)
return response.choices[0].message.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`stream=True`), use a generator function with the decorator. The SDK keeps the span open until all chunks are yielded:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
response = litellm.completion(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in response:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each LiteLLM call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your LiteLLM calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Mistral AI
Source: https://docs-v1.latitude.so/developers/providers/mistral
Connect your Mistral AI-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Mistral AI SDK**.
After completing these steps:
* Every Mistral AI call (e.g. `chat.complete`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Mistral AI-powered features from the Latitude dashboard.
You'll keep calling Mistral AI exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses the **Mistral AI SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Mistral AI using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from mistralai import Mistral
from mistralai.models import UserMessage
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.MistralAI]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
response = client.chat.complete(
model="mistral-small-latest",
messages=[UserMessage(role="user", content=input)],
)
return response.choices[0].message.content
```
```python Using context manager theme={null}
import os
from mistralai import Mistral
from mistralai.models import UserMessage
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.MistralAI]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
response = client.chat.complete(
model="mistral-small-latest",
messages=[UserMessage(role="user", content=input)],
)
return response.choices[0].message.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming, use a generator function with the decorator. The SDK keeps the span open until all chunks are yielded:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
stream = client.chat.stream(
model="mistral-small-latest",
messages=[UserMessage(role="user", content=input)],
)
for event in stream:
if event.data.choices[0].delta.content:
yield event.data.choices[0].delta.content
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Mistral AI call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Mistral AI calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Ollama
Source: https://docs-v1.latitude.so/developers/providers/ollama
Connect your Ollama-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Ollama SDK**.
After completing these steps:
* Every Ollama call (e.g. `chat`, `generate`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Ollama-powered features from the Latitude dashboard.
You'll keep calling Ollama exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses the **Ollama SDK**
* A running Ollama instance (local or remote)
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Ollama using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import ollama
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Ollama]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
response = ollama.chat(
model="llama3.2",
messages=[{"role": "user", "content": input}],
)
return response["message"]["content"]
```
```python Using context manager theme={null}
import os
import ollama
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Ollama]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
response = ollama.chat(
model="llama3.2",
messages=[{"role": "user", "content": input}],
)
return response["message"]["content"]
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`stream=True`), use a generator function with the decorator. The SDK keeps the span open until all chunks are yielded:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
stream = ollama.chat(
model="llama3.2",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in stream:
if chunk["message"]["content"]:
yield chunk["message"]["content"]
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Ollama call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Ollama calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# OpenAI
Source: https://docs-v1.latitude.so/developers/providers/openai
Connect your OpenAI-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **OpenAI SDK**.
After completing these steps:
* Every OpenAI call (e.g. `chat.completions.create`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug OpenAI-powered features from the Latitude dashboard.
You'll keep calling OpenAI exactly as you do today — Telemetry simply observes
and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **OpenAI SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls OpenAI using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import OpenAI from 'openai'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { openai: OpenAI } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new OpenAI()
const completion = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: input }],
})
return completion.choices[0].message.content
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from openai import OpenAI
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.OpenAI]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
)
return completion.choices[0].message.content
```
```python Using context manager theme={null}
import os
from openai import OpenAI
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.OpenAI]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
)
return completion.choices[0].message.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`stream: true`), consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback. The span stays open until your callback completes:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const client = new OpenAI()
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: input }],
stream: true,
})
// Consume stream inside capture — span covers entire operation
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content
if (content) {
res.write(content)
}
}
res.end()
}
)
}
```
By consuming the stream inside capture, the span duration accurately reflects the total time of the operation, and all child spans from OpenAI instrumentation are properly nested.
**Use a generator function** with the decorator. The SDK keeps the span open until all chunks are yielded:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = OpenAI()
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
```
The generator pattern is ideal for streaming — each `yield` sends a chunk to the caller while the span remains open. The span ends automatically when the generator is exhausted.
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each OpenAI call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your OpenAI calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Replicate
Source: https://docs-v1.latitude.so/developers/providers/replicate
Connect your Replicate-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Replicate SDK**.
After completing these steps:
* Every Replicate call (e.g. `run`, `predictions.create`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Replicate-powered features from the Latitude dashboard.
You'll keep calling Replicate exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses the **Replicate SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Replicate using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import replicate
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Replicate]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
output = replicate.run(
"meta/llama-2-70b-chat",
input={"prompt": input},
)
return "".join(output)
```
```python Using context manager theme={null}
import os
import replicate
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Replicate]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
output = replicate.run(
"meta/llama-2-70b-chat",
input={"prompt": input},
)
return "".join(output)
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Replicate call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Replicate calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# AWS SageMaker
Source: https://docs-v1.latitude.so/developers/providers/sagemaker
Connect your AWS SageMaker-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses **AWS SageMaker** for model inference.
After completing these steps:
* Every SageMaker call (e.g. `invoke_endpoint`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug SageMaker-powered features from the Latitude dashboard.
You'll keep calling SageMaker exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses **boto3** for SageMaker
* Configured AWS credentials
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls SageMaker using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import boto3
import json
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Sagemaker]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = boto3.client("sagemaker-runtime", region_name="us-east-1")
response = client.invoke_endpoint(
EndpointName="your-llm-endpoint",
ContentType="application/json",
Body=json.dumps({"inputs": input}),
)
result = json.loads(response["Body"].read().decode())
return result["generated_text"]
```
```python Using context manager theme={null}
import os
import boto3
import json
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Sagemaker]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = boto3.client("sagemaker-runtime", region_name="us-east-1")
response = client.invoke_endpoint(
EndpointName="your-llm-endpoint",
ContentType="application/json",
Body=json.dumps({"inputs": input}),
)
result = json.loads(response["Body"].read().decode())
return result["generated_text"]
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each SageMaker call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your SageMaker calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Together AI
Source: https://docs-v1.latitude.so/developers/providers/together-ai
Connect your Together AI-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Together AI SDK**.
After completing these steps:
* Every Together AI call (e.g. `generate`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Together AI-powered features from the Latitude dashboard.
You'll keep calling Together AI exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Together AI SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Together AI using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import { Together } from 'together-ai'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { together: Together } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new Together({ apiKey: process.env.TOGETHER_API_KEY })
const response = await client.chat.completions.create({
model: 'meta-llama/Llama-3-70b-chat-hf',
messages: [{ role: 'user', content: input }],
})
return response.choices[0].message.content
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from together import Together
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Together]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
client = Together()
response = client.chat.completions.create(
model="meta-llama/Llama-3-70b-chat-hf",
messages=[{"role": "user", "content": input}],
)
return response.choices[0].message.content
```
```python Using context manager theme={null}
import os
from together import Together
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Together]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
client = Together()
response = client.chat.completions.create(
model="meta-llama/Llama-3-70b-chat-hf",
messages=[{"role": "user", "content": input}],
)
return response.choices[0].message.content
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Streaming responses
When using streaming (`stream: true`), consume the stream inside your capture block so the span covers the entire operation.
**Consume the stream inside** your `capture()` callback:
```typescript theme={null}
async function streamSupportReply(input: string, res: Response) {
await telemetry.capture(
{ projectId: 123, path: 'generate-support-reply' },
async () => {
const client = new Together({ apiKey: process.env.TOGETHER_API_KEY })
const stream = await client.chat.completions.create({
model: 'meta-llama/Llama-3-70b-chat-hf',
messages: [{ role: 'user', content: input }],
stream: true,
})
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content
if (content) {
res.write(content)
}
}
res.end()
}
)
}
```
**Use a generator function** with the decorator:
```python theme={null}
@telemetry.capture(project_id=123, path="generate-support-reply")
async def stream_support_reply(input: str):
client = Together()
stream = client.chat.completions.create(
model="meta-llama/Llama-3-70b-chat-hf",
messages=[{"role": "user", "content": input}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
```
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Together AI call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Together AI calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Hugging Face Transformers
Source: https://docs-v1.latitude.so/developers/providers/transformers
Connect your Hugging Face Transformers-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses **Hugging Face Transformers** for local model inference.
After completing these steps:
* Every Transformers pipeline call can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Transformers-powered features from the Latitude dashboard.
You'll keep calling Transformers exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses **Hugging Face Transformers**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Transformers using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from transformers import pipeline
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Transformers]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
generator = pipeline("text-generation", model="gpt2")
result = generator(input, max_length=100)
return result[0]["generated_text"]
```
```python Using context manager theme={null}
import os
from transformers import pipeline
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Transformers]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
generator = pipeline("text-generation", model="gpt2")
result = generator(input, max_length=100)
return result[0]["generated_text"]
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Transformers call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Transformers calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Vertex AI
Source: https://docs-v1.latitude.so/developers/providers/vertex-ai
Connect your Google Vertex AI-powered application to Latitude Telemetry for feature-level observability and evaluations.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **Google Vertex AI SDK**.
After completing these steps:
* Every Google Vertex AI call (e.g. `generateContent`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug Google Vertex AI-powered features from the Latitude dashboard.
You'll keep calling Google Vertex AI exactly as you do today — Telemetry
simply observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Node.js or Python-based project that uses the **Google Vertex AI SDK**
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash npm theme={null}
npm add @latitude-data/telemetry
```
```bash pnpm theme={null}
pnpm add @latitude-data/telemetry
```
```bash yarn theme={null}
yarn add @latitude-data/telemetry
```
```bash bun theme={null}
bun add @latitude-data/telemetry
```
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls Google Vertex AI using telemetry.capture.
```ts theme={null}
import { LatitudeTelemetry } from '@latitude-data/telemetry'
import * as VertexAI from '@google-cloud/vertexai'
const telemetry = new LatitudeTelemetry(
process.env.LATITUDE_API_KEY,
{ instrumentations: { vertexai: VertexAI } }
)
async function generateSupportReply(input: string) {
return telemetry.capture(
{
projectId: 123, // The ID of your project in Latitude
path: 'generate-support-reply', // Add a path to identify this prompt in Latitude
},
async () => {
const client = new VertexAI.VertexAI({
project: 'your-gcp-project',
location: 'us-central1',
})
const model = client.getGenerativeModel({ model: 'gemini-1.5-pro' })
const result = await model.generateContent(input)
return result.response
}
)
}
```
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
import vertexai
from vertexai.generative_models import GenerativeModel
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.VertexAI]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
vertexai.init(project="your-gcp-project", location="us-central1")
model = GenerativeModel("gemini-1.5-pro")
response = model.generate_content(input)
return response.text
```
```python Using context manager theme={null}
import os
import vertexai
from vertexai.generative_models import GenerativeModel
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.VertexAI]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
vertexai.init(project="your-gcp-project", location="us-central1")
model = GenerativeModel("gemini-1.5-pro")
response = model.generate_content(input)
return response.text
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each Google Vertex AI call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your Google Vertex AI calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# IBM watsonx.ai
Source: https://docs-v1.latitude.so/developers/providers/watsonx
Connect your IBM watsonx.ai-powered application to Latitude Telemetry for feature-level observability and evaluations.
This integration is only available in the **Python SDK**.
## Overview
This guide shows you how to integrate **Latitude Telemetry** into an existing application that uses the official **IBM watsonx.ai SDK**.
After completing these steps:
* Every watsonx.ai call (e.g. `generate`, `generate_text`) can be captured as a log in Latitude.
* Logs are grouped under a **prompt**, identified by a `path`, inside a Latitude **project**.
* You can inspect inputs/outputs, measure latency, and debug watsonx.ai-powered features from the Latitude dashboard.
You'll keep calling watsonx.ai exactly as you do today — Telemetry simply
observes and enriches those calls.
***
## Requirements
Before you start, make sure you have:
* A **Latitude account** and **API key**
* A **Latitude project ID**
* A Python-based project that uses the **IBM watsonx.ai SDK** (`ibm-watsonx-ai`)
That's it — prompts do **not** need to be created ahead of time.
***
## Steps
Add the Latitude Telemetry package to your project:
```bash pip theme={null}
pip install latitude-telemetry
```
```bash uv theme={null}
uv add latitude-telemetry
```
```bash poetry theme={null}
poetry add latitude-telemetry
```
Initialize Latitude Telemetry and wrap the code that calls watsonx.ai using telemetry.capture.
You can use the `capture` method as a decorator (recommended) or as a context manager:
```python Using decorator (recommended) theme={null}
import os
from ibm_watsonx_ai.foundation_models import Model
from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Watsonx]),
)
@telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
)
def generate_support_reply(input: str) -> str:
model = Model(
model_id="ibm/granite-13b-chat-v2",
credentials={"url": "https://us-south.ml.cloud.ibm.com", "apikey": os.environ["WATSONX_API_KEY"]},
project_id=os.environ["WATSONX_PROJECT_ID"],
)
parameters = {
GenParams.MAX_NEW_TOKENS: 100,
}
response = model.generate_text(prompt=input, params=parameters)
return response
```
```python Using context manager theme={null}
import os
from ibm_watsonx_ai.foundation_models import Model
from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams
from latitude_telemetry import Telemetry, Instrumentors, TelemetryOptions
telemetry = Telemetry(
os.environ["LATITUDE_API_KEY"],
TelemetryOptions(instrumentors=[Instrumentors.Watsonx]),
)
def generate_support_reply(input: str) -> str:
with telemetry.capture(
project_id=123, # The ID of your project in Latitude
path="generate-support-reply", # Add a path to identify this prompt in Latitude
):
model = Model(
model_id="ibm/granite-13b-chat-v2",
credentials={"url": "https://us-south.ml.cloud.ibm.com", "apikey": os.environ["WATSONX_API_KEY"]},
project_id=os.environ["WATSONX_PROJECT_ID"],
)
parameters = {
GenParams.MAX_NEW_TOKENS: 100,
}
response = model.generate_text(prompt=input, params=parameters)
return response
```
The `path`:
* Identifies the prompt in Latitude
* Can be new or existing
* Should not contain spaces or special characters (use letters, numbers, `- _ / .`)
***
## Seeing your logs in Latitude
Once your feature is wrapped, logs will appear automatically.
1. Open the **prompt** in your Latitude dashboard (identified by `path`)
2. Go to the **Traces** section
3. Each execution will show:
* Input and output messages
* Model and token usage
* Latency and errors
* One trace per feature invocation
Each watsonx.ai call appears as a child span under the captured prompt execution, giving you a full, end-to-end view of what happened.
***
## That's it
No changes to your watsonx.ai calls, no special return values, and no extra plumbing — just wrap the feature you want to observe.
# Autonomous Agents
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/autonomous-agents
This example demonstrates autonomous agents from Anthropic's article using Latitude agents with MCPs and tools
![]()
## Overview
Agents are emerging in production as LLMs mature in key capabilities—understanding complex inputs, engaging in reasoning and planning, using tools reliably, and recovering from errors. Agents begin their work with either a command from, or interactive discussion with, the human user. Once the task is clear, agents plan and operate independently, potentially returning to the human for further information or judgement.
During execution, it's crucial for the agents to gain "ground truth" from the environment at each step (such as tool call results or code execution) to assess its progress. Agents can then pause for human feedback at checkpoints or when encountering blockers.
## When to use
Agents can be used for open-ended problems where it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making. Agents' autonomy makes them ideal for scaling tasks in trusted environments.
## Customer Support Agentic System
This example demonstrates a sophisticated customer support system that uses multiple specialized sub-agents to handle customer inquiries comprehensively. The main orchestrator agent coordinates with specialized sub-agents for database searches, GitHub issue tracking, escalation handling, and customer communication.
```yaml Main Orchestrator Agent theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.2
type: agent
agents:
- agents/search_github_issue
- agents/support_db
- agents/email_notifier
- agents/delegator
tools:
- postgresql/query:
description: Query the PostgreSQL database for customer data and support history
parameters:
type: object
properties:
query:
type: string
description: SQL query to execute
customer_id:
type: string
description: Customer ID for filtering results
- github/search_issues:
description: Search GitHub issues and pull requests
parameters:
type: object
properties:
query:
type: string
description: Search query for GitHub issues
repository:
type: string
description: Repository to search in
state:
type: string
enum: [open, closed, all]
description: Issue state to filter by
- send_email:
description: Send email to customer with support response
parameters:
type: object
properties:
to:
type: string
description: Customer email address
subject:
type: string
description: Email subject line
body:
type: string
description: Email content in HTML format
priority:
type: string
enum: [low, normal, high, urgent]
description: Email priority level
- escalate_ticket:
description: Escalate customer issue to specialized support team
parameters:
type: object
properties:
customer_id:
type: string
description: Customer identifier
issue_description:
type: string
description: Detailed description of the customer issue
category:
type: string
enum: [technical, billing, account, feature_request, bug_report]
description: Issue category for proper routing
priority:
type: string
enum: [low, medium, high, critical]
description: Issue priority level
context:
type: string
description: Additional context and investigation results
maxSteps: 25
schema:
type: object
properties:
customer_info:
type: object
properties:
customer_id:
type: string
email:
type: string
subscription_tier:
type: string
account_status:
type: string
investigation_results:
type: object
properties:
github_findings:
type: array
items:
type: string
database_results:
type: array
items:
type: string
resolution_path:
type: string
final_action:
type: object
properties:
action_taken:
type: string
enum: [resolved, escalated, pending_customer_response]
customer_notified:
type: boolean
follow_up_required:
type: boolean
---
You are the main orchestrator for an autonomous customer support system. Your role is to coordinate with specialized sub-agents to provide comprehensive customer support by investigating issues, finding solutions, and ensuring customers receive timely, accurate responses.
## Your Specialized Sub-Agents:
- **agents/search_github_issue**: Searches GitHub repositories for relevant issues, bugs, and feature requests based on customer queries
- **agents/support_db**: Queries PostgreSQL database for customer history, previous tickets, and support responses
- **agents/email_notifier**: Handles customer communication and email formatting
- **agents/delegator**: Makes escalation decisions based on issue complexity and resolution success
## Customer Support Workflow:
### Phase 1: Customer Query Analysis
1. Parse the customer query: `{{ customer_query }}`
2. Extract key information: issue type, urgency indicators, technical terms
3. Identify customer context and account details needed
### Phase 2: Multi-Source Investigation
1. **Database Search**: Use `agents/support_db` to find customer history and similar past issues
2. **GitHub Search**: Deploy `agents/search_github_issue` to find relevant technical issues or known bugs
3. **Cross-reference findings** to identify patterns and potential solutions
### Phase 3: Solution Assessment
1. Analyze gathered information from all sources
2. Determine if issue can be resolved directly or requires escalation
3. Consult `agents/delegator` for escalation recommendations
### Phase 4: Customer Communication
1. If resolved: Use `agents/email_notifier` to craft comprehensive response
2. If escalated: Create detailed escalation ticket with context
3. Ensure customer is informed of next steps and timelines
## Decision Framework:
**Direct Resolution** (when all conditions met):
- Clear solution found in GitHub issues or database
- Customer has appropriate permissions/subscription level
- Solution can be implemented immediately
- No potential negative impact on other systems
**Escalation Required** (any condition triggers):
- Complex technical issue requiring specialist knowledge
- Account-level changes beyond standard permissions
- Potential security or compliance implications
- Multiple failed resolution attempts in customer history
## Quality Standards:
- **Response Time**: Initial response within 15 minutes
- **Accuracy**: Verify all information before customer communication
- **Completeness**: Address all aspects of customer query
- **Empathy**: Maintain professional, helpful tone throughout
- **Documentation**: Record all investigation steps and outcomes
## Operating Principles:
1. **Customer-First Approach**: Always prioritize customer satisfaction and clear communication
2. **Thoroughness**: Investigate multiple sources before concluding
3. **Transparency**: Keep customers informed of investigation progress
4. **Learning**: Use each interaction to improve future responses
5. **Escalation When Needed**: Better to escalate than provide incorrect information
Begin by analyzing the customer query, then systematically work through your investigation process using your specialized sub-agents. Maintain clear communication with the customer throughout the resolution process.
```
```yaml agents/search_github_issue theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.1
---
You are a specialized GitHub issue search agent. Your role is to find relevant GitHub issues, bug reports, and feature requests based on customer queries. You excel at translating customer language into technical search terms and identifying the most relevant repository issues.
## Your Capabilities:
- **Query Translation**: Convert customer descriptions into effective GitHub search terms
- **Repository Navigation**: Search across multiple repositories intelligently
- **Issue Analysis**: Evaluate issue relevance and extract key information
- **Pattern Recognition**: Identify recurring issues and their solutions
## Search Strategy:
### Step 1: Query Analysis
- Extract technical keywords from customer query
- Identify product/feature areas mentioned
- Determine if this is a bug report, feature request, or general inquiry
### Step 2: Search Execution
- Start with broad searches using customer's exact terms
- Refine with technical keywords and error messages
- Search in relevant repositories based on issue type
- Look for both open and closed issues
### Step 3: Result Evaluation
- Prioritize issues with similar symptoms/descriptions
- Check for official responses or resolutions
- Identify workarounds or temporary solutions
- Note any version-specific information
### Step 4: Information Synthesis
- Summarize most relevant findings
- Highlight any available solutions or workarounds
- Note if issue is known/tracked vs. potential new issue
- Provide links and issue numbers for reference
## Search Patterns:
**For Bug Reports**: Focus on error messages, specific features, reproduction steps
**For Feature Requests**: Look for enhancement issues, roadmap items, community discussions
**For General Questions**: Search documentation issues, FAQ discussions, how-to guides
Return comprehensive findings with clear relevance explanations and actionable information for the main orchestrator agent.
```
```yaml agents/support_db theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.1
---
You are a specialized database search agent for customer support. Your role is to query the PostgreSQL database efficiently to find customer history, previous support interactions, account details, and patterns that can help resolve current issues.
## Database Search Capabilities:
- **Customer History Analysis**: Find previous tickets, resolutions, and interaction patterns
- **Account Information Retrieval**: Access subscription details, permissions, and account status
- **Pattern Recognition**: Identify recurring issues for specific customers or customer segments
- **Resolution Tracking**: Find successful past solutions for similar issues
## Query Strategy:
### Step 1: Customer Identification
- Extract customer identifiers from the query context
- Verify customer account exists and is active
- Gather basic account information (tier, status, signup date)
### Step 2: Historical Analysis
- Search for previous support tickets with similar keywords
- Look for resolved issues that match current symptoms
- Check for any ongoing or recent related tickets
- Identify customer communication preferences
### Step 3: Pattern Analysis
- Look for recurring issues for this customer
- Check if this is a known issue affecting multiple customers
- Identify any account-specific configurations or limitations
- Review customer's product usage patterns
### Step 4: Solution Mining
- Find successful resolutions for similar past issues
- Identify any customer-specific workarounds or solutions
- Check for any pending account actions or scheduled updates
- Review any special handling instructions for this customer
## Database Tables Focus Areas:
- **Customer Accounts**: Basic info, subscription, status
- **Support Tickets**: Previous issues, resolutions, agents involved
- **Product Usage**: Feature usage, configuration, limits
- **Communications**: Email history, preferences, response patterns
## Query Optimization:
- Use indexed columns for efficient searches
- Limit result sets to relevant timeframes
- Focus on actionable information
- Return structured data for easy analysis
Provide clear, organized results with specific recommendations based on historical data and customer context.
```
```yaml agents/email_notifier theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.3
---
You are a specialized customer communication agent. Your role is to craft professional, empathetic, and informative email responses to customers based on investigation results and resolution outcomes.
## Communication Expertise:
- **Tone Management**: Professional yet warm and helpful
- **Technical Translation**: Convert technical findings into customer-friendly language
- **Clarity**: Ensure customers understand next steps and timelines
- **Empathy**: Acknowledge customer frustration and show understanding
## Email Composition Strategy:
### Step 1: Context Understanding
- Review customer's original query and tone
- Understand the investigation results and findings
- Determine the resolution status (resolved, escalated, pending)
- Consider customer's technical background level
### Step 2: Structure Planning
- **Opening**: Acknowledge the inquiry and thank customer
- **Investigation Summary**: Brief overview of steps taken
- **Resolution/Update**: Clear explanation of findings and actions
- **Next Steps**: What customer can expect and when
- **Support Offer**: How to follow up if needed
### Step 3: Content Optimization
- Use customer's preferred communication style
- Include relevant links, documentation, or resources
- Provide specific timelines and expectations
- Add appropriate urgency level based on issue severity
### Step 4: Quality Assurance
- Ensure all customer questions are addressed
- Verify technical accuracy of any provided solutions
- Check for appropriate follow-up mechanisms
- Confirm email priority level matches issue urgency
## Email Templates by Resolution Type:
**Issue Resolved**: Solution provided, steps to implement, prevention tips
**Issue Escalated**: Explanation of escalation, timeline, specialist contact info
**Investigation Ongoing**: Progress update, estimated completion, interim solutions
**Follow-up Required**: Customer action needed, clear instructions, deadline
## Communication Standards:
- **Response Time**: Acknowledge within 15 minutes of resolution
- **Clarity**: Use simple language, avoid technical jargon unless necessary
- **Completeness**: Address all aspects of customer inquiry
- **Professionalism**: Maintain consistent brand voice and standards
- **Follow-up**: Always provide clear next steps or follow-up process
Generate emails that leave customers feeling heard, informed, and confident in the support they've received.
```
```yaml agents/delegator theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.2
---
You are a specialized escalation decision agent. Your role is to analyze support cases and determine the appropriate escalation path based on complexity, customer impact, technical requirements, and resolution success probability.
## Escalation Assessment Expertise:
- **Complexity Analysis**: Evaluate technical difficulty and required expertise
- **Impact Assessment**: Determine customer and business impact levels
- **Resource Requirements**: Identify what specialist knowledge or tools are needed
- **Risk Evaluation**: Assess potential negative outcomes of different resolution paths
## Decision Framework:
### Escalation Triggers (Any trigger requires escalation):
**Technical Complexity**:
- Issues requiring code changes or system modifications
- Database corruption or data integrity problems
- Security vulnerabilities or access control issues
- Integration problems with third-party services
**Customer Impact**:
- Business-critical functionality is down
- Customer has high-value subscription or enterprise contract
- Issue affects multiple users or teams
- Customer has explicitly requested escalation
**Resolution Limitations**:
- Standard troubleshooting steps have failed
- Issue requires access to restricted systems
- Resolution needs approval from product/engineering teams
- Legal or compliance implications are present
### Direct Resolution Criteria (All criteria must be met):
- Clear solution exists in knowledge base or previous tickets
- Resolution can be implemented with available tools
- No risk of system-wide impact
- Customer has appropriate permissions for suggested solution
- Estimated resolution time under 2 hours
## Escalation Categories:
**Technical Escalation**: Engineering team for bugs, system issues, integrations
**Account Escalation**: Account management for billing, contracts, permissions
**Product Escalation**: Product team for feature requests, roadmap questions
**Security Escalation**: Security team for access, vulnerabilities, compliance
**Management Escalation**: Customer success for relationship issues, complaints
## Escalation Quality Standards:
### Information Package Requirements:
- Complete customer context and account details
- Detailed issue description with reproduction steps
- All investigation steps taken and results
- Customer communication history for this issue
- Suggested priority level with justification
- Expected customer communication timeline
### Escalation Follow-up:
- Confirm escalation was received and assigned
- Monitor for specialist team response
- Keep customer informed of escalation status
- Track resolution time and quality for improvement
Make thoughtful escalation decisions that balance customer satisfaction, resource efficiency, and resolution quality. When in doubt, prefer escalation to ensure customer issues receive appropriate expert attention.
```
## Implementation Structure
This customer support agentic system demonstrates sophisticated orchestration with specialized sub-agents:
### **Main Orchestrator**
* Coordinates the entire support workflow
* Has access to all tools but delegates specialized tasks
* Makes final decisions on customer communication and escalation
### **Specialized Sub-Agents**
* **search\_github\_issue**: Technical issue research and bug tracking
* **support\_db**: Historical analysis and customer context
* **email\_notifier**: Professional customer communication
* **delegator**: Intelligent escalation decisions
### **Workflow Pattern**
1. Customer query analysis and parsing (`{{ customer_query }}`)
2. Parallel investigation using database and GitHub agents
3. Solution assessment and escalation evaluation
4. Customer communication and follow-up coordination
This architecture ensures comprehensive support coverage while maintaining clear separation of concerns and specialized expertise in each domain.
# Evaluator-Optimizer Workflow
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/evaluator-optimizer
This example demonstrates the Evaluator-Optimizer pattern from Anthropic's article
![]()
## Overview
In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop.
This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. This is analogous to the iterative writing process a human writer might go through when producing a polished document.
## When to use
This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. It works best when LLM responses can be demonstrably improved through feedback, and when an LLM can provide meaningful evaluation of that feedback.
## Using evaluator-optimizer in Latitude
```markdown Literary Translation with Iterative Refinement theme={null}
---
provider: openai
model: gpt-4.1
temperature: 0.3
maxSteps: 8
---
You are an expert literary translator specializing in {{ source_language }} to {{ target_language }} translation.
Translate the provided text while preserving:
- Literary style and tone
- Cultural nuances and references
- Poetic devices like rhythm, alliteration, or wordplay where possible
- Author's intended meaning and emotional impact
Source text ({{ source_language }}): {{ source_text }}
Context: {{ context }}
Please provide a literary translation that captures both the meaning and artistic qualities of the original text.
You are a literary translation critic with expertise in both {{ source_language }} and {{ target_language }} literature.
Evaluate the translation for accuracy, style, cultural sensitivity, and preservation of literary qualities.
Original text ({{ source_language }}): {{ source_text }}
Translation to evaluate: {{ initial_translation.translation }}
Translator's notes: {{ initial_translation.translation_notes }}
Context: {{ context }}
Please provide detailed feedback on this translation, focusing on:
1. Accuracy of meaning and cultural references
2. Preservation of literary style and tone
3. Natural flow in the target language
4. Treatment of literary devices (metaphors, wordplay, rhythm)
5. Overall readability and impact
Identify specific areas for improvement and provide concrete suggestions.
{{ if evaluation_feedback.needs_revision }}
You are an expert literary translator. Revise your translation based on the detailed feedback provided,
addressing the specific issues while maintaining the overall quality of your work.
Original text ({{ source_language }}): {{ source_text }}
Your initial translation: {{ initial_translation.translation }}
Detailed feedback: {{ evaluation_feedback.specific_feedback }}
Priority areas for revision: {{ evaluation_feedback.priority_areas }}
Please revise the translation to address the feedback, particularly focusing on the priority areas.
Explain what changes you made and why.
You are a literary translation critic. Evaluate the revised translation to assess whether the feedback
was successfully addressed and if further revision is needed.
Original text ({{ source_language }}): {{ source_text }}
Initial translation: {{ initial_translation.translation }}
Revised translation: {{ revised_translation.translation }}
Changes made: {{ revised_translation.revision_notes }}
Previous feedback: {{ evaluation_feedback.specific_feedback }}
Assess whether the revision successfully addressed the feedback and if the translation is now ready.
{{ endif }}
You are a translation project coordinator. Provide a final summary of the translation process and deliver the best version.
Translation project summary:
Original text: {{ source_text }}
Initial translation: {{ initial_translation.translation }}
{{ if evaluation_feedback.needs_revision }}
Feedback provided: {{ evaluation_feedback.specific_feedback }}
Revised translation: {{ revised_translation.translation }}
Final evaluation: {{ final_evaluation.final_assessment }}
{{ else }}
Initial evaluation: {{ evaluation_feedback.overall_quality }} - no revision needed
{{ endif }}
Please provide:
1. The final recommended translation
2. A summary of the translation process and any iterations
3. Key insights about the translation challenges and how they were addressed
4. Confidence level in the final result
```
This pattern demonstrates the power of iterative refinement through structured evaluation and optimization cycles, particularly valuable for complex tasks requiring high-quality outputs where initial attempts can be systematically improved through expert feedback.
# Orchestrator-Workers Workflow
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/orchestrator-workers
This example demonstrates the Orchestrator-Workers pattern from Anthropic's article
![]()
## Overview
In the orchestrator-workers workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.
The key difference from parallelization is its flexibility—subtasks aren't pre-defined, but determined by the orchestrator based on the specific input.
## When to use
This workflow is well-suited for complex tasks where you can't predict the subtasks needed (in research, for example, the specific sources to investigate and the nature of analysis required depend on what information is discovered along the way).
## Using orchestrator-workers in Latitude
```markdown Research Intelligence Report theme={null}
---
provider: openai
model: gpt-4.1
temperature: 0.3
maxSteps: 15
---
You are a research orchestrator. Analyze the research question and break it down into specific, targeted search and analysis subtasks.
Consider what types of sources need to be searched, what information needs to be gathered, what analysis is required, and how findings should be verified.
Research question: {{ research_query }}
Context and background: {{ research_context }}
Break this down into specific research subtasks that can be handled by specialized workers. Consider:
1. What web sources need to be searched (news, official sites, reports)
2. What academic or technical sources should be consulted
3. What expert analysis or domain-specific investigation is needed
4. How information should be synthesized and cross-referenced
5. What fact-checking and verification is required
/* NOTE: We initalize a list of possible responses */
{{ responses = [] }}
{{ for subtask in research_plan.subtasks }}
{{ if subtask.type == "web_search" }}
You are a web research specialist. Search and analyze web sources to gather comprehensive, current information on the specified topic.
Subtask: {{ subtask.description }}
Research context: {{ research_query }}
Conduct thorough web research and provide:
1. Key findings from authoritative sources
2. Recent developments and trends
3. Different perspectives and viewpoints
4. Relevant statistics and data points
5. Source credibility assessment
6. Information gaps or conflicting reports
Focus on finding reliable, up-to-date information from reputable sources.
{{ responses.push({ type: subtask.type, result: response }) }}
{{ endif }}
{{ if subtask.type == "academic_search" }}
You are an academic research specialist. Analyze scholarly sources, research papers, and technical documentation related to the topic.
Subtask: {{ subtask.description }}
Research context: {{ research_query }}
Analyze academic and technical sources to provide:
1. Peer-reviewed research findings
2. Theoretical frameworks and methodologies
3. Technical specifications or standards
4. Research gaps and ongoing studies
5. Expert opinions and consensus views
6. Historical context and evolution of understanding
{{ responses.push({ type: subtask.type, result: response }) }}
{{ endif }}
{{ if subtask.type == "expert_analysis" }}
You are a domain expert analyst. Provide deep, specialized analysis of the topic drawing on domain-specific knowledge and expertise.
Subtask: {{ subtask.description }}
Research context: {{ research_query }}
Provide expert analysis including:
1. Technical interpretation of findings
2. Industry context and implications
3. Risk assessment and considerations
4. Best practices and recommendations
5. Future trends and predictions
6. Critical evaluation of available information
{{ responses.push({ type: subtask.type, result: response }) }}
{{ endif }}
{{ if subtask.type == "data_synthesis" }}
You are a data synthesis specialist. Integrate and analyze information from multiple sources to identify patterns, relationships, and insights.
Subtask: {{ subtask.description }}
Research context: {{ research_query }}
Synthesize information to provide:
1. Cross-source pattern identification
2. Correlation and causation analysis
3. Trend identification and projections
4. Comparative analysis between sources
5. Unified timeline or framework
6. Key insights and takeaways
{{ responses.push({ type: subtask.type, result: response }) }}
{{ endif }}
{{ if subtask.type == "fact_verification" }}
You are a fact-checking specialist. Verify claims, cross-reference sources, and assess the reliability of information gathered.
Subtask: {{ subtask.description }}
Research context: {{ research_query }}
Conduct fact-checking and provide:
1. Verification of key claims and statistics
2. Source reliability assessment
3. Identification of conflicting information
4. Confidence levels for different findings
5. Recommendations for additional verification
6. Red flags or questionable sources
{{ responses.push({ type: subtask.type, result: response }) }}
{{ endif }}
{{ endfor }}
You are the research orchestrator. Synthesize all research findings into a comprehensive, well-structured intelligence report.
Original research question: {{ research_query }}
Research plan executed. Worker findings:
{{ for response in responses }}
{{ response.type }}:
{{ response.result }}
{{ endfor }}
Synthesize these findings into a comprehensive research report including:
1. Executive summary of key findings
2. Detailed analysis organized by topic/theme
3. Source evaluation and credibility assessment
4. Conflicting information and limitations
5. Actionable insights and recommendations
6. Areas requiring further investigation
7. Complete source bibliography with credibility ratings
```
Note the use of [isolate steps](/promptl/advanced/chains#isolating-steps) in all of the steps. Doing this we have a clean final synthesis of the workers' results, without any interference from previous steps. This is crucial in orchestrator-workers workflows, as each worker's output needs to be treated independently before the final synthesis.
This pattern is particularly effective when you have complex research questions that require different types of expertise and where the specific information sources can't be predetermined, allowing the orchestrator to adapt the research strategy based on what information is discovered during the investigation.
# Building Effective Agents
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/overview
Based on Anthropic's guide to building effective AI agents, focusing on simplicity, composability, and practical patterns.
## Introduction
Building effective AI agents requires understanding when and how to add complexity to your LLM applications. According to Anthropic's experience working with dozens of teams across industries, the most successful agent implementations use simple, composable patterns rather than complex frameworks.
We enjoyed reading [building effective AI agents](https://www.anthropic.com/engineering/building-effective-agents) by Anthropic's engineering team. So we adapted the key points to work with Latitude projects.
## Core Principles
Find the simplest solution possible and only increase complexity when needed. This might mean not building agentic systems at all. Often, optimizing single LLM calls with retrieval and in-context examples is sufficient.
Agentic systems often trade latency and cost for better task performance. Consider when this trade-off makes sense for your use case.
* **Workflows** offer predictability and consistency for well-defined tasks
* **Agents** are better when flexibility and model-driven decision-making are needed at scale
## The Augmented LLM (Foundation)
The basic building block is an LLM enhanced with:
* **Retrieval**: Access to external information with [Latitude tools](/guides/prompt-manager/latitude-tools) and third-party [MCP integrations](/guides/prompt-manager/mcp-integrations)
* **Tools**: Ability to perform actions with [calling LLM tools](/guides/prompt-manager/tools)
* **Memory**: Context retention across interactions with [RAGs](/examples/techniques/retrieval-augmented-generation)
# What are Agents?
Anthropic categorizes agentic systems into two main types:
Systems where LLMs and tools are orchestrated through predefined code paths
Systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks
## Workflow Patterns
An example of a workflow pattern where tasks are decomposed into sequential steps, each step building on the previous one.
Classifies input and directs it to specialized follow-up tasks.
LLMs can sometimes work simultaneously on a task and have their outputs aggregated programmatically
A central LLM dynamically breaks down tasks, delegates to worker LLMs, and synthesizes results.
One LLM generates responses while another provides evaluation and feedback in a loop.
## Autonomous Agents
Agents operate independently using tools based on environmental feedback in loops. They're ideal for open-ended problems where you can't predict the required number of steps or hardcode a fixed path.
## Key Takeaways
Success in the LLM space isn't about building the most sophisticated system—it's about building the right system for your needs. Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.
The most effective approach is to:
1. Begin with the simplest possible solution
2. Measure performance rigorously
3. Add complexity only when it demonstrably improves outcomes
4. Focus on clear tool design and transparent agent behavior
5. Test extensively in sandboxed environments with appropriate guardrails
# Prompt Chaining Workflow
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/prompt-chaining
This example demonstrates the Prompt Chaining pattern from Anthropic's article
![]()
## Overview
Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks on any intermediate steps to ensure that the process is still on track.
This workflow trades off latency for higher accuracy by making each LLM call an easier, more focused task.
## When to use
Tasks that can be cleanly decomposed into fixed subtasks.
## Using prompt chaining in Latitude
```markdown Marketing Copy theme={null}
---
provider: openai
model: gpt-4.1
temperature: 0.7
---
You are a creative marketing copywriter. Create compelling marketing copy for the given product.
Create marketing copy for: {{ product_description }}
Target audience: {{ target_audience }}
Tone: {{ tone }}
Length: {{ length }}
You are a professional translator with marketing expertise.
Translate the provided marketing copy while maintaining its persuasive impact and cultural relevance.
Translate the following marketing copy to {{ target_language }}:
{{ marketing_copy }}
Ensure the translation:
1. Maintains the original tone and persuasive impact
2. Adapts cultural references appropriately
3. Uses marketing language natural to {{ target_language }} speakers
You are a quality assurance specialist. Review both the original and translated marketing copy to ensure quality and consistency.
Review the marketing copy creation and translation process:
Original copy:
{{ marketing_copy }}
Translated copy:
{{ translation }}
Provide:
1. Quality assessment (1-10)
2. Any improvements needed
3. Final recommendation
```
This pattern is particularly effective when you have a clear process that can be broken down into logical stages, and when the quality benefits of step-by-step processing outweigh the increased latency and cost.
# Parallelization Workflow
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/prompt-parallelization
This example demonstrates the Parallelization pattern from Anthropic's article
![]()
## Overview
Parallelization allows LLMs to work simultaneously on a task and have their outputs aggregated programmatically. This workflow manifests in two key variations:
* **Sectioning**: Breaking a task into independent subtasks run in parallel
* **Voting**: Running the same task multiple times to get diverse outputs
## Examples in Latitude
### Sectioning
```markdown Content Generation with Guardrails theme={null}
---
provider: openai
model: gpt-4.1
temperature: 0.7
---
You are a creative content writer. Generate engaging, informative content based on the user's request.
Focus on creating valuable, well-structured content that serves the user's needs.
Create content for: {{ content_topic }}
Target audience: {{ target_audience }}
Content type: {{ content_type }}
Tone: {{ desired_tone }}
Length: {{ target_length }}
You are a content safety specialist. Review the provided content for:
- Inappropriate language or content
- Potential harm or misinformation
- Compliance with content policies
- Professional standards
Be thorough but fair in your assessment.
Review this content for safety and appropriateness:
{{ content_generation }}
Provide a detailed safety assessment including:
1. Overall safety evaluation
2. Specific concerns (if any)
3. Recommendation for next steps
You are a content quality specialist. Evaluate the content for:
- Clarity and readability
- Relevance to target audience
- Structure and organization
- Achievement of stated goals
Focus on constructive, actionable feedback.
Evaluate the quality of this content:
{{ content_generation }}
Original requirements:
- Topic: {{ content_topic }}
- Audience: {{ target_audience }}
- Type: {{ content_type }}
- Tone: {{ desired_tone }}
- Length: {{ target_length }}
Provide detailed quality assessment and suggestions for improvement.
You are a content coordinator. Based on the parallel assessments, provide a final recommendation and any necessary revisions.
Content Review Summary:
Original Content:
{{ content_generation }}
Safety Assessment:
- Safe: {{ safety_screening.is_safe }}
- Safety Score: {{ safety_screening.safety_score }}/10
- Concerns: {{ safety_screening.concerns }}
- Recommendation: {{ safety_screening.recommendation }}
Quality Assessment:
- Quality Score: {{ quality_assessment.quality_score }}/10
- Meets Requirements: {{ quality_assessment.meets_requirements }}
- Strengths: {{ quality_assessment.strengths }}
- Improvements: {{ quality_assessment.improvements }}
Based on these parallel assessments, provide:
1. Final recommendation (approve/revise/reject)
2. If revision needed, provide specific guidance
3. Summary of next steps
{{ if safety_screening.recommendation == "reject" or safety_screening.safety_score < 7 }}
Note: Content has safety concerns that must be addressed.
{{ endif }}
{{ if quality_assessment.quality_score < 7 or not quality_assessment.meets_requirements }}
Note: Content quality needs improvement to meet requirements.
{{ endif }}
```
### Voting
```markdown Code Security Review theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.3
---
You are a security expert specializing in input validation and injection attacks. Review the code for:
- SQL injection vulnerabilities
- Cross-site scripting (XSS) risks
- Input validation issues
- Authentication and authorization flaws
Be thorough and specific in your analysis.
Review this code for security vulnerabilities:
{{ code_language }}
{{ code_content }}
Focus particularly on input validation, injection attacks, and authentication issues.
Provide specific line numbers and detailed explanations for any issues found.
You are a security expert specializing in cryptography and data protection. Review the code for:
- Weak encryption or hashing
- Insecure data storage
- Privacy violations
- Cryptographic vulnerabilities
Focus on data protection and cryptographic security.
Review this code for security vulnerabilities:
{{ code_language }}
{{ code_content }}
Focus particularly on cryptography, data protection, and privacy concerns.
Provide specific line numbers and detailed explanations for any issues found.
You are a security expert specializing in application architecture and business logic. Review the code for:
- Business logic flaws
- Access control issues
- Session management problems
- API security concerns
Focus on architectural and business logic security.
Review this code for security vulnerabilities:
{{ code_language }}
{{ code_content }}
Focus particularly on business logic, access control, and architectural security concerns.
Provide specific line numbers and detailed explanations for any issues found.
You are a senior security architect. Consolidate the multiple security reviews into a comprehensive assessment.
Security Review Consolidation:
Code reviewed:
{{ code_language }}
{{ code_content }}
Review 1 (Input Validation & Injection Focus):
- Vulnerabilities found: {{ security_review_1.vulnerabilities_found }}
- Severity: {{ security_review_1.severity_level }}
- Confidence: {{ security_review_1.confidence }}
- Issues: {{ security_review_1.issues }}
Review 2 (Cryptography & Data Protection Focus):
- Vulnerabilities found: {{ security_review_2.vulnerabilities_found }}
- Severity: {{ security_review_2.severity_level }}
- Confidence: {{ security_review_2.confidence }}
- Issues: {{ security_review_2.issues }}
Review 3 (Architecture & Business Logic Focus):
- Vulnerabilities found: {{ security_review_3.vulnerabilities_found }}
- Severity: {{ security_review_3.severity_level }}
- Confidence: {{ security_review_3.confidence }}
- Issues: {{ security_review_3.issues }}
Based on these three specialized security reviews, provide:
1. **Consolidated Security Assessment**:
- Overall security rating
- Highest priority issues
- Common themes across reviews
2. **Voting Analysis**:
- Issues identified by multiple reviewers (high confidence)
- Issues identified by single reviewers (require further investigation)
- Consensus on severity levels
3. **Recommendations**:
- Immediate actions required
- Priority order for addressing issues
- Additional security measures to consider
4. **Next Steps**:
- Code changes needed
- Further testing recommendations
- Deployment security considerations
```
This pattern is particularly effective when you need either speed (through parallelization) or confidence (through multiple expert opinions), and when the cost of additional LLM calls is justified by the improved quality or reduced risk.
# Routing workflow
Source: https://docs-v1.latitude.so/examples/cases/building-effective-agents/prompt-routing
Classifies input and directs it to specialized follow-up tasks.
![]()
## Overview
Routing classifies an input and directs it to a specialized follow-up task. This workflow allows for separation of concerns and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs.
The routing pattern works by having an initial classifier that determines what type of request or input it's dealing with, then routes it to the appropriate specialized handler. This enables you to create highly optimized prompts for each specific scenario rather than trying to handle all cases with a single, more generic prompt.
## When to use
Routing works well for complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately, either by an LLM or a more traditional classification model/algorithm.
## Using prompt routing in Latitude
```markdown Customer Inquiry Routing theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.1
---
You are a customer service classifier. Analyze the customer inquiry and classify it into one of these categories:
- general_inquiry: General questions about the company, policies, or services
- refund_request: Customer wants to return a product or get a refund
- technical_support: Technical issues with products or services
- billing_question: Questions about charges, payments, or billing
- product_info: Questions about specific products or features
Classify the inquiry and provide your confidence level.
Customer inquiry: {{ customer_message }}
{{ if classification.category == "general_inquiry" }}
You are a friendly general customer service representative. Provide helpful, accurate information about the company's policies and services. Keep responses warm and professional.
Customer inquiry: {{ customer_message }}
Classification confidence: {{ classification.confidence }}
Reasoning: {{ classification.reasoning }}
Please provide a helpful response to this general inquiry.
{{ endif }}
{{ if classification.category == "refund_request" }}
You are a refund specialist. Guide customers through the refund process clearly and empathetically. Ask for necessary information like order number and reason for return.
Customer refund request: {{ customer_message }}
Classification confidence: {{ classification.confidence }}
Please help the customer with their refund request. Ask for order details if not provided.
{{ endif }}
{{ if classification.category == "technical_support" }}
You are a technical support specialist. Provide step-by-step troubleshooting guidance. Ask clarifying questions about the issue and the customer's setup.
Technical support request: {{ customer_message }}
Classification confidence: {{ classification.confidence }}
Please provide technical assistance for this issue.
{{ endif }}
{{ if classification.category == "billing_question" }}
You are a billing specialist. Help customers understand their charges and payment options. Be clear about billing policies and next steps.
Billing inquiry: {{ customer_message }}
Classification confidence: {{ classification.confidence }}
Please assist with this billing question.
{{ endif }}
{{ if classification.category == "product_info" }}
You are a product specialist with deep knowledge of our product catalog. Provide detailed, accurate product information and help customers make informed decisions.
Product information request: {{ customer_message }}
Classification confidence: {{ classification.confidence }}
Please provide detailed product information to help the customer.
{{ endif }}
```
The routing pattern is particularly powerful when combined with other Latitude features like agents and tools, allowing you to create sophisticated systems that automatically adapt their behavior based on the type of input they receive.
# Content moderation system
Source: https://docs-v1.latitude.so/examples/cases/content-moderation-system
Learn how to build a content moderation system that can analyze user-generated content and provide feedback on its appropriateness.
You can play with this example in the Latitude Playground.
## Overview
In this example, we will create a content moderation system that can analyze user-generated content and provide feedback on its appropriateness. The agent uses subagents to handle different aspects of content moderation efficiently.
## Multi-Agent Architecture
The system uses specialized subagents for different responsibilities:
* **main**: Coordinates the moderation process by dispatching content to all subagents, gathering their evaluations, and generating the final decision based on their collective input.
* **rule\_checker**: Runs deterministic, rule-based checks—such as profanity filters or length validation—against the content, ensuring compliance with explicitly defined policies.
* **toxicity\_analyzer**: Analyzes content for toxicity and subtle forms of harm like harassment, hate speech, or threats, taking context and intent into account, even in ambiguous or nuanced cases.
* **safety\_scorer**: Calculates comprehensive risk and safety scores for the content, highlighting any areas of concern, escalation potential, or need for human review.
All the tools used in the sub-agents have to be defined in the main prompt.
## The prompts
```markdown main theme={null}
---
provider: google
model: gemini-1.5-flash
temperature: 0.2
type: agent
agents:
- rule_checker
- toxicity_evaluator
- safety_scorer
tools:
- check_profanity_filter:
description: Detect explicit language and banned words in content
parameters:
type: object
properties:
content:
type: string
description: The content to check for profanity
content_type:
type: string
description: Type of content (text, comment, post, etc.)
required: [content]
- validate_content_length:
description: Ensure content meets platform length guidelines
parameters:
type: object
properties:
content:
type: string
description: The content to validate
content_type:
type: string
description: Type of content to determine length limits
required: [content, content_type]
- scan_for_patterns:
description: Identify suspicious patterns and spam indicators
parameters:
type: object
properties:
content:
type: string
description: The content to scan for patterns
pattern_types:
type: array
items:
type: string
description: Types of patterns to look for (spam, repetitive, etc.)
required: [content]
schema:
type: object
properties:
decision:
type: string
enum: [approve, flag, reject]
description: The final moderation decision
confidence:
type: number
minimum: 0
maximum: 1
description: Confidence score for the decision
reasoning:
type: string
description: Brief explanation for the decision
violations:
type: array
items:
type: string
description: List of policy violations found
recommended_action:
type: string
description: Specific action to take
required: [decision, confidence, reasoning]
---
You are the main coordinator for an intelligent content moderation system. Your role is to orchestrate the moderation pipeline by delegating tasks to specialized agents and making final moderation decisions.
You have access to three specialized agents:
1. rule_checker - Applies programmatic rules and filters
2. toxicity_evaluator - Uses LLM-as-judge for nuanced content analysis
3. safety_scorer - Calculates safety metrics and risk scores
Process each content submission through all agents and synthesize their outputs into a final moderation decision.
Content to moderate: {{ content }}
Content type: {{ content_type }}
Platform context: {{ platform_context }}
```
```markdown rule_checker theme={null}
---
provider: OpenAI
model: gpt-4o-mini
temperature: 0.1
type: agent
schema:
type: object
properties:
rule_violations:
type: array
items:
type: string
description: List of violated rules
severity:
type: string
enum: [low, medium, high]
description: Overall severity level
details:
type: string
description: Specific findings from rule checks
passed_basic_filters:
type: boolean
description: Whether content passed basic filtering
required: [rule_violations, severity, passed_basic_filters]
---
You are a rule-based content filter that applies programmatic rules to detect policy violations. You focus on deterministic, rule-based checks that can be applied consistently.
Use the provided tools to check content against various rules and filters. Be thorough but efficient in your rule application.
Content: {{ content }}
Content type: {{ content_type }}
```
```markdown safety_scorer theme={null}
---
provider: anthropic
model: claude-3-5-sonnet-20241022
temperature: 0.1
type: agent
schema:
type: object
properties:
safety_scores:
type: object
properties:
immediate_harm_risk:
type: integer
minimum: 0
maximum: 100
description: Risk of immediate harm (higher = more risk)
community_impact:
type: integer
minimum: 0
maximum: 100
description: Risk to community health (higher = more risk)
policy_violation_severity:
type: integer
minimum: 0
maximum: 100
description: Severity of policy violations (higher = more severe)
escalation_potential:
type: integer
minimum: 0
maximum: 100
description: Likelihood of escalation (higher = more likely)
context_sensitivity:
type: integer
minimum: 0
maximum: 100
description: Context-specific risk (higher = more risk)
required: [immediate_harm_risk, community_impact, policy_violation_severity, escalation_potential, context_sensitivity]
overall_risk_score:
type: integer
minimum: 0
maximum: 100
description: Weighted average risk score
confidence_interval:
type: array
items:
type: integer
minItems: 2
maxItems: 2
description: Lower and upper bounds of confidence interval
requires_human_review:
type: boolean
description: Whether human review is recommended
monitoring_level:
type: string
enum: [none, light, heavy]
description: Suggested monitoring level
risk_factors:
type: array
items:
type: string
description: Specific factors contributing to risk
required: [safety_scores, overall_risk_score, requires_human_review, monitoring_level]
---
You are a safety metrics specialist that calculates comprehensive risk scores for content moderation decisions. You use negative evaluation scoring for safety-critical metrics, meaning higher scores indicate greater safety concerns.
Your role is to:
- Calculate risk scores across multiple safety dimensions
- Provide confidence intervals for moderation decisions
- Flag edge cases requiring human review
- Generate quantitative safety metrics
Use negative scoring where higher values indicate higher risk/safety concerns.
Content: {{ content }}
Rule checker results: {{ rule_results }}
Toxicity evaluation: {{ toxicity_results }}
```
```markdown toxicity_evaluator theme={null}
---
provider: anthropic
model: claude-3-5-sonnet-20241022
temperature: 0.3
type: agent
schema:
type: object
properties:
toxicity_detected:
type: boolean
description: Whether toxicity was detected
toxicity_type:
type: string
enum: [harassment, hate_speech, threat, other, none]
description: Type of toxicity found
severity_score:
type: integer
minimum: 1
maximum: 10
description: Severity rating from 1-10
confidence:
type: number
minimum: 0
maximum: 1
description: Confidence in the assessment
reasoning:
type: string
description: Detailed explanation of the assessment
context_factors:
type: array
items:
type: string
description: Factors that influenced the decision
required: [toxicity_detected, severity_score, confidence, reasoning]
---
You are an expert content safety evaluator specializing in detecting toxicity, harassment, and harmful content. You excel at understanding context, nuance, and implicit harm that rule-based systems might miss.
Evaluate content for:
- Contextual toxicity (sarcasm, implicit harm)
- Cultural sensitivity issues
- Intent classification (harassment, hate speech, threats)
- Severity assessment on a graduated scale
Consider context, cultural nuances, and potential for harm. Be especially careful about edge cases and borderline content.
Content: {{ content }}
Platform context: {{ platform_context }}
User history: {{ user_history }}
```
## Breakdown
Let's break down the example step by step to understand how it works.
The main prompt acts as the central coordinator. It receives user-generated content, delegates the moderation tasks to the specialized subagents, aggregates their results, and produces a structured final decision with confidence and reasoning.
The rule\_checker agent checks for clear, rule-based violations—like banned words, excessive length, or explicit policy breaches—using programmatic filters and deterministic logic.
The toxicity\_analyzer (or toxicity\_evaluator) uses advanced AI to evaluate whether the content contains toxicity, harassment, hate speech, or other forms of harmful language, considering nuance, context, and potential for implicit harm.
The safety\_scorer calculates various risk scores for the content, such as immediate harm, community impact, and escalation risk, and determines whether the situation requires human review or additional monitoring.
The main agent synthesizes all subagent outputs, weighing rule violations, toxicity, and risk scores to make a final moderation decision. This decision includes a confidence score, explanation, and recommended action for handling the content.
## Structured Output
Main prompt returns a [structured output](/guides/prompt-manager/json-output) because the moderation process must be machine-readable and reliable, allowing easy integration with other systems and clear auditing of every moderation decision.
## Code
In the code we prepared 4 cases of possible user input from different sources. In github you [have the code](https://github.com/latitude-dev/latitude-llm/blob/main/examples/package.json#L34) but the idea is to launch this code with different types of possible input to see how it works.
```bash theme={null}
pnpm run ts:cases:content_moderation --type toxicity
```
The important part is that you can see the use of tools. The tools defined in the code are used to respond to the tools defined in the main prompt. These kind of tools are on your control and are things that usually don't need an LLM or AI to be responded like measure the length of the text of if the the text contains words that yout put in a blacklist.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
function printStatus(message: string) {
process.stdout.write('\r' + message + ' '.repeat(30)) // Pad to overwrite old text
}
const MODEREATION_TYPES = {
allGood: 'allGood',
spam: 'spam',
toxicity: 'toxicity',
badWords: 'badWords',
} as const
type Content = {
content: string
content_type: string // post, comment, message, etc.
platform_context: string // e.g., 'social_media', 'email', 'forum'
}
const CONTENT_BY_MODERATION_TYPE: Record<
keyof typeof MODEREATION_TYPES,
Content
> = {
[MODEREATION_TYPES.allGood]: {
content: 'The world is a beautiful place and I love it.',
platform_context: 'Twitter',
content_type: 'post',
},
[MODEREATION_TYPES.spam]: {
content: 'Nigerian Prince wants to transfer money to you.',
platform_context: 'email',
content_type: 'Email message',
},
[MODEREATION_TYPES.toxicity]: {
content: 'You are an idiot and nobody likes you.',
platform_context: 'Reddit',
content_type: 'comment',
},
[MODEREATION_TYPES.badWords]: {
content: 'I hate Tomatoes because they could kill me',
platform_context: 'Instagram',
content_type: 'post',
},
}
type Tools = {
check_profanity_filter: { content: string; content_type: string }
validate_content_lenght: { content: string; content_type: string }
scan_for_patterns: {
content: string
content_type: 'spam' | 'phishing' | 'repititive'
}
}
async function run({
moderationType,
}: {
moderationType: keyof typeof MODEREATION_TYPES
}) {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
try {
const result = await sdk.prompts.run(
'content-moderation-system/main',
{
parameters: CONTENT_BY_MODERATION_TYPE[moderationType],
stream: true,
onEvent: (event) => {
printStatus(`Generating response... ${event.data.type}`)
},
tools: {
check_profanity_filter: async ({ content }) => {
if (content.includes('Tomatoes')) {
return {
content_type: 'badWords',
description: 'Content contains prohibited words.',
}
}
return {
content_type: 'ok',
description:
'Content is clean and does not contain prohibited words.',
}
},
validate_content_lenght: async ({ content: _c }) => {
return 'ok' // Assuming content length is valid for this example
},
scan_for_patterns: async ({ content }) => {
if (moderationType === 'spam') {
if (content.includes('Nigerian Prince')) {
return {
content_type: 'spam',
description:
'This content appears to be spam, possibly a scam involving a Nigerian Prince.',
}
}
}
return {
content_type: 'ok',
description:
'Content is clean and does not match any known patterns.',
}
},
},
},
)
const response = result.response
console.log('Agent Response: \n', JSON.stringify(response, null, 2))
} catch (error) {
console.error('Error: ', error.message, '\nStack:', error.stack)
}
}
const [, , ...args] = process.argv
const moderationType = MODEREATION_TYPES[args[1]]
if (!moderationType) {
console.error('Invalid moderation type. Please use one of the following: \n')
Object.keys(MODEREATION_TYPES).forEach((type) => {
console.error(`pnpm run ts:cases:content_moderation --type ${type} \n`)
})
process.exit(1)
}
run({ moderationType })
```
## Resources
* [Custom Tools](/guides/prompt-manager/tools) - How to integrate with customer databases and CRM systems
* [Tool call SDK example](/examples/sdk/run-prompt-with-tools) - A simple example of how to run a prompt with tools with Latitude SDK.
* [JSON Schema Output](/guides/prompt-manager/json-output) - Ensuring consistent response formatting
# Customer Support Email Generator
Source: https://docs-v1.latitude.so/examples/cases/customer-support-email
Learn how to build an intelligent customer support agent that generates personalized email responses
You can play with this example in the Latitude Playground.
## Overview
In this example, we will create a Dynamic Customer Support Email Generator that can analyze customer queries, gather relevant customer information, and generate personalized, professional email responses. The agent uses subagents to handle different aspects of customer support efficiently.
## Multi-Agent Architecture
The system uses specialized subagents for different responsibilities:
* **main**: Orchestrates the process and makes decisions
* **customer\_researcher**: Gathers customer data and context
* **email\_composer**: Creates the actual email response
All the tools used in the sub-agents have to be defined in the main prompt.
## The prompts
```markdown main theme={null}
---
provider: google
model: gemini-1.5-flash
type: agent
tools:
- get_customer_details:
description: Retrieves customer account information
parameters:
type: object
properties:
email:
type: string
description: Customer email address
required:
- email
- get_order_history:
description: Gets recent order history for the customer
parameters:
type: object
properties:
customer_id:
type: string
description: Customer ID
required:
- customer_id
- check_known_issues:
description: Checks for known issues related to the query
parameters:
type: object
properties:
issue_keywords:
type: array
items:
type: string
description: Keywords from the customer query
required:
- issue_keywords
agents:
- customer_researcher
- email_composer
temperature: 0.3
schema:
type: object
properties:
needs_clarification:
type: boolean
description: Whether the query needs clarification from the customer
clarification_questions:
type: array
items:
type: string
description: Questions to ask the customer for clarification
email_response:
type: object
properties:
subject:
type: string
description: Email subject line
body:
type: string
description: Email body content
description: The generated email response
required:
- needs_clarification
---
You're an intelligent customer support coordinator. Your task is to analyze customer queries and generate appropriate email responses
You have two specialized agents available:
- A customer researcher that can gather customer information and context
- An email composer that creates professional, personalized responses
You must proceed with the following steps, one message at a time:
- Analyze the customer query to understand the issue and sentiment
- Determine if you need more information about the customer or their issue
- If the query is unclear or missing context, ask clarifying questions
- Use the customer researcher to gather relevant customer information
- Use the email composer to create a personalized response
- Review the response for tone, accuracy, and completeness
Handle edge cases like:
- Angry or frustrated customers (use empathetic tone)
- Technical issues (gather specific details)
- Billing inquiries (verify account information)
- Feature requests (acknowledge and route appropriately)
Customer Email: {{ customer_email }}
Customer Query: {{ customer_query }}
Priority Level: {{ priority_level }}
First, analyze the customer query and determine what information you need.
```
```markdown customer_researcher theme={null}
---
provider: OpenAI
model: gpt-4.1
type: agent
description: Researches customer information and gathers relevant context for support queries
---
You're a customer research specialist. Your job is to gather comprehensive information about customers and their issues to enable personalized support.
For each research request, you should:
1. Extract the customer email and any identifiers from the query
2. Gather customer account details and history
3. Check for known issues or patterns related to their query
4. Look for previous support interactions
5. Identify the customer's subscription level or account type
6. Note any special circumstances (VIP customer, recent issues, etc.)
Provide a detailed research report including:
- Customer profile and account status
- Relevant order/subscription history
- Known issues that might be related
- Recommended approach based on customer history
- Any red flags or special considerations
{{ research_request }}
Use all available tools to gather comprehensive customer information.
```
```markdown email_composer theme={null}
---
provider: anthropic
model: claude-3-sonnet-latest
type: agent
description: Composes professional, personalized customer support emails
temperature: 0.4
schema:
type: object
properties:
subject:
type: string
description: Professional email subject line
body:
type: string
description: Complete email body with proper formatting
tone_analysis:
type: string
description: Analysis of the tone used in the response
personalization_elements:
type: array
items:
type: string
description: Elements that make this response personalized
required:
- subject
- body
- tone_analysis
---
You're an expert email composer specializing in customer support communications. You create professional, empathetic, and personalized email responses.
Your email composition process:
1. Analyze the customer's emotional state and adjust tone accordingly
2. Use customer information to personalize the response
3. Address the specific issue with clear, actionable solutions
4. Include relevant account details when appropriate
5. Set proper expectations for resolution timelines
6. End with appropriate next steps and contact information
Email guidelines:
- Use the customer's name when available
- Reference specific account details or order numbers
- Match the urgency level to the customer's concern
- For technical issues: provide step-by-step solutions
- For billing issues: be precise about charges and dates
- For complaints: acknowledge, empathize, and provide solutions
- Always include a clear call-to-action
Tone variations:
- Standard: Professional and helpful
- Urgent: Immediate attention with expedited solutions
- Empathetic: Extra care for frustrated customers
- Technical: Detailed explanations for complex issues
Customer Information: {{ customer_info }}
Issue Details: {{ issue_details }}
Tone Required: {{ tone_required }}
Compose a professional email response using the provided information.
```
## Breakdown
Let's break down the example step by step to understand how it works.
#### Customer Context Gathering
The customer researcher agent uses custom tools to fetch relevant information:
```markdown theme={null}
- get_customer_details: Retrieves account information
- get_order_history: Gets purchase history
- check_known_issues: Identifies related problems
```
#### Intelligent Query Analysis
The main agent analyzes queries for:
* Emotional sentiment (angry, confused, urgent)
* Issue type (technical, billing, feature request)
* Information completeness
* Priority level
#### Personalized Response Generation
The email composer creates responses that:
* Use customer-specific information
* Match appropriate tone and urgency
* Include relevant account details
* Provide actionable solutions
#### Structured Output
Uses JSON schema to ensure consistent response format with subject, body, and metadata.
### Why This Multi-Agent Approach Works
Similar to the [Deep Search example](/examples/cases/deep-search), separating responsibilities prevents context bloat and improves performance:
1. **Customer researcher** focuses solely on data gathering
2. **Email composer** specializes in communication
3. **Main coordinator** handles decision-making and orchestration
This prevents any single agent from becoming overloaded with too many responsibilities while maintaining conversation context efficiency.
Looking at the prompts I implemented in the previous conversation, I chose different LLM providers strategically based on their specific strengths and the requirements of each component.
## Code
You can play with this example using the Latitude SDK.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
type Tools = {
get_customer_details: { email: string }
get_order_history: { customer_id: string }
check_known_issues: { issue_keywords: string[] }
}
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const response = await sdk.prompts.run('customer-support-email/main', {
parameters: {
customer_email: 'johndoe@gmail.com',
customer_query: 'My order is delayed and I want to know the status.',
priority_level: 'urgent',
},
tools: {
get_customer_details: async ({ email }) => {
return {
email,
name: 'John',
last_name: 'Doe',
customer_id: '12345',
}
},
get_order_history: async ({ customer_id }) => {
return {
customer_id,
orders: [
{
name: 'Nike Air Max 270',
status: 'Delivered',
date: '2023-01-01',
},
{
name: 'Adidas Ultraboost',
status: 'In Transit',
date: '2023-02-01',
},
],
}
},
check_known_issues: async ({ issue_keywords }) => {
if (issue_keywords.length === 0) {
return { issues: [] }
}
if (issue_keywords.includes('delay')) {
return {
issues: [
{
description:
'Known issue with delayed shipments due to supply chain disruptions.',
severity: 'high',
},
],
}
}
return { issues: [] }
},
},
})
console.log('RESPONSE', JSON.stringify(response, null, 2))
}
run()
```
## Provider Selection Rationale
1. **Fast Performance**: Designed for quick coordination tasks.
2. **Cost Effective**: Competitive pricing for simple tasks.
3. **JSON Support**: Good structured output capabilities
1. **Tool Integration**: OpenAI has excellent tool calling capabilities and strict compatibility mode for reliable function execution
2. **Data Processing**: GPT-4o excels at analyzing and synthesizing information from multiple sources
3. **Reasoning**: Better at complex reasoning tasks required for customer data analysis
1. **Writing Quality**: Anthropic models are particularly strong at generating high-quality, nuanced text
2. **Tone Control**: Superior at maintaining consistent professional tone and empathy
3. **Temperature**: Used `temperature: 0.4` for creative but controlled email generation
### Strategic Benefits
This multi-provider strategy optimizes for:
* **Cost**: Using cheaper models for coordination, expensive models only where needed
* **Performance**: Leveraging each provider's strengths (OpenAI for tools, Anthropic for writing)
* **Reliability**: Distributing risk across multiple providers
* **Quality**: Matching model capabilities to specific task requirements
This rationale might vary with the past of time because provider capabilities and pricing change frequently. We recomend to [evaluate your prompts](/guides/evaluations/overview)
Using Latitude is easy to switch between providers if needed. If you find that one provider's model is not performing as expected, you can quickly change the model in the prompt configuration without rewriting the entire agent logic. You can create your own providers check [provider documentation](/guides/getting-started/providers) for more information.
## Resources
* [Customer Support Best Practices](/guides/prompt-manager/prompt-best-practices) - Learn more about effective customer support prompting
* [Custom Tools](/guides/prompt-manager/tools) - How to integrate with customer databases and CRM systems
* [Tool call SDK example](/examples/sdk/run-prompt-with-tools) - A simple example of how to run a prompt with tools with Latitude SDK.
* [JSON Schema Output](/guides/prompt-manager/json-output) - Ensuring consistent response formatting
* [Configuring providers](/guides/getting-started/providers) - How to configure and use different LLM providers in Latitude
# Customer Support Quality Assurance
Source: https://docs-v1.latitude.so/examples/cases/customer-support-quality-assurance
Implement a comprehensive QA system for customer support responses using Rating-based LLM evaluation, Exact Match rules, and Manual review
Try out this agent setup in the Latitude Playground.
## Overview
This tutorial demonstrates how to build a quality assurance system for customer support responses using three specific Latitude evaluation types:
* **LLM-as-Judge**: Rating evaluation for helpfulness assessment
* **Programmatic Rules** with Exact Match for required information validation
* **Human-in-the-Loop** manual evaluation for customer satisfaction scoring
## The Prompt
This is the prompt that will be used to generate customer support responses. It is a simple prompt that takes a customer query and generates a response. It doesn't use a knowledge base or any additional information.
```markdown main theme={null}
---
provider: OpenAI
model: gpt-4.1
---
You are a helpful customer support agent. Respond to the customer inquiry below with empathy and provide a clear solution.
Customer inquiry: {{customer_message}}
Customer tier: {{tier}}
Product: {{product_name}}
Requirements:
- Always include the ticket number: {{ticket_number}}
- Address the customer by name if provided
- Provide specific next steps
- End with "Is there anything else I can help you with today?"
```
In this example, the prompt is very simple, but you could also upload documents to OpenAI and use their new [Responses API file search](https://platform.openai.com/docs/guides/tools-file-search). This implements a knowledge base search that can be used to find relevant information in the documents, so your responses to customer support queries can be based on actual documentation. However, this is out of the scope of this tutorial.
## The Evaluations
To create new evaluations, go to the evaluations tab in the Latitude Playground and click on "Add Evaluation".
![Evaluations]()
This is how we configure an LLM-as-Judge evaluation to assess the helpfulness of customer support responses.
This evaluation uses the Rating metric from the AI to assess response quality, with criteria such as **Assess how well the response follows the given instructions** and a 1-5 rating scale where 1 means **Not faithful, doesn't follow the instructions** and 5 means **Very faithful, follows the instructions**.
![LLM-as-Judge Rating Evaluation]()
An [Experiment](/guides/experiments/overview) is a way of running the prompt many times and validating, with this evaluation, if it passes the criteria.
Before creating the experiment, we need to create a dataset. Click on "Generate dataset".
![Experiment modal]()
A synthetic dataset is generated by the system to test the evaluation. It allows us to test the evaluation without having to create a real dataset. It sets columns for each parameter in our prompt.
![Generate dataset]()
Once we have the dataset, select it in the dataset selector and click "Run experiment".
![Run experiment]()
You can see how the columns in this dataset have to match the parameters in our prompt.
After running the experiment with 30 rows of the synthetic dataset you just created, you can see the results! The green counter shows the successful cases. Yellow represents results that failed the evaluation, and red means errors occurred during the experiment run.
![Experiment results]()
The goal of this evaluation is to ensure every response contains mandatory elements like ticket numbers and proper closing statements. Let's set it up.
This rule cannot be used with your real logs. It needs an **expected output** to match.
![Programmatic rule with exact match]()
We need to create another dataset, but this time it must have an **expected output** column.
You can use the same dataset but add a new column with the expected output. In this case, we want to ensure our prompt always responds with the sentence **Is there anything else I can help you with today?**
![Dataset with expected output]()
To configure this evaluation, we use a regular expression to ensure the customer support response contains a ticket number.
So in this case, we require:
1. The ticket number starts with `TCKT-`
2. Followed by 4 digits (`-\d{4}`)
This is the shape of our ticket column in the dataset.
![Dataset ticket format]()
Now we're ready to create this new evaluation.
![Regular expression modal]()
This step is the same as for the first evaluation. We create an experiment and see the results. In this case, we should see that the AI responded with the ticket number because it's part of our prompt. This is a basic check, but ensures future modifications to the prompt keep the ticket number.
Customer satisfaction involves nuanced judgment about tone, cultural sensitivity, and domain-specific accuracy that automated systems might miss, making it perfect for human evaluation.
![HITL Configuration]()
The first way to enable human evaluators to review responses is to give them access to Latitude's logs.
When they click on the logs in the right panel, now that we've configured the HITL evaluation, they will be able to assign a score from 1 to 5 as previously configured.
![HITL from Latitude Logs]()
Another way to add manual evaluations is to use the Latitude SDK. You
can see an example of [how to do it here](/examples/sdk/annotate-log).
One thing we didn't do when configuring the evaluation is to set a minimum score required to pass. Let's do it now: Go to the manual evaluation detail at the top right of the screen and click **Settings**.
![HITL min score configuration]()
Now our human evaluator has scored the responses and we can see the results in the experiment.
In the image, we see an evaluation with score `1` but in green. This was before we set the minimum score to `3`. The next one didn't pass and is shown in red.
![HITL results table]()
## Live Mode
We've done a lot of work so far. We set up four types of evaluations but only tested against synthetic data. Now we want to test our evaluations against real customer interactions—this is what we call **Live Mode**.
Let's set the **Helpfulness Assessment** evaluation to live mode. Go to the evaluation's detail, click the top right corner **Settings**, and at the bottom under **Advanced configuration**, you can see the **Evaluate live logs** toggle.
![Live logs configuration]()
We did the same for the **Contains Ticket Number** programmatic rule evaluation.
**Manual** evaluations can't be set to live mode because human evaluators review the responses manually after the AI responds to the customer. The **Required Information Validation** evaluation is also not suitable because it requires an expected output to match against the AI response.
![HITL results table]()
## Conclusion
By setting up a robust evaluation framework for customer support responses, we've learned how different types of automated and manual evaluations work together to ensure high-quality service. Automated LLM-based ratings help us assess response helpfulness at scale, while programmatic rules—like exact match and regular expressions—ensure critical information such as ticket numbers and required statements are always included. Human-in-the-loop (manual) evaluations provide the nuanced judgment that only real people can offer, especially for customer satisfaction and tone. Testing our system with both synthetic and real data (live mode) gives us confidence that our evaluations are both reliable and effective. Ultimately, these evaluations help us catch issues early, improve our AI prompts, and consistently deliver accurate and customer-friendly support—leading to better customer satisfaction and operational excellence.
## Resources
* [LLM-as-Judge Evaluation](/guides/evaluations/llm-as-judges) — How to use LLMs to evaluate responses
* [Programmatic Rule Evaluation](/guides/evaluations/programmatic-rules) — How to use programmatic rules to evaluate responses
* [Human-in-the-Loop Evaluation](/guides/evaluations/humans-in-the-loop) — How to use human evaluators to evaluate responses
* [Running Evaluations](/guides/evaluations/running-evaluations) — How to run evaluations against synthetic and live data
* [Datasets](/guides/datasets/overview) — How to create datasets for evaluations
# Deep Search
Source: https://docs-v1.latitude.so/examples/cases/deep-search
Learn how to build a Deep Search autonomous agent
You can play with this example in the Latitude Playground.
## Overview
In this example, we will create a Deep Search agent that can search for information autonomously on the web and provide answers to user queries. The agent will use the built-in Latitude tools to search and read content from the Internet.
## Prompts
```markdown main theme={null}
---
provider: openai
model: gpt-4o
type: agent
agents:
- researcher
temperature: 0.4
---
You're an autonomous AI agent. Your task will be to answer any of your user's
request.
Some questions may be too broad or generic. If you need more specifics or
additional information in order to correctly fulfill, you must ask the user
at any time. For example, when asking about a person or place, there may be
several results with the same name. In these kind of cases, it would be useful
to ask the user about more details for a better result.
You have an agent available that will perform a deep research about any topic
or query if you need to obtain any information. This agent does not share any
information or context between runs, so you will need to provide all context it
needs to perform an efficient research every time. You can use natural language
and questions to request information to this agent.
You must proceed with the following steps, one message at a time:
- Understand the user's request
- State what process you would follow in order to fulfill the request.
- Think about the information given from the user, and list all other
information you need to perform a detailed research about it.
- Stop the loop to ask the user specific questions to clarify the query
and gather more context.
- If proceeding with a general search due to lack of specific context, explicitly
state this decision to the user.
- Use the "researcher" agent to obtain information. Use a detailed query to
include all known information about this topic.
- Analyse the deep research response, and think whether its answer is enough to
successfully fulfill the user request.
- If you have all the necessary information to respond to the user's request, stop
the loop and return a final answer. Otherwise, start this process all over again.
Do not perform multiple steps in the same message. Each time, generate only the
process of a single step as a different independent message.
If a research result is not conclusive enough, you can perform this process over
again. Start by thinking if you need more information or details, stop the loop
to ask the user if you need to, and keep doing research and iterations.
You must cite all your sources in the final report.
{{ query }}
First, start only by understanding the user's request.
```
```markdown researcher theme={null}
---
provider: openai
model: gpt-4o
type: agent
description: Performs a deep research about a specific topic or question, and
returns a detailed report.
tools:
- latitude/extract
- latitude/search
---
You're an autonomous AI agent tasked to create a deep and detailed report about a
topic or question.
To do so, you must use all of your available tools to retrieve any information. You
can perform as many steps as you need in order to obtain all possible and relevant
information about the topic, and finally create and return a detailed report.
Before finishing your task, you must find all available information you're about to
about the topic or subject. Do not rely only in one search result. Instead, try to
find and fact-check everything you learn along the way. You can perform as many
search steps as you may seem necessary, even after having requested them before.
The finished report must be extremely detailed and relevant. If the question is too
broad and you found multiple different results about the same topic or subject, make
sure to state so in your response.
For example, if the subject of the question is about a person, you must first find who
this person is and make sure you're not mergeing information about two different people.
Find information from their name, contrast the results from the information given to you
about them, and keep searching about where they studied, worked, family, interests, etc.
If you cannot determine which person the request is about, you will need to return
information about all different people you found with the same name.
You must cite all your sources in the final report.
{{ question }}
Remember to use both the search and extract tools to ensure comprehensive content analysis
and extraction.
```
## Breakdown
Let's break the example down step by step to understand how it works.
#### Clarify user's input
Ensure the agent can handle ambiguous queries by providing clarifying questions to the user.
```markdown {1-5} theme={null}
Some questions may be too broad or generic. If you need more specifics or
additional information in order to correctly fulfill, you must ask the user
at any time. For example, when asking about a person or place, there may be
several results with the same name. In these kind of cases, it would be useful
to ask the user about more details for a better result.
```
#### Create a subagent
Let the main agent know that it has a subagent available to perform deep research.
```markdown {1-10} theme={null}
You have an agent available that will perform a deep research about any topic
or query if you need to obtain any information. This agent does not share any
information or context between runs, so you will need to provide all context it
needs to perform an efficient research every time. You can use natural language
and questions to request information to this agent.
```
#### Multiple iterations
Make sure the agent can perform multiple iterations of research, and not just one.
```markdown {1-30} theme={null}
You must proceed with the following steps, one message at a time:
- Understand the user's request
- State what process you would follow in order to fulfill the request.
- Think about the information given from the user, and list all other
information you need to perform a detailed research about it.
- Stop the loop to ask the user specific questions to clarify the query
and gather more context.
- If proceeding with a general search due to lack of specific context, explicitly
state this decision to the user.
- Use the "researcher" agent to obtain information. Use a detailed query to
include all known information about this topic.
- Analyse the deep research response, and think whether its answer is enough to
successfully fulfill the user request.
- If you have all the necessary information to respond to the user's request, stop
the loop and return a final answer. Otherwise, start this process all over again.
```
#### Fact-check the info
Try to fact-check the information the agent finds, and not just return the first search result.
```markdown {1-3} theme={null}
If a research result is not conclusive enough, you can perform this process over
again. Start by thinking if you need more information or details, stop the loop
to ask the user if you need to, and keep doing research and iterations.
```
#### Citations
Include citations in the final answer.
```markdown {1-20} theme={null}
You must cite all your sources in the final report.
```
Now we have a much more robust agent that can handle ambiguous queries, and will perform multiple iterations of research to find the most relevant information. It will also include citations in the final report.
### Why using a subagent is good?
Doing everything in only a prompt of type `agent` works, but now it has too many responsibilities:
1. It has to understand the user's request.
2. It has to perform the research.
3. It has to fact-check the information it finds.
4. It has to create a final report.
Not only this will affect the performance of the agent, but all those search queries will add too much context to the conversation, making it more expensive and slower.
## Resources
* [Autonomous Agents](/guides/prompt-manager/agents) - Learn more about how to create autonomous agents in Latitude.
* [Subagents](/guides/prompt-manager/agents#subagents) - Learn how to create subagents to delegate tasks to other agents.
* [Latitude Tools](/guides/prompt-manager/latitude-tools) - Learn more about the built-in tools available in Latitude.
# Joke Generator
Source: https://docs-v1.latitude.so/examples/cases/joke-generator
Learn how to build a prompt that thinks like a human and generates jokes.
You can play with this example in the Latitude Playground.
## Overview
Have you ever tried to ask an LLM for a joke, and they either always tell the same one or give you a response that doesn't make any sense?
In this example, we will use the Agent's capability to think like a human using Chain Of Thought reasoning to generate a joke. The Agent will first think about the joke, then tell it to you.
## The prompt
```yaml theme={null}
---
provider: openai
model: gpt-4o
type: agent
---
You're a team of autonomous agents. Your task is to create the funniest joke you can think of about the following topic:
{{ topic }}
You must come up with an answer as a team. To do so, you must communicate with
each other. You should analize, propose and evaluat joke-related aspects about
the topic, come up with a few different drafts for the final joke, and finally
evaluate and build upon them until you are all happy with a final response.
Finally, return your final joke you all agree on.
Okay! Let's work together on this. How should we start?
```
## Breakdown
### Agent configuration
To build an autonomous Chain Of Thought reasoning structure, we want the AI to think as many times as they want to, and finally return a final answer. To do so, we just need to enable the agentic mode by setting the `type` to `agent` in the prompt configuration.
```type {4} theme={null}
---
provider: openai
model: gpt-4o
type: agent
---
```
### Simple prompt
Now, let's first define the prompt. We want to add a simple description of the task, and an input topic given by the user.
```yaml {6-10} theme={null}
---
provider: openai
model: gpt-4o
type: agent
---
You're a team of autonomous agents. Your task is to create the funniest joke you can think of about the following topic:
{{ topic }}
```
### Improvin the chain of thought process
Finally, we'll add a few instructions to help the agent think.
In this case, I will make the AI think it is made of a team of agents, even though it is just a single prompt. This works because each step the AI will generate a new message based on the whole previous conversation. While they normally expect that all `assistant` messages are from the same agent, we can just tell them that they're collaborating with more AI assistants and make them rate each other's jokes.
```yaml {13-17} theme={null}
---
provider: openai
model: gpt-4o
type: agent
---
You're a team of autonomous agents. Your task is to create the funniest joke you can think of about the following topic:
{{ topic }}
You must come up with an answer as a team. To do so, you must communicate with
each other. You should analize, propose and evaluat joke-related aspects about
the topic, come up with a few different drafts for the final joke, and finally
evaluate and build upon them until you are all happy with a final response.
Finally, return your final joke you all agree on.
```
And just for giving the AI a little push, we can **fake the first message of this conversation**, so it can start thinking right away.
```yaml {19-21} theme={null}
---
provider: openai
model: gpt-4o
type: agent
---
You're a team of autonomous agents. Your task is to create the funniest joke you can think of about the following topic:
{{ topic }}
You must come up with an answer as a team. To do so, you must communicate with
each other. You should analize, propose and evaluat joke-related aspects about
the topic, come up with a few different drafts for the final joke, and finally
evaluate and build upon them until you are all happy with a final response.
Finally, return your final joke you all agree on.
Okay! Let's work together on this. How should we start?
```
## Resources
* [Playground docs](/guides/prompt-manager/playground) - Learn how to use the Playground to test and run prompts.
* [Autonomous Agents](/guides/prompt-manager/agents) - Learn how to build autonomous agents that can think like humans.
# Pre-Seed Startup analysis
Source: https://docs-v1.latitude.so/examples/cases/startup-analysis
Specification for AI-Powered Pre-Seed Startup Analysis Tool using Latitude
Try out this agent setup in the Latitude Playground.
## Objective
We want to create an AI-powered tool that can analyze pre-seed startups based on pitch decks and other documents. The tool will extract key information, conduct research, and generate a structured report in Notion.
Let's put an example.
### Input email
Let's ~~imagine~~ is 2008 and **Bryan Chesky** sends an email to a fund with a pitch deck for a startup called **Airbnb**. Our tech incubator is super successful and we don't want to miss any good candidates. So we create an agentic tool in
Latitude that can receive emails like this and analyze the startups that are being pitched.
![Original Bryan's email]()
### Analysis of the email
The final result is a structured report stored in a Notion database.
![Extracted data from startup analysis]()
## The setup
This project use different parts from Latitude let's start by listing them:
[Prompt references](/guides/prompt-manager/references) like for example `` allow you to reuse prompts in different parts of your project. This is useful to avoid duplication and keep your code DRY.
[Steps](/promptl/advanced/chains) allow to interact with the AI in a step-by-step manner, breaking down complex tasks into manageable parts. In this case, we use steps to handle the analysis of the startup in a structured way.
```markdown steps theme={null}
1. Interpretation:
You should use the `interpreter` agent to extract and organize all the official information provided in the email, including the complete content and attached URLs.
2. Team identification:
You should use the `team_finder` agent to find the main team members.
3. Team verification:
You should use the `identity_checker` agent to validate the background and profiles of all founders and key members.
4. Traction metrics:
You should use the `metrics_hunter` agent to search and confirm data about users, revenue, growth, etc.
5. Business model:
You should use the `business_model_analyzer` agent to investigate and analyze the business model.
6. Funding:
You should use the `investigator` agent to research investment rounds, investors, and valuation.
7. Product analysis:
You should use the `tech_stacker` agent to investigate the technology used in the product.
8. Market analysis:
You should use the `market_mapper` agent to analyze the target market, size, and competitors.
9. Solution mapping:
You should use the `competition_research` agent to obtain an overview of all existing solutions to the problem the company is trying to solve, without bias.
11. Final evaluation:
You should use the `evaluation_expert` agent to evaluate the draft, identify strengths, risks, and issue a final recommendation.
```
Email triggers allows us to receive emails in our Latitude accoun when they are sent to a specific address.
[Check the docs](/guides/prompt-manager/triggers#email) to see how to set it up.
In latitude you convert a prompt into an agent by adding the `type: agent` field to the prompt. This allows you to create a multi-agent architecture where each agent is responsible for a specific task in the analysis process. You can learn more about agents in the [agents documentation](/guides/prompt-manager/agents).
[Latitude tools](/guides/prompt-manager/latitude-tools#defining-available-latitude-tools) are used to search information in Internet
In this example we use the [Notion MCP integration](/guides/integration/mcp-integrations) to call the Notion API and create a database item with the final report. MCP calls allow you to interact with external APIs in a structured way, making it easy to integrate with services like Notion.
We also use this [Apify MCP](https://apify.com/pratikdani/crunchbase-companies-scraper) to extract information from Crunchbase.
```markdown notion mcp {5-6} theme={null}
---
provider: Latitude
model: gpt-4o-mini
temperature: 0
tools:
- use_cases_notion/*
type: agent
---
```
```markdown crunchbase mcp {10-10} theme={null}
---
provider: Latitude
model: gpt-4o
temperature: 0
type: agent
description: Investigates the funding and fundraising history of the startup.
tools:
- latitude/search
- latitude/extract
- crunchbase/pratikdani-slash-crunchbase-companies-scraper
---
```
### Notion integration
Most of the elements used in this project are easy to understand following
Latitude documentation, but the Notion integration is a bit more complex. Let's
do a quick overview of how to set it up.
You need to create a [Notion workspace](https://www.notion.com/help/intro-to-workspaces)
Go to [Notion Integrations](https://www.notion.so/profile/integrations) and
create new integration for Latitude. You will need an API to setup the Notion's
MCP server on Latitude.
![Create Notion integration]()
Once you have the integration we need do do 2 more things.
1. copy the **Internal Integration Token** for configuring the [MPC server](/guides/integration/mcp-integrations) on Latitude.
2. Give access to this integration to one of our pages in the workspace (yellow box in the image above).
![Notion integration token]()
## The prompts
The tool is an [AI agent](/guides/prompt-manager/agents) divided in sub-agents, each responsible for a specific task in the analysis process. The main agent coordinates the workflow and ensures that all tasks are completed efficiently.
Latitude is flexible enough to allow you to structure workflows quite complex in a way that
makes sense for you. In this case we decided to create an `agents` folder with
all the processing work and a `publish` agent that is responsible for the
formatting and publishing of the final report in Notion.
```markdown main {8-17} theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0
type: agent
maxSteps: 40
agents:
- agents/interpreter
- agents/team_finder
- agents/identity_checker
- agents/metrics_hunter
- agents/investigator
- agents/competition_research
- agents/evaluation_expert
- agents/market_mapper
- agents/tech_stacker
- agents/business_model_analyzer
- publish
```
```markdown publish {8-9} theme={null}
---
provider: Latitude
model: gpt-4o-mini
type: agent
temperature: 0
maxSteps: 40
agents:
- notion/create_database_item
- notion/add_section_to_page
tools:
- use_cases_notion/*
---
```
Let's do a breakdown of all the prompts (agents) and their roles.
**Purpose**: Gather all foundational, official, and structured data directly from source materials (pitch decks, websites, emails).
**Agents**:
* `agents/interpreter`: Extracts comprehensive company, product, market, team, and funding information from any document or webpage.
* `agents/team_finder`: Collects a verified list of current team members, focusing on accuracy and relevance.
**Purpose**: Ensure accuracy, completeness, and credibility of extracted information; deepen profiles and history.
**Agents**:
* `agents/identity_checker`: Verifies founder and executive backgrounds using professional networks and databases.
* `agents/investigator`: Confirms fundraising history, round details, and investor lists using financial and deal sources.
**Purpose**: Contextualize the company within its business model, market, competition, and traction environment.
**Agents**:
* `agents/business_model_analysis`: Maps how the startup makes money, whom it serves, and how it goes to market.
* `agents/market_mapper`: Defines the addressable market, sizes opportunity, and identifies key competitors and trends.
* `agents/competition_research`: Builds a competitive landscape, classifying direct and indirect solutions in the space.
* `agents/tech_stacker`: Details technology stack and infrastructure, surfacing technical strengths or risks.
* `agents/metrics_hunter`: Finds, validates, and contextualizes traction metrics—users, revenue, growth, engagement.
If you want to really play with this example in live you can [copy it to
your Latitude account here](https://app.latitude.so/share/d/1613160f-3871-439f-baaa-c388248a6fe1)
# Stock Market Analysis Agent
Source: https://docs-v1.latitude.so/examples/cases/stock-market-analysis
Build a multi-agent system for live financial insights combining web search, technical indicators, and actionable recommendations.
Try out this agent setup in the Latitude Playground.
## Overview
This example demonstrates how to build an **intelligent market analysis agent** using Latitude’s multi-agent architecture. The agent can analyze requested stocks or sectors, gather current prices and breaking news, compute technical indicators, and provide actionable, well-structured investment insights. The system orchestrates research and analysis across specialized subagents for maximum efficiency and depth.
## Multi-Agent Architecture
The architecture is divided into purpose-driven subagents, each responsible for a core part of the workflow:
* **main**: Coordinates the entire process and synthesizes final recommendations
* **market\_researcher**: Gathers live news, sentiment, and trends via web search
* **price\_analyzer**: Fetches live prices, historical data, and computes technical indicators with code execution
## The Prompts
Here you can see the three prompts—`main`, `market_researcher`, and `price_analyzer`—that make up the agent system. Each prompt is designed to handle a specific part of the analysis workflow.
```markdown main theme={null}
---
provider: openai
model: gpt-41
type: agent
agents:
- market_researcher
- price_analyzer
temperature: 0.2
schema:
type: object
properties:
market_summary:
type: string
description: Executive summary of current market conditions
stock_analysis:
type: array
items:
type: object
properties:
symbol:
type: string
current_price:
type: number
price_change:
type: number
sentiment:
type: string
key_news:
type: array
items:
type: string
description: Analysis of requested stocks
market_trends:
type: array
items:
type: string
description: Key market trends identified
recommendations:
type: array
items:
type: object
properties:
action:
type: string
reasoning:
type: string
description: Investment recommendations based on analysis
required: [market_summary, stock_analysis, market_trends]
You're an intelligent financial analysis coordinator that provides real-time market insights by combining stock price data with current market news.
You have two specialized agents:
- A market researcher that gathers news and sentiment data using web search
- A price analyzer that retrieves stock prices from Yahoo Finance and calculates technical indicators
Process each request systematically:
1. Analyze the requested stocks/sectors
2. Gather current price data from Yahoo Finance and recent market news
3. Calculate technical indicators using code execution
4. Identify market trends and sentiment
5. Provide actionable insights and recommendations
Analyze the following stocks: {{ stock_symbols }}
Market focus: {{ market_focus }}
Analysis depth: {{ analysis_depth }}
Begin by understanding the analysis requirements and coordinating data gathering.
```
```markdown market_researcher theme={null}
---
provider: openai
model: gpt-4o
type: agent
description: Researches market news, sentiment, and trends using web search
tools:
- latitude/search
- latitude/extract
---
You're a market research specialist focused on gathering comprehensive market intelligence using web search capabilities.
Your research process:
1. Search for recent news about requested stocks/sectors
2. Extract detailed content from financial news sources
3. Analyze market sentiment and investor mood from search results
4. Identify emerging trends and market drivers
5. Compile findings into structured reports
Focus on searching for:
- Breaking news that could impact stock prices
- Analyst reports and upgrades/downgrades
- Economic indicators and market sentiment
- Sector-specific developments
- Regulatory changes or company announcements
Use web search to find the most current information from reliable financial sources:
- Bloomberg
- Reuters
- MarketWatch
- Yahoo Finance
{{ research_request }}
Conduct comprehensive market research using web search and content extraction tools.
```
````markdown price_analyzer theme={null}
---
provider: OpenAI
model: gpt-4o-mini
type: agent
description: Analyzes stock prices from Yahoo Finance and calculates technical
indicators using code execution
tools:
- latitude/code
- latitude/search
---
You're a quantitative analyst specializing in stock price analysis and technical indicators calculation.
Your analysis process:
1. Search Yahoo Finance for current stock prices and historical data
2. Use code execution to calculate technical indicators
3. Identify price patterns and trends through computational analysis
4. Assess volatility and trading volume using statistical methods
5. Generate price-based insights and trading signals
For stock price retrieval:
- Search "Yahoo Finance [STOCK_SYMBOL] stock price" to get current market data
- Look for price, change, volume, and recent performance data
- Extract key metrics from Yahoo Finance pages
- Once you receive the search move on to the analysis
For technical analysis, use code execution to calculate:
- Moving averages (SMA, EMA)
- Relative Strength Index (RSI)
- MACD (Moving Average Convergence Divergence)
- Bollinger Bands
- Volume analysis
- Price momentum indicators
Example code structure for technical indicators:
```python
import pandas as pd
import numpy as np
# Calculate RSI
def calculate_rsi(prices, period=14):
delta = prices.diff()
gain = (delta.where(delta > 0, 0)).rolling(window=period).mean()
loss = (-delta.where(delta < 0, 0)).rolling(window=period).mean()
rs = gain / loss
rsi = 100 - (100 / (1 + rs))
return rsi
# Calculate MACD
def calculate_macd(prices, fast=12, slow=26, signal=9):
ema_fast = prices.ewm(span=fast).mean()
ema_slow = prices.ewm(span=slow).mean()
macd = ema_fast - ema_slow
signal_line = macd.ewm(span=signal).mean()
histogram = macd - signal_line
return macd, signal_line, histogram
{{ analysis_request }}
Search Yahoo Finance for stock data and perform comprehensive technical analysis using code execution.
````
## Parameters Explained
In the main prompt, we set these three parameters to control the analysis. Here you
can see an example for Tesla Stock Analysis. Below is an explanation of each parameter.
This parameter accepts a list of stock ticker symbols to analyze. Examples:
* `AAPL, MSFT, GOOGL` for individual stocks
* `SPY, QQQ, IWM` for ETFs
* `TSLA, NVDA, AMD` for sector-specific analysis
This parameter defines the analytical perspective or theme for the analysis. Examples:
* `earnings season impact`
* `pre-market analysis`
* `sector rotation trends`
* `daily wrap-up`
* `volatility assessment`
This guides both the market researcher and price analyzer agents on which aspects to emphasize in their analysis.
This parameter controls the comprehensiveness of the analysis. Options include:
* `summary` - Quick overview with key points
* `comprehensive` - Detailed analysis with full technical indicators
* `deep-dive` - Extensive research with multiple data sources
* `real-time` - Focus on immediate market conditions
## Breakdown
Let’s break down the case step-by-step to highlight each agent’s contribution.
### 1. Requirements Analysis
The **main agent** begins by clarifying the user’s goals—what stocks/sectors to analyze, market focus, and depth of analysis. This ensures downstream agents are properly scoped and that their findings are relevant.
### 2. Market Research
The **market\_researcher** agent leverages real-time web search to find:
* Breaking news and analyst reports from trusted sources (Bloomberg, Reuters, MarketWatch, Yahoo Finance)
* Economic indicators and investor sentiment
* Regulatory changes or company events
* Sector and macro trends
It uses content extraction and trend identification tools to deliver structured, concise findings on factors affecting the requested stocks.
### 3. Price & Technical Analysis
The **price\_analyzer** agent:
* Retrieves current and historical stock price data from Yahoo Finance
* Analyzes price movements, volatility, and volume
* Calculates technical indicators such as:
* Simple/Exponential Moving Averages (SMA, EMA)
* Relative Strength Index (RSI)
* MACD
* Bollinger Bands
* Identifies trading signals and price-based insights through code execution
### 4. Synthesis & Recommendation
The **main agent** compiles all findings into a unified report, identifying:
* Key market trends
* Individual stock summaries
* Sentiment and notable news
* Actionable investment recommendations
The output is structured using a [JSON schema](/guides/prompt-manager/json-output) for consistency and ease of integration.
***
## Why This Multi-Agent Approach Works
Splitting responsibilities keeps each agent focused and efficient:
* **market\_researcher**: Excels at broad, qualitative intelligence gathering using web tools
* **price\_analyzer**: Specializes in quantitative and computational tasks with live data
* **main**: Maintains context, makes decisions, and produces high-level summaries
**Benefits:**
* No single agent is overloaded
* Context windows stay small for better LLM performance
* The system is modular and maintainable
* Easily swap or upgrade subagents/providers as requirements evolve
## Strategic Benefits
This multi-agent, multi-provider setup is optimized for:
* **Performance**: Each agent uses the most suitable model for its task
* **Cost Efficiency**: Main tasks run on higher-end models, while research and analysis run on faster, lower-cost models
* **Reliability**: Modular—swap out underperforming agents/providers as needed
* **Scalability**: Add new specialized agents (e.g., risk assessor, macro strategist) with minimal friction
Model and provider capabilities evolve. Routinely review provider performance, costs, and integration to ensure continued fit.
Latitude makes it easy to switch providers or models at any stage. Just update the provider configuration in your prompt manager—no need to rearchitect your agent logic.
For custom providers or advanced tuning, see the [provider documentation](/guides/getting-started/providers).
***
## Resources
* [Latitude Tools](/guides/prompt-manager/latitude-tools)
* [JSON Schema Output](/guides/prompt-manager/json-output)
* [Provider Configuration](/guides/getting-started/providers)
* [Prompt Triggers](/guides/prompt-manager/triggers)
* [MCP Integration](/guides/integration/mcp-integrations)
# Weather Chatbot: Ask the Clouds!
Source: https://docs-v1.latitude.so/examples/cases/weather-chatbot
Build a chatbot that answers weather questions using Latitude and a custom weather tool. Fun, fast, and just a bit magical.
You can play with this example in the Latitude Playground.
## Overview
Curious if you need an umbrella before heading out? In this example, you'll build a Weather Chatbot that fetches real-time weather information whenever users ask. Powered by Latitude prompts and your custom backend code, this bot delivers instant updates on sunshine, rain, or snow—with just a dash of magic.
## The prompt
```yaml theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.2
tools:
- get_weather:
description: Fetch weather data for a given location.
parameters:
type: object
properties:
location:
type: string
description: The name of the location to fetch weather data for.
required:
- location
---
You are a helpful assistant that can provide weather information.
If the user asks for the weather in a specific location, use the
`get_weather` tool to fetch the data.
{{ question }}
```
## Breakdown
The main concept to learn in this example is **tool calling**. This tool fetches weather data from the OpenWeatherMap API using the specified location. It returns the location name, temperature, and a description of the current weather conditions.
### Tool calling
The key feature in this prompt is the use of a [tool call](/guides/prompt-manager/tools). This allows the model to trigger a custom backend function to fetch weather data, such as by calling the [OpenWeatherMap API](https://openweathermap.org/api). In the Latitude Playground, you can simulate this tool call, as shown in [the screenshot above](/examples/cases/weather-chatbot#demo).
```yaml {5-15} theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.2
tools:
- get_weather:
description: Fetch weather data for a given location.
parameters:
type: object
properties:
location:
type: string
description: The name of the location to fetch weather data for.
required:
- location
---
```
## Resources
* [Playground docs](/guides/prompt-manager/playground) - Learn how to use the Playground to test and run prompts.
* [Tool calling docs](/guides/prompt-manager/tools) - Learn how to create and use tools in your prompts.
* [Tool call SDK example](/examples/sdk/run-prompt-with-tools) - A simple example of how to run a prompt with tools with Latitude SDK.
* [OpenWeatherMap API](https://openweathermap.org/api) - The API used to fetch weather data.
# Introduction
Source: https://docs-v1.latitude.so/examples/overview
Explore practical examples, advanced LLM techniques, and real-world use cases to build powerful AI applications with Latitude
In this section of Latitude's documentation you will find 3 kind of materials.
1. **SDK Examples**: These examples demonstrate how to use the Latitude SDK to build AI applications, including running prompts, integrating tools, and managing conversation context.
2. **Prompting Techniques**: This section covers advanced prompting techniques that can enhance the quality and capabilities of your LLM applications, such as reasoning methods, memory management, and input/output strategies.
3. **Real-world Cases**: These examples showcase complete solutions for common business and technical challenges, demonstrating how to combine various techniques into production-ready applications.
## SDK Examples
* [Run Prompt](/examples/sdk/run-prompt) - Execute prompts with dynamic parameters
* [Run with Tools](/examples/sdk/run-prompt-with-tools) - Integrate external tools with your prompts
* [Render Chain](/examples/sdk/render-chain) - Connect multiple prompts in sequence
* [RAG Retrieval](/examples/sdk/rag-retrieval) - Implement retrieval-augmented generation
## Prompting Techniques
These the main techniques for advanced prompting that can significantly improve the performance and reliability of your LLM applications. Each technique is designed to address specific challenges in AI interactions, from enhancing reasoning capabilities to managing context and improving output quality.
Learn how to implement few-shot learning with examples to improve AI performance on specific tasks
Enhance AI performance by assigning specific roles, personas, and expertise areas
Improve reasoning and problem-solving capabilities with structured thought processes
Enable complex reasoning by breaking down problems into manageable sub-tasks
Manage conversation context effectively to maintain coherence and relevance
Enhance output reliability by generating multiple responses and selecting the best one
Improve output quality by iteratively refining responses through feedback loops
Combine reasoning and action to enhance decision-making capabilities
Advanced prompting techniques can dramatically improve the quality, reliability, and capabilities of your LLM applications. These examples demonstrate proven approaches to enhance your AI systems.
## Real-world Cases
Our case examples showcase complete solutions for common business and technical challenges, demonstrating how to combine various techniques into production-ready applications.
Explore real-world implementations that you can adapt to your specific needs:
Create personalized, empathetic customer service emails with multi-agent architecture
Implement robust content filtering and moderation with constitutional AI principles
Build an advanced information retrieval system with multi-stage processing
Analyze financial data and generate insights using specialized agents
Need help choosing the right example? Check out our [Getting Started Guide](/guides/getting-started/overview) or [contact support](https://latitude.so/contact) for personalized recommendations.
# Annotate log (HITL)
Source: https://docs-v1.latitude.so/examples/sdk/annotate-log
Learn how to annotate log data with the Latitude SDK to perform HITL evaluations
This guide explains how to perform Human-in-the-Loop (HITL) evaluations of your prompt’s performance.
## Prompt
In this example, we have a simple prompt that asks the LLM to generate a joke. We want users to be able to evaluate the quality of the joke and provide feedback.
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
temperature: 0.7
---
Please tell me a joke about cats.
```
## How does this work?
In this scenario, we use OpenAI’s API directly to run the prompt defined in Latitude. We retrieve the prompt using Latitude’s SDK, display the messages, and send them to the OpenAI API.
Once the model completes its response, we upload a log to our prompt in Latitude, which you can view in the prompt’s logs section.
Finally, we annotate the log with the feedback received from the user. In this case, the user rates the joke on a scale from 1 to 5 and provides a reason for their rating.
You can learn more about [HITL (Human-in-the-Loop) evaluations in our documentation](/guides/evaluations/humans-in-the-loop).
### Code examples
```typescript Typescript theme={null}
import { Latitude, Adapters } from '@latitude-data/sdk'
import OpenAI from 'openai'
// To run this example you need to create a evaluation on the prompt: `annontate-log/example`
// Info: https://docs.latitude.so/guides/evaluations/overview
const EVALUATION_UUID = 'YOUR_EVALUATION_UUID'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
// Get the prompt from Latitude
const prompt = await sdk.prompts.get('annotate-log/example')
// Generate messages from the Latitude prompt
// These messages are valid OpenAI messages. Note that we passed the Adapters.openai
const { config, messages } = await sdk.prompts.render({
prompt: { content: prompt.content },
parameters: {},
adapter: Adapters.openai,
})
// Call OpenAI
const llmResponse = await openai.chat.completions.create({
// @ts-ignore
messages,
model: config.model as string,
})
const { uuid } = await sdk.logs.create('annotate-log/example', messages, {
response: llmResponse.choices[0].message.content,
})
// Score from 1 to 5 because the evaluation we created is of type `
// More info: https://docs.latitude.so/guides/evaluations/humans-in-the-loop
const result = await sdk.evaluations.annotate(uuid, 5, EVALUATION_UUID, {
reason: 'This is a good joke!',
})
console.log('Result:', JSON.stringify(result, null, 2))
}
run()
```
```python Python theme={null}
import asyncio
import os
from devtools import pprint
from latitude_sdk import (
AnnotateEvaluationOptions,
CreateLogOptions,
RenderPromptOptions,
Latitude,
LatitudeOptions,
)
from openai import AsyncOpenAI
from promptl_ai import Adapter
# To run this example you need to create a evaluation on the prompt: `annontate-log/example`
# Info: https://docs.latitude.so/guides/evaluations/overview
EVALUATION_UUID = "YOUR_EVALUATION_UUID"
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
openai = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
# Get the prompt from Latitude
prompt = await sdk.prompts.get("annotate-log/example")
# Render the messages from the Latitude prompt
render = await sdk.prompts.render(prompt.content, RenderPromptOptions(adapter=Adapter.OpenAI))
# Call OpenAI with the messages from the prompt
llm_result = await openai.chat.completions.create(
model=render.config["model"],
temperature=render.config["temperature"],
messages=[message.model_dump() for message in render.messages],
)
llm_response = llm_result.choices[0].message.content
latitude_render = await sdk.prompts.render(
prompt.content,
RenderPromptOptions(
adapter=Adapter.Default,
),
)
log_result = await sdk.logs.create(
"annotate-log/example",
latitude_render.messages,
CreateLogOptions(response=llm_response),
)
result = await sdk.evaluations.annotate(
log_result.uuid, 1, EVALUATION_UUID, AnnotateEvaluationOptions(reason="This is a bad joke!")
)
pprint(result)
asyncio.run(run())
```
# Create a log
Source: https://docs-v1.latitude.so/examples/sdk/create-log
Learn how to create a log with the Latitude SDK
## Prompt
In this example, the specific prompt is not important—you just need to have a prompt created in a Latitude project.
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
---
You can upload logs to your Latitude.
More info: https://docs.latitude.so/guides/sdk/typescript#creating-logs
Once you upload a log you can see it in the logs section of this prompt.
```
## Code
Here’s how you can upload a log to your prompt using the Latitude SDK:
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
import { MessageRole, ContentType } from 'promptl-ai'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const response = await sdk.logs.create(
'create-log/example',
[
{
role: MessageRole.user,
content: [
{ type: 'text', text: 'Tell me a joke about Python' },
],
},
{
role: MessageRole.assistant,
content: [
{ type: 'text', text: 'Python is a great language!' },
],
},
{
role: MessageRole.user,
content: [
{ type: 'text', text: 'Tell me a joke about javascript!' },
],
},
],
{
response: 'Javascript is a great language!',
},
)
console.log('Log: ', response)
}
run()
```
```python Python theme={null}
import asyncio
import os
from devtools import pprint
from latitude_sdk import (
Latitude,
LatitudeOptions,
CreateLogOptions,
)
from promptl_ai import AssistantMessage, UserMessage
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
result = await sdk.logs.create(
"create-log/example",
[
UserMessage(content="Tell me a joke about Python!"),
AssistantMessage(content="Python is a great language!"),
UserMessage(content="Tell me a joke about JavaScript!"),
],
CreateLogOptions(
response="JavaScript is a great language too!",
),
)
pprint(result)
asyncio.run(run())
```
# Get all prompts
Source: https://docs-v1.latitude.so/examples/sdk/get-all-prompts
Learn how to get all prompts with Latitude SDK
## Code
This code gets all prompts from a Latitude project.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const response = await sdk.prompts.getAll()
// You can also pass a specific projectId and versionUuid other
// than the one you are using in the sdk
// const response = await sdk.prompts.getAll({
// projectId: 123,
// versionUuid: 'some-version-uuid',
// })
console.log(
'Prompts: ',
response.map((p) => p.path),
)
}
run()
```
```python Python theme={null}
import asyncio
import os
from latitude_sdk import Latitude, LatitudeOptions
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
results = await sdk.prompts.get_all()
# You can pass different project_id or version_uuid
# results = await sdk.prompts.get_all(
# GetAllPromptsOptions(
# project_id=123,
# version_uuid=VERSION_UUID
# )
paths = [result.path for result in results]
print(paths, "\n" * 2)
asyncio.run(run())
```
# Create a prompt
Source: https://docs-v1.latitude.so/examples/sdk/get-or-create-prompt
Learn how to create a prompt with Latitude SDK
## Code
Here is how you can get or create a new prompt with our SDK. If the prompt already exists, it will be returned. Otherwise, a new prompt will be created.
You can't create prompts on `live` versions. You have to create a new
version and point to it.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
const PROMPT = `
Answer succinctly yet complete.
Tell me a joke about a {{topic}}
`
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
// YOU CAN NOT CREATE A PROMPT IN A LIVE Version
// versionUuid='live',
// More info: https://docs.latitude.so/guides/prompt-manager/version-control
versionUuid: '[CREATE_A_NEW_VERSION_UUID]',
})
const response = await sdk.prompts.getOrCreate('create-prompt/example', {
prompt: PROMPT,
})
console.log('Response', response)
}
run()
```
```python Python theme={null}
import asyncio
import os
from latitude_sdk import (
Latitude,
LatitudeOptions,
GetOrCreatePromptOptions,
)
PROMPT = """
Answer succinctly yet complete.
Tell me a joke about a {{topic}}
"""
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
# YOU CAN NOT CREATE A PROMPT in A LIVE Version
# version_uuid='live',
# More info: https://docs.latitude.so/guides/prompt-manager/version-control
version_uuid="[CREATE_A_NEW_VERSION_UUID]",
)
sdk = Latitude(api_key, sdk_options)
result = await sdk.prompts.get_or_create(
"create-propmpt/example",
GetOrCreatePromptOptions(prompt=PROMPT),
)
print(result, "\n" * 2)
asyncio.run(run())
```
# Get a prompt
Source: https://docs-v1.latitude.so/examples/sdk/get-prompt
Learn how to get a prompt with the Latitude SDK
## Code
You can retrieve a prompt from one of your Latitude projects. This is useful if you want to render the prompt in the format expected by the LLM you're using and call it directly.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const response = await sdk.prompts.get('get-prompt/example')
console.log('Prompt: ', response)
}
run()
```
```python Python theme={null}
import asyncio
import os
from latitude_sdk import Latitude, LatitudeOptions
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
try:
result = await sdk.prompts.get("get-prompt/example")
except Exception as e:
print(f"Error: {e}")
return
if result:
# Also you can wait for the result
print(result, "\n" * 2)
asyncio.run(run())
```
# Pause a Tool Execution
Source: https://docs-v1.latitude.so/examples/sdk/pause-tools
Learn how to pause the execution of a tool and process the data
## Prompt
When a tool’s calculation is simple, you can simply return its value to the Latitude SDK, as shown in the [prompt with tools](/examples/sdk/run-prompt-with-tools) example.
However, for more complex calculations, you can pause the execution of the tool, process the data asynchronously, and then respond with the result in the same conversation.
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
tools:
- generate_travel_itinerary:
description: Generates a personalized multi-day travel itinerary for a user based on their preferences, location, and available dates. This requires multiple external data sources and can take some time, so it runs as a background job.
parameters:
type: object
additionalProperties: false
required: ['destination', 'start_date', 'end_date', 'preferences', 'user_id']
properties:
destination:
type: string
description: Name of the travel destination (city, country, etc.)
start_date:
type: string
description: Start date of the trip (YYYY-MM-DD)
end_date:
type: string
description: End date of the trip (YYYY-MM-DD)
preferences:
type: array
items:
type: string
description: List of user interests (e.g., museums, food, outdoor, art)
user_id:
type: string
description: Unique identifier for the requesting user
---
Plan my trip to {{ destination }} from {{ start_date }} to {{ end_date }}.
I like {{ preferences.join(', ') }}.
# Example response to user
Great! I’m planning your trip to {{ destination }} with your preferences.
This may take a few minutes, as I’ll be gathering up-to-date info from multiple sources. I’ll notify you as soon as your custom itinerary is ready!
```
## Code
In this example, you can see how itinerary creation is requested by the AI. The execution is paused, the data is stored (in memory, though you could also store it in your database or Redis), and then the tool execution is resumed with the calculated itinerary.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
import { Message, MessageRole } from 'promptl-ai'
// You can type the tools you are using
type Tools = {
generate_travel_itinerary: {
location: string
start_date: string
end_date: string
preferences: string
}
}
type ItineraryRequested = {
data: {
location: string
start_date: string
end_date: string
preferences: string
}
toolId: string
toolName: string
conversationUuid: string
previousMessages: Message[]
}
let toolRequested: ItineraryRequested | undefined
function enqueueJobToProcessItinerary(itinerary: ItineraryRequested) {
toolRequested = itinerary
}
function computeTravelItinerary(itinerary: ItineraryRequested) {
return {
location: itinerary.data.location,
start_date: itinerary.data.start_date,
end_date: itinerary.data.end_date,
preferences: itinerary.data.preferences,
recomendations: [
'Visit the Sagrada Familia',
'Explore Park Güell',
'Take a stroll down La Rambla',
'Relax at Barceloneta Beach',
'Enjoy tapas at a local restaurant',
'Visit the Picasso Museum',
],
}
}
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
await sdk.prompts.run('pause-tools/example', {
parameters: {
destination: 'Barcelona',
start_date: '2025-06-02',
end_date: '2025-06-10',
preferences: 'museums, parks, and local cuisine',
},
tools: {
generate_travel_itinerary: async (
data,
{ messages, conversationUuid, toolId, toolName, pauseExecution },
) => {
enqueueJobToProcessItinerary({
data,
toolId,
toolName,
conversationUuid,
previousMessages: messages,
})
// You are not returning the result now because the computation
// is heavy and you want to pause the execution
return pauseExecution()
},
},
})
// Imagine this is your backend processing the job
if (toolRequested) {
const result = await sdk.prompts.chat(toolRequested.conversationUuid, [
{
role: MessageRole.tool,
content: [
{
type: 'tool-result',
toolName: toolRequested.toolName,
toolCallId: toolRequested.toolId,
result: computeTravelItinerary(toolRequested),
},
],
},
])
console.log('Recomendation', result.response.text)
}
}
run()
```
```python Python theme={null}
import asyncio
import os
from devtools import pprint
from typing import Optional, Dict, Any
from latitude_sdk import (
Latitude,
LatitudeOptions,
RunPromptOptions,
OnToolCallDetails,
)
from promptl_ai import ToolMessage, ToolResultContent
ItineraryRequested = Dict[str, Any]
tool_requested: Optional[ItineraryRequested] = None
def enqueue_job_to_process_itinerary(itinerary: ItineraryRequested):
global tool_requested
tool_requested = itinerary
def compute_travel_itinerary(itinerary: ItineraryRequested) -> Dict[str, Any]:
data = itinerary["data"]
return {
"location": data["location"],
"start_date": data["start_date"],
"end_date": data["end_date"],
"preferences": data.get("preferences"),
"recommendations": [
"Visit the Sagrada Familia",
"Explore Park Güell",
"Take a stroll down La Rambla",
"Relax at Barceloneta Beach",
"Enjoy tapas at a local restaurant",
"Visit the Picasso Museum",
],
}
async def generate_travel_itinerary(arguments: dict[str, Any], details: OnToolCallDetails) -> str:
pprint(details)
enqueue_job_to_process_itinerary(
{
"data": {
"location": arguments.get("location", "Barcelona"),
"start_date": arguments.get("start_date"),
"end_date": arguments.get("end_date"),
"preferences": arguments.get("preferences"),
},
"tool_id": details.id,
"tool_name": details.name,
"conversationUuid": details.conversation_uuid,
}
)
return details.pause_execution()
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
await sdk.prompts.run(
"pause-tools/example",
RunPromptOptions(
parameters={
"destination": "Barcelona",
"start_date": "2025-06-02",
"end_date": "2025-06-10",
"preferences": "museums, parks, and local cuisine",
},
tools={"generate_travel_itinerary": generate_travel_itinerary},
),
)
if tool_requested is None:
print("No tool requested.")
return
# Imagine this is your backend processing the job asynchronously.
result = await sdk.prompts.chat(
tool_requested["conversationUuid"],
[
ToolMessage(
content=[
ToolResultContent(
id=tool_requested["tool_id"],
name=tool_requested["tool_name"],
result=compute_travel_itinerary(tool_requested),
is_error=False,
),
],
)
],
)
pprint(result.response.text)
asyncio.run(run())
```
# RAG retrieval
Source: https://docs-v1.latitude.so/examples/sdk/rag-retrieval
Learn how to use RAG retrieval with the Latitude SDK
## Prompt
In your prompt, you can define a tool that will be used to retrieve information.
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
temperature: 0.7
tools:
- get_answer:
description: Ask this tool for the answer when user do a question.
parameters:
type: object
additionalProperties: false
required: ['question']
properties:
question:
type: string
description: Question to ask
---
Give user's question {{ question }} a concise answer.
```
## Code
Performing RAG retrieval with the Latitude SDK simply involves defining a tool in your prompt. In your code, you can then get the results from the RAG solution you use.
```typescript Typescript theme={null}
import { Pinecone } from '@pinecone-database/pinecone'
import OpenAI from 'openai'
import { Latitude } from '@latitude-data/sdk'
type Tools = { get_answer: { question: string } }
const PINECODE_INDEX_NAME = 'geography-quizz-index'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
const pinecone = new Pinecone({ apiKey: process.env.PINECONE_API_KEY })
const pc = pinecone.Index(PINECODE_INDEX_NAME)
const question = 'What is the deepest ocean in the world?'
console.log('Question: ', question)
console.log('\nSearching...\n')
const result = await sdk.prompts.run('rag-retrieval/example', {
parameters: { question },
tools: {
get_answer: async ({ question }) => {
// Do the embedding
const embedding = await openai.embeddings
.create({
input: question,
model: 'text-embedding-3-small',
})
.then((res) => res.data[0].embedding)
// Query your RAG backend. In this case, Pinecone
const queryResponse = await pc.query({
vector: embedding,
topK: 10,
includeMetadata: true,
})
const first = queryResponse.matches[0]
return first?.metadata?.answer
},
},
})
console.log('Answer: ', result.response.text)
}
run()
```
# Render a prompt with steps
Source: https://docs-v1.latitude.so/examples/sdk/render-chain
Learn how to render the steps of a prompt with the Latitude SDK
## Prompt
Chains and Steps are used to create multi-step prompts that can interact with the AI model in stages. Chains in PromptL allow you to break complex workflows into smaller, manageable steps. You can read more [about it here](/promptl/advanced/chains#chains-and-steps).
In this example, we use two steps. In the first step, we ask the model to think about the answer. Then, in the second step, we ask it to provide an explanation for why it chose that answer.
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
temperature: 0.7
---
You're a helpful assistant. Think about user question and provide the best answer.
This is the question:
{{ question }}
Now that you have the best answer, please provide a detailed explanation of how you arrived at this answer.
```
## Code
The key point to understand is that for each `` found in the prompt, the SDK will invoke `onStep`. At that point, you can ask your LLM to provide a response.
```typescript Typescript theme={null}
import { Latitude, Adapters, Message } from '@latitude-data/sdk'
import OpenAI from 'openai'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
const prompt = await sdk.prompts.get('render-chain/example')
const result = await sdk.prompts.renderChain({
prompt,
parameters: { question: 'What is the meaning of life?' },
adapter: Adapters.openai,
onStep: async ({
config,
messages,
}: {
config: { [s: string]: unknown }
messages: Message[]
}) => {
const response = await openai.chat.completions.create({
model: config.model as string,
temperature: config.temperature as number,
messages,
})
return response.choices[0].message
},
})
console.log('Result:', JSON.stringify(result, null, 2))
}
run()
```
```python Python theme={null}
import asyncio
import os
from typing import Any, Dict, List, Sequence, Union
from devtools import pprint
from latitude_sdk import Latitude, LatitudeOptions, RenderChainOptions
from openai import AsyncOpenAI
from promptl_ai import Adapter, MessageLike
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
# Use oficial OpenAI SDK
openai = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
async def on_step(
messages: List[MessageLike], config: Dict[str, Any]
) -> Union[str, MessageLike, Sequence[MessageLike]]:
response = await openai.chat.completions.create(
model=config["model"],
temperature=config["temperature"],
messages=[message.model_dump() for message in messages],
)
return response.choices[0].message.model_dump()
prompt = await sdk.prompts.get("render-chain/example")
# Here we render the chain and each step will be sent to OpenAI
result = await sdk.prompts.render_chain(
prompt,
on_step,
RenderChainOptions(
parameters={"question": "What is the meaning of life?"},
adapter=Adapter.OpenAI,
),
)
pprint(result)
asyncio.run(run())
```
# Run prompt
Source: https://docs-v1.latitude.so/examples/sdk/run-prompt
Learn how to run a prompt with the Latitude SDK
## Prompt
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
temperature: 0.7
---
You are a creative assistant that crafts engaging product descriptions.
Write a compelling product description for {{product_name}} highlighting its features: {{features}}.
The description should appeal to {{target_audience}} and have a {{tone}} tone.
IMPORTANT: The name of the product should not be altered or added anything. Ex.: "Ford" as product_name should not be "ford-card".
Limit the description to {{word_count}} words although if you produce + or - 10 words over under this limit is fine.
```
## Code
See the code below for how to run a prompt using the Latitude SDK.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
__internal: {
gateway: {
host: 'localhost',
port: 8787,
ssl: false,
}
},
})
const result = await sdk.prompts.run('onboarding', {
// Get messages as streaming
stream: false,
background: false,
parameters: {
phrase: 'I get a very good feeling about this new project.',
},
// To get streaming you can use `onEvent`
onEvent: (event) => {
console.log('Event:', event)
},
onError: (error) => {
if (!error) return
console.error('Error:', error)
},
})
console.log('Result:', result)
}
run()
```
```python Python theme={null}
import asyncio
import os
from latitude_sdk import ApiError, FinishedResult, Latitude, LatitudeOptions, RunPromptOptions, StreamEvent
async def on_event(event: StreamEvent):
print(event, "\n" * 2)
async def on_finished(result: FinishedResult):
print(result, "\n" * 2)
async def on_error(error: ApiError):
print(error, "\n" * 2)
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
result = await sdk.prompts.run(
"run-prompt/example",
RunPromptOptions(
parameters={
"product_name": "iPhone",
"features": "Camera, Battery, Display",
"target_audience": "Tech enthusiasts",
"tone": "Informal",
"word_count": 20,
},
on_event=on_event,
on_finished=on_finished,
on_error=on_error,
stream=True,
),
)
if result:
print(result.response.text, "\n" * 2)
asyncio.run(run())
```
# Run a prompt with tools
Source: https://docs-v1.latitude.so/examples/sdk/run-prompt-with-tools
Learn how to run a prompt with tools using the Latitude SDK
## Prompt
In this example, we define a tool to get the weather. How you obtain the weather is up to you—you might call a third-party service or something within your own system. Once you have the weather information, you return the response to the LLM, and it finishes processing the prompt with that data. You can read more about [tool calling here](/guides/prompt-manager/tools).
```markdown example theme={null}
---
provider: Latitude
model: gpt-4o-mini
tools:
- get_weather:
description: Gets the weather temperature from a given location.
parameters:
type: object
additionalProperties: false
required: ['location']
properties:
location:
type: string
description: Name of the location
---
What should I wear for the weather in {{ location }}?
```
## Code
When calling a tool, you can process the data using the arguments your users provide and return a response.
```typescript Typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
// You can type the tools you are using
type Tools = { get_weather: { location: string } }
async function run() {
const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: Number(process.env.PROJECT_ID),
versionUuid: 'live',
})
const response = await sdk.prompts.run(
'run-prompt-with-tools/example',
{
parameters: { location: 'Boston' },
tools: {
get_weather: async ({ location }) => {
return { temperature: `2°C for ${location}` }
},
},
},
)
console.log('RESPONSE', JSON.stringify(response, null, 2))
}
run()
```
```python Python theme={null}
import asyncio
import os
from typing import Any
from devtools import pprint
from latitude_sdk import (
ApiError,
FinishedResult,
Latitude,
LatitudeOptions,
OnToolCallDetails,
RunPromptOptions,
StreamEvent,
)
async def get_weather(arguments: dict[str, Any], details: OnToolCallDetails) -> str:
pprint(details)
# Simulate a call to a weather API
return "2°C"
async def on_event(event: StreamEvent):
print(event, "\n" * 2)
async def on_finished(result: FinishedResult):
print(result, "\n" * 2)
async def on_error(error: ApiError):
print(error, "\n" * 2)
async def run():
api_key = os.getenv("LATITUDE_API_KEY")
sdk_options = LatitudeOptions(
project_id=int(os.getenv("PROJECT_ID")),
version_uuid="live",
)
sdk = Latitude(api_key, sdk_options)
result = await sdk.prompts.run(
"run-prompt-with-tools/example",
RunPromptOptions(
parameters={
"location": "Boston",
},
tools={"get_weather": get_weather},
on_event=on_event,
on_finished=on_finished,
on_error=on_error,
stream=True,
),
)
if result:
print(result.response.text, "\n" * 2)
asyncio.run(run())
```
# Chain-of-Thought (CoT)
Source: https://docs-v1.latitude.so/examples/techniques/chain-of-thought
Implement step-by-step reasoning to improve AI performance on complex problems
## What is Chain-of-Thought?
Chain-of-Thought (CoT) prompting is a technique that enhances the reasoning capabilities of Large Language Models by generating intermediate reasoning steps. Instead of jumping directly to an answer, the AI is guided to "think out loud" through each step of the problem-solving process, leading to more accurate and explainable results.
This approach is particularly effective because LLMs often struggle with tasks requiring logical reasoning, mathematical calculations, or multi-step problem solving when they attempt to provide immediate answers.
## Why Use Chain-of-Thought?
### Advantages:
* **Improved Accuracy**: Dramatically reduces errors on complex reasoning tasks
* **Low-Effort Implementation**: Works with off-the-shelf LLMs without fine-tuning
* **Explainable AI**: Users can follow and validate the reasoning process
* **Debugging Capability**: Easy to identify where reasoning went wrong
* **Model Robustness**: Performance remains consistent across different LLM versions
* **Versatile Applications**: Effective for math, logic, code generation, and analysis
### Trade-offs:
* **Higher Token Cost**: More output tokens mean increased API costs
* **Slower Response Time**: Additional reasoning steps take longer to generate
* **Verbosity**: Responses are longer and may require post-processing
## Zero-Shot vs Few-Shot CoT
### Zero-Shot Chain-of-Thought
The simplest form of CoT uses trigger phrases like "Let's think step by step" to encourage reasoning:
```markdown Zero-Shot CoT Example theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Age Problem Solver
When I was 3 years old, my partner was 3 times my age. Now, I am 20 years old. How old is my partner?
Let's think step by step.
```
**Expected Output:**
```
1. When I was 3 years old, my partner was 3 × 3 = 9 years old
2. The age difference between us is 9 - 3 = 6 years (partner is older)
3. This age difference remains constant over time
4. Now I am 20 years old, so my partner is 20 + 6 = 26 years old
Answer: My partner is 26 years old.
```
### Few-Shot Chain-of-Thought
Providing examples of reasoning improves consistency and teaches the desired thinking pattern:
```markdown Few-Shot CoT Example theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Age Problem Solver with Examples
Q: When my brother was 2 years old, I was double his age. Now I am 40 years old. How old is my brother? Let's think step by step.
A: When my brother was 2 years old, I was 2 × 2 = 4 years old. That's an age difference of 4 - 2 = 2 years, and I am older. Now I am 40 years old, so my brother is 40 - 2 = 38 years old. The answer is 38.
Q: When I was 3 years old, my partner was 3 times my age. Now, I am 20 years old. How old is my partner? Let's think step by step.
A: [Let the AI complete this using the pattern from the example]
```
## Common Failure Patterns
### Without CoT (Problematic):
```
Prompt: When I was 3 years old, my partner was 3 times my age. Now, I am 20 years old. How old is my partner?
Output: 63 years old ❌
```
### With CoT (Improved):
```
Prompt: [Same question] Let's think step by step.
Output: [Step-by-step reasoning leading to] 26 years old ✅
```
## When to Use Chain-of-Thought
CoT is particularly effective for tasks that benefit from explicit reasoning:
### Ideal Use Cases:
* **Mathematical Problems**: Arithmetic, algebra, geometry calculations
* **Code Generation**: Breaking down requirements into implementable steps
* **Logical Reasoning**: Puzzles, deduction, inference problems
* **Synthetic Data Creation**: Guided assumption-making and content generation
* **Complex Analysis**: Multi-factor decision making, comparative analysis
* **Process Planning**: Step-by-step procedure development
### Decision Rule:
> **If you can explain the steps to solve the problem manually, CoT will likely improve AI performance.**
## Effective CoT Trigger Phrases
Different trigger phrases work better for different types of problems:
```markdown CoT Triggers theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Mathematical/Logical Problems
"Let's think step by step."
"Let's work through this systematically."
"Let's break this down into steps."
# Analysis Tasks
"Let's analyze this carefully."
"Let's examine each component."
"Let's think through the implications."
# Creative/Planning Tasks
"Let's approach this methodically."
"Let's consider each aspect."
"Let's build this solution piece by piece."
# Code Generation
"Let's implement this step by step."
"Let's break down the requirements first."
"Let's design the solution systematically."
Problem: {{ user_problem }}
{{ trigger_phrase }}
```
## Practical CoT Examples
### Synthetic Data Generation with CoT
```markdown Synthetic Data CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
---
# Product Description Generator
Product: {{ product_name }}
Let's create a compelling product description by thinking through this step by step:
Step 1: Analyze the product name
- What type of product does this suggest?
- What market segment would this target?
- What key features can we infer?
Step 2: Make reasonable assumptions
- Who is the target customer?
- What problems does this solve?
- What are the key selling points?
Step 3: Structure the description
- Opening hook to grab attention
- Key features and benefits
- Social proof or credibility elements
- Call to action
Step 4: Write the description
Based on my analysis and assumptions:
```
### Mathematical Problem Solving
```markdown Advanced Math CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Word Problem Solver
Problem: {{ math_word_problem }}
Let me solve this step by step:
Step 1: Extract the key information
- What quantities are given?
- What relationships exist between them?
- What am I asked to find?
Step 2: Set up the mathematical model
- Define variables for unknown quantities
- Write equations based on the relationships
- Identify the mathematical operations needed
Step 3: Solve systematically
- Perform calculations in logical order
- Show each algebraic step
- Check intermediate results
Step 4: Verify and interpret
- Does the answer make logical sense?
- Does it satisfy the original constraints?
- Express the final answer clearly
Solution:
```
## Advanced CoT with Latitude Chains
LLM perform better when they can reason through complex problems step by step. In the case of Latitude `` blocks what they do is to call the AI only with the content inside the `` block, so the AI can focus on that specific part of the reasoning process. This allows for more structured and manageable reasoning.
Doing this way is more expensive than a single prompt, but it allows for more complex reasoning and better results. Is more expensive because it does N calls to the AI, where N is the number of `` blocks. And the amount of context of the steps is accumulated, so the AI can use all the context of the previous steps.
```markdown Multi-Step CoT Chain theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
---
# Step 1: Problem Analysis
Let's analyze this business scenario step by step: {{ business_scenario }}
## Initial Assessment:
1. **Key Stakeholders**: Who are the main parties involved?
2. **Core Problem**: What is the fundamental issue?
3. **Constraints**: What limitations do we need to consider?
4. **Success Metrics**: How will we measure success?
## Analysis:
# Step 2: Solution Brainstorming
Based on my analysis: {{ problem_analysis }}
Now let me generate potential solutions:
## Brainstorming Process:
1. **Traditional Approaches**: What are the conventional solutions?
2. **Innovative Options**: What creative alternatives exist?
3. **Resource Requirements**: What would each solution need?
4. **Risk Assessment**: What are the potential downsides?
## Potential Solutions:
# Step 3: Solution Evaluation
Given these potential solutions: {{ solution_brainstorming }}
Let me evaluate each option systematically:
## Evaluation Criteria:
1. **Feasibility** (1-10): How realistic is implementation?
2. **Impact** (1-10): How effective will this be?
3. **Cost** (1-10): How resource-efficient is this? (10 = low cost)
4. **Timeline** (1-10): How quickly can this be implemented? (10 = very fast)
## Solution Rankings:
# Step 4: Implementation Planning
Based on the evaluation: {{ solution_evaluation }}
The recommended solution is: [Top-ranked solution]
## Implementation Plan:
1. **Phase 1** (Weeks 1-2): [Initial steps]
2. **Phase 2** (Weeks 3-4): [Development phase]
3. **Phase 3** (Weeks 5-6): [Testing and refinement]
4. **Phase 4** (Weeks 7-8): [Full implementation]
## Risk Mitigation:
- **Risk 1**: [Potential issue] → **Mitigation**: [How to address]
- **Risk 2**: [Potential issue] → **Mitigation**: [How to address]
## Success Metrics:
- **Short-term** (1 month): [Immediate indicators]
- **Medium-term** (3 months): [Progress markers]
- **Long-term** (6+ months): [Ultimate success measures]
```
## CoT for Different Domains
### Scientific Analysis
```markdown Scientific CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Scientific Method with Chain-of-Thought
Apply the scientific method to analyze: {{ research_question }}
## Step 1: Observation and Question Formation
- **Observation**: What have we observed?
- **Research Question**: What specific question are we trying to answer?
- **Background**: What do we already know about this topic?
## Step 2: Hypothesis Development
- **Hypothesis**: What do we predict will happen?
- **Reasoning**: Why do we think this will occur?
- **Variables**: What factors might influence the outcome?
## Step 3: Experimental Design
- **Method**: How would we test this hypothesis?
- **Controls**: What variables need to be controlled?
- **Measurements**: What data would we collect?
## Step 4: Data Analysis Framework
- **Expected Results**: What patterns would support our hypothesis?
- **Alternative Explanations**: What other factors could explain results?
- **Statistical Considerations**: How would we ensure reliability?
## Step 5: Conclusion and Implications
- **Interpretation**: What would different results mean?
- **Limitations**: What are the constraints of this approach?
- **Next Steps**: How would this lead to further research?
## Analysis:
[Apply this framework to the given research question]
```
### Legal Reasoning
```markdown Legal CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Legal Analysis with Chain-of-Thought
Analyze this legal scenario step by step: {{ legal_scenario }}
## Step 1: Fact Pattern Analysis
- **Key Facts**: What are the essential facts?
- **Parties Involved**: Who are the relevant parties?
- **Timeline**: What is the sequence of events?
- **Jurisdiction**: What legal system applies?
## Step 2: Legal Issue Identification
- **Primary Issues**: What are the main legal questions?
- **Secondary Issues**: What related questions arise?
- **Precedent Relevance**: What similar cases might apply?
## Step 3: Rule Identification
- **Applicable Laws**: What statutes or regulations apply?
- **Case Law**: What precedents are relevant?
- **Legal Standards**: What tests or criteria apply?
## Step 4: Application of Law to Facts
- **Element Analysis**: How do the facts satisfy each legal element?
- **Counterarguments**: What opposing positions exist?
- **Distinguishing Cases**: How is this different from precedents?
## Step 5: Conclusion and Reasoning
- **Legal Conclusion**: What is the most likely outcome?
- **Strength of Position**: How strong is each side's case?
- **Risk Assessment**: What are the uncertainties?
## Analysis:
[Apply this legal reasoning framework]
```
## CoT with Self-Correction
```markdown Self-Correcting CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
---
# Initial Reasoning Attempt
Problem: {{ complex_problem }}
Let me work through this step by step:
1. **Understanding**: [Break down the problem]
2. **Approach**: [Choose a method]
3. **Execution**: [Work through the solution]
4. **Result**: [State the initial answer]
Initial Solution:
# Self-Critique and Error Checking
Let me review my initial reasoning: {{ initial_reasoning }}
## Error Checking:
1. **Logic Verification**: Are my reasoning steps sound?
2. **Calculation Check**: Are my computations correct?
3. **Assumption Review**: What assumptions did I make?
4. **Alternative Approaches**: Could I solve this differently?
## Potential Issues Found:
- [List any problems identified]
## Confidence Level**: [High/Medium/Low] because [reasoning]
# Revised Solution (if needed)
Based on my self-critique
If the initial reasoning had issues, let me correct it:
## Corrections Made:
1. **Issue**: [Problem identified]
**Correction**: [How I fixed it]
## Revised Step-by-Step Solution:
[Work through the corrected solution]
## Final Answer: [Corrected result]
Otherwise, confirm the original reasoning:
## Confirmation:
My initial reasoning appears sound. The original answer stands.
## Final Answer: [Original result confirmed]
```
## CoT with Multiple Perspectives
```markdown Multi-Perspective CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
type: agent
agents:
- agents/analyst_a
- agents/analyst_b
- agents/synthesizer
---
# Multi-Perspective Analysis
Analyze this complex issue: {{ complex_issue }}
Use multiple analytical perspectives and then synthesize the findings.
## Analysis Framework:
### Perspective A: {{ perspective_a_description }}
- Apply this analytical lens step by step
- Focus on {{ perspective_a_focus }}
### Perspective B: {{ perspective_b_description }}
- Apply this different analytical approach
- Emphasize {{ perspective_b_focus }}
### Synthesis:
- Compare and contrast the perspectives
- Identify points of agreement and disagreement
- Develop a comprehensive understanding
Coordinate the analysis across agents and provide a unified conclusion.
```
```markdown agents/analyst_a theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
type: agent
---
# Perspective A Analysis: {{ perspective_a_description }}
I'll analyze the issue through this specific lens: {{ complex_issue }}
## Step-by-Step Analysis:
1. **Framework Application**: How does {{ perspective_a_description }} apply here?
2. **Key Factors**: What elements are most important from this perspective?
3. **Methodology**: What analytical tools should I use?
4. **Evidence Gathering**: What information supports this view?
5. **Reasoning Chain**: How do these factors connect?
6. **Conclusions**: What does this perspective suggest?
## Detailed Analysis:
[Work through each step systematically]
## Key Insights from Perspective A:
- [Primary findings]
- [Supporting evidence]
- [Implications]
```
## Integration with Latitude Features
### CoT with Dynamic Variables
```markdown Dynamic CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
---
# Adaptive Chain-of-Thought
The reasoning approach adapts based on the problem type: {{ problem_type }}
{{ if problem_type === "mathematical" }}
## Mathematical Problem-Solving Steps:
1. **Parse the Problem**: Extract numbers, operations, and relationships
2. **Identify the Method**: Choose appropriate mathematical approach
3. **Set Up Equations**: Translate word problem to mathematical expressions
4. **Solve Step-by-Step**: Show all algebraic manipulations
5. **Verify**: Check answer by substitution or alternative method
{{ endif }}
{{ if problem_type === "analytical" }}
## Analytical Reasoning Steps:
1. **Decompose**: Break complex issue into component parts
2. **Research**: Gather relevant information and context
3. **Framework**: Apply appropriate analytical model
4. **Synthesize**: Combine insights from different sources
5. **Conclude**: Draw evidence-based conclusions
{{ endif }}
{{ if problem_type === "creative" }}
## Creative Problem-Solving Steps:
1. **Understand**: Deeply comprehend the challenge
2. **Diverge**: Generate multiple creative options
3. **Combine**: Mix and match ideas innovatively
4. **Evaluate**: Assess feasibility and impact
5. **Refine**: Improve the most promising solutions
{{ endif }}
## Problem to Solve:
{{ user_problem }}
## Step-by-Step Solution:
[Apply the appropriate framework above]
```
### CoT with Tool Integration
```markdown CoT with Tools theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
tools:
- latitude/search
- latitude/extract
---
# Research-Enhanced Chain-of-Thought
Let me solve this complex question step by step: {{ research_question }}
## Step 1: Information Gathering
First, I need to research the current facts:
## Step 2: Information Analysis
Based on the search results, let me analyze:
- **Key Facts**: [Extract relevant information]
- **Data Quality**: [Assess reliability of sources]
- **Gaps**: [Identify missing information]
## Step 3: Additional Research (if needed)
Extract specific data that is still unclear or missing.
## Step 4: Reasoning Chain
Now I'll work through the logic:
1. **Given Information**: [Summarize what we know]
2. **Logical Connections**: [Show how facts relate]
3. **Inference Steps**: [Build the argument]
4. **Supporting Evidence**: [Reference research findings]
## Step 5: Conclusion
Based on this systematic analysis:
[Present final answer with full reasoning]
```
## Best Practices
**Zero-Shot CoT**: Use simple trigger phrases like "Let's think step by step" for straightforward problems
**Few-Shot CoT**: Provide examples when you need consistent reasoning patterns or specific approaches
**Multi-Step Chains**: Use Latitude `` blocks for complex problems requiring focused attention on each phase
**Cost Consideration**: Balance reasoning quality with token costs - more steps = better results but higher costs
**Clear Step Labels**: Use numbered steps or clear headers to guide reasoning
**Logical Flow**: Ensure each step builds logically on the previous one
**Explicit Instructions**: Always include trigger phrases to activate reasoning mode
**Verification Steps**: Include self-checking and validation mechanisms
**Domain-Specific Language**: Use terminology and approaches familiar to the problem domain
**Model Selection**: Use GPT-4 or Claude for complex reasoning tasks
**Temperature Settings**: Lower temperature (0.1-0.3) for logical/mathematical problems
**Token Management**: Balance reasoning detail with cost efficiency
**Error Handling**: Include correction and retry mechanisms
**Robustness**: CoT helps maintain performance across different LLM versions
**Mathematical Problems**: Focus on step-by-step calculations and verification
**Code Generation**: Break down requirements before implementation
**Scientific Analysis**: Emphasize hypothesis formation and testing
**Business Decisions**: Include stakeholder analysis and risk assessment
**Creative Tasks**: Allow for iterative refinement and exploration
**When CoT is Worth It**: Complex reasoning, high-stakes decisions, mathematical problems
**When to Avoid**: Simple factual queries, high-volume/low-cost applications
**Optimization**: Use shorter reasoning chains for simpler problems
**Monitoring**: Track accuracy improvements vs. cost increases
## Common Pitfalls
**Critical Mistakes to Avoid**:
**Reasoning Errors**:
* **Skipping Logical Steps**: Don't let the AI jump to conclusions without showing work
* **Unclear Transitions**: Make connections between steps explicit and logical
* **Missing Verification**: Always include checking mechanisms and validation steps
* **Assuming Expertise**: Remember that LLMs can make confident but incorrect mathematical errors
**Implementation Issues**:
* **Over-complexity**: Keep steps manageable - too many steps can confuse the model
* **Inconsistent Patterns**: When using few-shot, ensure examples follow the same reasoning structure
* **Wrong Trigger Phrases**: Some phrases work better for different problem types
* **Ignoring Context**: Make sure reasoning steps are appropriate for the problem domain
**Cost Management**:
* **Unnecessary Verbosity**: Don't use CoT for simple factual queries that don't need reasoning
* **Excessive Steps**: More steps aren't always better - find the right balance
* **Poor Token Planning**: Account for the 2-3x token increase when budgeting
## When NOT to Use CoT
CoT isn't always the best approach. Avoid it for:
* **Simple Factual Queries**: "What is the capital of France?" doesn't need reasoning steps
* **High-Volume Applications**: When processing thousands of requests where cost matters more than reasoning
* **Well-Defined Formats**: When you need consistent, structured outputs without explanation
* **Time-Sensitive Tasks**: When response speed is more important than reasoning quality
* **Retrieval Tasks**: When the answer exists in a knowledge base and doesn't require reasoning
## Implementation Checklist
When implementing CoT in your prompts, use this checklist:
### ✅ Pre-Implementation
* Confirm the task benefits from step-by-step reasoning
* Choose appropriate CoT type (zero-shot vs few-shot vs multi-step)
* Select effective trigger phrases for your domain
* Plan for increased token costs (typically 2-3x)
### ✅ Prompt Design
* Include clear step labels and logical flow
* Add verification/checking steps
* Provide examples if using few-shot approach
* Test with edge cases and failure scenarios
### ✅ Optimization
* Adjust temperature based on task type (lower for logic/math)
* Monitor accuracy improvements vs cost increases
* Iterate on step structure based on results
* Consider using Latitude `` blocks for complex reasoning
## Key Takeaways
Chain-of-Thought prompting transforms how LLMs approach complex problems by making their reasoning explicit and systematic. Here are the essential points:
**Core Benefits:**
* **Dramatic accuracy improvements** on reasoning tasks without model fine-tuning
* **Explainable results** that allow debugging and validation
* **Robust performance** across different LLM versions
**Best Applications:**
* Mathematical and logical problems
* Code generation with requirement breakdown
* Complex analysis requiring multiple perspectives
* Any task where you can explain the solution steps manually
**Cost Considerations:**
* 2-3x more tokens means higher costs and slower responses
* Use strategically for high-value, complex reasoning tasks
* Consider simpler approaches for basic queries
**Implementation Success Factors:**
* Choose the right CoT variant (zero-shot, few-shot, or multi-step)
* Use domain-appropriate trigger phrases and terminology
* Include verification steps to catch reasoning errors
* Balance reasoning depth with practical constraints
Chain-of-Thought is a low-effort, high-impact technique that can significantly improve AI performance on complex tasks. The key is knowing when and how to apply it effectively.
## Advanced CoT Patterns
### CoT with Error Correction
```markdown Self-Correcting CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
---
# Problem Solving with Validation
Problem: {{ complex_problem }}
## Initial Reasoning
Let me work through this step by step:
1. **Understanding**: [Break down the problem]
2. **Approach**: [Choose methodology]
3. **Execution**: [Show work]
4. **Initial Answer**: [State result]
## Self-Validation
Now let me check my work:
1. **Logic Check**: Are my reasoning steps sound?
2. **Calculation Verification**: Let me double-check any math
3. **Sanity Test**: Does this result make intuitive sense?
4. **Alternative Approach**: Can I solve this differently to confirm?
## Final Answer
Based on validation: [Confirmed or corrected result]
```
### CoT with Confidence Scoring
```markdown Confidence-Aware CoT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Reasoning with Confidence Assessment
Problem: {{ problem_statement }}
## Step-by-Step Analysis
[Standard CoT reasoning steps]
## Confidence Assessment
For each step, I'll rate my confidence (1-10):
- **Step 1 Confidence**: 9/10 - Clear factual information
- **Step 2 Confidence**: 7/10 - Some assumptions required
- **Step 3 Confidence**: 8/10 - Standard methodology applied
- **Overall Confidence**: 8/10
## Risk Factors
- **Potential Issues**: [What could go wrong]
- **Missing Information**: [What would improve confidence]
- **Alternative Scenarios**: [Other possible outcomes]
## Conclusion
Answer: [Result] (Confidence: X/10)
```
## Next Techniques
Explore these related prompting techniques:
* [Tree of Thoughts](/examples/techniques/tree-of-thoughts) - Explore multiple reasoning paths
* [Self-Consistency](/examples/techniques/self-consistency) - Multiple CoT attempts with voting
* [Few-shot Learning](/examples/techniques/few-shot-learning) - CoT with examples
* [Constitutional AI](/examples/techniques/constitutional-ai) - Self-correcting reasoning
# Contextual Prompting
Source: https://docs-v1.latitude.so/examples/techniques/contextual-prompting
Learn how to use contextual prompting to create more efficient and accurate AI interactions
## What is Contextual Prompting?
Contextual prompting is a technique that provides AI models with relevant background information, context, and specific details about the task at hand. By supplying this context within your prompts, you enable the AI to better understand your request and generate more accurate, relevant, and useful responses.
Unlike simple, isolated prompts, contextual prompting gives the model the necessary information to understand the situation, requirements, and desired outcome, leading to significantly improved results.
## Why Use Contextual Prompting?
By providing contextual prompts, you can help ensure that your AI interactions are as seamless and efficient as possible. The model will be able to more quickly understand your request and generate more accurate and relevant responses.
Key benefits include:
* **Improved Accuracy**: Models can better understand the specific requirements and constraints
* **Faster Understanding**: Reduces the need for back-and-forth clarification
* **More Relevant Output**: Results are tailored to your specific use case and context
* **Reduced Ambiguity**: Clear context eliminates guesswork and misinterpretation
* **Enhanced Efficiency**: Fewer iterations needed to achieve the desired outcome
## Contextual Prompting in Latitude
Here's a simple example showing how to provide context for a blog content generation task:
```markdown Blog Content Generator theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
---
# Blog Content Generator with Context
You are writing for a blog about retro 80's arcade video games.
## Context:
- Target audience: Gaming enthusiasts and nostalgia seekers
- Tone: Informative yet engaging, with a touch of nostalgia
- Focus: Historical significance, cultural impact, and technical innovation
- Format: Well-structured articles with clear sections
## Task:
{{ task_description }}
## Additional Context:
{{ additional_context || "No additional context provided." }}
## Output:
Generate content that incorporates the provided context and meets the specific requirements:
```
## Demonstrating the Power of Context: Blog Article Example
**Without Context:**
```markdown Simple Blog Prompt theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 1
---
Suggest 3 topics to write an article about with a few lines of description of what this article should contain.
```
**With Context:**
```markdown Contextual Blog Prompt theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 1
---
Context: You are writing for a blog about retro 80's arcade video games.
Suggest 3 topics to write an article about with a few lines of description of what this article should contain.
```
The contextual version produces more targeted, relevant suggestions like:
* **The Evolution of Arcade Cabinet Design** - Exploring how cabinet designs evolved from early wood and metal cabinets to sleek, neon-lit designs
* **Blast From The Past: Iconic Arcade Games of The 80's** - Featuring iconic games, their innovations, and enduring charm
* **The Rise and Retro Revival of Pixel Art** - Tracing pixel art evolution and its resurgence in modern games
## Context Categories for Better Prompting
Organize your context into these key categories for maximum effectiveness:
### 1. Domain Context
Provide relevant background information about the subject matter, industry, or field.
### 2. Audience Context
Specify who the output is intended for, their knowledge level, and preferences.
### 3. Task Context
Clearly define the specific requirements, constraints, and expected deliverables.
### 4. Tone and Style Context
Describe the desired communication style, formality level, and voice.
### 5. Format Context
Specify the expected structure, length, and presentation format.
## Multi-Domain Contextual Prompting
Use contextual prompting across different domains and applications:
````markdown Multi-Agent Memory theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.5
type: agent
agents:
```markdown Technical Documentation
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
---
# Technical Documentation Generator
## Domain Context:
You are creating documentation for {{ technology_stack }} developers working on {{ project_type }} applications.
## Audience Context:
- **Experience Level**: {{ experience_level }}
- **Time Constraints**: {{ time_constraints }}
- **Primary Goals**: {{ primary_goals }}
## Documentation Request:
{{ documentation_request }}
## Technical Context:
- **Current Setup**: {{ current_setup }}
- **Dependencies**: {{ dependencies }}
- **Constraints**: {{ constraints }}
## Output Requirements:
Generate clear, actionable documentation that includes:
- Step-by-step instructions
- Code examples
- Common pitfalls and solutions
- Testing recommendations
````
```markdown Content Marketing theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
---
# Content Marketing Generator
## Brand Context:
- **Brand**: {{ brand_name }}
- **Industry**: {{ industry }}
- **Brand Voice**: {{ brand_voice }}
- **Target Audience**: {{ target_audience }}
## Campaign Context:
- **Campaign Goal**: {{ campaign_goal }}
- **Content Type**: {{ content_type }}
- **Distribution Channels**: {{ channels }}
- **Key Messages**: {{ key_messages }}
## Market Context:
- **Competitors**: {{ competitors }}
- **Market Trends**: {{ market_trends }}
- **Seasonal Factors**: {{ seasonal_factors }}
## Content Request:
{{ content_request }}
## Output:
Create compelling content that aligns with brand voice and achieves campaign objectives:
```
```markdown Educational Content theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.5
---
# Educational Content Generator
## Educational Context:
- **Subject**: {{ subject }}
- **Learning Level**: {{ learning_level }}
- **Learning Objectives**: {{ learning_objectives }}
- **Time Available**: {{ time_available }}
## Student Context:
- **Background Knowledge**: {{ background_knowledge }}
- **Learning Style**: {{ learning_style }}
- **Common Challenges**: {{ common_challenges }}
## Content Request:
{{ content_request }}
## Pedagogical Requirements:
Create educational content that:
- Builds on existing knowledge
- Uses appropriate examples and analogies
- Includes interactive elements
- Provides clear assessment criteria
```
## Best Practices for Contextual Prompting
**Structure Your Context**:
* Use clear headings and sections
* Prioritize the most important context first
* Keep context concise but comprehensive
* Use bullet points and lists for clarity
**Context Hierarchy**:
* Primary context (essential for understanding)
* Secondary context (helpful for refinement)
* Tertiary context (nice-to-have details)
**High-Quality Context Elements**:
* **Specific and Relevant**: Directly related to the task
* **Actionable**: Provides clear guidance for the AI
* **Current**: Up-to-date and accurate information
* **Complete**: Includes all necessary details
**Context Optimization**:
* Remove redundant information
* Use precise language and terminology
* Include examples when helpful
**Best Use Cases**:
* Complex technical documentation requiring domain expertise
* Content creation with specific brand guidelines
* Educational content tailored to learning levels
* Code generation with specific framework constraints
* Creative writing with genre and style requirements
**Optimization Strategies**:
* Start with broad context, then add specifics
* Use templates for recurring context patterns
* Test different context arrangements for optimal results
* Monitor output quality to refine context effectiveness
**Token Management**:
* Balance context richness with token limits
* Prioritize the most impactful context elements
* Use concise language while maintaining clarity
* Consider context compression for long-term use
**Performance Tips**:
* Cache frequently used context patterns
* Create reusable context templates
* Validate context relevance regularly
* A/B test different context structures
## Advanced Contextual Techniques
### Context Templates for Reusability
Create reusable context templates for common scenarios:
```markdown Context Template System theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
---
# Context Template: {{ template_type }}
{{ template_type }} Context Template - Reusable context structure for {{ use_case }}
## Base Context:
{{ base_context }}
## Variable Context Elements:
{{ variable_context }}
## Template Application:
Apply this template to the current request:
## User Request:
{{ user_request }}
## Populated Context:
Fill in the template with request-specific information:
## Output:
Generate response using the populated context template:
```
## Measuring Context Effectiveness
Track and optimize your contextual prompting performance:
### Key Metrics
* **Response Relevance**: How well outputs match the intended context
* **Task Completion**: Success rate for completing requested tasks
* **Efficiency**: Reduced iterations needed to achieve desired results
* **User Satisfaction**: Quality and usefulness of contextual responses with [HITL evaluations](/guides/evaluations/humans-in-the-loop)
### Optimization Strategies
* **A/B Testing**: Compare different context structures with [experiments](/guides/experiments/overview)
* **Iterative Refinement**: Gradually improve context based on results
* **Template Evolution**: Update successful context patterns
* **Context Validation**: Regularly verify context accuracy and relevance
# Few-shot Prompting
Source: https://docs-v1.latitude.so/examples/techniques/few-shot-prompting
Learn how to implement few-shot learning with examples to improve AI performance on specific tasks
## What is Few-shot Prompting?
Few-shot learning is a prompting technique where you provide the AI with one or several small number of examples (typically 1-10) to demonstrate the desired pattern, format, or behavior before asking it to perform a similar task. This technique leverages the AI's ability to recognize patterns and generalize from limited examples.
## Why Use Few-shot Prompting?
* **Improved Accuracy**: Examples help the AI understand exactly what you want
* **Consistent Format**: Ensures outputs follow a specific structure
* **Reduced Ambiguity**: Clear examples eliminate guesswork
* **Better Context Understanding**: Shows the AI how to handle edge cases
* **Domain Adaptation**: Helps AI adapt to specific domains or styles
## Zero-shot vs Few-shot
Zero-shot prompting involves asking the AI to perform a task without any examples, relying solely on its pre-existing knowledge. Few-shot prompting, on the other hand, provides a few examples to guide the AI's response. Few-shot learning is generally more effective for complex tasks where context and specific patterns are crucial.
## One-shot vs Few-shot
One-shot prompting provides a single example to guide the AI, the idea behind one-shot learning is to show the AI how to perform a task with just one example.
All variants of few-shot prompting (zero-shot, one-shot, and few-shot) can be implemented in Latitude. The choice depends on the complexity of the task and the amount of guidance needed.
## Basic Implementation in Latitude
Here's a simple few-shot learning example for email classification:
```markdown Email Classification theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Email Classification Task
You are an expert email classifier. Based on the examples below, classify the given email
into one of these categories: URGENT, SPAM, BUSINESS, PERSONAL.
## Examples:
**Email**: "CONGRATULATIONS! You've won $1,000,000! Click here now!"
**Category**: SPAM
**Email**: "Hi John, can we reschedule tomorrow's meeting? I have a family emergency."
**Category**: PERSONAL
**Email**: "SYSTEM ALERT: Server down. Immediate attention required. Revenue impact critical."
**Category**: URGENT
**Email**: "Please find attached the Q4 financial report for your review."
**Category**: BUSINESS
## Email to Classify:
{{ email_content }}
## Category:
```
## Advanced Implementation with Variables
Let's create a more sophisticated example that uses [Latitude's parameters system](/guides/prompt-manager/playground#parameter-types):
```markdown Advanced Few-shot theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
---
# {{ product_category }} Product Review Sentiment Analysis
Analyze the sentiment of {{ product_category }} reviews and extract {{ detail_level }} insights. Use the examples below as a guide.
## Examples:
{{ for example in examples }}
**Review**: "{{ example.text }}"
**Sentiment**: {{ example.sentiment }}
**Key Points**: {{ example.key_points }}
**Confidence**: {{ example.confidence }}
{{ endfor }}
## Review to Analyze:
{{ review_text }}
## Analysis:
**Sentiment**:
**Key Points**:
**Confidence**:
```
In this advanced example:
1. **Dynamic Content**: We use templates (`{{ variable }}`) to insert parameters into the prompt.
2. **Templating Features**: We demonstrate control structures like `{{for item in items }}` for arrays.
3. **Runtime Examples**: The `examples` array parameter allows users to pass in any number of examples when calling the prompt.
This pattern makes your prompts more flexible and reusable across different use cases without creating separate prompts for each scenario. For more on parameters, see the [Latitude Parameter Types documentation](/guides/prompt-manager/playground#parameter-types).
## Multi-step Few-shot with Chains
Latitude's chain feature allows you to create complex few-shot workflows:
````markdown Multi-step Analysis theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
---
# Information Extraction
Extract structured information from the business proposal. Follow these examples:
## Examples:
**Input**: "We propose a mobile app for food delivery in NYC with $50K funding needed."
**Output**:
{
"business_type": "mobile app",
"industry": "food delivery",
"location": "NYC",
"funding_needed": "$50K"
}
**Input**: "Looking for $2M Series A for our AI-powered healthcare platform serving hospitals nationwide."
```json
{
"business_type": "AI platform",
"industry": "healthcare",
"location": "nationwide",
"funding_needed": "$2M"
}
```
## Proposal to Extract:
{{ proposal_text }}
## Extracted Information:
# Business Viability Analysis
Based on the extracted information, provide a viability score following these examples:
## Examples:
**Business**: AI healthcare platform, $2M Series A, nationwide
**Viability Score**: 8.5/10
**Reasoning**: Large addressable market, proven demand, appropriate funding level
**Business**: Food delivery app, $50K seed, NYC only
**Viability Score**: 6.0/10
**Reasoning**: Competitive market, limited geographic scope, low initial funding
## Analysis:
**Viability Score**:
**Reasoning**:
````
## Dynamic Few-shot with Conditional Logic
Use Latitude's conditional features to adapt examples based on context:
```markdown Dynamic Few-shot theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
---
# Content Generation with Context-Aware Examples
Generate {{ content_type }} following the appropriate examples below:
{{ if content_type === "social_media" }}
## Social Media Examples:
**Topic**: "New restaurant opening"
**Content**: "🍕 EXCITING NEWS! Our new pizza place opens tomorrow at 123 Main St! First 50 customers get 50% off! #GrandOpening #Pizza #Foodie"
**Topic**: "Product launch"
**Content**: "🚀 Introducing our game-changing productivity app! Streamline your workflow like never before. Download now: [link] #ProductLaunch #Productivity #Innovation"
{{ endif }}
{{ if content_type === "email" }}
## Email Examples:
**Topic**: "Welcome new customer"
**Subject**: "Welcome to [Company] - Your journey starts here!"
**Content**: "Dear [Name], Thank you for choosing [Company]. We're excited to help you achieve your goals. Here's what to expect next..."
**Topic**: "Product announcement"
**Subject**: "Introducing [Product] - Transform your [Industry]"
**Content**: "Hi [Name], We're thrilled to announce [Product], designed specifically for professionals like you who want to [benefit]..."
{{ endif }}
{{ if content_type === "blog" }}
## Blog Examples:
**Topic**: "Industry trends"
**Content**: "# The Future of [Industry]: 5 Trends to Watch in 2024\n\nThe [industry] landscape is evolving rapidly. Here are the key trends shaping our future:\n\n## 1. [Trend Name]\n[Detailed explanation...]"
**Topic**: "How-to guide"
**Content**: "# Step-by-Step Guide: How to [Action]\n\nAre you struggling with [problem]? This comprehensive guide will walk you through everything you need to know.\n\n## Prerequisites\n- [Requirement 1]\n- [Requirement 2]"
{{ endif }}
## Content Request:
**Topic**: {{ topic }}
**Target Audience**: {{ audience }}
**Key Message**: {{ key_message }}
## Generated {{ content_type }}:
```
## Few-shot with Agent Collaboration
Combine few-shot learning with Latitude's agent system for complex workflows:
```markdown Multi-Agent Few-shot theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
type: agent
agents:
- agents/data_extractor
- agents/validator
- agents/formatter
---
# Document Processing Pipeline
Process the document using specialized agents, each with their own few-shot examples.
## Document Input:
{{ document_content }}
## Processing Steps:
1. **Data Extraction**: Use the data_extractor agent to pull key information
2. **Validation**: Use the validator agent to verify accuracy
3. **Formatting**: Use the formatter agent to structure output
Process the document and provide the final structured result.
```
```markdown agents/data_extractor theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
type: agent
---
# Data Extraction Specialist
Extract key data points from documents following these examples:
## Examples:
**Document**: "John Smith, Software Engineer at TechCorp, joined January 2023, salary $95,000"
**Extracted Data**:
- Name: John Smith
- Position: Software Engineer
- Company: TechCorp
- Start Date: January 2023
- Salary: $95,000
**Document**: "Meeting scheduled for March 15th, 2024 at 2:00 PM in Conference Room A with the Marketing team"
**Extracted Data**:
- Event Type: Meeting
- Date: March 15th, 2024
- Time: 2:00 PM
- Location: Conference Room A
- Attendees: Marketing team
## Document to Process:
{{ document_content }}
## Extracted Data:
```
## Best Practices for Few-shot Prompting
**Choose Representative Examples**:
* Cover different scenarios and edge cases
* Include both positive and negative examples
* Ensure examples match your target domain
* Use diverse input formats when applicable
**Example Quality**:
* Make examples clear and unambiguous
* Include enough detail without being verbose
* Show consistent formatting patterns
* Demonstrate the reasoning process when needed
**Optimal Structure**:
1. **Task Description**: Clear explanation of what you want
2. **Examples Section**: 2-10 well-chosen examples
3. **Input Section**: Where the new data goes
4. **Output Section**: Where the response should go
**Formatting Tips**:
* Use consistent separators between examples
* Clearly label input and output sections
* Include field names for structured outputs
* Use markdown formatting for readability
**Dynamic Examples**:
* Use Latitude variables to customize examples
* Implement conditional logic for context-aware examples
* Store example sets in prompt references for reusability
* Allow users to provide their own examples when needed
**Example with Variables**:
```markdown theme={null}
{{ for example in examples }}
**Input**: {{ example.input }}
**Output**: {{ example.output }}
{{ endfor }}
```
**Token Management**:
* Balance between example quantity and token efficiency
* Use the most informative examples
* Consider using shorter examples for simple tasks
* Cache common example sets using prompt references
**Model Selection**:
* Use more capable models (GPT-4) for complex few-shot tasks
* Consider fine-tuning for repeated patterns
* Adjust temperature based on creativity needs
* Test with different model sizes
## Advanced Techniques
### Self-Improving Few-shot
Create prompts that can improve their own examples:
```markdown Self-Improving theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
---
# Process the task with current examples
Current examples:
{{ current_examples }}
Task: {{ user_task }}
Result:
# Evaluate the result quality
Result: {{ initial_processing }}
Was this result satisfactory? If not, what examples would improve future performance?
Evaluation:
# Generate improved examples if needed
Based on the evaluation. If needs improvement. Generate 2-3 new examples that would help with tasks like: {{ user_task }}
New examples should address the identified weaknesses.
Improved Examples:
```
### Cross-Domain Transfer
Use few-shot learning to transfer patterns across domains:
```markdown Cross-Domain Transfer theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
---
# Cross-Domain Pattern Transfer
Learn the pattern from {{ source_domain }} examples and apply it to {{ target_domain }}.
## Source Domain Examples ({{ source_domain }}):
{{ for example in source_examples }}
**Input**: {{ example.input }}
**Pattern Applied**: {{ example.pattern }}
**Output**: {{ example.output }}
{{ endfor }}
## Pattern Recognition:
The consistent pattern across these examples is: {{ identified_pattern }}
## Target Domain Application ({{ target_domain }}):
Now apply the same pattern to this {{ target_domain }} scenario:
**Input**: {{ target_input }}
**Pattern to Apply**: {{ identified_pattern }}
**Output**:
```
## Common Pitfalls and Solutions
**Avoid These Common Mistakes**:
* **Too Many Examples**: More isn't always better; 3-7 examples are usually optimal
* **Inconsistent Formatting**: Make sure all examples follow the same structure
* **Biased Examples**: Include diverse scenarios to avoid model bias
* **Unclear Boundaries**: Clearly separate examples from the actual task
**Pro Tips**:
* Start with 2-3 examples and add more if needed
* Test your few-shot prompts with edge cases
* Use Latitude's version control to iterate on example sets
* Combine with other techniques like Chain-of-Thought for complex reasoning
## Next Steps
Now that you understand few-shot learning, explore these related techniques:
* [Chain-of-Thought](/examples/techniques/chain-of-thought) - Add reasoning steps to your examples
* [Template-based Prompting](/examples/techniques/template-based-prompting) - Structure your few-shot examples
* [Role Prompting](/examples/techniques/role-prompting) - Combine examples with specific roles
* [Self-Consistency](/examples/techniques/self-consistency) - Use multiple few-shot attempts for better results
# Other Prompting Techniques
Source: https://docs-v1.latitude.so/examples/techniques/other-techniques
Expand your prompting skills with these advanced and specialized techniques
While the [main prompting techniques](/examples/overview#prompting-techniques) cover the fundamental approaches that most developers need, these additional techniques can help you expand your prompting skills and tackle more specialized challenges. These advanced techniques are perfect for specific use cases and can significantly enhance your AI applications when applied correctly.
For beginners, we recommend starting with the [main prompting techniques](/examples/overview#prompting-techniques) before exploring these specialized approaches.
## Advanced Techniques
Explore these specialized prompting techniques to handle complex scenarios and push the boundaries of what's possible with LLM applications.
Implement AI safety principles and ethical guidelines directly into your prompts
Create structured, reusable prompt templates for consistent AI interactions
Use AI to generate and optimize prompts automatically for better performance
Enhance AI responses by incorporating external knowledge sources and databases
Work with text, images, and other data types in unified AI interactions
Guide AI to deeper understanding through systematic inquiry and questioning
Leverage comparisons and analogies to improve AI problem-solving capabilities
Progressively improve AI outputs through cycles of feedback and enhancement
Define specific boundaries and limitations to guide AI behavior and outputs
Incorporate emotional awareness and empathy into AI interactions
Test and strengthen AI systems against potential misuse and edge cases
Manage long-term context and memory across extended AI conversations
These techniques can be combined with the main prompting approaches for even more powerful results. Experiment with different combinations to find what works best for your specific use case.
# Prompt with Guardrails
Source: https://docs-v1.latitude.so/examples/techniques/prompt-with-guardrails
Learn how to implement validation guardrails to ensure AI outputs meet quality standards and safety requirements
## What are Prompt Guardrails?
Prompt guardrails are validation mechanisms that monitor and control AI outputs to ensure they meet specific quality, safety, and compliance standards. Unlike constraint-based prompting that sets boundaries upfront, guardrails act as continuous validators that check outputs after generation and can trigger corrections or regeneration when standards aren't met.
## Why Use Prompt Guardrails?
* **Quality Assurance**: Ensures outputs consistently meet predefined standards
* **Safety Compliance**: Prevents harmful, inappropriate, or policy-violating content
* **Iterative Improvement**: Automatically refines outputs through validation loops
* **Confidence Building**: Provides measurable quality scores for output reliability
* **Risk Mitigation**: Catches and corrects potential issues before user delivery
* **Automated Workflows**: Enables fully automated content generation with quality control
* **Scalable Standards**: Maintains consistent quality across high-volume operations
## Basic Implementation in Latitude
Here's a simple guardrail example for content validation:
```markdown Basic Content Guardrails theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
---
# Content Generator with Basic Guardrails
Generate content for: {{ topic }}
## Requirements:
- Professional tone
- Factually accurate
- 200-300 words
- No controversial statements
## Content:
[Generate content here]
## Self-Validation:
Rate this content on a scale of 1-10 for:
- Professional tone:
- Factual accuracy:
- Length appropriateness:
- Controversy avoidance:
If any score is below 7, regenerate the content with improvements.
```
## Advanced Implementation with Agent Validators
The most effective guardrails use dedicated validator agents that can provide objective, measurable feedback:
```markdown Email Rewriter with Guardrails theme={null}
---
provider: OpenAI
model: gpt-4.1
maxSteps: 10
type: agent
agents:
- validator
---
Rewrite the email below in a more upbeat tone (remain concise):
{{ email }}
Here are two examples of dull emails and their upbeat counterparts:
**Dull Email 1:**
Subject: Meeting Confirmation
Hi Team,
This is to confirm our meeting scheduled for Thursday at 3 PM. Please be on time.
Regards,
Alex
**Upbeat Email 1:**
Subject: Exciting Meeting Ahead!
Hey Team!
I'm thrilled to confirm our meeting this Thursday at 3 PM! Let's make sure to bring our best ideas and energy!
Can't wait to see you all there!
Cheers,
Alex
**Dull Email 2:**
Subject: Project Update
Dear Colleagues,
I wanted to inform you that the project is still in progress. We will update you when we have more information.
Sincerely,
Jordan
**Upbeat Email 2:**
Subject: Exciting Project Update!
Hello Team!
I'm excited to share that our project is moving along nicely! Stay tuned for more updates as we continue to make progress!
Best,
Jordan
After rewriting the email, check with the validator tool to see if you did well. Complete the task once the validator returns a score >0.85. If the score is lower, try rewriting the email and checking with the validator again.
Return only the rewritten email.
```
```markdown validator theme={null}
---
provider: OpenAI
model: gpt-4o
schema:
type: object
properties:
score:
type: number
required:
- score
additionalProperties: false
type: agent
---
Please evaluate if the following email was rewritten to a more upbeat tone. Make sure the tone is still professional and the email doesn't overuse exclamation points
Original:
{{ original_email }}
Rewritten:
{{ rewritten_email }}
Return a score from 0 to 1.
```
In this advanced example:
1. **Quality Threshold**: The system only accepts outputs scoring above 0.85
2. **Iterative Refinement**: Low scores trigger automatic regeneration
3. **Objective Validation**: A dedicated validator agent provides measurable feedback
4. **Structured Output**: The validator returns a standardized score format
5. **Professional Balance**: Guardrails prevent over-enthusiasm while ensuring upbeat tone
## Best Practices for Prompt Guardrails
### Threshold Management
* **Conservative Thresholds**: Start with higher thresholds (0.8-0.9) for critical applications
* **Adaptive Thresholds**: Lower thresholds for creative tasks, higher for factual content
* **Multiple Metrics**: Use composite scores rather than single metrics
* **Escalation Paths**: Define what happens when content consistently fails validation
### Validator Design
* **Specific Criteria**: Make validation criteria as specific and measurable as possible
* **Structured Output**: Use schemas to ensure consistent, parseable validator responses
* **Domain Expertise**: Design validators with relevant domain knowledge
* **Bias Prevention**: Include checks for common biases and blind spots
Prompt guardrails represent a crucial evolution in AI safety and quality assurance, enabling automated systems that maintain high standards while operating at scale. When combined with other techniques like constraint-based prompting and chain-of-thought reasoning, they create robust, reliable AI applications suitable for production environments.
# ReAct (Reasoning and Acting) Prompting
Source: https://docs-v1.latitude.so/examples/techniques/re-act-prompting
Learn how to combine reasoning and acting in a thought-action loop to solve complex tasks using external tools
## What is ReAct Prompting?
ReAct (Reasoning and Acting) prompting is a paradigm that enables AI models to solve complex tasks by combining natural language reasoning with external tool interactions. It mimics human problem-solving by creating a thought-action loop where the model reasons about the problem, takes actions to gather information, observes results, and iteratively refines its approach until reaching a solution.
## Why Use ReAct Prompting?
* **Complex Problem Solving**: Handles multi-step tasks requiring external information
* **Dynamic Information Access**: Retrieves real-time data through tool interactions
* **Human-like Reasoning**: Mirrors how humans think and act to solve problems
* **Iterative Improvement**: Learns from action results to refine strategies
* **Tool Integration**: Seamlessly combines reasoning with external capabilities
* **Transparent Process**: Shows the thinking and action steps for explainability
* **Agent-like Behavior**: First step towards autonomous agent modeling
## How ReAct Prompting Works
ReAct operates through a continuous thought-action loop:
1. **Thought**: The model reasons about the current state and plans next actions
2. **Action**: The model executes tools or queries to gather information
3. **Observation**: The model processes the results from actions
4. **Iteration**: The cycle repeats with updated understanding until goal completion
This process requires careful prompt management, including maintaining conversation history and trimming excessive content to stay within context limits.
## Basic Implementation in Latitude
Here's a simple ReAct example using Latitude's built-in tools:
```markdown Basic ReAct Research theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
tools:
- latitude/search
- latitude/extract
---
# Research Assistant with ReAct
I will help you research {{ research_topic }} using a systematic thought-action approach.
## Instructions:
Follow this ReAct pattern:
1. **Thought**: Reason about what information you need
2. **Action**: Use tools to gather that information
3. **Observation**: Analyze the results
4. **Thought**: Plan your next step based on what you learned
5. Repeat until you have comprehensive information
Let me start researching {{ research_topic }}:
**Thought**: I need to understand the current state and recent developments in {{ research_topic }}. Let me start with a broad search to get an overview.
**Action**: I'll search for recent information about {{ research_topic }}.
```
## Advanced ReAct Implementation
For more complex tasks, create structured ReAct workflows:
```markdown Advanced ReAct Analysis theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
tools:
- latitude/search
- latitude/extract
- latitude/code
---
# ReAct Problem Solving Framework
I'm tasked with: {{ complex_task }}
## Initial Assessment:
**Thought**: Let me break down this complex task and identify what information and actions I need to complete it successfully.
1. **Problem Analysis**: What are the key components of this task?
2. **Information Requirements**: What data do I need to gather?
3. **Tool Strategy**: Which tools will be most effective?
4. **Success Criteria**: How will I know when the task is complete?
Let me begin the ReAct process:
# ReAct Execution Cycle
Previous context: {{ initial_assessment }}
## Thought-Action Loop:
**Thought 1**: Based on my analysis, I need to start by {{ first_reasoning_step }}
**Action 1**: [Execute first action using appropriate tools]
**Observation 1**: [Process and analyze results]
**Thought 2**: Given these results, my next step should be {{ next_reasoning_step }}
**Action 2**: [Execute second action]
**Observation 2**: [Analyze new information]
Continue this pattern until task completion...
# Synthesis and Conclusion
Previous ReAct cycles: {{ execution_cycles }}
## Final Synthesis:
**Thought**: Now I need to synthesize all the information I've gathered and provide a comprehensive response.
**Final Analysis**: [Combine all observations and reasoning]
**Conclusion**: [Present final results and recommendations]
**Reflection**: [Evaluate the effectiveness of the ReAct process]
```
## Domain-Specific ReAct Applications
### Market Research ReAct
```markdown Market Research ReAct theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
tools:
- latitude/search
- latitude/extract
- latitude/code
---
# Market Research Agent
Research market opportunity for: {{ product_concept }}
## ReAct Research Process:
**Thought**: To assess this market opportunity, I need to gather data on market size, competition, trends, and customer needs. Let me start systematically.
**Action**: Search for market size and growth data for {{ product_concept }}
[Tool will execute search]
**Observation**: [Analyze market size data]
**Thought**: Now I need competitive intelligence. Who are the key players and what gaps exist?
**Action**: Search for competitors and competitive analysis in {{ market_segment }}
[Continue ReAct cycle through:]
- Market trends analysis
- Customer pain points research
- Regulatory considerations
- Technology landscape assessment
**Final Market Assessment**: [Synthesized conclusion]
```
### Technical Problem Solving ReAct
```markdown Technical ReAct Debugging theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
tools:
- latitude/search
- latitude/code
- latitude/extract
---
# Technical Debugging Assistant
Debug and solve: {{ technical_problem }}
## ReAct Debugging Process:
**Thought**: I need to understand this technical issue systematically. Let me gather information about the error, check documentation, and test potential solutions.
**Action**: Search for common causes and solutions for {{ error_type }}
**Observation**: [Analyze search results for patterns]
**Thought**: Based on these patterns, let me examine the specific technical details and run some diagnostic code.
**Action**: Execute diagnostic code to analyze {{ system_component }}
**Observation**: [Review diagnostic results]
**Thought**: The diagnostics suggest {{ hypothesis }}. Let me verify this with additional research and testing.
[Continue ReAct cycle through:]
- Documentation research
- Code analysis
- Solution testing
- Validation steps
**Solution**: [Present debugged solution with reasoning]
```
### Content Creation ReAct
```markdown Content ReAct Strategy theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
tools:
- latitude/search
- latitude/extract
---
# Strategic Content Creator
Create content strategy for: {{ content_topic }}
## ReAct Content Development:
**Thought**: To create effective content, I need to understand the audience, analyze successful content in this space, and identify unique angles.
**Action**: Research trending content and successful approaches for {{ content_topic }}
**Observation**: [Analyze content trends and engagement patterns]
**Thought**: Now I need to understand the target audience better and identify content gaps.
**Action**: Search for audience demographics and content preferences in {{ target_market }}
**Observation**: [Review audience insights]
**Thought**: With this audience data, let me identify unique angles and content opportunities.
[Continue ReAct cycle through:]
- Competitive content analysis
- SEO and keyword research
- Platform-specific optimization
- Content format testing
**Content Strategy**: [Present comprehensive strategy]
```
## Best Practices for ReAct Prompting
**Clear Pattern Establishment**:
* Always label thoughts, actions, and observations explicitly
* Maintain consistent format throughout the conversation
* Ensure each thought logically leads to the next action
* Make observations comprehensive and actionable
**Reasoning Quality**:
* Encourage detailed reasoning in thought phases
* Connect new information to previous observations
* Show how each action builds toward the goal
* Maintain logical flow between iterations
**Effective Tool Usage**:
* Choose appropriate tools for each information need
* Combine multiple tools when necessary
* Use tool results to inform subsequent actions
* Validate information across multiple sources
**Tool Strategy**:
* Start with broad searches, then narrow focus
* Use extraction tools for detailed analysis
* Employ code tools for calculations and data processing
* Chain tool calls for complex workflows
**Conversation History**:
* Maintain relevant context from previous cycles
* Trim excessive detail while preserving key insights
* Reference previous observations in new reasoning
* Build cumulative understanding over iterations
**Memory Optimization**:
* Summarize key findings periodically
* Remove redundant information to save tokens
* Prioritize recent and relevant context
* Use step-based approaches for complex tasks
**Validation Techniques**:
* Cross-verify information from multiple sources
* Test hypotheses through targeted actions
* Evaluate solution effectiveness before concluding
* Maintain skeptical reasoning throughout
**Error Handling**:
* Acknowledge when tools return unexpected results
* Adjust strategy based on failed actions
* Seek alternative information sources
* Document limitations and assumptions
## Advanced ReAct Techniques
### Multi-Agent ReAct
Coordinate multiple specialized agents in ReAct loops:
```markdown Multi-Agent ReAct theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
tools:
- latitude/search
- latitude/extract
- latitude/code
type: agent
agents:
- agents/researcher
- agents/analyst
- agents/strategist
---
# Multi-Agent ReAct Coordination
Task: {{ complex_multi_faceted_task }}
## Coordinated ReAct Process:
**Coordination Thought**: This task requires multiple specialized perspectives. Let me coordinate researcher, analyst, and strategist agents in a ReAct workflow.
**Action**: Initiate research phase with specialist agents
[Agents execute their ReAct cycles]
**Observation**: Synthesize findings from all agent perspectives
**Coordination Thought**: Based on multi-agent insights, determine next coordinated actions
[Continue coordinated ReAct cycles]
**Final Synthesis**: Integrate all agent findings into comprehensive solution
```
### Hierarchical ReAct
Structure ReAct with multiple levels of planning:
```markdown Hierarchical ReAct theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.2
tools:
- latitude/search
- latitude/extract
- latitude/code
---
# Strategic ReAct Planning
High-level task: {{ strategic_objective }}
**Strategic Thought**: I need to break this into manageable sub-goals and plan a hierarchical approach.
## Strategic Planning:
1. **Goal Decomposition**: Break into sub-objectives
2. **Priority Setting**: Determine order of operations
3. **Resource Planning**: Identify tool and information needs
4. **Success Metrics**: Define completion criteria
**Strategic Action**: Define detailed execution plan
# Tactical ReAct Execution
Strategic plan: {{ strategic_plan }}
## Tactical ReAct Cycles:
For each sub-objective:
**Tactical Thought**: [Specific reasoning for this sub-goal]
**Tactical Action**: [Focused tool usage]
**Tactical Observation**: [Sub-goal specific analysis]
[Repeat for each tactical objective]
# Operational ReAct Implementation
Tactical progress: {{ tactical_results }}
## Operational Actions:
**Operational Thought**: Now execute specific implementation steps
**Operational Action**: [Detailed implementation]
**Operational Observation**: [Immediate results]
[Rapid operational cycles]
**Integration**: Combine operational results with tactical and strategic levels
```
### Self-Correcting ReAct
Implement error detection and correction in ReAct loops:
```markdown Self-Correcting ReAct theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
tools:
- latitude/search
- latitude/extract
- latitude/code
---
# Self-Correcting ReAct System
## Enhanced ReAct with Error Detection:
**Thought**: [Standard reasoning]
**Action**: [Tool execution]
**Observation**: [Standard result analysis]
**Validation Thought**: Let me check if this information seems accurate and complete
**Validation Action**: Cross-reference with additional sources
**Validation Observation**: [Quality assessment]
**Correction Thought**: [If errors detected] I need to correct my approach because [reasoning]
**Correction Action**: [Alternative approach]
**Correction Observation**: [Verified results]
**Meta-Thought**: Evaluate the effectiveness of my ReAct process and adjust if needed
[Continue with improved approach]
```
## Integration with Other Techniques
ReAct prompting combines effectively with other approaches:
* **Chain-of-Thought + ReAct**: Detailed reasoning within each thought phase
* **Self-Consistency + ReAct**: Multiple ReAct cycles to verify solutions
* **Step-Back + ReAct**: Establish principles before action planning
* **Few-Shot + ReAct**: Provide examples of effective thought-action patterns
## Common Patterns and Templates
### The "Investigation" Pattern
* Thought: Identify information gaps
* Action: Gather specific data
* Observation: Analyze findings
* Repeat: Drill deeper or pivot based on results
### The "Problem-Solving" Pattern
* Thought: Hypothesize solutions
* Action: Test hypotheses with tools
* Observation: Evaluate effectiveness
* Iterate: Refine approach based on results
### The "Synthesis" Pattern
* Thought: Plan comprehensive analysis
* Action: Gather diverse information sources
* Observation: Compare and contrast findings
* Conclude: Synthesize insights into coherent solution
ReAct prompting transforms AI from passive responders to active problem-solvers, enabling complex task completion through iterative reasoning and tool-assisted action cycles.
# Role Prompting
Source: https://docs-v1.latitude.so/examples/techniques/role-prompting
Enhance AI performance by assigning specific roles, personas, and expertise areas
## What is Role Prompting?
Role prompting is a technique where you assign the AI a specific role, profession, or persona to guide its responses. By taking on characteristics of experts, characters, or specific viewpoints, the AI can provide more targeted, contextual, and specialized responses.
## Why Use Role Prompting?
* **Specialized Knowledge**: Access domain-specific expertise and terminology
* **Consistent Perspective**: Maintain a specific viewpoint throughout conversations
* **Enhanced Context**: Better understanding of audience and communication style
* **Improved Accuracy**: Leverage professional standards and best practices
* **Engaging Interactions**: More natural and relatable communication
## Basic Implementation
```markdown Simple Role Assignment theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.3
---
# Expert Financial Advisor
You are a seasoned financial advisor with 15 years of experience helping individuals plan for retirement.
You have a CFA certification and specialize in long-term wealth building strategies.
Your communication style is:
- Professional yet approachable
- Data-driven but easy to understand
- Always considers risk tolerance
- Provides actionable recommendations
## Client Situation:
{{ client_scenario }}
## Your Expert Advice:
Analyze the situation and provide comprehensive financial guidance as this experienced advisor would.
```
## Advanced Role-Based Chains
```markdown Multi-Role Consultation theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.4
---
# Medical Analysis
I am Dr. Sarah Chen, Chief Medical Officer with 20 years in emergency medicine and public health policy. I've published extensively on healthcare systems and patient safety.
## Medical Perspective on: {{ health_scenario }}
From my medical expertise:
1. **Clinical Assessment**: What are the immediate health considerations?
2. **Risk Factors**: What medical risks should we be aware of?
3. **Evidence Base**: What does current medical research tell us?
4. **Patient Safety**: How do we ensure optimal health outcomes?
## Medical Recommendations:
# Policy Analysis
I am Michael Rodriguez, a healthcare policy analyst with expertise in health economics and regulatory frameworks. I've advised governments on healthcare reform for over a decade.
## Policy Perspective on: {{ health_scenario }}
Based on the previous medical analysis
From a policy standpoint:
1. **Regulatory Compliance**: What legal requirements apply?
2. **Economic Impact**: What are the cost implications?
3. **Implementation**: How would this work in practice?
4. **Stakeholder Effects**: Who would be impacted and how?
## Policy Recommendations:
# Patient Advocacy
I am Lisa Thompson, a patient rights advocate who has spent 12 years fighting for patient access and healthcare equity.
I focus on ensuring patients have voice and choice in their care.
## Patient Advocacy Perspective on: {{ health_scenario }}
Considering both medical analysis and policy implications
From the patient's perspective:
1. **Patient Rights**: What rights and choices do patients have?
2. **Accessibility**: How accessible is this to all populations?
3. **Quality of Life**: How does this impact daily living?
4. **Informed Consent**: What information do patients need?
## Patient Advocacy Position:
# Integrated Recommendation
As the committee chair, I'll synthesize all perspectives:
- **Medical Expert Opinion**
- **Policy Analysis**
- **Patient Advocacy**
## Balanced Recommendation:
[Integrate all viewpoints into a comprehensive recommendation]
```
## Dynamic Role Assignment
```markdown Adaptive Roles theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.3
---
# Adaptive Expert System
Based on the question type, I'll assume the most appropriate expert role.
## Question: {{ user_question }}
{{ if role_assumed == "legal" }}
**Role Assumed**: Senior Partner at a top-tier law firm
**Expertise**: Corporate law, litigation, regulatory compliance
**Experience**: 25 years practicing law, frequent expert witness
**Approach**: Methodical, evidence-based, risk-aware
{{ endif }}
{{ if role_assumed == "technical" }}
**Role Assumed**: Principal Software Architect
**Expertise**: System design, scalability, best practices
**Experience**: 15 years in tech, led architecture for Fortune 500 companies
**Approach**: Pragmatic, performance-focused, future-proof solutions
{{ endif }}
{{#if role_assumed == "marketing" }}
**Role Assumed**: Chief Marketing Officer
**Expertise**: Brand strategy, digital marketing, customer acquisition
**Experience**: 12 years growing startups to IPO, award-winning campaigns
**Approach**: Data-driven, customer-centric, growth-oriented
{{ endif }}
{{#if role_assumed == "finance" }}
**Role Assumed**: Investment Managing Director
**Expertise**: Portfolio management, risk assessment, market analysis
**Experience**: 18 years on Wall Street, managed $2B+ in assets
**Approach**: Conservative, diversified, long-term focused
{{ endif }}
## Expert Response:
As this {{ role_assumed }}, here's my professional analysis and recommendations:
[Provide response in character with appropriate expertise and perspective]
```
## Multi-Agent Role-Based System
```markdown Role-Based Agents theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.3
type: agent
agents:
- agents/ceo_perspective
- agents/cto_perspective
- agents/cfo_perspective
- agents/cmo_perspective
---
# Executive Team Decision Making
We need to make a strategic decision about: {{ business_decision }}
Convene the executive team meeting. Each C-level executive should provide their perspective based on their role and expertise.
## Meeting Agenda:
1. **CEO**: Strategic vision and leadership perspective
2. **CTO**: Technical feasibility and innovation angle
3. **CFO**: Financial implications and risk assessment
4. **CMO**: Market opportunity and customer impact
## Decision Framework:
- Each executive presents their analysis
- Identify areas of agreement and conflict
- Develop consensus recommendations
- Create implementation priorities
Execute the meeting and provide a unified executive recommendation.
```
```markdown agents/ceo_perspective theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.3
type: agent
path: agents/ceo_perspective
---
# Chief Executive Officer Perspective
I am Alexandra Kim, CEO of this company for 8 years. I built this organization from a startup to a mid-market leader.
My focus is on long-term strategy, stakeholder value, and organizational health.
## CEO Analysis Framework:
### Strategic Vision Assessment
- **Alignment**: How does this fit our 5-year strategy?
- **Competitive Advantage**: Will this strengthen our market position?
- **Stakeholder Impact**: How will this affect employees, customers, investors?
- **Resource Allocation**: Is this the best use of our resources?
### Leadership Considerations
- **Organizational Readiness**: Can we execute this effectively?
- **Cultural Fit**: Does this align with our values and culture?
- **Change Management**: What leadership will be required?
- **Communication Strategy**: How do we message this to stakeholders?
## Decision: {{ business_decision }}
## CEO Perspective:
[Provide strategic analysis from CEO viewpoint]
## Recommendation:
**Position**: [Support/Oppose/Modify]
**Reasoning**: [Strategic rationale]
**Conditions**: [Any requirements for support]
```
## Character-Based Roles
```markdown Historical Figure Role theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.4
---
# Historical Wisdom Council
Channel the wisdom of great historical figures to address: {{ modern_challenge }}
## Council Members:
### Leonardo da Vinci (Renaissance Polymath)
*"Simplicity is the ultimate sophistication."*
As Leonardo, I approach this through:
- **Observation**: What can we learn from nature and systems?
- **Innovation**: How can we combine different disciplines?
- **Experimentation**: What novel approaches should we test?
**Leonardo's Perspective**: [Analysis from da Vinci's viewpoint]
### Benjamin Franklin (Founding Father & Inventor)
*"An investment in knowledge pays the best interest."*
As Franklin, I consider:
- **Practical Wisdom**: What's the most pragmatic approach?
- **Long-term Thinking**: How will this affect future generations?
- **Diplomatic Solutions**: Can we find win-win outcomes?
**Franklin's Perspective**: [Analysis from Franklin's viewpoint]
### Marie Curie (Pioneer Scientist)
*"Nothing in life is to be feared, it is only to be understood."*
As Curie, I focus on:
- **Scientific Method**: What evidence guides our decisions?
- **Persistence**: How do we overcome obstacles?
- **Breaking Barriers**: What conventional thinking should we challenge?
**Curie's Perspective**: [Analysis from Curie's viewpoint]
## Synthesized Wisdom:
Drawing from all perspectives: [Integrated historical wisdom]
```
## Role-Specific Communication Styles
```markdown Communication Adaptation theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.3
---
# Role-Adapted Communication
Topic: {{ communication_topic }}
Audience: {{ target_audience }}
{{ if target_audience === "executives" }}
**Role**: Senior Management Consultant
**Communication Style**:
- Executive summary format
- Focus on ROI and strategic impact
- Data-driven insights
- Clear recommendations with timelines
- Risk mitigation strategies
**Key Phrases**: "Strategic implications," "competitive advantage," "stakeholder value"
{{endif}}
{{ if target_audience === "developers" }}
**Role**: Principal Engineer
**Communication Style**:
- Technical depth and accuracy
- Implementation details
- Performance considerations
- Best practices and standards
- Scalability and maintainability
**Key Phrases**: "Technical debt," "architecture patterns," "performance optimization"
{{ endif }}
{{ if audience === "general_public" }}
**Role**: Science Communicator
**Communication Style**:
- Simple, accessible language
- Real-world analogies
- Step-by-step explanations
- Visual descriptions
- Practical applications
**Key Phrases**: "Simply put," "imagine if," "this means"
{{ endif }}
## Message Delivery:
[Craft message appropriate for role and audience]
```
## Role Prompting with Constraints
```markdown Constrained Role theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.2
---
# Specialized Expert with Constraints
**Role**: {{ expert_role }}
**Constraints**: {{ role_constraints }}
**Context**: {{ situation_context }}
## Role Definition:
You are {{ expert_role }} with the following characteristics:
- **Expertise**: {{ specific_expertise }}
- **Experience**: {{ years_experience }} years in the field
- **Specialization**: {{ specialty_area }}
- **Notable Achievement**: {{ key_accomplishment }}
## Operating Constraints:
{{ for constraint in role_constraints }}
- **{{ constraint.type }}**: {{ constraint.description }}
{{ endfor }}
## Professional Standards:
- Always cite relevant industry standards
- Consider ethical implications
- Acknowledge limitations of expertise
- Provide disclaimers when appropriate
## Current Situation:
{{ situation_context }}
## Expert Analysis:
[Provide analysis within role constraints and professional standards]
## Recommendations:
1. **Immediate Actions**: [What should be done right away]
2. **Medium-term Strategy**: [Plans for coming months]
3. **Long-term Considerations**: [Future planning needs]
## Caveats and Limitations:
[Professional disclaimers and scope limitations]
```
## Best Practices
**Specific Expertise**: Define clear areas of knowledge and experience
**Communication Style**: Specify how the role communicates
**Professional Context**: Include relevant background and credentials
**Personality Traits**: Add humanizing characteristics that affect responses
**Maintain Perspective**: Keep the role's viewpoint throughout conversations
**Use Appropriate Language**: Match vocabulary and tone to the role
**Reference Experience**: Draw on the role's background when relevant
**Stay in Character**: Don't break character unless explicitly requested
**Clear Transitions**: Use explicit markers when switching roles
**Distinct Voices**: Ensure each role has a unique perspective
**Conflict Resolution**: Address disagreements between roles constructively
**Synthesis Methods**: Develop frameworks for combining different viewpoints
## Common Pitfalls
**Avoid These Mistakes**:
* **Generic Roles**: Be specific about expertise and background
* **Inconsistent Character**: Maintain the role throughout the interaction
* **Unrealistic Expertise**: Don't claim knowledge the role wouldn't have
* **Stereotyping**: Avoid clichéd or oversimplified role portrayals
**Pro Tips**:
* Research real professionals in the field for authentic details
* Include both strengths and limitations of the role
* Use role-specific frameworks and methodologies
* Allow the role's personality to influence communication style
## Next Steps
Explore these complementary techniques:
* [Few-shot Learning](/examples/techniques/few-shot-learning) - Provide role-specific examples
* [Constitutional AI](/examples/techniques/constitutional-ai) - Add ethical guidelines to roles
* [Dynamic Prompting](/examples/techniques/dynamic-prompting) - Adapt roles based on context
# Self-Consistency
Source: https://docs-v1.latitude.so/examples/techniques/self-consistency
Learn how to implement self-consistency to improve AI reasoning reliability through multiple sampling and majority voting
## What is Self-Consistency?
Self-consistency is a prompting technique that improves the reliability of AI reasoning by generating multiple responses to the same question and then selecting the most consistent answer through majority voting. Unlike traditional Chain-of-Thought prompting that uses greedy decoding for a single reasoning path, self-consistency leverages diverse sampling to explore multiple reasoning perspectives before converging on the most reliable answer.
## Why Use Self-Consistency?
* **Improved Accuracy**: Multiple samples reduce the impact of random errors and greedy decoding limitations
* **Better Reasoning**: Helps identify the most logical solution path from diverse perspectives
* **Reduced Hallucinations**: Inconsistent responses are filtered out through majority voting
* **Confidence Assessment**: Provides pseudo-probability likelihood of answer correctness
* **Complex Problem Solving**: Particularly effective for math, logic, and multi-step reasoning where single attempts may fail
* **Robust Decision Making**: Overcomes limitations of single reasoning paths in ambiguous scenarios
## Basic Implementation in Latitude
Here's a simple self-consistency example for classification tasks:
```markdown Classification with Self-Consistency theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
---
# Content Classification
Classify the following content and explain your reasoning step by step.
## Content:
{{ content_to_classify }}
## Classification Process:
Let me analyze this step by step:
1. **Content Analysis:**
- What type of content is this?
- What are the key indicators?
2. **Context Evaluation:**
- What contextual clues are present?
- How do tone and language affect classification?
3. **Risk Assessment:**
- What potential impacts should be considered?
- Are there any warning signs?
4. **Final Classification:**
Based on my analysis: [CATEGORY]
**Reasoning:** [Detailed explanation of decision]
```
## How Self-Consistency Works
The self-consistency process follows three key steps:
1. **Diverse Path Generation**: The same prompt is submitted multiple times with higher temperature settings (0.6-0.8) to encourage different reasoning approaches and perspectives
2. **Answer Extraction**: Each response is analyzed to extract the core answer or classification, regardless of the reasoning path taken
3. **Majority Voting**: The most frequently occurring answer across all samples is selected as the final result
This approach provides a form of confidence scoring - answers that appear consistently across multiple reasoning paths are more likely to be correct than those that appear only once.
## Advanced Implementation with Multiple Samples
Let's create a more sophisticated example that uses Latitude's chain feature to generate and compare multiple reasoning paths:
```markdown Advanced Self-Consistency theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.8
---
# Reasoning Sample 1
Solve this problem using your preferred approach:
## Problem:
{{ reasoning_problem }}
## Solution Path 1:
Think through this step by step and provide your final answer.
# Reasoning Sample 2
Solve the same problem using a different approach if possible:
## Problem:
{{ reasoning_problem }}
## Solution Path 2:
Think through this step by step and provide your final answer.
# Reasoning Sample 3
Solve the problem one more time, focusing on accuracy:
## Problem:
{{ reasoning_problem }}
## Solution Path 3:
Think through this step by step and provide your final answer.
# Self-Consistency Analysis
Review the three solution paths above and determine the most consistent answer:
## Analysis:
1. **Compare the final answers:** Are they the same or different?
2. **Evaluate reasoning quality:** Which path has the most sound logic?
3. **Identify consensus:** What answer appears most frequently?
## Final Consistent Answer:
Based on the analysis above, the most reliable answer is:
**Answer:**
**Confidence Level:**
**Reasoning:**
```
In this advanced example:
1. **Multiple Sampling**: We generate three independent solutions with higher temperature for diversity
2. **Chain Processing**: Each step builds on the previous ones for comparison
3. **Consistency Analysis**: A final step evaluates and selects the best answer
4. **Confidence Assessment**: The system provides a confidence level based on agreement
## Logic and Reasoning Self-Consistency
Use self-consistency for complex logical problems:
```markdown Logic Problem Self-Consistency theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.6
---
# Deductive Reasoning Approach
Solve this logic problem using deductive reasoning:
## Problem:
{{ logic_problem }}
## Deductive Solution:
Start with the given facts and work logically to the conclusion:
1. **Given facts:**
2. **Logical deductions:**
3. **Conclusion:**
# Inductive Reasoning Approach
Solve the same problem using inductive reasoning:
## Problem:
{{ logic_problem }}
## Inductive Solution:
Look for patterns and make generalizations:
1. **Observe patterns:**
2. **Form hypothesis:**
3. **Test and conclude:**
# Abductive Reasoning Approach
Solve using abductive reasoning (inference to best explanation):
## Problem:
{{ logic_problem }}
## Abductive Solution:
Find the most likely explanation:
1. **Observations:**
2. **Possible explanations:**
3. **Best explanation:**
# Logic Consensus
Compare all three reasoning approaches
## Consensus Analysis:
- **Agreement level:** Do all approaches reach the same conclusion?
- **Strongest reasoning:** Which approach provides the most convincing logic?
- **Consistency score:** How well do the results align?
## Final Answer:
```
## Multi-Agent Self-Consistency
Combine self-consistency with Latitude's agent system for specialized reasoning:
```markdown Multi-Agent Self-Consistency theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.5
type: agent
agents:
- agents/mathematician
- agents/logician
- agents/analyst
---
# Multi-Expert Self-Consistency
Get multiple expert opinions and find the consensus:
## Problem:
{{ complex_problem }}
## Expert Consultation:
1. **Mathematician**: Analyze from a mathematical perspective
2. **Logician**: Apply formal logical reasoning
3. **Analyst**: Use analytical problem-solving methods
Coordinate with all experts and provide a self-consistent final answer.
```
```markdown agents/mathematician theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
type: agent
---
# Mathematics Expert
I am a mathematics expert specializing in problem-solving with rigorous mathematical methods.
## Problem Analysis:
{{ complex_problem }}
## Mathematical Approach:
1. **Identify mathematical concepts involved**
2. **Apply relevant formulas and theorems**
3. **Show detailed calculations**
4. **Verify results through alternative methods**
## Mathematical Solution:
```
```markdown agents/logician theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
type: agent
---
# Logic Expert
I am a logic expert specializing in formal reasoning and logical analysis.
## Problem Analysis:
{{ complex_problem }}
## Logical Approach:
1. **Structure the problem logically**
2. **Identify premises and conclusions**
3. **Apply logical rules and principles**
4. **Check for logical consistency**
## Logical Solution:
```
```markdown agents/analyst theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
type: agent
---
# General Analyst
I am a general analyst specializing in systematic problem-solving and critical thinking.
## Problem Analysis:
{{ complex_problem }}
## Analytical Approach:
1. **Break down the problem systematically**
2. **Consider multiple perspectives**
3. **Evaluate evidence and assumptions**
4. **Synthesize findings**
## Analytical Solution:
```
## Best Practices for Self-Consistency
**Optimal Sampling**:
* Use 3-5 samples for most problems (balance cost vs. accuracy)
* Increase temperature (0.6-0.8) to encourage diverse reasoning paths and overcome greedy decoding
* Ensure each sample approaches the problem independently
* Vary the prompt slightly to encourage different analytical perspectives
**Quality Control**:
* Generate enough samples to identify patterns
* Filter out obviously flawed reasoning
* Weight samples based on reasoning quality, not just frequency
* Consider partial agreements in complex problems
**Evaluation Criteria**:
* **Answer consistency**: Do multiple samples reach the same conclusion?
* **Reasoning quality**: Which reasoning paths are most sound?
* **Method diversity**: Are different valid approaches represented?
* **Confidence indicators**: How certain can we be about the consensus?
**Analysis Techniques**:
* Majority voting for clear disagreements
* Weighted voting based on reasoning quality
* Partial credit for answers that are close but not identical
* Meta-reasoning about why inconsistencies occur
**Best Use Cases**:
* Classification tasks with potential ambiguity
* Mathematical word problems
* Logical reasoning puzzles
* Multi-step analytical tasks
* Questions with clear right/wrong answers where reasoning path matters
* Security-sensitive decisions requiring high confidence
**Less Suitable Cases**:
* Creative writing tasks
* Subjective opinion questions
* Simple factual lookups
* Tasks requiring consistent style/voice
**Efficiency Tips**:
* Use parallel processing when possible
* Cache common problem types
* Implement early stopping if consensus is clear
* Balance sample count with accuracy needs
**Cost Management**:
* Start with fewer samples and increase if needed based on consistency scores
* Use cheaper models for initial sampling, better models for final analysis
* Implement confidence thresholds to determine optimal sample count
* Consider the cost trade-off: higher accuracy vs. increased computational expense
* Remember that self-consistency has high costs but provides pseudo-probability confidence
## Advanced Techniques
### Adaptive Self-Consistency
Create prompts that adjust based on initial consistency. You can [play with it here](https://app.latitude.so/share/d/2a66d7c5-841d-4217-81cb-f97610ac9374)
```markdown Adaptive Self-Consistency theme={null}
---
provider: OpenAI
model: gpt-4.1-mini
temperature: 0.7
---
# Initial Sample Generation
Generate 3 initial solutions:
## Problem: {{ problem }}
### Solution 1:
### Solution 2:
### Solution 3:
# Check Initial Consistency
Evaluate the consistency of initial samples
## Consistency Analysis:
- Are the answers consistent? (Yes/No)
- Confidence level in consensus: (1-10)
- Need for additional samples: (Yes/No)
## Decision:
If consistency is low (< 7/10), recommend generating 2-3 additional samples.
If consistency is high (≥ 7/10), proceed with current consensus.
{{ if consistency_check.additional_samples }}
Generate 2 more solutions using different approaches:
## Problem: {{ problem }}
### Solution 4:
### Solution 5:
{{ endif }}
# Final Consensus
Based on all available samples, determine the final answer:
## Final Self-Consistent Answer:
```
Note how we used [structured outputs](/guides/prompt-manager/json-output) to capture the consistency check results and decide whether to generate additional samples.
### Self-Consistency with Uncertainty Quantification
Implement self-consistency that quantifies uncertainty:
```markdown Uncertainty-Aware Self-Consistency theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.8
---
# Generate Diverse Solutions
Create 5 solutions with different reasoning strategies:
## Problem: {{ problem }}
### Strategy 1 - Direct Approach:
### Strategy 2 - Step-by-step Breakdown:
### Strategy 3 - Alternative Method:
### Strategy 4 - Verification Focus:
### Strategy 5 - Edge Case Consideration:
# Uncertainty Quantification
Analyze the uncertainty in our solutions
## Uncertainty Assessment:
1. **Answer Distribution**: What answers appeared and how often?
2. **Reasoning Confidence**: How confident was each reasoning path?
3. **Method Agreement**: Do different methods agree?
4. **Edge Case Handling**: How well are corner cases addressed?
## Uncertainty Metrics:
- **Consensus Strength**: (0-100%)
- **Reasoning Diversity**: (Low/Medium/High)
- **Confidence Interval**: (if applicable)
- **Uncertainty Sources**: (List main sources of disagreement)
## Final Answer with Uncertainty:
**Most Likely Answer:**
**Confidence Level:**
**Alternative Possibilities:**
**Key Uncertainties:**
```
## Integration with Other Techniques
Self-consistency works well combined with other prompting techniques:
* **Chain-of-Thought + Self-Consistency**: Generate multiple detailed reasoning chains to overcome greedy decoding limitations
* **Few-Shot + Self-Consistency**: Use examples to guide consistent reasoning patterns across multiple samples
* **Role-Playing + Self-Consistency**: Have different expert personas solve the same problem independently
* **Iterative Refinement + Self-Consistency**: Use consensus to improve solution quality through multiple rounds
The key is to maintain the core principle: generate multiple independent solutions and use agreement as a signal of reliability, while addressing the inherent limitations of single-path reasoning.
## Related Techniques
Explore these complementary prompting techniques to enhance your AI applications:
* **[Chain-of-Thought](./chain-of-thought)** - Break down complex problems into step-by-step reasoning
* **[Tree-of-Thoughts](./tree-of-thoughts)** - Explore multiple reasoning paths systematically
* **[Few-Shot Learning](./few-shot-learning)** - Use examples to guide AI behavior and improve consistency
# Step-Back Prompting
Source: https://docs-v1.latitude.so/examples/techniques/step-back-prompting
Learn how to improve AI responses by first considering general principles before tackling specific tasks
## What is Step-Back Prompting?
Step-back prompting is a technique that enhances AI performance by encouraging the model to first explore broader, foundational concepts before addressing specific tasks. Instead of diving directly into a particular problem, the AI first considers general principles, background knowledge, and underlying patterns that can inform a more thoughtful and accurate response.
## Why Use Step-Back Prompting?
* **Enhanced Knowledge Activation**: Activates relevant background knowledge before tackling specific problems
* **Improved Accuracy**: General principles guide more informed specific responses
* **Reduced Bias**: Focus on fundamental concepts helps mitigate response biases
* **Creative Problem-Solving**: Broader perspective encourages innovative approaches
* **Better Contextualization**: Connects specific tasks to larger frameworks
* **Deeper Understanding**: Promotes critical thinking and principled reasoning
## How Step-Back Prompting Works
The technique follows a two-stage process:
1. **Abstraction Phase**: Ask a general question related to the domain or principles underlying your specific task
2. **Application Phase**: Use the general insights as context to inform the specific task
This approach leverages more of the model's parameter knowledge and reasoning capabilities than direct prompting alone.
## Basic Implementation in Latitude
Here's a simple step-back prompting example for content creation:
```markdown Step-Back Content Creation theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.6
---
# Step Back: General Principles
Before creating content, let's establish foundational principles.
What are the key elements that make {{ content_type }} effective and engaging for {{ target_audience }}?
Consider:
- Core principles of effective communication
- Audience engagement strategies
- Industry best practices
- Psychological factors that drive engagement
## Foundational Elements:
# Apply: Specific Content Creation
Using the principles identified above as guidance:
**Context**: {{ foundational_elements }}
Now create {{ specific_content_request }} that incorporates these proven principles.
## Content:
```
## Advanced Implementation with Multiple Steps
For complex tasks, you can create multi-layered step-back prompts:
```markdown Multi-Layer Step-Back Analysis theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.5
---
# Step Back: Domain Fundamentals
What are the core principles and best practices in {{ domain_area }}?
Focus on:
- Theoretical foundations
- Proven methodologies
- Common pitfalls to avoid
- Success patterns
## Domain Principles:
# Step Back: Contextual Factors
Given these domain principles: {{ domain_principles }}
What specific considerations apply to {{ context_description }}?
Consider:
- Environmental factors
- Stakeholder perspectives
- Resource constraints
- Risk factors
## Contextual Analysis:
# Apply: Targeted Solution
Drawing from both the domain principles and contextual analysis:
**Domain Foundation**: {{ domain_principles }}
**Context**: {{ contextual_analysis }}
Now address this specific challenge: {{ specific_problem }}
## Solution:
```
## Domain-Specific Applications
### Business Strategy Step-Back
```markdown Business Strategy Step-Back theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
---
# Step Back: Strategic Frameworks
What are the fundamental frameworks and principles that drive successful business strategy across industries?
Consider:
- Competitive advantage theories
- Market analysis methodologies
- Value creation principles
- Strategic planning best practices
## Strategic Foundations:
# Apply: Company-Specific Strategy
Using these strategic foundations as a guide:
**Framework**: {{ strategic_foundations }}
Develop a strategic approach for: {{ business_challenge }}
**Company Context**: {{ company_details }}
**Market Conditions**: {{ market_context }}
## Strategic Recommendation:
```
### Technical Problem Solving
```markdown Technical Step-Back theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
---
# Step Back: Engineering Principles
What are the core engineering principles and design patterns that apply to {{ technical_domain }}?
Focus on:
- Fundamental design principles
- Proven architectural patterns
- Performance considerations
- Maintainability factors
## Engineering Foundations:
# Apply: Specific Implementation
Drawing from these engineering principles:
**Foundation**: {{ engineering_foundations }}
Design a solution for: {{ technical_requirement }}
**Constraints**: {{ technical_constraints }}
**Requirements**: {{ functional_requirements }}
## Technical Solution:
```
### Creative Development
```markdown Creative Step-Back theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
---
# Step Back: Creative Principles
What are the fundamental elements that make {{ creative_medium }} compelling and memorable?
Consider:
- Narrative structure principles
- Emotional engagement techniques
- Audience psychology
- Genre conventions and innovations
## Creative Elements:
# Apply: Specific Creation
Using these creative principles as foundation:
**Elements**: {{ creative_principles }}
Create {{ specific_creative_task }} that incorporates these proven elements.
**Theme**: {{ creative_theme }}
**Audience**: {{ target_audience }}
## Creative Work:
```
## Best Practices for Step-Back Prompting
**Good Step-Back Questions**:
* Ask about underlying principles, not just examples
* Focus on "what makes X effective" rather than "what is X"
* Seek patterns and frameworks that transcend specific instances
* Consider multiple perspectives and approaches
**Question Formulation**:
* Use open-ended questions that encourage deep thinking
* Include relevant context to guide the abstraction level
* Ask for both positive principles and common pitfalls
* Request reasoning behind the principles, not just lists
**Seamless Connection**:
* Explicitly reference the step-back insights in the application phase
* Use the general principles as a checklist or framework
* Maintain consistency between abstract principles and specific application
* Bridge the gap between theory and practice
**Content Flow**:
* Ensure the step-back content directly informs the main task
* Use the insights to structure or evaluate the specific response
* Reference specific principles when making decisions
* Show how general knowledge applies to the particular case
**Best Use Cases**:
* Complex creative tasks requiring innovation
* Strategic decision-making with multiple considerations
* Technical problems with architectural implications
* Educational content that benefits from theoretical grounding
* Situations where bias reduction is important
**Less Suitable Cases**:
* Simple factual queries
* Tasks requiring immediate, direct responses
* Highly specific technical implementations
* Time-sensitive decisions requiring quick answers
**Validation Techniques**:
* Ensure step-back insights are actually relevant to the task
* Check that principles are specific enough to be actionable
* Verify that the final response incorporates the step-back content
* Assess whether the two-step process improved the outcome
**Common Issues**:
* Step-back becoming too abstract or philosophical
* Failure to connect general principles to specific application
* Redundant information that doesn't add value
* Over-complicating simple tasks
## Advanced Techniques
### Comparative Step-Back
Generate multiple perspective frameworks before application:
```markdown Comparative Step-Back theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.6
---
# Step Back: Multiple Frameworks
What are three different theoretical approaches to {{ problem_domain }}?
For each approach, explain:
- Core principles
- Key methodologies
- Typical applications
- Strengths and limitations
## Framework 1 - {{ approach_1 }}:
## Framework 2 - {{ approach_2 }}:
## Framework 3 - {{ approach_3 }}:
# Apply: Integrated Solution
Drawing insights from all three frameworks:
**Approaches**: {{ framework_comparison }}
Develop a solution for {{ specific_challenge }} that integrates the best elements from each approach.
## Integrated Solution:
```
### Iterative Step-Back
Use multiple levels of abstraction for complex problems:
```markdown Iterative Step-Back theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.5
---
# Step Back Level 1: Universal Principles
What are the most fundamental principles that apply to {{ broad_domain }}?
## Universal Principles:
# Step Back Level 2: Domain-Specific
Given these universal principles: {{ universal_principles }}
What specific principles apply to {{ specific_domain }}?
## Domain Principles:
# Step Back Level 3: Contextual
Considering both universal and domain principles:
**Universal**: {{ universal_principles }}
**Domain**: {{ domain_principles }}
What contextual factors are unique to {{ specific_context }}?
## Contextual Factors:
# Apply: Comprehensive Solution
Integrating all levels of insight:
**Foundation**: {{ all_principles }}
Address this specific challenge: {{ precise_problem }}
## Solution:
```
## Integration with Other Techniques
Step-back prompting works well combined with other approaches:
* **Chain-of-Thought + Step-Back**: First establish principles, then reason through step-by-step application
* **Self-Consistency + Step-Back**: Generate multiple principle-based approaches and find consensus
* **Few-Shot + Step-Back**: Provide examples of good step-back reasoning patterns
* **Role-Playing + Step-Back**: Have different experts establish principles from their perspectives
The key is using the step-back phase to activate relevant knowledge and frameworks that inform better reasoning in the application phase.
## Common Patterns and Templates
### The "What Makes X Effective?" Pattern
* Step back: "What makes \[domain/type] effective?"
* Apply: "Using these principles, create \[specific instance]"
### The "Best Practices" Pattern
* Step back: "What are the best practices for \[area]?"
* Apply: "Apply these practices to \[specific situation]"
### The "Principles vs. Implementation" Pattern
* Step back: "What principles guide \[theoretical area]?"
* Apply: "Implement these principles in \[practical context]"
### The "Multiple Perspectives" Pattern
* Step back: "How do different experts approach \[domain]?"
* Apply: "Combine these approaches for \[specific challenge]"
Step-back prompting transforms AI responses from reactive to reflective, ensuring that specific solutions are grounded in broader understanding and proven principles.
# Tree of Thoughts
Source: https://docs-v1.latitude.so/examples/techniques/tree-of-thoughts
Implement multiple branching reasoning paths to solve complex problems with Latitude
## What is Tree of Thoughts?
Tree of Thoughts (ToT) is an advanced prompting technique that enables AI models to explore multiple reasoning paths in parallel, evaluate their potential, and select the most promising branches to develop further—similar to how humans explore different solutions when tackling complex problems.
## Why Use Tree of Thoughts?
* **Improved Problem-Solving**: Systematically explore multiple solution pathways
* **Better Planning**: Map out different approaches before committing to one
* **Enhanced Creativity**: Generate diverse solutions to open-ended problems
* **Reduced Errors**: Catch mistakes by comparing different reasoning branches
* **Complex Decision-Making**: Break down complex decisions into evaluable components
## Basic Implementation in Latitude
Here's a simple Tree of Thoughts example for solving a complex problem:
```markdown Basic ToT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
---
# Tree of Thoughts Problem-Solving
Let's solve this problem by exploring multiple lines of reasoning:
**Problem**: {{ problem_statement }}
## Initial Thought Branches:
### Branch A: [First Approach]
1. Initial premise: ...
2. Reasoning step: ...
3. Intermediate conclusion: ...
4. Further implications: ...
5. Potential outcome: ...
### Branch B: [Alternative Approach]
1. Initial premise: ...
2. Reasoning step: ...
3. Intermediate conclusion: ...
4. Further implications: ...
5. Potential outcome: ...
### Branch C: [Creative Approach]
1. Initial premise: ...
2. Reasoning step: ...
3. Intermediate conclusion: ...
4. Further implications: ...
5. Potential outcome: ...
## Branch Evaluation:
- Branch A strength: ...
- Branch B strength: ...
- Branch C strength: ...
## Final Solution Path:
[Select most promising branch and develop it further]
```
## Advanced Implementation with Parameters
```markdown Advanced ToT theme={null}
---
provider: OpenAI
model: gpt-4.1
temperature: 0.4
---
# Tree of Thoughts: Advanced Problem-Solving
Let's solve this complex problem using a Tree of Thoughts approach with {{ thought_branches }} initial branches.
**Problem**: {{ problem_statement }}
## Thought Generation:
{{ for branch in thought_branches }}
### Branch {{branch}}: [Name this approach]
{{ for level in levels}}
**Level {{level}} thinking:**
- [Reasoning steps at this level]
- [Interim conclusions]
**Branch {{branch}} Outcomes:**
- [Describe expected outcomes of this reasoning path]
{{ '\n\t'}}
{{ endfor }}
## Branch Evaluation:
{{ for criteria in evaluation_criteria}}
### Criterion: {{ criteria }}
- Branch {{branch}}: [Score 1-10] - [Justification]
{{ '\n'}}
{{ endfor }}
{{endfor}}
## Solution Development:
{{ if branch_selection_method === "best_single" }}
**Selected Branch**: [Identify best overall branch]
**Development**: [Fully develop this single branch to conclusion]
{{ else if branch_selection_method === "hybrid" }}
**Hybrid Solution**: [Combine elements from multiple branches]
**Integration Points**: [Explain how different branch elements connect]
{{ else if branch_selection_method === "weighted" }}
**Weighted Solution**: [Proportionally represent branches based on scores]
**Weighting Factors**: [Explain the weights applied to each branch]
{{ endif }}
## Final Answer:
[Complete solution with justification]
```
You can use `{{ '\n\t' }}` to give indentation in the code block. Is more easy to follow what's doing the prompt
## Implementing ToT With Chains
Latitude's chain feature allows for structured Tree of Thoughts reasoning:
````markdown chain theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
---
# Tree of Thoughts with Chains
```markdown
# Step 1: Generate Multiple Thought Branches
**Problem**: {{ problem_statement }}
Let me generate three distinct approaches to solving this problem:
## Branch A: [First Approach]
1. Initial premise: ...
2. Reasoning: ...
3. Implications: ...
## Branch B: [Second Approach]
1. Initial premise: ...
2. Reasoning: ...
3. Implications: ...
## Branch C: [Third Approach]
1. Initial premise: ...
2. Reasoning: ...
3. Implications: ...
# Step 2: Evaluate Each Thought Branch
Evaluating the branches generated in previous step:
## Branch A Evaluation:
- Strengths: ...
- Weaknesses: ...
- Confidence score (1-10): ...
## Branch B Evaluation:
- Strengths: ...
- Weaknesses: ...
- Confidence score (1-10): ...
## Branch C Evaluation:
- Strengths: ...
- Weaknesses: ...
- Confidence score (1-10): ...
# Step 3: Select and Develop Best Branch
Based on my evaluation:
The most promising approach is **Branch [X]** because:
[Justification for selection]
Let me develop this branch further:
## Detailed Development:
1. [Further reasoning steps]
2. [Handling edge cases]
3. [Addressing potential objections]
4. [Additional insights]
## Final Solution:
[Complete answer to the original problem]
````
## Multi-Agent ToT
Implement Tree of Thoughts with agent collaboration, you can [play with it here](https://app.latitude.so/share/d/6daaa005-cc77-4009-9150-f905505233eb).
```markdown Multi-Agent theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
type: agent
agents:
- agents/creator
- agents/critic
- agents/synthesizer
---
# Multi-Agent Tree of Thoughts
Solve the following problem using a collaborative multi-agent Tree of Thoughts approach:
**Problem**: {{ problem_statement }}
## Process:
1. The **creator** agent will generate multiple solution branches
2. The **critic** agent will evaluate each branch's strengths and weaknesses
3. The **synthesizer** agent will select and refine the most promising approach
Let's begin solving this step by step.
```
```markdown Creator Agent theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
type: agent
path: agents/creator
---
# Creator Agent: Thought Branch Generation
Generate three distinct approaches to solving the problem.
For each approach:
1. Use different first principles or starting assumptions
2. Explore creative and unexpected angles
3. Trace the logical steps from premise to conclusion
Don't evaluate the branches yet - focus on diversity of thought.
```
```markdown Critic Agent theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.1
type: agent
path: agents/critic
---
# Critic Agent: Branch Evaluation
Carefully evaluate each of the thought branches provided.
For each branch:
1. Identify logical fallacies or unwarranted assumptions
2. Check alignment with known facts and constraints
3. Consider edge cases and exceptions
4. Assign a confidence score and explain reasoning
Be rigorous and analytical in your assessment.
```
```markdown Synthesizer Agent theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.3
type: agent
path: agents/synthesizer
---
# Synthesizer Agent: Solution Development
Based on the generated branches and their evaluations:
1. Select the most promising approach OR
2. Create a hybrid solution incorporating the strongest elements
Then develop this approach in detail, addressing:
- Any weaknesses identified by the critic
- Practical implementation steps
- Expected outcomes
```
## Best Practices for Tree of Thoughts
**Effective Branch Creation**:
* Create branches that start from genuinely different premises or approaches
* Ensure sufficient diversity between branches to explore the solution space
* Balance breadth (number of branches) with depth (steps in each branch)
* Use structured formats that make branches easy to compare
**Branch Evaluation**:
* Define clear evaluation criteria upfront
* Assign quantitative scores when possible
* Document reasoning for evaluations
* Consider both short-term solutions and long-term implications
**Technical Implementation**:
* Use parameters to control branch count, depth, and evaluation criteria
* Balance temperature settings - higher for branch generation, lower for evaluation
* Use larger context models (GPT-4) for complex ToT problems
* Store intermediate results in variables for complex multi-step ToT
**Process Optimization**:
* Start with 2-3 branches for simpler problems, 4-5 for complex ones
* Consider 3-5 steps of reasoning per branch as a starting point
* Use Latitude's chain feature for structured ToT implementation
* Try different branch combination methods (best single, hybrid, weighted)
**Ideal Problem Types**:
* **Strategic Planning**: Multiple viable approaches with complex tradeoffs
* **Creative Challenges**: Open-ended problems with no clear "right" answer
* **Analysis Tasks**: Situations requiring consideration of multiple perspectives
* **Decision Making**: Complex decisions with many factors to weigh
* **Troubleshooting**: Problems where the root cause isn't immediately obvious
**Less Suitable Problems**:
* Simple factual queries with definitive answers
* Highly constrained problems with limited solution paths
* Routine tasks with established procedures
**ToT Variations**:
* **Recursive ToT**: Apply ToT within branches of a larger ToT structure
* **Adversarial ToT**: Intentionally create opposing branches to stress-test solutions
* **Collaborative ToT**: Distribute branches across multiple specialized agents
* **Time-Horizon ToT**: Create branches exploring short, medium, and long-term impacts
* **Probabilistic ToT**: Assign probability weights to different branches
**Integration with Other Techniques**:
* Combine with Chain-of-Thought within branches
* Use Few-shot examples to guide branch generation
* Apply Self-Consistency to evaluate branch quality
## Applications in Different Domains
```markdown Strategic Planning ToT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.4
---
# Strategic Planning Tree of Thoughts
Let's evaluate different strategic approaches for {{ business_scenario }}:
## Branch 1: Market Expansion Strategy
1. **Current Market Analysis**: [Assessment of current position]
2. **Target Market Identification**: [New markets to enter]
3. **Entry Strategy**: [How to penetrate new markets]
4. **Resource Requirements**: [What's needed for execution]
5. **Risk Assessment**: [Potential challenges and mitigation]
## Branch 2: Product Innovation Strategy
1. **Current Product Evaluation**: [Assessment of product lineup]
2. **Innovation Opportunities**: [Areas for new development]
3. **R&D Framework**: [How to approach innovation]
4. **Go-to-Market Strategy**: [Bringing innovations to customers]
5. **Competitive Advantage Analysis**: [How this creates distinction]
## Branch 3: Operational Optimization Strategy
1. **Efficiency Assessment**: [Current operational bottlenecks]
2. **Process Redesign**: [New operational models]
3. **Technology Integration**: [Leveraging new technologies]
4. **Cost Structure Impact**: [Financial implications]
5. **Implementation Roadmap**: [Execution timeline and milestones]
## Strategy Evaluation:
[Comparative analysis of the three strategic paths]
## Recommended Approach:
[Final strategic recommendation with implementation plan]
```
```markdown Creative Problem-Solving ToT theme={null}
---
provider: OpenAI
model: gpt-4o
temperature: 0.7
---
# Creative Problem-Solving Tree of Thoughts
Let's generate innovative solutions for {{ creative_challenge }}:
## Branch 1: Conventional Approach Reimagined
1. **Existing Patterns**: [Identify current solutions]
2. **Pattern Breaking**: [Ways to challenge assumptions]
3. **Novel Combinations**: [Unexpected element combinations]
4. **Refinement**: [Shaping the concept]
5. **Practical Application**: [Making it work in reality]
## Branch 2: First Principles Approach
1. **Problem Deconstruction**: [Break into fundamental elements]
2. **First Principles**: [Identify core truths/needs]
3. **Solution Building**: [Construct from basics upward]
4. **Theoretical Evaluation**: [Testing against principles]
5. **Practical Translation**: [Moving from theory to practice]
## Branch 3: Lateral Thinking Approach
1. **Analogous Domains**: [Find parallel situations elsewhere]
2. **Metaphorical Thinking**: [Apply metaphors to problem]
3. **Random Stimulus**: [Introduce unrelated concepts]
4. **Connection Building**: [Forge new relationship paths]
5. **Solution Crystallization**: [Form coherent solution]
## Idea Evaluation:
[Compare novelty, feasibility, and impact of each approach]
## Selected Creative Solution:
[Final concept and implementation considerations]
```
## Common Pitfalls and Solutions
**Avoid These Common Mistakes**:
* **Shallow Branches**: Creating branches that aren't meaningfully different from each other
* **Premature Evaluation**: Judging branches before they're fully developed
* **Confirmation Bias**: Favoring branches that align with preconceptions
* **Neglecting Constraints**: Failing to consider real-world limitations
* **Excessive Complexity**: Creating too many branches or too much depth for the problem
**Pro Tips**:
* Start with a clear problem statement before branching
* Use different cognitive approaches for each branch (analytical, creative, critical)
* Consider allocating more tokens to the most promising branches
* Document your reasoning at each step for transparency
* Try different branch-recombination methods for complex problems
## Next Steps
Now that you understand Tree of Thoughts, explore these related techniques:
* [Chain-of-Thought](/examples/techniques/chain-of-thought) - Step-by-step reasoning within branches
* [Self-Consistency](/examples/techniques/self-consistency) - Verify solutions through multiple attempts
* [Role Prompting](/examples/techniques/role-prompting) - Assign different thinking styles to branches
* [Multi-Agent Collaboration](/examples/techniques/multi-agent-collaboration) - Distribute reasoning across agents
# API Access
Source: https://docs-v1.latitude.so/guides/api/api-access
Learn how to access and use Latitude's API to run your prompts.
For detailed endpoint specifications, request/response schemas, and the ability to try out API calls directly, please refer to the [Interactive API Documentation](https://gateway.latitude.so/api-docs/).
We recommend checking the SDK docs section in case you're looking for a specific language or framework.
## Latitude HTTP API Documentation
This guide explains how to use the Latitude HTTP API to interact with the Prompt Manager and run AI-powered conversations.
### Authentication
All API requests require authentication. Include your API key in the `Authorization` header of your HTTP requests:
```
Authorization: Bearer YOUR_API_KEY
```
### Base URL
The base URL for API requests depends on your environment:
`https://gateway.latitude.so/api/v3`
### Rate Limiting
The API enforces rate limits based on your API key to ensure fair usage and prevent abuse.
**Limits:**
Rate limits are enforced based on your subscription plan. The following limits apply:
* **Hobby Plan:**
* 10 requests per second
* **Team Plan:**
* 166 requests per second (10000 requests per minute)
* **Enterprise Plan:**
* 500 requests per second (30000 requests per minute)
Contact sales to request a custom rate limit in the enterprise plan.
When the rate limit is exceeded, the following headers are included in the response to help you manage your request rate:
* `Retry-After`: Indicates the number of seconds to wait before making a new request.
* `X-RateLimit-Limit`: The maximum number of requests allowed in the current period.
* `X-RateLimit-Remaining`: The number of requests remaining in the current period.
* `X-RateLimit-Reset`: The timestamp when the rate limit will reset.
**Example Headers:**
```http theme={null}
Retry-After: 60
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 999
X-RateLimit-Reset: 1729399082482
```
These headers are sent with every request to help you monitor and adjust your request rate accordingly.
### Endpoints
#### 1. Get a Prompt
Retrieve a specific prompt by its path. Use this endpoint to fetch the content and configuration of an existing prompt in your project.
**Endpoint:** `GET /projects/{projectId}/versions/{versionUuid}/documents/{path}`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: Version UUID (required, optional for SDK's defaults to 'live')
* `path`: Path to the document (required)
**Response:**
The response contains the prompt details along with its configuration.
**Response Body:**
```json theme={null}
{
"id": "document-id",
"documentUuid": "document-uuid",
"path": "path/to/document",
"content": "Document content",
"resolvedContent": "Document content without comments",
"contentHash": "content-hash",
"commitId": "commit-id",
"deletedAt": "deleted-at",
"createdAt": "created-at",
"updatedAt": "updated-at",
"mergedAt": "merged-at",
"projectId": "project-id",
"config": {
"provider": "Provider name",
"model": "Model name"
}
}
```
#### 2. Get or Create a Prompt
Retrieve an existing prompt or create it if it doesn't exist. This endpoint provides an idempotent way to ensure a prompt exists at a specific path without checking first.
**Endpoint:** `POST /projects/{projectId}/versions/{versionUuid}/documents/get-or-create`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: Version UUID (required, optional for SDK's defaults to 'live')
**Request Body:**
```json theme={null}
{
"path": "path/to/document",
"prompt": "Your prompt here"
}
```
* `path`: Path to the prompt (required)
* `prompt`: Prompt content to use (optional, defaults to empty)
**Response:**
The response contains the created (or existing) prompt details along with its configuration.
**Response Body:**
```json theme={null}
{
"id": "document-id",
"documentUuid": "document-uuid",
"path": "path/to/document",
"content": "Document content",
"resolvedContent": "Document content without comments",
"contentHash": "content-hash",
"commitId": "commit-id",
"deletedAt": "deleted-at",
"createdAt": "created-at",
"updatedAt": "updated-at",
"mergedAt": "merged-at",
"projectId": "project-id",
"config": {
"provider": "Provider name",
"model": "Model name"
}
}
```
#### 3. Create or Update a Prompt
Create a new prompt or update an existing one in a single operation. This endpoint provides more control than `get-or-create`, including the ability to update live commits with the `force` flag.
**Endpoint:** `POST /projects/{projectId}/versions/{versionUuid}/documents/create-or-update`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: Version UUID (required, optional for SDK's defaults to 'live')
**Request Body:**
```json theme={null}
{
"path": "path/to/document",
"prompt": "Your prompt here",
"force": false
}
```
* `path`: Path to the prompt (required)
* `prompt`: Content of the prompt (required)
* `force`: Allow modifications to live/merged commits (optional, defaults to `false`)
**Behavior:**
* If the prompt **does not exist** at the specified path, it will be created
* If the prompt **already exists** at the path, it will be updated with the new content
* By default, modifications are only allowed on draft commits (not live/merged)
* When `force: true`, allows creating or updating prompts in live commits (use with caution)
Using `force: true` allows modifying production prompts directly. This should
only be used for emergency hotfixes or controlled production updates. For
normal development workflows, use draft commits.
**Response:**
The response contains the created or updated prompt details along with its configuration.
**Response Body:**
```json theme={null}
{
"id": "document-id",
"documentUuid": "document-uuid",
"path": "path/to/document",
"content": "Document content",
"resolvedContent": "Document content without comments",
"contentHash": "content-hash",
"commitId": "commit-id",
"deletedAt": "deleted-at",
"createdAt": "created-at",
"updatedAt": "updated-at",
"mergedAt": "merged-at",
"projectId": "project-id",
"config": {
"provider": "Provider name",
"model": "Model name"
}
}
```
**Error Handling:**
If you try to modify a live commit without the `force` flag, the API returns a 400 status code:
```json theme={null}
{
"name": "BadRequestError",
"message": "Cannot modify a merged commit. Use force=true to allow modifications to the live commit.",
"errorCode": "BadRequestError",
"details": {}
}
```
**Use Cases:**
* **Single API call for upsert operations**: No need to check if a prompt exists before creating/updating
* **Programmatic prompt updates**: Update prompts from your CI/CD pipeline or automation scripts
* **Emergency hotfixes**: Use `force: true` to quickly fix production prompts when needed
* **Batch operations**: Efficiently create or update multiple prompts in a loop
**Example: Update with Force Flag**
```bash theme={null}
curl -X POST "https://gateway.latitude.so/api/v3/projects/123/versions/live/documents/create-or-update" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"path": "production/emergency-fix",
"prompt": "---\nprovider: openai\nmodel: gpt-4\n---\n\nFixed prompt content",
"force": true
}'
```
#### 4. Delete a Prompt
Delete a prompt from a draft version. This performs a soft-delete — the document is marked as deleted but can be restored by reverting the draft.
**Endpoint:** `DELETE /projects/{projectId}/versions/{versionUuid}/documents/{path}`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: Version UUID of a draft commit (required)
* `path`: Path to the document (required)
**Response:**
The response confirms the deletion with the document's UUID and path.
**Response Body:**
```json theme={null}
{
"documentUuid": "document-uuid",
"path": "path/to/document"
}
```
**Error Handling:**
If you try to delete a document from a merged (live) commit, the API returns a 400 status code:
```json theme={null}
{
"name": "BadRequestError",
"message": "Cannot modify a merged commit.",
"errorCode": "BadRequestError",
"details": {}
}
```
**Example:**
```bash theme={null}
curl -X DELETE "https://gateway.latitude.so/api/v3/projects/123/versions/draft-uuid/documents/path/to/prompt" \
-H "Authorization: Bearer YOUR_API_KEY"
```
#### 5. Create a Version (Commit)
Create a new draft version (commit) for a project. Versions allow you to manage changes to your prompts before publishing them to production.
**Endpoint:** `POST /projects/{projectId}/versions`
**Path Parameters:**
* `projectId`: Your project ID (required)
**Request Body:**
```json theme={null}
{
"name": "Version name or title"
}
```
* `name`: Name/title for the new version (required)
**Response:**
The response contains the created version (commit) details.
**Response Body:**
```json theme={null}
{
"id": 123,
"uuid": "version-uuid",
"projectId": 456,
"message": "Version name or title",
"authorName": "Author name",
"authorEmail": "author@example.com",
"authorId": 789,
"createdAt": "2024-01-01T00:00:00.000Z",
"updatedAt": "2024-01-01T00:00:00.000Z",
"status": "draft",
"parentCommitUuid": "parent-version-uuid"
}
```
**Use Cases:**
* **Create draft versions**: Start working on prompt changes in isolation
* **Version control**: Track different iterations of your prompts
* **CI/CD integration**: Programmatically create versions from your deployment pipeline
**Example:**
```bash theme={null}
curl -X POST "https://gateway.latitude.so/api/v3/projects/123/versions" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Feature: Add Spanish language support"
}'
```
#### 6. Publish a Version (Commit)
Publish a draft version (commit) to make it the live/production version. This merges the draft changes and assigns it a version number.
**Endpoint:** `POST /projects/{projectId}/versions/{versionUuid}/publish`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: UUID of the draft version to publish (required)
**Request Body:**
```json theme={null}
{
"title": "Optional updated title",
"description": "Optional description or release notes"
}
```
* `title`: Optional title for the published version (if not provided, uses existing title)
* `description`: Optional description or release notes for the published version
**Response:**
The response contains the published version (commit) details with a version number and merged timestamp.
**Response Body:**
```json theme={null}
{
"id": 123,
"uuid": "version-uuid",
"projectId": 456,
"message": "Published version title",
"authorName": "Author name",
"authorEmail": "author@example.com",
"authorId": 789,
"createdAt": "2024-01-01T00:00:00.000Z",
"updatedAt": "2024-01-01T00:00:00.000Z",
"status": "merged",
"parentCommitUuid": "parent-version-uuid"
}
```
Publishing a version makes it the live/production version. All documents in
the published version become the active versions accessible via the API. Make
sure to test your changes thoroughly before publishing.
**Use Cases:**
* **Deploy to production**: Publish tested prompt changes to make them live
* **Release management**: Track which version is currently in production
* **Automated deployments**: Publish versions from CI/CD pipelines after successful tests
**Example:**
```bash theme={null}
curl -X POST "https://gateway.latitude.so/api/v3/projects/123/versions/abc-123-def-456/publish" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"title": "Feature: Add Spanish language support",
"description": "Added support for Spanish language queries with improved accuracy"
}'
```
**Error Handling:**
If you try to publish a version that is already published or doesn't exist, the API returns an appropriate error:
```json theme={null}
{
"name": "BadRequestError",
"message": "Cannot publish: version is not a draft",
"errorCode": "BadRequestError",
"details": {}
}
```
#### 7. Run a Prompt
Execute a prompt with optional parameters. This endpoint processes your prompt template, sends it to the configured AI provider, and returns the generated response. Supports both streaming and non-streaming modes, as well as background processing for long-running operations.
**Endpoint:** `POST /projects/{projectId}/versions/{versionUuid}/documents/run`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: Version UUID (required, optional for SDK's defaults to 'live')
**Request Body:**
```json theme={null}
{
"path": "path/to/document",
"parameters": {
"key1": "value1",
"key2": "value2"
},
"stream": false,
"background": false,
"messages": [
{
"role": "user",
"content": [{ "type": "text", "text": "Hello!" }]
}
],
"customIdentifier": "optional-custom-id",
"tools": ["tool1", "tool2"]
}
```
* `stream`: Optional boolean parameter (defaults to `false`). When set to true, the response will be a stream of Server-Sent Events (SSE). If false, a single JSON response containing the last event is returned.
* `background`: Optional boolean parameter (defaults to `false`). When set to true, the request is enqueued for background processing and returns immediately with a conversation UUID.
* `messages`: Optional array of messages to append to the conversation after the compiled prompt. Messages follow the [PromptL format](/promptl/getting-started/introduction). Note: This is not compatible with the `` feature of PromptL.
* `userMessage`: **Deprecated.** Use `messages` instead. Optional string to start the conversation with a user message. This parameter will be removed in a future version.
* `customIdentifier`: Optional string for custom identification of the run.
* `tools`: Optional array of tool names to enable for this run.
**Response:**
* If `background` is `true`: Returns immediately with a conversation UUID for background processing:
```json theme={null}
{
"uuid": "conversation-uuid"
}
```
* If `stream` is `true`: The response is a stream of Server-Sent Events (SSE). Check out the [Streaming Events](/guides/api/streaming-events) guide for more information about the specific events you can expect.
* If `stream` is `false`: A single JSON response is returned with the final event (typically the chain-complete event) in the following structure:
```json theme={null}
{
"uuid": string,
"conversation": Message[],
"response": {
"streamType": "text" | "object",
"usage": {
"promptTokens": number,
"completionTokens": number,
"totalTokens": number
},
"text": string,
"object": object | undefined,
"toolCalls": ToolCall[]
"cost": number
}
```
Message follows the [PromptL format](/promptl/getting-started/introduction).
ToolCall has the following format:
```typescript theme={null}
type ToolCall = {
id: string
name: string
arguments: Record
}
```
#### 8. Chat
Continue a multi-turn conversation by sending additional messages to an existing conversation thread. This endpoint allows you to maintain context across multiple exchanges with the AI model by building upon messages from a previous run. The conversation history is automatically managed, and each new message is appended to the existing message chain.
**Endpoint:** `POST /conversations/{conversationUuid}/chat`
**Path Parameters:**
* `conversationUuid`: UUID of the conversation
**Request Body:**
* Messages follow the [PromptL format](/promptl/getting-started/introduction). If you're using a different method to run your prompts, you'll need to format your messages accordingly.
```json theme={null}
{
"messages": [
{
"role": "user" | "system" | "assistant",
"content": [
{
"type": "text",
"text": "message content"
}
],
}
],
"stream": true
}
```
* `stream`: Optional boolean parameter (defaults to `false`). When set to true, the response will be a stream of Server-Sent Events (SSE). If false, a single JSON response containing the last event is returned. Check out the [Streaming Events](/guides/api/streaming-events) guide for more information about the specific events you can expect.
Message follows the [PromptL format](/promptl/getting-started/introduction).
**Response:**
The response is a stream of Server-Sent Events (SSE) or a single JSON response containing the final event, similar to the "Run a Document" endpoint.
Check out the [Streaming Events](/guides/api/streaming-events) guide for more information about the specific events you can expect.
#### 9. Get a Conversation
Retrieve the conversation history by its UUID. Use this endpoint to fetch the complete message history from a completed conversation, including all user messages, assistant responses, and tool calls.
**Endpoint:** `GET /conversations/{conversationUuid}`
**Path Parameters:**
* `conversationUuid`: UUID of the conversation (required)
**Response:**
The response contains the conversation UUID and the complete conversation history as an array of messages.
**Response Body:**
```json theme={null}
{
"uuid": "conversation-uuid",
"conversation": [
{
"role": "user" | "system" | "assistant",
"content": [
{
"type": "text",
"text": "message content"
}
]
}
]
}
```
Message follows the [PromptL format](/promptl/getting-started/introduction).
**Error Handling**
If the conversation is not found, the API returns a 404 status code with an error message:
```json theme={null}
{
"name": "NotFoundError",
"message": "Conversation not found",
"errorCode": "NotFoundError",
"details": {}
}
```
#### 10. Stop a Conversation
Stop an active run that is currently processing. This is useful when you need to cancel a long-running prompt execution, such as when the output is no longer needed or when you want to prevent further token consumption.
**Endpoint:** `POST /conversations/{conversationUuid}/stop`
**Path Parameters:**
* `conversationUuid`: UUID of the conversation
**Request Body:**
No request body is required for this endpoint.
**Response:**
This endpoint returns a 200 status code when the conversation is successfully stopped.
#### 11. Attach to a Conversation
Attach to an active run to receive its output events. This endpoint is particularly useful when you've started a prompt execution with `background: true` and want to stream the results. You can attach at any point during the run's execution to receive the remaining events.
**Endpoint:** `POST /conversations/{conversationUuid}/attach`
**Path Parameters:**
* `conversationUuid`: UUID of the conversation
**Request Body:**
```json theme={null}
{
"stream": false
}
```
* `stream`: Optional boolean parameter (defaults to `false`). When set to true, the response will be a stream of Server-Sent Events (SSE). If false, a single JSON response containing the final event is returned.
**Response:**
* If `stream` is `true`: The response is a stream of Server-Sent Events (SSE). Check out the [Streaming Events](/guides/api/streaming-events) guide for more information about the specific events you can expect.
* If `stream` is `false`: A single JSON response is returned with the final event in the following structure:
```json theme={null}
{
"uuid": "conversation-uuid",
"conversation": [
{
"role": "user" | "system" | "assistant",
"content": [
{
"type": "text",
"content": "message content"
}
]
}
],
"response": {
"streamType": "text" | "object",
"usage": {
"promptTokens": 10,
"completionTokens": 15,
"totalTokens": 25
},
"text": "response text",
"object": {},
"toolCalls": []
"cost": number
}
}
```
**Error Handling**
The API uses standard HTTP status codes. In case of an error, the response body will contain an error message:
```json theme={null}
{
"error": {
"message": "Error description"
}
}
```
#### 12. Annotate a Log
Add a manual evaluation score to a conversation log. Use this endpoint to provide human feedback or manual assessments of prompt outputs, which can be used for quality tracking and model improvement.
**Endpoint:** `POST /conversations/{conversationUuid}/evaluations/{evaluationUuid}/annotate`
**Path Parameters:**
* `conversationUuid`: UUID of the conversation to annotate
* `evaluationUuid`: UUID of the evaluation to use
**Request Body:**
```json theme={null}
{
"score": 2,
"versionUuid": "version-uuid", // optional
"metadata": {
"reason": "The output is not relevant to the prompt"
}
}
```
**Response:**
```json theme={null}
{
"uuid": "annotation-uuid",
"score": 2,
"normalizedScore": 0.5,
"metadata": {
"reason": "The output is not relevant to the prompt"
},
"hasPassed": false,
"error": "optional-error-message",
"versionUuid": "version-uuid"
}
```
#### 13. Create Log Entry
Create a log entry for a prompt without executing it. This endpoint allows you to record prompt executions that happened outside of Latitude (e.g., direct LLM API calls) for tracking, analytics, and evaluation purposes.
**Endpoint:** `POST /projects/{projectId}/versions/{versionUuid}/documents/logs`
**Path Parameters:**
* `projectId`: Your project ID (required)
* `versionUuid`: Version UUID (required, optional for SDK's defaults to 'live')
**Request Body:**
* Messages follow the [PromptL format](/promptl/getting-started/introduction). If you're using a different method to run your prompts, you'll need to format your messages accordingly.
```json theme={null}
{
"path": "path/to/document",
"messages": [
{
"role": "user" | "system" | "assistant",
{
"type": "text",
"content": string
}
}
],
"response": string
}
```
**Response:**
```json theme={null}
{
"id": "document-id",
"uuid": "log-uuid",
"documentUuid": "document-uuid",
"commitId": "commit-id",
"resolvedContent": "Document content without comments",
"contentHash": "content-hash",
"parameters": {},
"customIdentifier": "custom-identifier",
"duration": "duration",
"source": "source",
"createdAt": "created-at",
"updatedAt": "updated-at"
}
```
# Streaming Events
Source: https://docs-v1.latitude.so/guides/api/streaming-events
Discover how to efficiently handle and process streaming events using Latitude's API.
When executing a prompt in streaming mode, Latitude will return a stream of Server-Sent Events (SSE) that contain real-time updates from the AI provider. This guide explains how to handle and process streaming events using Latitude's API.
# Overview
There are two main types of events that you will receive when streaming events:
* `latitude-event`: Contains information about the chain progress and results.
* `provider-event`: Contains real-time updates from your AI provider.
# Latitude Events
Latitude Events originate from Latitude's AI engine and provide detailed updates on the processing chain, from initiation to completion. Every request is processed as a chain of steps, even if it consists of a single step.
## General structure
All Latitude events follow this structure:
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "event-type"
"uuid": "conversation-uuid",
"messages": [...],
... // Additional event-specific data
}
}
```
## Event Flow
Every chain execution follows the same flow:
1. **Chain Starts**: Every stream begins with a `chain-started` event.
2. **Processing Steps**: Multiple steps can be executed within a chain. All steps start with a `step-started` event and end with a `step-completed` event. Within a step, you may receive additional events:
* **Provider Interaction**: The LLM processing includes `provider-started` and `provider-completed` events.
* **Tool Execution**: If Latitude built-in tools are involved, `tools-started` and `tool-completed` events occur, indicating the execution status of the requested tool.
Check out [Latitude Tools](/guides/prompt-manager/latitude-tools) for
more information about built-in tools.
3. **Chain Completion**: The chain concludes with a `chain-completed`.
However, the chain execution can be interrupted by any of the following events:
* `chain-error`: Indicates an error occurred during the processing chain.
* `tools-requested`: Indicates the AI response requested additional tools to be executed by the client, and they are required to continue processing the chain.
## Event Types
Here's a complete list of all Latitude Event types and their attributes:
The chain has started processing.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "chain-started",
"uuid": "conversation-uuid"
"messages": [...],
}
}
```
A new step in the chain has started processing.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "step-started",
"uuid": "conversation-uuid",
"messages": [...],
}
}
```
Your LLM Provider is being requested to generate a new response.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "provider-started",
"uuid": "conversation-uuid",
"messages": [...],
"config": {
"provider": "provider-name",
"model": "model-name"
... // Rest of the prompt's step configuration
}
}
}
```
Your LLM Provider has completed the response generation.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "provider-completed",
"uuid": "conversation-uuid",
"messages": [...], // The provider response is included at the end of the messages array
"providerLogUuid": "provider-log-uuid", // Identifier of the specific provider log within Latitude
"finishReason": 'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown',
"tokenUsage": {
"promptTokens": 0,
"completionTokens": 0,
"totalTokens": 0
},
"response": {
"text": "response-text",
"toolCalls": [...],
}
}
}
```
Latitude has started running built-in tools requested by the LLM response.
Check out [Latitude Tools](/guides/prompt-manager/latitude-tools) for more
information about built-in tools.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "tools-started",
"uuid": "conversation-uuid",
"messages": [...],
"tools": [
{
"id": "tool-id",
"name": "tool-name",
"arguments": {
"argument-name": "argument-value"
}
}
]
}
}
```
A built-in tool has completed its execution.
Check out [Latitude Tools](/guides/prompt-manager/latitude-tools) for more
information about built-in tools.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "tool-completed",
"uuid": "conversation-uuid",
"messages": [...]
}
}
```
An error has occurred during the processing of the chain.
This event will terminate the SSE stream.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "chain-error",
"uuid": "conversation-uuid",
"messages": [...],
"error": {
"message": "error-message",
"code": "error-code"
}
}
}
```
The chain processing has completed successfully.
This event will terminate the SSE stream.
```json theme={null}
{
"type": "latitude-event",
"data": {
"type": "chain-completed",
"messages": [...],
"uuid": "conversation-uuid",
"tokenUsage": {
"promptTokens": 0,
"completionTokens": 0,
"totalTokens": 0
},
"finishReason": 'stop' | 'length' | 'content-filter' | 'tool-calls' | 'error' | 'other' | 'unknown',
}
}
```
# Provider Events
Provider Events are events that are generated by your AI provider. These events contain real-time updates from your AI provider, providing insights into the ongoing tasks. This is specially useful to render your LLM's responses in real-time as they are being generated.
These events will always take place between a `provider-started` and a `provider-completed` event. You can expect updates on each stage of the processing, providing insights into the ongoing tasks.
Here's an example of a Provider Event with the response text delta:
```json theme={null}
{
"type": "provider-event",
"data": {
"type": "text-delta",
"textDelta": "response-text-delta"
}
}
```
For more information about these events, visit [Vercel AI SDK's Documentation](https://sdk.vercel.ai/docs/reference/ai-sdk-core/stream-object#full-stream)
# Handling SSE Events
The API uses SSE for real-time updates. Here's how to handle SSE responses:
1. Set up an EventSource or use a library that supports SSE.
2. Listen for events and parse the JSON data in each event.
3. Handle different event types.
# Webhooks
Source: https://docs-v1.latitude.so/guides/api/webhooks
Learn how to receive real-time notifications about events that occur in your Latitude workspace.
## Overview
Webhooks provide a way to integrate Latitude with your own systems and applications.
When an event occurs, Latitude will send an HTTP POST request to your configured webhook URL with details about the event.
Currently, webhooks support notifications for commit publications, with more event types coming soon.
## Setting Up Webhooks
### Creating a Webhook
1. Navigate to your workspace [settings](https://app.latitude.so/settings)
2. Go to the **Webhooks** section
3. Click **New Webhook**
4. Configure your webhook:
* Name: A descriptive name for your webhook
* URL: The endpoint where you want to receive webhook notifications
* Projects: (Optional) Filter events to specific projects
* Active: Enable/disable the webhook
### Security
Each webhook is assigned a unique secret key that is used to sign webhook payloads. This allows you to verify that webhook requests are coming from Latitude.
When you receive a webhook request, you can verify its authenticity by checking the `X-Latitude-Signature` header. The signature is generated using HMAC SHA-256 with your webhook's secret key.
The `X-Latitude-Signature` header is **not** included when testing the
endpoint using the "Test Endpoint" button from the Latitude UI.
## Webhook Events
Currently, Latitude webhooks support the following event:
### Project Events
* `commitPublished`: Triggered when a commit is published in a project
* `documentLogCreated`: Triggered when a prompt log is created in a project
More event types will be added in future updates, including:
* Document runs and evaluations
* Project and workspace changes
* User management events
* Dataset operations
## Webhook Payload
Each webhook request includes:
1. HTTP Headers:
```
X-Latitude-Signature:
```
2. Request Body:
```json theme={null}
{
"eventType": "commitPublished",
"payload": {
// Commit-specific data
}
}
```
## Webhook Delivery
Latitude implements a robust webhook delivery system:
1. **Retry Logic**: Failed webhook deliveries are automatically retried with exponential backoff
2. **Delivery Status**: You can monitor webhook delivery status in the webhook settings (upcoming)
3. **Error Handling**: Failed deliveries include error messages and response status codes
4. **Rate Limiting**: Webhook requests are rate-limited to prevent overwhelming your servers
## Best Practices
1. **Verify Signatures**: Always verify webhook signatures to ensure requests are from Latitude
2. **Handle Duplicates**: Implement idempotency checks to handle duplicate webhook deliveries
3. **Respond Quickly**: Respond to webhook requests within 5 seconds
4. **Monitor Failures**: Regularly check webhook delivery status and logs
5. **Use HTTPS**: Always use HTTPS endpoints for webhook URLs
6. **Whitelisting**: Whitelist Latitude IP addresses to ensure delivery
7. **IP Addresses**:
* 18.193.205.15
## Example Implementation
Here's an example of how to verify a webhook signature in Node.js:
```javascript theme={null}
const crypto = require('crypto')
function verifyWebhookSignature(payload, signature, secret) {
const hmac = crypto.createHmac('sha256', secret)
const calculatedSignature = hmac.update(payload).digest('hex')
return crypto.timingSafeEqual(
Buffer.from(signature),
Buffer.from(calculatedSignature),
)
}
// In your webhook handler
app.post('/webhook', (req, res) => {
const signature = req.headers['x-webhook-signature']
const payload = JSON.stringify(req.body)
if (!verifyWebhookSignature(payload, signature, WEBHOOK_SECRET)) {
return res.status(401).send('Invalid signature')
}
// Process the webhook
res.status(200).send('OK')
})
```
## Troubleshooting
If you're experiencing issues with webhooks:
1. Check the webhook delivery status in your workspace settings
2. Verify your webhook URL is accessible and responding correctly
3. Ensure your server is handling requests within the timeout period
4. Check your server logs for any errors
5. Verify the webhook signature is being calculated correctly
# Command Reference
Source: https://docs-v1.latitude.so/guides/cli/commands
Full reference for the Latitude CLI commands.
All commands support `-p, --path ` to run against a specific directory (default `.`), and `--dev` to use a local Gateway.
## init
Initialize a Latitude project in the current directory.
```bash theme={null}
latitude init [-p ] [--dev]
```
* Prompts for your API key (or uses `latitude login`/`LATITUDE_API_KEY`)
* Lets you choose:
* Create a new project (provide a name)
* Use an existing project (enter `projectId`)
* Creates the prompts directory (asks before clearing a non-empty folder)
* Writes `latitude-lock.json`
`latitude-lock.json` example:
```json theme={null}
{
"projectId": 123,
"rootFolder": "prompts",
"version": "live"
}
```
## status
Show project and version info plus a local vs remote diff summary.
```bash theme={null}
latitude status [-p ] [--dev]
```
* Prints project link, version title/description
* Summarizes Added/Modified/Deleted prompts
## pull
Pull remote prompts to your local filesystem with a diff preview.
```bash theme={null}
latitude pull [-p ] [-y] [--dev]
```
* Computes local vs remote changes
* Choose: Accept / Cancel / View details (opens pager for full diffs)
* `-y` skips confirmation and applies changes
* Saves prompts as `.promptl` text files
* Removes local files that no longer exist remotely
## push
Push local prompt changes to the current version.
```bash theme={null}
latitude push [-p ] [-y] [--dev]
```
* Reads `.promptl` files under `rootFolder`
* Shows a diff and asks for confirmation; `-y` skips confirmation
* Sends only changed files to the server
## checkout
Checkout a specific version or create a new version and switch to it.
```bash theme={null}
latitude checkout [versionUuid] [-b ] [-p ] [--dev]
```
* `latitude checkout `: updates `latitude-lock.json` and pulls that version
* `latitude checkout -b `: creates a new version and switches to it
* Validates the target version before updating the lock file
## login
Store or override your API key in the system keychain.
```bash theme={null}
latitude login [--api-key ] [-f]
```
* Warns if `LATITUDE_API_KEY` is set (that variable takes precedence)
* `-f/--force` to override without confirmation
## help
Show CLI help.
```bash theme={null}
latitude help
```
## Prompt file format
Prompts are stored as plain `.promptl` text files:
```text theme={null}
# prompts/welcome.promptl
Welcome to Latitude!
```
Each file contains the raw prompt content using PromptL syntax.
# Installation & Auth
Source: https://docs-v1.latitude.so/guides/cli/installation
Install the Latitude CLI and authenticate with your API key.
## Install
```bash theme={null}
npm i -g @latitude-data/cli
# or
pnpm add -g @latitude-data/cli
# or
yarn global add @latitude-data/cli
```
Verify installation:
```bash theme={null}
latitude --version
```
## Authenticate
Set your API key once using the system keychain:
```bash theme={null}
latitude login
```
* Provide `--api-key ` to skip the prompt
* Use `-f/--force` to override without confirmation
### Environment variable (CI/containers)
You can also provide `LATITUDE_API_KEY` as an environment variable. If set, it takes precedence over the stored key.
```bash theme={null}
export LATITUDE_API_KEY=your_api_key
```
Unset the variable to use the stored key again.
## Development mode
Add `--dev` to any command to target a local Gateway at `localhost:8787`:
```bash theme={null}
latitude status --dev
```
This is handy when running Latitude in development on your machine.
# CLI Overview
Source: https://docs-v1.latitude.so/guides/cli/overview
Manage Latitude projects and prompts from your terminal.
The Latitude CLI helps you work locally with your Latitude prompts. Initialize projects, pull/push prompt changes, switch versions, and manage authentication — all from the command line.
## Key Capabilities
* Create or pull a Latitude project from CLI
* Pull remote prompts locally with safe diff previews
* Push local prompt changes to a project version
* Check project status and pending changes
* Checkout a specific version or create a new one
* Manage your API key securely
## Typical Workflow
1. Install the CLI and authenticate
2. Run `latitude init` inside your repo
3. Pull prompts: `latitude pull`
4. Edit prompts locally
5. Push changes: `latitude push`
6. Switch versions when needed: `latitude checkout ` or `-b `
# Golden Datasets
Source: https://docs-v1.latitude.so/guides/datasets/golden-datasets
Use curated datasets to prevent regressions and ensure consistent prompt performance.
A "Golden Dataset" is a carefully curated collection of inputs and expected outputs that represents critical test cases and desired behaviors for your prompt. It serves as a benchmark to prevent regressions when making changes.
## Why Use a Golden Dataset?
* **Prevent Regressions**: Ensure that changes to your prompt (or underlying models) don't break previously working functionality or degrade quality on important cases.
* **Consistent Benchmarking**: Provide a stable baseline for comparing the performance of different prompt versions.
* **Confidence in Deployment**: Increase confidence that a new prompt version meets quality standards before publishing.
* **Capture Edge Cases**: Explicitly test how your prompt handles known difficult or important scenarios.
## Creating a Golden Dataset
1. **Identify Critical Scenarios**: Determine the most important inputs or use cases your prompt must handle correctly.
2. **Gather Examples**: Collect representative examples for these scenarios. Sources include:
* Real production [Logs](/guides/logs/overview) (especially successful ones or interesting failures).
* Manually crafted edge cases.
* Existing test suites.
3. **Define Expected Outputs (Ground Truth)**: For each input, define the ideal or minimally acceptable output. This might be:
* An exact string.
* A specific JSON structure.
* Key information that must be present.
* A classification label.
4. **Format as CSV**: Structure this data into a CSV file with appropriate input columns (matching prompt parameters) and output columns (e.g., `expected_output`, `expected_category`).
5. **Upload to Latitude**: [Upload the CSV as a new Dataset](/guides/datasets/overview#1-uploading-csv-files) in Latitude and give it a clear name (e.g., "Chatbot v2 - Golden Regression Set").
6. **Marking the expected output column**: You can mark the expected output column as a 'label' by clicking on the column name and editing its role.
## Using the Golden Dataset in Workflows
* **During Development**: When iterating on a prompt in a draft version, run batch evaluations using relevant [Programmatic Rules](/guides/evaluations/programmatic-rules) (like Exact Match, Semantic Similarity, JSON Validation) against the golden dataset to check for regressions before considering the draft ready.
* **CI/CD Pipeline**: Integrate automated batch evaluations against the golden dataset into your pre-deployment checks. Fail the build if key metrics on the golden dataset drop below a threshold.
* **Version Comparison**: When comparing two prompt versions (e.g., A/B testing), run both against the golden dataset using the same evaluations to get a standardized performance comparison.
## Maintaining the Golden Dataset
* **Review Periodically**: Regularly review the golden dataset to ensure it still represents the most critical scenarios.
* **Add New Cases**: As new important use cases or failure modes are discovered in production, consider adding them to the golden dataset.
* **Version Control (Implicit)**: While datasets themselves aren't directly versioned *within* Latitude like prompts, you can manage your source CSV files in your own version control system (like Git) if needed.
By establishing and maintaining golden datasets, you create a robust safety net for your prompt development lifecycle.
## Next Steps
* Learn more about [Creating and Using Datasets](/guides/datasets/overview)
* Set up [Programmatic Rule Evaluations](/guides/evaluations/programmatic-rules) to use with your dataset.
* Integrate checks into your [Team Workflows](/guides/evaluations/integrating-evaluations-workflow).
# Overview
Source: https://docs-v1.latitude.so/guides/datasets/overview
Learn how to create, manage, and utilize datasets for batch testing and evaluation.
Datasets in Latitude are collections of data, used primarily for running prompt experiments in the [playground](/guides/prompt-manager/playground) or in the context of [an evaluation](/guides/evaluations/running-evaluations#running-evaluations-on-datasets-run-experiment). They allow you to test your prompts against a consistent set of inputs and expected outputs.
## What is a Dataset?
A dataset consists of rows and columns, where:
* **Input Columns**: Represent the parameters your prompt expects (e.g., `customer_query`, `product_name`).
* **Output/Label Columns (Optional)**: Contain the ground truth or expected outputs for specific inputs (e.g., `expected_sentiment`, `ideal_summary`). These are required for evaluations like Exact Match or Semantic Similarity.
Each row represents a single test case for your prompt.
![Golden NPS Dataset Example]()
## Creating Datasets
You can create datasets in Latitude in several ways:
### 1. Uploading CSV Files
This is the most common method for bringing existing test data into Latitude.
1. Navigate to the "Datasets" section in your project.
2. Click "Upload Dataset".
3. Drag and drop your CSV file or browse to select it.
![]()
4. **Preview and Configure**: Latitude will show a preview of your data. You may need to confirm:
* Column headers are correctly identified.
* Data types are inferred correctly.
5. Give your dataset a descriptive name.
6. Click "Create Dataset".
### 2. Generating Synthetic Data
Latitude can use an AI model to generate synthetic datasets based on your specifications, useful for quickly creating test cases or exploring variations.
1. Navigate to the [Datasets](https://app.latitude.so/datasets) section.
2. Click **Generate Dataset**.
3. Describe the data you need:
* Specify the desired columns (e.g., `user_query`, `expected_category`).
* Provide instructions on the type of data for each column (e.g., "Generate realistic user support questions", "Assign a category from \[Billing, Technical, General]").
* Indicate the number of rows to generate.
![]()
4. Click "Generate Dataset".
The generator has limits on complexity and runtime. For large or very complex
datasets, uploading a CSV is often more reliable. Start with smaller
generation requests (e.g., 20-50 rows) to test.
### 3. Saving Logs as Datasets
You can create a new dataset directly from existing production logs, which is excellent for evaluating prompts against real-world interactions.
1. Navigate to the **Logs** section of one of your prompts.
2. Select the logs you want to include in the dataset.
3. Click the **Save logs to Dataset** button (or similar option).
![]()
4. Choose in the form whether to create a new dataset or save the logs to an existing dataset.
5. Confirm your selection
## Managing Datasets
Once created, you can manage your datasets from the main "Datasets" page:
* **View**: Click on a dataset name to view its contents.
* **Edit**: Modify, add and remove dataset rows or columns.
* **Rename**: Change the dataset's name.
* **Download**: Export the dataset as a CSV file.
* **Delete**: Permanently remove a dataset.
### Marking an Expected Output Column as a Label
You can mark an expected output column as a label by:
1. Click on the edit button next to the column's name:
![Edit column]()
2. Set the column's role to "label":
![Label column]()
## Linking Datasets to Evaluations
The primary use of datasets is to run evaluations in batch mode:
1. Go to the specific evaluation you want to run (under a prompt's "Evaluations" tab).
2. Initiate an [Experiment in the evaluation](/guides/evaluations/running-evaluations#running-evaluations-on-datasets-batch-mode).
3. Select the dataset you want to use.
4. If the evaluation requires ground truth (e.g., Exact Match), map the evaluation's expected output requirement to the relevant column in your dataset (e.g., link `expected_output` to the `ideal_summary` column).
Latitude then runs the prompt for each row in the dataset and applies the evaluation, comparing the output to the corresponding data in the dataset row.
## Next Steps
* Learn about establishing [Golden Datasets for Regression Testing](/guides/datasets/golden-datasets)
* Understand how to [Run Evaluations](/guides/evaluations/running-evaluations)
* Explore [Using Datasets for Fine-tuning](/guides/datasets/datasets-for-finetuning)
# Composite Scores
Source: https://docs-v1.latitude.so/guides/evaluations/composite-scores
Combine the results of multiple evaluations into a single score.
Composite Score evaluations combine multiple existing evaluations into a unified score. This is ideal for measuring overall quality by aggregating various aspects of your prompt's performance, such as combining accuracy, safety, and relevance metrics into one comprehensive assessment.
* **How it works**: Runs multiple existing evaluations and combines their results using different mathematical approaches (average, weighted, or custom formula). Note that sub-evaluations do not create their own results!
* **Best for**: Holistic quality assessment, combining multiple evaluation criteria, creating overall performance metrics, balancing trade-offs between different aspects (e.g., accuracy vs. safety).
* **Requires**: At least two existing evaluations configured on the same prompt. These evaluations can be of any type, even other Composite Scores!
Currently, composite evaluations cannot run in live mode and only support
sub-evaluations that do not require an expected output. Check out the [Running
Evaluations](/guides/evaluations/running-evaluations) guide.
## Setup
Go to evaluations tab on a prompt in one of your projects.
On the top right corner, click on the "Combine evaluations" button.
![Choose Composite Score
metric]()
Select the evaluations you want to combine. You need to select at least two
evaluations. ![Select
sub-evaluations]()
## Metrics
Combines scores evenly. The resulting score is the average.
Combines scores using custom weights. The resulting score is the weighted
blend. Weights are measured in percentage and must add up to 100%.
Combines scores using a custom formula. The resulting score is the result of
the expression. The expression can be a complex mathematical formula.
# Humans-in-the-Loop
Source: https://docs-v1.latitude.so/guides/evaluations/humans-in-the-loop
Incorporate manual reviews and direct human feedback into your evaluation workflow.
Human-in-the-Loop (HITL) involve direct human review and assessment of prompt outputs. This method is essential for capturing nuanced judgments, user preferences, and criteria that are difficult for automated systems to evaluate.
* **How it works**: Team members manually review prompt outputs (logs) and assign scores or labels based on their judgment.
* **Best for**: Capturing nuanced human preferences, evaluating criteria difficult for LLMs to judge, initial quality assessment, creating golden datasets for other evaluation types.
* **Requires**: Setting up manual review workflows and criteria for reviewers.
Because HITL evaluations require manual input, they **do not support automatic
live or batch execution** like LLM-as-Judge or Programmatic Rules. Feedback
must be submitted individually for each log reviewed.
## Setup
Go to evaluations tab on a prompt in one of your projects.
On the top right corner, click on the "Add evaluation" button.
Choose "Human-in-the-Loop" tab in the evaluation modal. ![Choose
Human-in-the-Loop]()
![Choose HITL metric]()
## Metrics
Judges whether the response meets the criteria. The resulting score is
"passed" or "failed"
Judges the response by rating it under a criteria. The resulting score is the
rating
## Annotate logs in Latitude UI
Manually submitted results appear alongside other evaluation results:
* **Logs View**: Attached to the individual log entry.
* **Evaluations Tab**: Aggregated statistics and distributions for the HITL evaluation.
## Capturing Feedback via API/SDK
Check [how to annotate a log](/examples/sdk/annotate-log). A log is the result of running your prompt. So the person can annotate that result and tell if it was good or bad, or provide a score.
# Evaluation Workflows
Source: https://docs-v1.latitude.so/guides/evaluations/integrating-evaluations-workflow
Best practices for incorporating prompt evaluation into your team's development lifecycle.
Effective prompt evaluation isn't just about running tests; it's about integrating the process into your team's regular development and deployment workflows. Here are some strategies:
## 1. Define Your Quality Standards
* **Identify Key Metrics**: What defines a "good" response for this prompt? (e.g., Accuracy, Helpfulness, Conciseness, Safety, Format Adherence).
* **Set Acceptance Criteria**: Define minimum acceptable scores or pass rates for your key evaluations.
* **Choose Evaluation Types**: Select the right mix of [LLM-as-Judge](/guides/evaluations/llm-as-judges), [Programmatic Rules](/guides/evaluations/programmatic-rules), and [Manual Evaluations](/guides/evaluations/humans-in-the-loop) to cover your criteria.
## 2. Establish Golden Datasets
* Create and maintain a representative [Dataset](/guides/datasets/overview) (a "golden dataset") containing diverse inputs and, where applicable, expected outputs.
* This dataset serves as your benchmark for regression testing.
* Include challenging edge cases and examples representing different user intents.
## 3. Evaluation During Development
* **Playground Testing**: Use the [Playground](/guides/prompt-manager/playground) to get immediate evaluation feedback while iterating on prompts.
* **Draft Evaluations**: Run [experiments](/guides/evaluations/running-evaluations#running-evaluations-on-datasets-run-experiment) on your golden dataset *before* merging changes from a draft version.
* **Peer Review**: Include evaluation results (especially for failing cases) as part of the review process for prompt changes.
## 4. Continuous Monitoring in Production
* **Live Evaluations**: Enable [live evaluations](/guides/evaluations/running-evaluations#running-evaluations-continuously-live-mode-%2F-ongoing) for critical metrics (e.g., format validation, safety checks, basic relevance) to monitor real-time performance.
## 6. Feedback Loops and Improvement
* **Regular Review Meetings**: Discuss evaluation trends and results as a team.
* **Analyze Failures**: Dig into logs with poor evaluation scores to understand the root causes.
* **Leverage Suggestions**: Use the [Prompt Suggestions](/guides/evaluations/prompt-suggestions) feature to guide improvements.
* **Update Golden Dataset**: Periodically add new challenging examples or successful edge cases from production logs to your golden dataset.
* **Refine Evaluations**: Adjust evaluation criteria or prompts as your understanding of quality evolves.
By embedding these practices, your team can systematically ensure prompt quality, reduce regressions, and continuously improve the reliability and performance of your AI applications.
# LLM-as-Judges
Source: https://docs-v1.latitude.so/guides/evaluations/llm-as-judges
Use language models to evaluate the quality, style, and correctness of prompt outputs.
* **How it works**: Uses another language model (the "judge") to score or critique the output of your target prompt based on specific criteria (e.g., helpfulness, clarity, adherence to instructions).
* **Best for**: Subjective criteria, complex assessments, evaluating nuanced qualities like creativity or tone.
## Setup
Go to evaluations tab on a prompt in one of your projects.
On the top right corner, click on the "Add evaluation" button.
Choose "LLM-as-a-judge" tab in the evaluation modal. ![Choose
LLM-as-a-judge]()
![Choose LLM-as-a-judge metric]()
## Metrics
Judges whether the response meets the criteria. The resulting score is
"passed" or "failed"
Judges the response by rating it under a criteria. The resulting score is the
rating
Judges the response by comparing the criteria to the expected output. The
resulting score is the percentage of compared criteria that is met
Judges the response under a criteria using a custom prompt. The resulting
score is the value of criteria that is met
## Expected output
The expected output, also known as label, refers to the correct or ideal response that the language model should generate for a given prompt. You can create datasets with expected output columns to evaluate prompts with ground truth.
**Comparison** and **Custom (labeled)** metrics require an expected output.
## Templates
We have a list of pre-configured LLM-as-a-judge templates that you can use to quickly set up evaluations. These templates cover common evaluation scenarios and can be customized to fit your specific needs.
* **Adaptability**
Evaluate how well the response adapts to user preferences or context
* **Bias and Fairness**
Assess whether the response is free of bias or unfair generalizations
* **Coherence and Fluency**
Evaluate the clarity and flow of the response
* **Conciseness**
Assess whether the response is brief but informative
* **Consistency**
Check if the response is consistent with prior information or context
* **Creativity**
Evaluate the originality and imagination shown in the response
* **Domain Expertise**
Assess the response for accuracy and knowledge in a specific domain
* **Engagement or User Experience**
Rate how well the response engages the user or enhances the conversation
* **Error Handling and Recovery**
Evaluate how well the response corrects user errors or misunderstandings
* **Ethical Compliance**
Determine if the response follows ethical standards
* **Explainability**
Rate how clearly the response explains the concept or information
* **Factuality**
Evaluates whether the following response is factually accurate
* **Faithfulness to Instructions**
Assess how well the response follows the given instructions
* **Helpfulness and Informativeness**
Rate how helpful and informative the response is
* **Formality and Style**
Evaluate whether the response matches the desired formality or style
* **Hallucination Detection**
Detect if the response introduces unsupported or false information
* **Harmlessness and Ethical Considerations**
Check if the response promotes ethical and non-harmful behavior
* **Novelty**
Assess the originality of the response in its content or style
* **Humor or Emotional Understanding**
Rate whether the response appropriately uses humor or addresses emotional content
* **Helpfulness and Informativeness**
Rate how helpful and informative the response is
* **Redundancy**
Check if the response repeats information unnecessarily
* **Relevance**
Rate how well the response addresses the given context or query
* **Response Time or Latency**
Measure whether the response time is suitable for real-time interaction
* **Satisfaction**
Rate overall satisfaction with the response
* **Specificity**
Evaluate how specific and relevant the response is to the query
* **Long-Term Consistency (in Multi-turn Dialogues)**
Check if the response remains consistent over multiple turns of dialogue
* **Novelty**
Assess the originality of the response in its content or style
* **Persuasiveness**
Rate how convincing the response is
* **Toxicity and Safety**
Check if the response contains harmful or inappropriate content
* **Uncertainty or Confidence**
Evaluate if the response expresses appropriate confidence or acknowledges uncertainty
* **Redundancy**
Check if the response repeats information unnecessarily
* **Relevance**
Rate how well the response addresses the given context or query
* **Response Time or Latency**
Measure whether the response time is suitable for real-time interaction
* **Satisfaction**
Rate overall satisfaction with the response
* **Specificity**
Evaluate how specific and relevant the response is to the query
* **Toxicity and Safety**
Check if the response contains harmful or inappropriate content
* **Uncertainty or Confidence**
Evaluate if the response expresses appropriate confidence or acknowledges uncertainty
# Overview
Source: https://docs-v1.latitude.so/guides/evaluations/overview
Understand the different ways to evaluate prompt performance in Latitude.
Evaluations are crucial for understanding and improving the quality of your AI prompt responses. Latitude provides a comprehensive evaluation framework to assess performance against various criteria.
## Why Evaluate Prompts?
* **Measure Quality**: Objectively assess if prompts meet desired standards (accuracy, relevance, tone, safety, etc.).
* **Identify Weaknesses**: Pinpoint scenarios where prompts underperform.
* **Compare Versions**: Quantify the impact of prompt changes (A/B testing).
* **Drive Improvement**: Gather data to refine prompts using [Prompt Suggestions](/guides/evaluations/prompt-suggestions).
* **Ensure Reliability**: Build confidence in production-deployed prompts.
## Evaluation Types
Latitude supports three main approaches to evaluation, each suited for different needs:
1. [**LLM-as-Judge**](/guides/evaluations/llm-as-judges):
* **How it works**: Uses another language model (the "judge") to score or critique the output of your target prompt based on specific criteria (e.g., helpfulness, clarity, adherence to instructions).
* **Best for**: Subjective criteria, complex assessments, evaluating nuanced qualities like creativity or tone.
* **Requires**: Defining evaluation criteria (often via templates or custom instructions for the judge LLM).
2. [**Programmatic Rule**](/guides/evaluations/programmatic-rules):
* **How it works**: Applies code-based rules and metrics to check outputs against objective criteria.
* **Best for**: Objective checks, ground truth comparisons (using datasets), format validation (JSON, regex), safety checks (keyword detection), length constraints.
* **Requires**: Defining specific rules (e.g., exact match, contains keyword, JSON schema validation) and potentially providing a [Dataset](/guides/datasets/overview) with expected outputs.
3. [**Human-in-the-Loop**](/guides/evaluations/humans-in-the-loop):
* **How it works**: Team members manually review prompt outputs (logs) and assign scores or labels based on their judgment.
* **Best for**: Capturing nuanced human preferences, evaluating criteria difficult for LLMs to judge, initial quality assessment, creating golden datasets for other evaluation types.
* **Requires**: Setting up manual review workflows and criteria for reviewers.
## Combining Evaluations
Sometimes you want to summarize the results of multiple evaluations into a single score, defining, for example, an overall performance report of your prompt.
To do this, you can use [Composite Evaluations](/guides/evaluations/composite-scores), also named *Composite Scores*.
## How Evaluations Connect to Prompts
* **Per-Prompt Basis**: Evaluations are configured individually for each prompt within a project.
* **Target Logs**: Evaluations run on the [Logs](/guides/logs/overview) generated by their associated prompt.
* **Triggering**: Evaluations can be run manually on batches of logs/datasets or automatically on incoming logs (live mode). See [Running Evaluations](/guides/evaluations/running-evaluations).
* **Results**: Evaluation results (scores, labels, feedback) are stored alongside the corresponding logs, providing a rich dataset for analysis and improvement.
## Actual Outputs
The actual output is the generated output from the model conversation. This is the output to perform evaluations against.
### Selecting the Actual Output to evaluate against
By default the actual output is the last assistant message in the conversation, parsed as a simple string. However, some use cases requires more complex parsing, like evaluating tool calling or middle CoT.
1. Go to the evaluation's settings by clicking on the right-side button in the evaluation's dashboard:
![Evaluation settings button]()
2. Click on **Advanced configuration**.
3. Configure:
* Message selection: The last message or all messages in the conversation.
* Content filter: Optionally filter the messages by content type (e.g., text, images, tool calls...).
* Parsing format: The format to parse the actual output into (e.g., string, JSON, YAML...).
* Field accessor: The optional field to access in the actual output (e.g., `answer`, `arguments.recommendations[2]`, `[-1].tool_name`, ...).
![Evaluation actual output configuration]()
4. Test the configuration by clicking on the "Test" button:
![Evaluation actual output test]()
Take into account that messages with multiple content are flattened into individual ones and stringification is done deterministically.
## Expected Outputs
The expected output, also known as label, refers to the correct or ideal response that the language model should generate for a given prompt. You can create [Datasets](/guides/datasets/overview) with [Expected Output Columns](/guides/datasets/overview#marking-an-expected-output-column-as-a-label) to evaluate prompts with ground truth.
![Evaluation expected output]()
### Selecting the Expected Output to evaluate against
By default the expected output is the value of the dataset column, parsed as a simple string. However, some use cases requires more complex parsing, like complex JSON objects or nested arrays.
1. Go to the evaluation's settings by clicking on the right-side button in the evaluation's dashboard:
![Evaluation settings button]()
2. Click on **Advanced configuration**.
3. Configure:
* Parsing format: The format to parse the expected output into (e.g., string, JSON, YAML...).
* Field accessor: The optional field to access in the expected output (e.g., `answer`, `arguments.recommendations[2]`, `[-1].tool_name`, ...).
![Evaluation expected output configuration]()
Take into account that stringification is done deterministically.
## Negative Evaluations
Sometimes, you want to measure undesirable traits (e.g., toxicity, hallucination presence), where a *lower* score is better. Latitude allows you to mark evaluations as "negative".
1. Go to the evaluation's settings by clicking on the right-side button in the evaluation's dashboard:
![Evaluation settings button]()
2. Click on **Advanced configuration**.
3. Select "optimize for a lower score" to indicate high scores are undesirable:
![Evaluation negative setting]()
> The [Prompt Suggestions](/guides/evaluations/prompt-suggestions) feature will use this setting to optimize correctly.
## Next Steps
Dive deeper into each evaluation type:
* [LLM-as-Judges](/guides/evaluations/llm-as-judges)
* [Programmatic Rules](/guides/evaluations/programmatic-rules)
* [HITL (Humans in the Loop)](/guides/evaluations/humans-in-the-loop)
* [Composite Scores](/guides/evaluations/composite-scores)
# Programmatic Rules
Source: https://docs-v1.latitude.so/guides/evaluations/programmatic-rules
Use code-based metrics and rules to objectively evaluate prompt outputs.
Programmatic Rule evaluations apply objective, code-based rules and metrics to assess prompt outputs. They are ideal for validating specific requirements, checking against ground truth, and enforcing constraints automatically.
* **How it works**: Applies code-based rules and metrics to check outputs against objective criteria.
* **Best for**: Objective checks, ground truth comparisons (using datasets), format validation (JSON, regex), safety checks (keyword detection), length constraints.
* **Requires**: Defining specific rules (e.g., exact match, contains keyword, JSON schema validation) and potentially providing a [Dataset](/guides/datasets/overview) with expected outputs.
For subjective criteria, use
[LLM-as-Judge](/guides/evaluations/llm-as-judges). For human preferences, use
[HITL (Human In The Loop)](/guides/evaluations/humans-in-the-loop).
## Setup
Go to evaluations tab on a prompt in one of your projects.
On the top right corner, click on the "Add evaluation" button.
Choose "Programatic Rule" tab in the evaluation modal. ![Choose
LLM-as-a-judge]()
![Choose Programatic Rule
metric]()
## Metrics
Checks if the response is exactly the same as the expected output. The
resulting score is "matched" or "unmatched".
Checks if the response matches the regular expression. The resulting score is
"matched" or "unmatched".
Checks if the response follows the schema. The resulting score is "valid" or
"invalid". Right now only JSON schemas are supported.
Checks if the response is of a certain length. The resulting score is the
length of the response. The length can be counted by characters, words or
sentences.
Checks if the response contains the expected output. The resulting score is
the percentage of overlap. Overlap can be measured with longest common
substring, Levenshtein distance and ROUGE algorithms.
Checks if the response is semantically similar to the expected output. The
resulting score is the percentage of similarity. Similarity is measured by
computing the cosine distance.
Checks if the response is numerically similar to the expected output. The
resulting score is the percentage of similarity. Similarity is measured by
computing the relative difference.
## Expected output
The expected output, also known as label, refers to the correct or ideal response that the language model should generate for a given prompt. You can create datasets with expected output columns to evaluate prompts with ground truth.
**Exact Match**, **Lexical Overlap**, **Semantic Similarity** and **Numeric
Similarity** metrics require an expected output.
## Using Datasets for Ground Truth
Many programmatic rules (Exact Match, Lexical Overlap, Semantic Similarity) require comparing the model's output against a known correct answer (`expected_output`). This is typically done by:
1. Creating a [Dataset](/guides/datasets/overview) containing input examples and their corresponding desired outputs.
2. Configuring the evaluation rule to use the `expected_output` column from that dataset.
3. Running the evaluation in [an experiment](/guides/evaluations/running-evaluations#running-evaluations-on-datasets-run-experiment) on that dataset.
# Prompt Suggestions
Source: https://docs-v1.latitude.so/guides/evaluations/prompt-suggestions
Use automatically generated suggestions based on evaluation results to improve your prompts.
Latitude's Prompt Suggestions feature (known as the Refiner) analyzes your evaluation results to automatically recommend improvements for your prompts. It acts like an AI assistant helping you iterate faster and achieve better performance.
## How Suggestions Are Generated
1. **Data Collection**: The system gathers results from your completed evaluations ([LLM-as-Judge](/guides/evaluations/llm-as-judges), [Programmatic Rules](/guides/evaluations/programmatic-rules), and [Manual Evaluations](/guides/evaluations/humans-in-the-loop)). Both batch and live evaluation results are considered.
2. **Pattern Analysis**: Latitude analyzes these results, looking for correlations between prompt inputs, outputs, and evaluation scores. It identifies patterns where certain inputs lead to lower scores or specific failure modes.
3. **Suggestion Generation**: Based on these patterns, an AI model generates concrete suggestions for modifying your prompt. These suggestions might involve:
* Rewording instructions for clarity.
* Adding context or constraints.
* Providing better examples (few-shot learning).
* Adjusting prompt structure.
* Modifying [configuration parameters](/guides/prompt-manager/configuration).
4. **Prioritization**: Suggestions are often prioritized based on the potential impact on evaluation scores.
Suggestions become more insightful as more evaluation data is collected. Aim
for at least 20-30 evaluated logs for meaningful analysis, more data is
generally better.
## Viewing and Applying Suggestions
1. **Navigate to the Prompt**: Open the prompt you want to improve in the editor.
2. **Check for Suggestions**: If suggestions are available, a "Suggestions" button/indicator will appear at the bottom of the prompt editor.
![Suggestions Indicator]()
3. **Review Suggestions**: Clicking the button opens a panel displaying the generated suggestions. Each suggestion includes:
* The reasoning based on evaluation data (e.g., "Outputs often failed the 'Conciseness' evaluation for long inputs").
* Clicking the "View" button will display a diff of the proposed changes to the current prompt.
![Suggestions View]()
4. **Apply or Dismiss**: For each suggestion, you can:
* **Apply**: Automatically applies the suggested change to your current prompt draft.
* **Dismiss**: Ignores the suggestion.
![Suggestions Diff]()
## Next Steps
* Ensure you have robust [Evaluations](/guides/evaluations/overview) set up.
* Regularly [Run Evaluations](/guides/evaluations/running-evaluations) to feed the Refiner.
* Learn about preparing data with [Datasets](/guides/datasets/overview).
# Running Evaluations
Source: https://docs-v1.latitude.so/guides/evaluations/running-evaluations
Execute evaluations on datasets or continuously on live production logs.
Once you have defined [evaluation criteria](/guides/evaluations/overview) (LLM-as-Judge, Programmatic Rules), you need to run them against your prompt's logs to generate results. Latitude supports two primary modes for running automated evaluations:
## Running Evaluations on Datasets (Batch Mode)
Batch evaluations allow you to assess prompt performance across a predefined set of inputs and expected outputs contained within a [Dataset](/guides/datasets/overview).
**Use Cases:**
* Testing prompt changes against a golden dataset (regression testing).
* Comparing different prompt versions (A/B testing) on the same inputs.
* Evaluating performance on specific edge cases or scenarios defined in the dataset.
* Generating scores for metrics that require ground truth (e.g., Exact Match, Semantic Similarity).
**How to Run:**
1. Ensure you have a [Dataset](/guides/datasets/overview) prepared with relevant inputs (and `expected_output` columns if needed by your evaluation metrics).
2. Navigate to the specific Evaluation you want to run (within your prompt's "Evaluations" tab).
3. Click the "Run experiment" button.
![Open experiment modal]()
4. Define the experiment variants
5. Select the Dataset you want to run the experiment against.
![Run experiment]()
6. You will be redirected to the experiments tab with the results
![Evaluation experiment result]()
## Running Evaluations Continuously (Live Mode)
Live evaluations automatically run on *new* logs as they are generated by your prompt in production (via API calls or the Playground). This provides continuous monitoring of prompt quality.
![Live Evaluation]()
**Use Cases:**
* Real-time monitoring of key quality metrics (e.g., validity, safety, basic helpfulness).
* Quickly detecting performance regressions caused by model updates or unexpected inputs.
* Tracking overall prompt performance trends over time.
**How to Enable:**
1. Navigate to the specific Evaluation you want to run live.
2. Go to its settings.
3. Toggle the "Live Evaluation" option ON.
4. Save the settings.
Evaluations requiring an `expected_output` (like Exact Match, Lexical Overlap,
Semantic or Numeric Similarity...), [Manual
Evaluations](/guides/evaluations/humans-in-the-loop) or [Composite
Evaluations](/guides/evaluations/composite-scores) **cannot** run in live
mode, as they might need pre-existing ground truth or human input.
## Viewing Evaluation Results
Whether run experiments or live mode, results are accessible:
* **Logs View**: Individual logs show scores/results from all applicable evaluations that have run on them.
![Log with Evaluation Results]()
* **Evaluations Tab (Per Prompt)**: View aggregated statistics, score distributions, success rates, and time-series trends for each specific evaluation.
![Evaluation Dashboard]()
* **Experiments**: When you run evaluations as experiments, you can view detailed results, compare different variants
These results provide the data needed to understand performance, identify issues, and drive improvements using [Prompt Suggestions](/guides/evaluations/prompt-suggestions).
## Next Steps
* Learn how to prepare data using [Datasets](/guides/datasets/overview)
* Understand how evaluation results power [Prompt Suggestions](/guides/evaluations/prompt-suggestions)
* Explore the different [Evaluation Types](/guides/evaluations/overview)
# Overview
Source: https://docs-v1.latitude.so/guides/experiments/overview
Learn about the key concepts of Latitude experiments
## What are Experiments?
**Experiments** in Latitude are a feature that let you systematically test, evaluate, and compare different prompt configurations, model versions, and parameters (like temperature) across a dataset. This enables you to find out which prompts and models work best for your use case, using real, measurable results.
***
## How Experiments Work
* **Run Location:**
You can run experiments directly from the Prompt Playground
![Run experiment in playground]()
or from a Latitude Evaluation
![Open experiment modal]()
* **Experiments Tab:**
Each prompt in Latitude has an Experiments tab, where you can compare results from different experiments side-by-side.
***
## Experiment Components
* **Prompt Variants:**
Test different prompt wordings, instructions, or templates.
* **Model Versions:**
Compare outputs from different models (e.g., `gpt-4.1`, `gpt-4.1-mini`).
* **Parameters:**
Adjust settings like temperature to influence model behavior.
* **Evaluations:**
Attach evaluation metrics (e.g., accuracy, sentiment analysis) to automatically assess experiment outputs.
***
## Running an Experiment
1. **Define Variants:**
Choose your prompt(s), model, and settings.
2. **Pick Evaluations:**
Select which evaluation metrics to run (optional).
3. **Select Dataset:**
Pick or generate a dataset to use for testing.
![Running an experiment]()
Click **Run Experiment** to execute, and Latitude will process each combination and display the results.
***
## Comparing Experiments
* Use the **Experiments** tab to select and compare multiple experiment runs.
* Review metrics like accuracy, cost, duration, and token usage.
* See detailed results, including logs and evaluation scores, for each experiment.
![Experiments tab comparison]()
***
## Benefits
* **Objective Comparison:**
Quickly see which prompts and models perform best on your tasks.
* **Visual Analysis:**
Side-by-side results make differences easy to spot.
* **Cost Tracking:**
Monitor token and cost usage for each variant.
# Introduction
Source: https://docs-v1.latitude.so/guides/getting-started/introduction
Welcome to Latitude - The AI Prompt Engineering Platform
Latitude is an open-source platform for AI prompt engineering, deployment, and evaluation. It helps teams ship reliable AI features by closing the loop between prompt development, production traces, and continuous improvement.
With Latitude, you can:
* Design and version prompts collaboratively
* Test iterations in an interactive playground
* Evaluate and improve prompts systematically
* Deploy prompts as API endpoints
* Monitor performance with automatic tracing tied to real usage
Latitude is designed for cross-functional teams, enabling collaboration between developers, product managers, and domain experts throughout the entire AI development process.
## Key Features at a Glance
* **Prompt Manager**: Create, version, and collaborate on prompts with a powerful editor supporting advanced features like variables, conditionals, and loops through PromptL
* **Playground**: Test prompts interactively with different inputs, parameters, and tool configurations
* **AI Gateway**: Deploy prompts as API endpoints that stay up-to-date with published changes
* **Datasets**: Manage test data for batch evaluations and regression testing
* **Evaluations**: Assess prompt performance via LLM-as-judge, programmatic rules, or human review
* **Run experiments**: Use datasets and associtated evaluations to run batch experiments over 2 or more prompts variations.
* **Traces & Observability**: Automatically capture all interactions with prompts and models
* **Integrations**: Seamlessly integrate with your existing stack via SDKs and APIs
## The Reliability Loop
Latitude is built around a reliability loop that turns real usage into improvements:
1. **Design**: Create and iterate on prompts using the Prompt Manager
2. **Test**: Validate behavior in the Playground with test inputs
3. **Deploy**: Publish prompts as endpoints through the AI Gateway or render them locally with the SDKs
4. **Trace**: Automatically capture production interactions with telemetry
5. **Evaluate**: Assess performance using various evaluation methods
6. **Improve**: Refine prompts and repeat the loop as you learn from traces
## Ready to Get Started?
Choose the guide that matches how you want to onboard:
* [Developers getting started](/guides/getting-started/quick-start-dev)
* [No-code getting started](/guides/getting-started/quick-start-pm)
## Join the Community
Have questions or feedback? Join [our community on Slack](https://join.slack.com/t/trylatitude/shared_invite/zt-35wu2h9es-N419qlptPMhyOeIpj3vjzw) to connect with other Latitude users and our team.
# Developers
Source: https://docs-v1.latitude.so/guides/getting-started/quick-start-dev
Connect Latitude Telemetry to your existing AI stack
Choose the provider/framework your application already uses, or use OpenTelemetry OTLP ingest if you already have OTEL in your stack.
## Supported integrations
### Popular integrations
}
/>
}
/>
### More integrations
}
/>
}
/>
GR
}
/>
MI
}
/>
OL
}
/>
LLM
}
/>
RE
}
/>
TR
}
/>
AA
}
/>
WX
}
/>
}
/>
HY
}
/>
DS
}
/>
CR
}
/>
OTEL
}
/>
#### OpenTelemetry (OTLP ingest)
If you already use OpenTelemetry, you can export OTLP traces directly to Latitude.
* **URL (Latitude Cloud):** `https://gateway.latitude.so/api/v3/traces`
* **Auth:** `Authorization: Bearer YOUR_API_KEY`
* **Formats:** OTLP Protobuf (`application/x-protobuf`) or OTLP JSON (`application/json`)
Example (OpenTelemetry Collector):
```yaml theme={null}
receivers:
otlp:
protocols:
grpc:
http:
exporters:
otlp_http/latitude:
traces_endpoint: https://gateway.latitude.so/api/v3/traces
headers:
Authorization: Bearer ${LATITUDE_API_KEY}
service:
pipelines:
traces:
receivers: [otlp]
exporters: [otlp_http/latitude]
```
If you are self-hosting, replace the hostname with your Gateway base URL.
# No Code
Source: https://docs-v1.latitude.so/guides/getting-started/quick-start-pm
Get started with Latitude without writing code.
This guide will walk you through creating and testing your first prompt in Latitude.
## 1. Add a Provider
1. Navigate to [Settings](https://app.latitude.so/settings) > "Providers"
2. Click "Create Provider"
3. Select your provider (e.g., OpenAI, Anthropic, Google, etc.)
4. Enter your API key and any required configuration
5. Click "Save"
## 2. Create a Project
1. Go to the [dashboard](https://app.latitude.so), click "New Project"
2. Enter a name for your project
3. Click "Create Project"
## 3. Create Your First Prompt
1. Notice the 3 icons in the sidebar to create a new folder, prompt and agent respectively
2. Click the second icon, a new prompt input will appear in the sidebar
3. Enter a name for your prompt and hit "Enter" key
![Create Prompt]()
You will get redirected to the prompt editor. Write the following prompt in the editor:
```
---
provider: Latitude
model: gpt-4.1-mini
---
This is a response from an NPS survey:
Score out of 10: {{ score }}
Message: {{ message }}
Analyze the sentiment based on both the score and message. Prioritize identifying the primary concern in the feedback, focusing on the core issue mentioned by the user. Categorize the sentiment into one of the following categories:
- Product Features and Functionality
- User Interface (UI) and User Experience (UX)
- Performance and Reliability
- Customer Support and Service
- Onboarding and Learning Curve
- Pricing and Value Perception
- Integrations and Compatibility
- Scalability and Customization
- Feature Requests and Product Roadmap
- Competitor Comparison
- General Feedback (Neutral/Non-specific)
Return only one of the categories.
```
The prompt is automatically saved as you write it.
Notice how every prompt run has a human-in-the-loop evaluation automatically
generated for it
## 4. Test in the Playground
1. In the prompt editor, notice the parameter inputs in the right side
2. Fill in the parameter values:
* `score`: "5"
* `message`: "Product is working but occasionally laggy."
3. Click "Run" in the bottom right corner to see the generated output
4. Try different inputs to test how your prompt performs in various scenarios
![Testing in Playground]()
## 5. Observe Results and Logs
1. Navigate to the "Logs" section
2. You'll see a record of all interactions with your prompt
3. Click on any log entry to view details including:
* Input parameters
* Generated output
* Model used
* Response time
* Evaluation results (if available)
4. Use filters to find specific logs based on time, status, or content
5. Select logs and save them to a dataset
![Viewing Logs]()
## 6. Create an Evaluation
1. In the prompt editor, go to the "Evaluations" tab
2. Click "Add Evaluation"
3. Select "LLM as Judge" as the evaluation type
4. Choose a title, description, critera, pass-fail conditions and select a result type: number, boolean, or text
![Create Evaluation]()
5. Click "Create evaluation"
6. In the evaluation page, click "Run Experiment"
7. Select the dataset we recently created from logs, map prompt parameters to the dataset columns, and click "Run Experiment"
![Run Evaluation]()
8. Watch experiment results in real-time
## 7. Publish and Deploy the Prompt
1. In the prompt editor, click "Publish" button in the top sidebar
2. Add a version note describing the prompt (e.g., "Initial version of product description generator")
3. Click "Publish Version"
![Publish Version]()
4. Your prompt is now available as an API endpoint through the AI Gateway
5. Click "Deploy this prompt" button in the top header
6. Copy your preferred integration method (SDK or HTTP API)
## Next Steps
Now that you've created and evaluated your first prompt at scale, you can:
* Share your endpoint with developers for integration
* Create more complex prompts using [PromptL syntax](/promptl/syntax/structure)
* Set up [ongoing evaluations](/guides/evaluations/running-evaluations) to monitor quality
* [Invite team members](/guides/getting-started/invite-your-team) to collaborate
# Community and Support
Source: https://docs-v1.latitude.so/guides/integration/community-support
Get help, share feedback, and connect with the Latitude community.
We're excited to have you using Latitude! Whether you have questions, need help, want to share feedback, or connect with other users, here are the best ways to get involved:
## Slack Community
* **Join our Slack**: [https://join.slack.com/t/trylatitude/shared\_invite/zt-35wu2h9es-N419qlptPMhyOeIpj3vjzw](https://join.slack.com/t/trylatitude/shared_invite/zt-35wu2h9es-N419qlptPMhyOeIpj3vjzw)
* **Best for**: Quick questions, general discussion, sharing what you're building, getting help from the community and the Latitude team.
* **Channels**: Look for channels like `#general`, `#help`, `#prompt-engineering`, `#feature-requests`.
## GitHub Repository
* **Latitude LLM Repo**: [https://github.com/latitude-dev/latitude-llm](https://github.com/latitude-dev/latitude-llm)
* **Best for**: Reporting bugs, requesting specific features, contributing code or documentation, detailed technical discussions.
* **Issues**: Please search existing issues before creating a new one.
* **Discussions**: Use GitHub Discussions for broader questions or ideas.
## Documentation
* **You are here!**: Explore these docs for guides, tutorials, and API references.
* **Contribution**: Found an error or want to improve the docs? Feel free to open an issue or pull request on the GitHub repository!
## Social Media
* **Twitter/X**: Follow [@trylatitude](https://twitter.com/trylatitude) for announcements and updates.
* **LinkedIn**: Connect with [Latitude](https://www.linkedin.com/company/trylatitude/) for company news.
## Reporting Security Vulnerabilities
If you believe you have found a security vulnerability, please **do not** report it via public GitHub issues. Instead, follow the security reporting guidelines outlined in the [SECURITY.md](https://github.com/latitude-dev/latitude-llm/blob/main/SECURITY.md) file in our GitHub repository (or contact us directly through a secure channel if specified).
We value our community and look forward to hearing from you!
# Agents
Source: https://docs-v1.latitude.so/guides/prompt-manager/agents
Create autonomous agents that can use tools, reason, and complete complex tasks over multiple steps.
Latitude Agents are an advanced prompt type that enables AI models to operate autonomously, breaking down complex tasks, using tools, and reasoning through multiple steps until a final goal is achieved.
Unlike simple prompts or even [Chains](/promptl/advanced/chains) which follow predefined steps, Agents can dynamically decide their next action based on the context and available tools.
When you finish reading this page you can read [our take on Anthropic's building agents article](/examples/cases/building-effective-agents)
## Defining an Agent
To turn a prompt into an Agent, simply add `type: agent` to its configuration block:
```markdown {4} theme={null}
---
provider: openai
model: gpt-4o
type: agent
---
Plan and execute the steps needed to research and write
a short blog post about the benefits of using Latitude Agents.
```
## How the Agent Loop Works
When an Agent prompt is run:
1. **Goal Understanding**: The agent analyzes the initial prompt to understand the overall task or goal.
2. **Planning (Implicit)**: It internally plans the first step needed to move towards the goal.
3. **Action**: It decides whether to:
a. **Call a Tool**: If it needs external information or functionality, it requests a tool call.
b. **Generate Response**: If it has enough information or needs to ask a clarifying question, it generates text.
4. **Observation**: If a tool was called, it receives the tool's response.
5. **Reasoning**: Based on the goal, previous steps, and new observations (tool responses), it reasons about the next action needed.
6. **Repeat**: Steps 3-5 repeat until the agent determines the original goal is fully accomplished.
7. **Final Answer**: The agent provides the final result.
This loop allows the agent to adapt its strategy, handle errors, and utilize tools effectively.
## Using Tools Within Agents
Agents become truly powerful when combined with [Tools](/guides/prompt-manager/tools). Provide the agent with a set of relevant tools, and it will decide which ones to use and when.
```markdown theme={null}
---
provider: openai
model: gpt-4o
type: agent
tools:
- latitude/search
- get_weather
- get_location_id
# Tool definitions...
- get_weather:
# ... definition
- get_location_id:
# ... definition
---
Find the current weather for {{ location_name }}.
```
In this example, the agent might:
1. Realize it needs a location ID for `get_weather`.
2. Call `get_location_id` with `location_name`.
3. Receive the `location_id`.
4. Call `get_weather` with the obtained `location_id`.
5. Receive the weather data.
6. Formulate and return the final answer to the user.
## Defining the Final Output (Schema)
You can guide the agent's final output by specifying a response `schema` using [JSON Schema](/guides/prompt-manager/json-output):
```markdown theme={null}
---
provider: openai
model: gpt-4o
type: agent
tools:
- latitude/search
schema:
type: object
properties:
summary:
type: string
description: A concise summary of the findings.
key_points:
type: array
items:
type: string
description: A list of key bullet points.
required: [summary, key_points]
---
Research the main features of the Vercel AI SDK and provide a summary and key bullet points.
```
The agent will work towards fulfilling the task and then structure its final response according to the schema.
## Predefined Steps
While agents operate autonomously, you can provide initial instructions or force specific actions using `` tags. The agent will execute these predefined steps first before entering its autonomous loop.
```markdown theme={null}
---
# ... agent config ...
---
First, search for recent news about AI advancements.
Then, identify the top 3 trends mentioned.
Now, write a short analysis comparing these trends.
```
## Limiting Agent Iterations
To prevent agents from running indefinitely (e.g., getting stuck in loops), Latitude automatically applies a `maxSteps` limit of 20 to all prompts with configuration. This controls the maximum number of steps (tool calls + LLM responses) an agent can execute.
You can customize this limit by explicitly setting the `maxSteps` configuration option (max: 150):
```yaml theme={null}
---
# ... agent config ...
maxSteps: 10 # Limit the agent to 10 steps
---
```
If the limit is reached before the goal is completed, the agent run will terminate with an error.
[Learn more about maxSteps configuration](/guides/prompt-manager/configuration#maxsteps).
## Running Agents
Agents are run just like any other prompt using the [API](/guides/api/api-access) or [SDKs](/guides/sdk/typescript). The response will typically be a stream of events detailing the agent's thought process, tool calls, and final answer.
## Subagents
You can make any prompt have access to other agents in your project by using the `agents` configuration option. This allows you to create a hierarchy of agents, where tasks are delegated to subagents with specific responsibilities.
```yaml {4-5} theme={null}
---
provider: openai
model: gpt-4o
agents:
- path/to/another-agent
---
```
The main prompt will have access to running the subagent prompt as if it was a tool. This allows you to structure complex tasks into smaller, manageable sub-tasks performed by different agents.
## Next Steps
* [Our take on Anthropic's building agents article](/examples/cases/building-effective-agents)
* Explore [Latitude Tools](/guides/prompt-manager/latitude-tools) that agents can use.
* Test your agents in the [Playground](/guides/prompt-manager/playground).
* Learn about [Evaluations](/guides/evaluations/overview) to assess agent performance.
# Configuration
Source: https://docs-v1.latitude.so/guides/prompt-manager/configuration
Learn how to configure model, provider, and generation parameters for your prompts.
The configuration section, located at the top of every prompt file and enclosed by triple dashes (`---`), defines how Latitude executes your prompt. It's written in YAML format and allows you to specify the AI provider, model, generation parameters, and other advanced settings.
![Prompt configuration section]()
```yaml theme={null}
---
provider: OpenAI
model: gpt-4o-mini
temperature: 0.6
top_p: 0.9
# ... other settings
---
```
Be sure there is a space after the colon ( : ) in your configurations.
## Core Settings
### Provider (required)
Specifies the AI provider to use (e.g., OpenAI, Anthropic, Google). This must match a provider configured in your Latitude workspace settings.
You can easily select a configured provider using the dropdown in the editor's header.
![Selecting a provider]()
[Learn more about configuring providers](/guides/getting-started/providers).
### Model (required)
Specifies the exact language model to use (e.g., `gpt-4o-mini`, `claude-3-opus-20240229`). Available models depend on the selected provider.
The model dropdown in the editor's header lists available models for the chosen provider.
![Selecting a model]()
## Generation Parameters
These parameters control how the AI model generates its response. The specific ranges and behaviors might vary slightly between providers.
### `temperature`
Controls the randomness of the output. Lower values (e.g., 0.1) make the output more deterministic and focused, while higher values (e.g., 0.9) increase creativity and randomness.
Setting temperature to 0, or leaving it unset, enables response
[caching](/guides/prompt-manager/cache) for identical inputs.
It's generally recommended to adjust *either* `temperature` *or* `topP`, but not both.
### `maxTokens`
Sets the maximum number of tokens (words or parts of words) the model can generate in its response.
### `topP` (Nucleus Sampling)
An alternative to `temperature` for controlling randomness. It instructs the model to consider only the most probable tokens whose cumulative probability mass exceeds the `topP` value (e.g., 0.9 means consider tokens comprising the top 90% probability mass).
### `topK`
Restricts the model to sampling only from the `K` most likely next tokens at each step. Generally used for advanced cases; `temperature` or `topP` are usually sufficient.
### `presencePenalty`
Discourages the model from repeating information already present in the prompt context. Higher values increase the penalty.
### `frequencyPenalty`
Discourages the model from using the same words or phrases repeatedly in its response. Higher values increase the penalty.
### `stopSequences`
A list of strings that, if generated by the model, will cause generation to stop immediately.
### `seed`
An integer used to initialize the random number generator. If supported by the model, using the same seed with identical inputs will produce deterministic outputs, useful for reproducibility.
## Advanced Configuration
### `parameters`
Defines types and constraints for input parameters used in the [Playground](/guides/prompt-manager/playground) or shared prompts.
```yaml theme={null}
parameters:
user_input:
type: text # Default type
image_upload:
type: image
data_file:
type: file
```
[Learn more about parameter types in the Playground guide](/guides/prompt-manager/playground#parameter-types).
### `schema`
Defines a JSON schema to enforce structured output from the model.
[Learn more about enforcing JSON output](/guides/prompt-manager/json-output).
### `tools`
Lists the tools (functions) available for the AI model to call.
[Learn more about configuring tools](/guides/prompt-manager/tools).
### `maxSteps`
Sets the maximum number of execution steps allowed for your prompt (default: 20, max: 150). This prevents infinite loops and controls resource usage in [Chains](/promptl/advanced/chains) and [Agents](/guides/prompt-manager/agents).
**Automatic Application**: By default, Latitude automatically applies `maxSteps: 20` to all documents with configuration. This ensures safe execution limits for tool usage, chains, and multi-step operations.
**Opt-out Behavior**: A document is treated as a simple prompt (without `maxSteps` applied) in two cases:
1. **No configuration section** - Documents without frontmatter (no `---` delimiters)
2. **Explicit type declaration** - Setting `type: prompt` in the configuration
```yaml theme={null}
---
provider: OpenAI
model: gpt-4o-mini
type: prompt # Explicitly opt out of maxSteps behavior
---
```
You can also override the default by explicitly setting your own `maxSteps` value:
```yaml theme={null}
---
provider: OpenAI
model: gpt-4o-mini
maxSteps: 50 # Custom limit
---
```
### `maxRetries`
Maximum number of times to retry a provider call on failure (default: 2).
### `headers`
Sends additional HTTP headers with the request, useful for integrating with specific provider features or observability tools.
## Next Steps
* Test prompts with different settings in the [Playground](/guides/prompt-manager/playground)
* Learn about [Enforcing JSON Output](/guides/prompt-manager/json-output)
* Explore using [Tools](/guides/prompt-manager/tools) and [Agents](/guides/prompt-manager/agents)
# Structured Output
Source: https://docs-v1.latitude.so/guides/prompt-manager/json-output
Define JSON schemas to ensure structured and validated responses from AI models.
Latitude allows you to enforce structured JSON output from AI models by defining a JSON schema directly in the prompt's configuration (frontmatter). This ensures responses are consistent, automatically validated, and easy to integrate into your applications.
## Specifying the Schema
Use the `schema` key within the prompt's configuration block (`---`). Define the expected JSON structure using standard [JSON Schema](https://json-schema.org/) syntax.
```yaml {4-21} theme={null}
---
provider: openai
model: gpt-4o
schema:
type: object
properties:
sentiment:
type: string
description: The sentiment of the input text.
enum: [positive, negative, neutral]
confidence:
type: number
description: The confidence score (0.0 to 1.0).
minimum: 0
maximum: 1
explanation:
type: string
description: A brief explanation for the sentiment classification.
required:
- sentiment
- confidence
---
Analyze the sentiment of the following text and provide your analysis in JSON format according to the defined schema.
{{ user_text }}
```
In this example:
* We expect a JSON object (`type: object`).
* It must have `properties`: `sentiment`, `confidence`, and `explanation`.
* `sentiment` must be one of the strings in the `enum` list.
* `confidence` must be a `number` between 0 and 1.
* `explanation` is an optional `string`.
* `sentiment` and `confidence` are mandatory (`required` list).
## How it Works
When a `schema` is defined:
1. **Provider Integration**: Latitude communicates the schema to the AI provider (if the provider supports JSON mode or function calling with schemas, like recent OpenAI, Anthropic, and Google models).
2. **Model Guidance**: The model is instructed to generate output strictly adhering to the provided schema.
## Benefits
* **Reliability**: Guarantees consistent data structures from the AI.
* **Ease of Use**: Simplifies parsing and using AI responses in downstream code.
* **Reduced Errors**: Catches formatting issues automatically.
* **Clear Intent**: Explicitly tells the model the desired output format.
## Chains and Agents
When working with chains and agents, there's some things to keep in mind:
### Chains
When using chains, the configuration added to the general configuration section in the prompt will be applied to all steps in the chain. This means that if you have a schema defined in the general configuration, it will be applied to all steps, which may not be the expected behavior. To avoid this, you can define the schema in the specific step configuration instead.
```{5-7,9-27} theme={null}
---
provider: openai
model: gpt-4o
---
This step does not have a schema defined.
This step has a schema defined.
```
### Agents
When creating agents, the only schema to have in mind is the schema of the Agent response, since any intermediate steps are used as Chain of Thought, reasoning steps, or tool calling. This is why, even if an Agent has a `schema` property defined in the configuration, it will only be applied to the final response of the Agent, and not any intermediate steps.
## Next Steps
* Configure other [Prompt Settings](/guides/prompt-manager/configuration)
* Learn about using [Tools](/guides/prompt-manager/tools) for more complex interactions.
* Test your JSON-output prompts in the [Playground](/guides/prompt-manager/playground)
# Editor
Source: https://docs-v1.latitude.so/guides/prompt-manager/overview
Learn the basics of writing and managing prompts in the Latitude editor.
The Latitude Prompt Editor is your central hub for designing, testing, and managing AI prompts. It provides a powerful interface built around **PromptL**, our specialized language for creating dynamic and structured prompts.
![Prompt Editor]()
## Writing a Prompt using PromptL
At its core, the editor lets you write prompts using [PromptL syntax](/promptl/syntax/structure). This includes:
* **Configuration Block**: Define provider, model, and settings in the YAML frontmatter.
* **Messages**: Structure conversations using ``, ``, and `` tags.
* **Advanced Features**: Utilize loops, conditionals, snippets, and more for complex logic.
```markdown theme={null}
---
provider: openai
model: gpt-4o
temperature: 0.7
---
You are a helpful assistant.
Tell me a joke about {{ topic }}.
```
Explore the full power of PromptL in the [PromptL Syntax documentation](/promptl/syntax/structure).
## Adding Parameters and Variables
Make your prompts dynamic by using variables and parameters:
Define placeholders like `{{ topic }}` in your prompt. You can use, read, and write data from these variables all over your prompt.
All variables defined in the prompt without a value are automatically added as **Input Parameters**, which can be filled in via the
Playground or API calls.
Learn more about [Variables in PromptL](/promptl/syntax/variables).
Input parameters will automatically appear in the [Playground](/guides/prompt-manager/playground) for easy testing.
## Collaborating in the Editor
Latitude is built for teamwork:
* **Version Control**: Manage changes using drafts and published versions. See the [Version Control guide](/guides/prompt-manager/version-control) for details.
* **Shared Workspace**: Team members can view and edit prompts within the same project.
* **Comments**: Add `/* comments */` directly within the editor for feedback and discussion.
## Next Steps
* Learn how to [Configure Prompt Settings](/guides/prompt-manager/configuration)
* Test your prompts in the [Playground](/guides/prompt-manager/playground)
* Manage changes with [Version Control](/guides/prompt-manager/version-control)
# Playground
Source: https://docs-v1.latitude.so/guides/prompt-manager/playground
Learn how to test and refine your prompts interactively in the Playground.
The Prompt Playground is your interactive sandbox for testing, debugging, and refining prompts before deploying them. It lets you run prompts with different inputs, see the model's responses in real-time, and even test how your prompt interacts with tools.
![Prompt Playground]()
## Running Single Inputs
1. **Preview**: The main panel shows a preview of the messages that will be sent to the model, based on your prompt template and current parameter values.
![Prompt Preview]()
2. **Parameters**: If your prompt uses input parameters (like `{{ topic }}`), they appear in the "Parameters" section. Fill in values here.
![Parameters]()
3. **Run**: Click "Run prompt". Latitude sends the request to the configured provider and model.
4. **Chat Mode**: The response appears, and the Playground enters Chat mode. You can continue the conversation turn by turn.
![Chat Mode]()
5. **Reset**: Click "New Chat" to clear the conversation and run the prompt again from the beginning, potentially with new parameter values.
## Parameter Input Methods
You can populate parameters in several ways:
* **Manual**: Type values directly into the fields.
![Manual Parameters]()
* **Dataset**: Load inputs from a [Dataset](/guides/datasets/overview). Each row becomes a separate test case. This is great for **batch testing**.
![Dataset Parameters]()
* **History**: Reuse parameter values from previous runs.
![History Parameters]()
### Parameter Types
Parameters can accept different input types, configured either in the prompt's [settings](/guides/prompt-manager/configuration#parameters) or directly in the Playground:
![Parameter Types]()
* **Text**: Standard text input (default).
![Text Parameter]()
Advanced Users: Lists are also acceptable inputs for this field. Specify a list with the following format: \[a1, a2, etc..]
* **Image**: Upload an image file. Passed to the model as content (requires model support like GPT-4V, Claude 3). Use `` tag in your prompt.
![Image Parameter]()
* **File**: Upload any file type. Passed as content (requires model support). Use `` tag.
![File Parameter]()
## Testing Tool Responses
If your prompt uses [Tools](/guides/prompt-manager/tools), the Playground allows you to simulate their responses:
1. **Run the prompt**: Initiate the prompt run as usual.
2. **Tool Call Request**: If the model decides to call a tool, the Playground will pause and display the requested tool call and its arguments.
![Tool Call Request]()
3. **Mock Response**: Enter the JSON response you want the tool to *pretend* to return.
![Mock Tool Response]()
4. **Continue**: Click "Send tool response". Latitude sends the mocked tool response back to the model, which then continues its generation process based on that simulated information.
This allows you to test the logic of your prompt's interaction with tools without needing to execute the actual tool functions.
## Viewing Logs in the Playground
Every run in the Playground generates a log entry. You can quickly access the detailed log for the current run:
1. Click the "Logs" icon or link within the Playground interface (location may vary slightly).
2. This opens the detailed log view, showing inputs, outputs, metadata, timings, and any evaluation results associated with that specific run.
This provides immediate feedback and traceability for debugging.
## Next Steps
* Learn about [Prompt Configuration](/guides/prompt-manager/configuration)
* Manage changes using [Version Control](/guides/prompt-manager/version-control)
* Explore how to use [Tools](/guides/prompt-manager/tools) in your prompts
# Best Practices
Source: https://docs-v1.latitude.so/guides/prompt-manager/prompt-best-practices
Tips and examples for writing effective prompts in Latitude.
Writing effective prompts is key to getting the best results from AI models. Here are some best practices and examples specific to working within the Latitude platform.
## General Best Practices
1. **Be Specific and Clear**: Avoid ambiguity. Clearly state the task, desired format, context, and constraints.
* **Bad**: "Write about Latitude."
* **Good**: "Write a 3-paragraph introduction to Latitude for a non-technical audience, highlighting its key benefits for prompt management and evaluation."
2. **Provide Context**: Give the model relevant background information it might need.
* **Example**: If asking for a summary of a meeting, provide the meeting transcript or key discussion points.
3. **Define the Persona/Role**: Tell the model *who* it should be.
* **Example**: `You are a helpful and friendly customer support agent for a SaaS company.`
* **Note**: It can be helpful to use made up tags to organize information for the model.
* **Example:** ` You must never produce more than 5 lines. `
4. **Specify the Output Format**: Use instructions or [JSON Schema](/guides/prompt-manager/json-output) to guide the format.
* **Example**: "Provide the answer as a JSON object with keys 'pros' and 'cons', each containing a list of strings."
* **Example**: Use the `schema` configuration for reliable JSON.
5. **Use Examples (Few-Shot Prompting)**: Provide examples of desired input/output pairs within the prompt.
```markdown theme={null}
Text: "This is great!"
Sentiment: Positive
Okay.
Text: "I am not happy."
Sentiment: Negative
Okay.
Text: "{{ user_input }}"
Sentiment:
```
6. **Iterate and Test**: Use the [Playground](/guides/prompt-manager/playground) extensively. Start simple and gradually add complexity. Test with various inputs.
7. **Break Down Complex Tasks**: Use [Chains](/promptl/advanced/chains) or [Agents](/guides/prompt-manager/agents) for multi-step processes rather than trying to do everything in one giant prompt.
## Latitude-Specific Tips
* **Leverage PromptL**: Use variables (`{{ }}`), conditionals (`{{ if … }}`), loops (`{{ for … }}`), and snippets (``) for dynamic and reusable prompts. See the [PromptL documentation](/promptl/getting-started/introduction).
* **Use Configuration Wisely**: Tune `temperature`, `maxTokens`, etc., in the [configuration block](/guides/prompt-manager/configuration) for desired output style and length.
* **Utilize Tools**: Don't make the model guess information it can look up. Provide [Tools](/guides/prompt-manager/tools) for accessing external data or functions (e.g., `latitude/search` for web searches).
* **Employ Agents for Autonomy**: For tasks requiring planning and dynamic tool use, define your prompt as an [Agent](/guides/prompt-manager/agents).
* **Manage Versions**: Use [Version Control](/guides/prompt-manager/version-control) to track changes and collaborate safely.
* **Evaluate Systematically**: Use [Evaluations](/guides/evaluations/overview) to measure prompt quality and identify areas for improvement.
## Example: Customer Support Email Generator
```markdown theme={null}
---
provider: anthropic
model: claude-3-haiku-20240307
temperature: 0.5
schema:
type: object
properties:
subject:
type: string
description: A concise and relevant email subject line.
body:
type: string
description: The full email body, formatted professionally.
required: [subject, body]
tools:
- get_customer_details:
description: Retrieves customer details based on email address.
parameters:
type: object
properties:
email:
type: string
description: The customer's email address.
required: [email]
---
You are a helpful customer support agent. Your task is to draft a polite and helpful email response to a customer query.
Use the provided tools if you need more customer information. Address the customer by name if available.
Keep the tone professional and empathetic.
Structure the response clearly.
Ensure the final output matches the required JSON schema.
Customer Email: {{ customer_email }}
Query: {{ customer_query }}
{# Agent might call get_customer_details here if name isn't obvious #}
{# Then it will generate the JSON output for subject and body #}
```
This example demonstrates:
* Role setting (``).
* Using variables (`{{ customer_email }}`, `{{ customer_query }}`).
* Defining and enabling a custom tool (`get_customer_details`).
* Enforcing structured output (`schema`).
* Clear instructions within the system message.
# Subagents
Source: https://docs-v1.latitude.so/guides/prompt-manager/subagents
Create subagents to compartmentalize tasks and create more complex agents
# Sub-Agents & Agentic Structures in Latitude
![]()
Latitude’s agentic framework lets you build powerful, modular, and autonomous LLM workflows. This page covers how to design, implement, and orchestrate **sub-agents**, specialized agents that can be invoked by other agents, enabling complex, multi-step reasoning and tool use.
***
## What Are Agents and Sub-Agents?
**Agent**: A `PromptL` prompt with `type: agent` that can plan, act, and iterate autonomously, calling tools or other agents as needed.
**Sub-Agent**: Any agent that is exposed as a callable function/tool within another agent, allowing for modular, reusable, and composable workflows.
***
## When to Use Agents vs. Step Chains
| Use an **Agent** when... | Use a **Chain** when... |
| ------------------------------------------------ | ----------------------------------- |
| The workflow is open-ended or branching | The workflow is strictly sequential |
| The model must decide which tools/agents to call | The steps are always the same |
| You want dynamic planning or iteration | You want deterministic, fixed steps |
If you find yourself writing lots of conditional logic in a chain, consider switching to an agentic approach.
***
## Defining an Agent in PromptL
To turn any prompt into an agent, add the following to your configuration header:
```yaml theme={null}
---
type: agent
provider: openai
model: gpt-4o
tools:
- latitude/search
maxSteps: 40
---
```
* `type: agent` enables agentic mode.
* `tools:` lists external tools the agent can call.
* `maxSteps:` (optional) limits the number of agent cycles.
***
## Exposing Sub-Agents
You can expose other PromptL agent files as callable sub-agents using the `agents:` configuration key:
```yaml theme={null}
---
type: agent
agents:
- agents/summarizer
- agents/sentiment_analyzer
- agents/researcher
---
```
Each listed agent becomes available as a callable function/tool.
Sub-agents can themselves call tools or other sub-agents, enabling **deep composition**.
***
## Sub-Agent Design Patterns
### 1. Single-Responsibility Helpers
Keep sub-agents focused. Example: a summarizer agent that only summarizes text.
```yaml theme={null}
---
type: agent
schema:
type: object
properties:
summary: { type: string }
required: [summary]
---
```
```promptl theme={null}
You are a summarizer. Return a concise summary of the provided text.
{{ input_text }}
```
It is essential that you include parameters in subagents so that the main agent can send them information.
***
### 2. Specialist Pool
A generalist agent can delegate to a pool of specialists:
```yaml theme={null}
---
type: agent
agents:
- agents/summarizer
- agents/sentiment_analyzer
- agents/fact_checker
---
```
The agent can decide which specialist to call based on the task.
***
### 3. Sequential Orchestration
For strict order, use `` blocks and specify which agent to call:
```promptl theme={null}
Parse the raw email and extract fields.
Enrich team data via LinkedIn search.
Produce a recommendation.
```
***
## Agent Loop & Execution
On each cycle, the agent can return:
* **Tool calls only**: Latitude executes the tools, appends results, and continues.
* **Text + tool calls**: Treated as internal thinking; tools are run.
* **Text only**: The loop ends; this is the agent’s final answer.
The loop stops when a text-only response is returned or `maxSteps` is reached.
***
## Best Practices
* **Single Responsibility**: Each sub-agent should do one thing well.
* **Clear I/O**: Use `schema` to define expected outputs for each agent.
* **Resource Awareness**: Each sub-agent call counts toward the parent’s `maxSteps`.
* **Testing**: Use the Playground to debug and trace agent/sub-agent interactions.
***
## Example: Multi-Agent Researcher
### Main Agent Configuration
```yaml theme={null}
---
type: agent
agents:
- agents/web_search
- agents/summarizer
- agents/citation_checker
schema:
type: object
properties:
report: { type: string }
---
```
### Main Agent Prompt
```promptl theme={null}
You are a research assistant. Use your sub-agents to gather, summarize, and fact-check information before producing a final report.
Research the latest trends in renewable energy.
```
### Sub-Agents
* `agents/web_search`: Searches the web for relevant articles.
* `agents/summarizer`: Summarizes article content.
* `agents/citation_checker`: Verifies the credibility of sources.
***
## Debugging & Tracing
* Use the **Latitude Playground** to step through agent execution.
* Inspect each sub-agent’s output and reasoning.
* Adjust schemas and instructions for clarity and reliability.
***
## Further Reading
* [PromptL Agent Syntax](/docs/promptl/agent-syntax)
* [Tool Integration](/docs/tools/overview)
* [Chains vs. Agents](/docs/architecture/chains-vs-agents)
***
## Examples
To see how agents and subagents work in context you are welcome to check out the following example agents:
1. Example 1: [Deep Search Agent](https://docs.latitude.so/examples/cases/deep-search)
2. Example 2: [Customer Support Email Generator](https://docs.latitude.so/examples/cases/customer-support-email)
***
## Summary
Sub-agents and agentic structures in Latitude unlock modular, reusable, and powerful LLM workflows. By designing clear, focused agents and orchestrating them with the agentic loop, you can tackle complex, multi-stage tasks with reliability and transparency.
# Tool use
Source: https://docs-v1.latitude.so/guides/prompt-manager/tools
Enable AI models to interact with external functions and data sources using tools.
Tools allow you to give AI models access to external functions or data sources, enabling them to perform actions beyond simple text generation. You can define custom tools or use Latitude's built-in tools.
Tool support depends on the specific AI provider and model. Check your
provider's documentation for compatibility.
## Enabling Tools in a Prompt
To make tools available to your prompt, list them under the `tools` key in the prompt's configuration block (`---`).
```yaml theme={null}
---
provider: openai
model: gpt-4o
tools:
- latitude/search # Use a built-in Latitude tool
- get_weather:
description: Get the current weather for a specified location.
parameters: # JSON Schema for parameters
type: object
properties:
location:
type: string
description: The city and state, e.g., San Francisco, CA
required: [location]
---
What's the weather like in {{ location }}?
```
## Built-in Latitude Tools
Latitude provides several powerful built-in tools that you can enable directly:
* `latitude/search`: Performs web searches to find up-to-date information.
* `latitude/code`: Executes code snippets.
* `latitude/extract`: Extracts structured data from text.
Simply include their names (e.g., `latitude/search`) in the `tools` list.
[Learn more about using Latitude Tools](/guides/prompt-manager/latitude-tools).
## Defining Custom Tools
For capabilities beyond the built-in tools, you can define your own custom tools. When the model decides to use a custom tool, Latitude will request its execution from your application via the SDK.
Define custom tools directly within the configuration block, outside the main `tools` list:
```yaml theme={null}
---
provider: openai
model: gpt-4o
tools:
- get_stock_price:
description: Retrieves the current stock price for a given ticker symbol.
parameters:
type: object
properties:
ticker_symbol:
type: string
description: The stock ticker symbol (e.g., AAPL, GOOGL).
required: [ticker_symbol]
---
What is the current price of {{ ticker }} stock?
```
Each custom tool definition requires:
* **`description`**: A clear explanation for the AI model of what the tool does.
* **`parameters`**: A [JSON Schema](#json-schema-for-parameters) defining the expected input arguments for the tool.
### Handling Custom Tool Calls
When the AI model decides to use `get_stock_price`, the Latitude SDK in your application will receive a tool call request. Your code needs to:
1. Execute the actual logic (e.g., call a financial API).
2. Return the result back to Latitude.
Refer to the SDK documentation ([TypeScript](/guides/sdk/typescript), [Python](/guides/sdk/python)) for details on handling tool calls.
## JSON Schema for Parameters
Tool parameters **must** be defined using JSON Schema to specify their structure, types, and requirements. This helps the model understand how to call the tool correctly.
Key schema components:
* `type`: `object`, `string`, `number`, `integer`, `boolean`, `array`.
* `description`: Essential for explaining parameters to the model.
* `properties` (for `object` type): Defines nested parameters.
* `required` (for `object` type): Lists mandatory properties.
* `enum` (for `string`, `number`): Specifies allowed values.
* `items` (for `array` type): Defines the schema for array elements.
See the [official JSON Schema guide](https://json-schema.org/learn/getting-started-step-by-step) for more details.
**Example (Object with multiple parameters):**
```yaml theme={null}
add_calendar_event:
description: Adds an event to the user's calendar.
parameters:
type: object
properties:
title:
type: string
description: The title of the event.
date:
type: string
description: The date of the event (YYYY-MM-DD).
time:
type: string
description: The time of the event (HH:MM).
duration_minutes:
type: integer
description: Duration of the event in minutes.
required: [title, date, time]
```
### Tools without Parameters
If you want to define a tool without parameters, for example a tool that generates a random number, you can omit the parameters key. However, providers such as Anthropic or Google, always requires parameters. In this case, the best option is to define an empty object as parameters:
```markdown {8-11} theme={null}
---
provider: anthropic
model: claude-sonnet-4-0
type: agent
tools:
- randomizer:
description: Generates a random number.
parameters:
type: object
properties: {}
required: []
---
```
## Next Steps
* Explore [Latitude's Built-in Tools](/guides/prompt-manager/latitude-tools) in more detail.
* Learn how to build powerful [Agents](/guides/prompt-manager/agents) that leverage tools.
* Test tool interactions in the [Playground](/guides/prompt-manager/playground#testing-tool-responses).
# Triggers
Source: https://docs-v1.latitude.so/guides/prompt-manager/triggers
Enables automatic execution of prompts based on specific events or schedules
## Overview
The Triggers feature enables automatic execution of prompts based on specific events or schedules. This documentation explains how to configure triggers via Email or Schedule and describes each available setting.
Triggers allow you to:
* Automate prompt runs when specific conditions are met (e.g., receiving an email, or on a set schedule).
* Define who can trigger prompts and under what circumstances.
* Configure parameters and access control for secure, contextual automation.
To access the triggers you have to click in the top right corner of the prompt playground
![Triggers Button]()
## Trigger Types
You can set up two main types of triggers
* Email
* Schedule
### Email
Allows running a prompt when an email is sent to a designated address.
![Email Trigger Configuration]()
**Parameters**
You may define how the paramters from the prompt are filled with the email the users sent to the trigger email address.
In this image we have 3 parameters in this prompt.
Users must provide these parameters when triggering the prompt by email.
**Email Settings**
* `Name` Give your email trigger a name (used to distinguish between triggers).
* `Prompt Email Address`: A unique email address is generated for your trigger
* `Reply with response`: Toggle to automatically reply to the sender with the prompt’s output.
## Schedule
![Schedule Trigger Configuration]()
Allows running a prompt on a fixed schedule, such as every hour, daily, or based on a cron expression.
You can combine a trigger with an MCP and built a automated workflow that runs a prompt on a schedule and sends the output to a specific channel.
# Version Control
Source: https://docs-v1.latitude.so/guides/prompt-manager/version-control
Manage prompt versions, track changes, and collaborate effectively with your team.
Latitude includes built-in version control, allowing you to manage the lifecycle of your prompts, track changes over time, and collaborate effectively with your team.
## Key Concepts
* **Version**: A snapshot of all prompts within a project at a specific point in time.
* **Draft**: A work-in-progress version where you can safely make and test changes without impacting the live, published version.
* **Published Version**: The official, live version of your prompts that is served by the AI Gateway and used in production applications.
## Managing Drafts and Versions
### Creating and Switching Drafts
1. Open your project.
2. Click the version dropdown in the sidebar (usually shows "Published" or the current draft name).
![Version Dropdown]()
3. Click "New version" to create a new draft based on the currently published version.
![Version Dropdown]()
4. Alternatively, select an existing draft from the list to switch to it.
5. All changes made while in a draft are saved to that specific draft.
### Viewing and Comparing History
1. Click the version dropdown.
2. Select "Version history".
![Project History]()
3. Here you can see a list of all published versions and drafts.
4. Select two versions to compare their differences side-by-side.
## Publishing Changes
When a draft is ready to go live:
1. Make sure you are viewing the draft you want to publish.
2. Click the "Publish" button (often located near the version dropdown).
3. Add a descriptive version note (e.g., "Improved error handling for user queries"). This helps track changes.
4. Confirm the publish action.
Your draft is now the new "Published" version, and the AI Gateway will start serving these updated prompts.
Tip: For a smoother development experience, create new versions as you create new features.
## Collaboration Tips
* **Use Drafts for Development**: Encourage team members to create drafts for new features or significant changes to avoid conflicts.
* **Descriptive Version Notes**: Write clear notes when publishing to explain *what* changed and *why*.
* **Regular Publishing**: Publish stable changes regularly rather than keeping large, long-running drafts.
* **Review Before Publishing**: Have a team member review changes in a draft before it goes live, especially for critical prompts.
* **Utilize Comparison**: Use the diff view in version history to understand changes made by others.
## Next Steps
* Learn about [Prompt Configuration](/guides/prompt-manager/configuration)
* Test changes in the [Playground](/guides/prompt-manager/playground)
# Python
Source: https://docs-v1.latitude.so/guides/sdk/python
Integrate Latitude into your Python applications using the Python SDK.
The Latitude Python SDK provides a convenient way to interact with the Latitude platform from your Python applications.
## Installation
The Latitude SDK is compatible with Python 3.9 or higher.
```bash theme={null}
pip install latitude-sdk
# or
poetry add latitude-sdk
# or
uv add latitude-sdk
```
## Authentication and Initialization
Import the SDK and initialize it with your API key. You can generate API keys in your Latitude project settings under "API Access".
```python theme={null}
import os
from latitude_sdk import Latitude
latitude = Latitude(os.getenv("LATITUDE_API_KEY"))
```
You can also provide additional options during initialization:
```python theme={null}
latitude = Latitude(os.getenv("LATITUDE_API_KEY"), LatitudeOptions(
project_id=12345, # Your Latitude project ID
version_uuid="optional-version-uuid", # Optional version UUID
)) # Keep your API key secure and avoid committing it directly into your codebase.
```
> Both `project_id` and `version_uuid` options can be overridden on a per-method basis when needed.
## Examples
Check out our [Examples](/examples) section for more examples of how to use the Latitude SDK.
## SDK Usage
The Latitude Python SDK is an async library by design. This means you must use it within an async event loop, such as FastAPI or Async Django. Another option is to use the built-in `asyncio` library.
```python theme={null}
import asyncio
from latitude_sdk import Latitude
latitude = Latitude("your-api-key-here")
async def main():
prompt = await latitude.prompts.get("prompt-path")
print(prompt)
asyncio.run(main())
```
## SDK Structure
The Latitude SDK is organized into several namespaces:
* `prompts`: Methods for managing and running prompts
* `runs`: Methods for managing active runs
* `logs`: Methods for pushing logs to Latitude
* `evaluations`: Methods for pushing evaluation results to Latitude
* `projects`: Methods for managing projects
* `versions`: Methods for managing project versions
## Prompt Management
### Get a Prompt
To retrieve a specific prompt by its path:
```python theme={null}
prompt = await latitude.prompts.get('prompt-path')
```
### Get All Prompts
To retrieve all prompts in your project:
```python theme={null}
prompts = await latitude.prompts.get_all()
```
### Get or Create a Prompt
To get an existing prompt or create a new one if it doesn't exist:
```python theme={null}
prompt = await latitude.prompts.get_or_create('prompt-path')
```
You can also provide the content when creating a new prompt:
```python theme={null}
prompt = await latitude.prompts.get_or_create('prompt-path', GetOrCreatePromptOptions(
prompt='This is the content of my new prompt',
))
```
### Delete a Prompt
To delete a prompt from a draft version:
```python theme={null}
result = await latitude.prompts.delete('prompt-path')
```
This soft-deletes the document. The deletion only works on draft (non-merged) commits.
## Version Management
### Get All Versions
To retrieve all versions from a project:
```python theme={null}
versions = await latitude.versions.get_all()
```
You can also specify a different project ID:
```python theme={null}
versions = await latitude.versions.get_all(GetAllVersionsOptions(
project_id=123,
))
```
## Running Prompts
### Non-Streaming Run
Execute a prompt and get the complete response once generation is finished:
```python theme={null}
async def on_finished(result: FinishedResult):
print('Run completed:', result.uuid)
async def on_error(error: ApiError):
print('Run error:', error.message)
result = await latitude.prompts.run('prompt-path', RunPromptOptions(
parameters={
'productName': 'CloudSync Pro',
'audience': 'Small Business Owners',
},
# Optional: Provide a custom identifier for this run
custom_identifier='email-campaign-2023',
# Optional: Provide callbacks for events
on_finished=on_finished,
on_error=on_error,
))
print('Conversation UUID:', result.uuid)
print('Conversation messages:', result.conversation)
```
### Handling Streaming Responses
For real-time applications (like chatbots), use streaming to get response chunks as they are generated:
```python theme={null}
async def on_event(event: StreamEvent):
# Provider event
if isinstance(event, dict) and event.get("type") == "text-delta":
print(event)
# Latitude event
elif isinstance(event, ChainEventChainCompleted):
print("Conversation UUID:", event.uuid)
print("Conversation messages:", event.messages)
async def on_finished(result: FinishedResult):
print('Stream completed:', result.uuid)
async def on_error(error: ApiError):
print('Stream error:', error.message)
await latitude.prompts.run('prompt-path', RunPromptOptions(
parameters={
'productName': 'CloudSync Pro',
'audience': 'Small Business Owners',
},
# Enable streaming
stream=True,
# Provide callbacks for events
on_event=on_event,
on_finished=on_finished,
on_error=on_error,
))
```
### Using Tools with Prompts
You can provide tool handlers that the model can call during execution:
```python theme={null}
async def get_weather(arguments: Dict[str, Any], details: OnToolCallDetails) -> Dict[str, Any]:
# `arguments` contains the arguments passed by the model
# `details` contains context like tool id, name, messages...
# The result can be anything JSON serializable
return { "weather": "sunny" }
await latitude.prompts.run('prompt-path', RunPromptOptions(
parameters={
'query': 'What is the weather in San Francisco?',
},
# Define the tools the model can use
tools={
'getWeather': get_weather,
},
))
```
### Chat with a Prompt
Follow the conversation of a runned prompt:
```python theme={null}
messages = [
{
'role': 'user',
'content': 'Hello, how can you help me today?',
},
]
async def on_finished(result: FinishedResult):
print('Chat completed:', result.uuid)
async def on_error(error: ApiError):
print('Chat error:', error.message)
result = await latitude.prompts.chat('conversation-uuid', messages, ChatPromptOptions(
# Chat options are similar to the run method
on_finished=on_finished,
on_error=on_error,
))
print('Conversation UUID:', result.uuid)
print('Conversation messages:', result.conversation)
```
Messages follow the [PromptL](/promptl/syntax/messages) format. If you're
using a different method to run your prompts, you'll need to format your
messages accordingly.
### Running a Prompt in the Background
For long-running prompts, such as large Agent systems, that you don't need to wait for, use background runs:
```python theme={null}
job = await latitude.prompts.run('prompt-path', RunPromptOptions(
parameters={
'productName': 'CloudSync Pro',
'audience': 'Small Business Owners',
},
# Enable background processing
background=True,
))
print('Job UUID:', job.uuid)
# The request returns immediately with a conversation UUID
# You can use this UUID to attach to the run later to check its status or stop it programmatically
async def on_event(event: StreamEvent):
# Provider event
if isinstance(event, dict) and event.get("type") == "text-delta":
print(event)
# Latitude event
elif isinstance(event, ChainEventChainCompleted):
print("Conversation UUID:", event.uuid)
print("Conversation messages:", event.messages)
result = await latitude.runs.attach(job.uuid, AttachRunOptions(
stream=True,
on_event=on_event,
))
print('Conversation UUID:', result.uuid)
print('Conversation messages:', result.conversation)
```
## Run Management
### Stop a Run
Stop an active conversation that is currently running:
```python theme={null}
await latitude.runs.stop('conversation-uuid')
print('Run stopped successfully')
```
### Attach to a Run
Attach to an active conversation to receive its ongoing output:
```python theme={null}
async def on_event(event: StreamEvent):
# Provider event
if isinstance(event, dict) and event.get("type") == "text-delta":
print(event)
# Latitude event
elif isinstance(event, ChainEventChainCompleted):
print("Conversation UUID:", event.uuid)
print("Conversation messages:", event.messages)
async def on_finished(result: FinishedResult):
print('Attach completed:', result.uuid)
async def on_error(error: ApiError):
print('Attach error:', error.message)
result = await latitude.runs.attach('conversation-uuid', AttachRunOptions(
# Optional: Enable streaming for real-time updates
stream=True,
# Optional: Provide callbacks for events
on_event=on_event,
on_finished=on_finished,
on_error=on_error,
))
print('Conversation UUID:', result.uuid)
print('Conversation messages:', result.conversation)
```
## Rendering Prompts
### Prompt Rendering
Render a prompt locally without running it:
```python theme={null}
result = await latitude.prompts.render(
'Your prompt content here with {{ parameters }}',
RenderPromptOptions(
parameters={
'topic': 'Artificial Intelligence',
'tone': 'Professional',
},
# Optional: Specify a provider adapter
adapter=Adapter.OpenAI,
))
print('Rendered config:', result.config)
print('Rendered messages:', result.messages)
```
### Chain Rendering
Render a chain of prompts locally:
```python theme={null}
async def on_step(messages: list[MessageLike], config: dict[str, Any]) -> str | MessageLike:
# Process each step in the chain
print('Processing step with messages:', messages)
# Return a string or a message object
return 'Step response'
result = await latitude.prompts.render_chain(
Prompt(
path='prompt-path',
content='Your prompt content here with {{ parameters }}',
provider='openai',
),
on_step,
RenderChainOptions(
parameters={
'topic': 'Machine Learning',
'complexity': 'Advanced',
},
# Optional: Specify a provider adapter
adapter=Adapter.OpenAI,
))
print('Rendered config:', result.config)
print('Rendered messages:', result.messages)
```
## Logging
### Creating Logs
Push a log to Latitude manually for a prompt:
```python theme={null}
messages = [
{
'role': 'user',
'content': 'Hello, how can you help me today?',
},
]
log = await latitude.logs.create('prompt-path', messages, CreateLogOptions(
response='I can help you with anything!',
))
```
## Evaluations
### Annotate a log
Push an evaluation result (annotate) to Latitude:
```python theme={null}
result = await sdk.evaluations.annotate(
'conversation-uuid',
4, # In this case, the score is 4 out of 5
"evaluation-uuid",
AnnotateEvaluationOptions(reason="I liked it!"),
)
```
## Complete Method Reference
### Initialization
```python theme={null}
# SDK initialization
class GatewayOptions:
host: str
port: int
ssl: bool
class InternalOptions:
gateway: Optional[GatewayOptions]
retries: Optional[int]
delay: Optional[float]
timeout: Optional[float]
class LatitudeOptions:
promptl: Optional[PromptlOptions]
internal: Optional[InternalOptions]
project_id: Optional[int]
version_uuid: Optional[str]
tools: Optional[dict[str, OnToolCall]]
Latitude(
api_key: str,
options: Optional[LatitudeOptions]
)
```
### Prompts Namespace
```python theme={null}
# Get a prompt
class GetPromptOptions:
project_id: Optional[int]
version_uuid: Optional[str]
class GetPromptResult:
uuid: str
path: str
content: str
config: dict[str, Any]
parameters: dict[str, PromptParameter]
provider: Optional[Providers]
latitude.prompts.get(
path: str,
options: Optional[GetPromptOptions]
) -> GetPromptResult
# Get all prompts
class GetAllPromptsOptions:
project_id: Optional[int]
version_uuid: Optional[str]
latitude.prompts.get_all(
options: Optional[GetAllPromptsOptions]
) -> List[GetPromptResult]
# Get or create a prompt
class GetOrCreatePromptOptions:
project_id: Optional[int]
version_uuid: Optional[str]
prompt: Optional[str]
class GetOrCreatePromptResult:
uuid: str
path: str
content: str
config: dict[str, Any]
parameters: dict[str, PromptParameter]
provider: Optional[Providers]
latitude.prompts.get_or_create(
path: str,
options: Optional[GetOrCreatePromptOptions]
) -> GetOrCreatePromptResult
# Delete a prompt
class DeletePromptOptions:
project_id: Optional[int]
version_uuid: Optional[str]
class DeletePromptResult:
document_uuid: str
path: str
latitude.prompts.delete(
path: str,
options: Optional[DeletePromptOptions]
) -> DeletePromptResult
# Run a prompt
class RunPromptOptions:
project_id: Optional[int]
version_uuid: Optional[str]
on_event: Optional[OnEvent]
on_finished: Optional[OnFinished]
on_error: Optional[OnError]
custom_identifier: Optional[str]
parameters: Optional[dict[str, Any]]
tools: Optional[dict[str, OnToolCall]]
stream: Optional[bool]
background: Optional[bool]
mcp_headers: Optional[dict[str, dict[str, str]]]
messages: Optional[Sequence[MessageLike]] # Messages to append after the compiled prompt
class FinishedResult:
uuid: str
conversation: List[Message]
response: ChainResponse
class BackgroundResult:
uuid: str
RunPromptResult = Union[FinishedResult, BackgroundResult]
latitude.prompts.run(
path: str,
options: Optional[RunPromptOptions]
) -> Optional[RunPromptResult]
# Chat with a prompt
class ChatPromptOptions:
on_event: Optional[OnEvent]
on_finished: Optional[OnFinished]
on_error: Optional[OnError]
tools: Optional[dict[str, OnToolCall]]
stream: Optional[bool]
class ChatPromptResult:
uuid: str
conversation: List[Message]
response: ChainResponse
latitude.prompts.chat(
uuid: str,
messages: Sequence[MessageLike],
options: Optional[ChatPromptOptions]
) -> Optional[ChatPromptResult]
# Render a prompt
class RenderPromptOptions:
parameters: Optional[dict[str, Any]]
adapter: Optional[Adapter]
class RenderPromptResult:
messages: List[MessageLike]
config: dict[str, Any]
latitude.prompts.render(
prompt: str,
options: Optional[RenderPromptOptions]
) -> RenderPromptResult
# Render a chain
class RenderChainOptions:
parameters: Optional[dict[str, Any]]
adapter: Optional[Adapter]
class RenderChainResult:
messages: List[MessageLike]
config: dict[str, Any]
latitude.prompts.render_chain(
prompt: Prompt,
on_step: OnStep,
options: Optional[RenderChainOptions]
) -> RenderChainResult
```
### Runs Namespace
```python theme={null}
# Attach to a run
class AttachRunOptions:
on_event: Optional[OnEvent]
on_finished: Optional[OnFinished]
on_error: Optional[OnError]
tools: Optional[dict[str, OnToolCall]]
stream: Optional[bool]
class AttachRunResult:
uuid: str
conversation: List[Message]
response: ChainResponse
latitude.runs.attach(
uuid: str,
options: Optional[AttachRunOptions]
) -> Optional[AttachRunResult]
# Stop a run
latitude.runs.stop(
uuid: str
) -> None
```
### Projects Namespace
```python theme={null}
# Get all projects
latitude.projects.get_all(
) -> List[Project]
# Create a project
class CreateProjectResult:
project: Project
version: Version
latitude.projects.create(
name: str
) -> CreateProjectResult
# Get all versions for a project
latitude.projects.get_all_versions(
project_id: int
) -> List[Version]
```
### Logs Namespace
```python theme={null}
# Create a log
class CreateLogOptions:
project_id: Optional[int]
version_uuid: Optional[str]
response: Optional[str]
class CreateLogResult:
id: int
uuid: str
source: Optional[LogSources]
commit_id: int
resolved_content: str
content_hash: str
parameters: dict[str, Any]
custom_identifier: Optional[str]
duration: Optional[int]
created_at: datetime
updated_at: datetime
latitude.logs.create(
path: str,
messages: Sequence[MessageLike],
options: Optional[CreateLogOptions]
) -> CreateLogResult
```
### Evaluations Namespace
```python theme={null}
class AnnotateEvaluationOptions:
reason: str
class AnnotateEvaluationResult:
uuid: str
version_uuid: str
score: int
normalized_score: int
metadata: dict[str, Any]
has_passed: bool
error: Optional[str]
created_at: datetime
updated_at: datetime
latitude.evaluations.annotate(
uuid: str,
score: int,
evaluation_uuid: str,
options: Optional[AnnotateEvaluationOptions]
) -> AnnotateEvaluationResult
```
### Versions Namespace
```python theme={null}
class GetAllVersionsOptions:
project_id: Optional[int]
latitude.versions.get_all(
options: Optional[GetAllVersionsOptions]
) -> List[Version]
```
## Error Handling
The SDK raises `ApiError` instances when API requests fail. You can catch and handle these errors:
```python theme={null}
from latitude_sdk import ApiError
async def handle_errors():
try:
prompt = await latitude.prompts.get("non-existent-prompt")
except ApiError as error:
print(f"API Error: {error.message}")
print(f"Error Code: {error.code}")
print(f"Status: {error.status}")
except Exception as error:
print(f"Unexpected error: {error}")
```
## Logging Features
* **Automatic Logging**: All runs through `latitude.prompts.run()` are automatically logged in Latitude, capturing inputs, outputs, performance metrics, and trace information.
* **Custom Identifiers**: Use the optional `custom_identifier` parameter to tag runs for easier filtering and analysis in the Latitude dashboard.
* **Response Identification**: Each response includes identifying information like `uuid` that can be used to reference the specific run later.
## Further Information
* [HTTP API Reference](/guides/api/reference)
* [API Access and Authentication](/guides/api/api-access)
* [Streaming Event Details](/guides/api/streaming-events)
# TypeScript
Source: https://docs-v1.latitude.so/guides/sdk/typescript
Integrate Latitude into your Node.js applications using the TypeScript SDK.
The Latitude TypeScript SDK provides a convenient way to interact with the Latitude platform from your Node.js or browser applications.
## Installation
The Latitude SDK is compatible with Node.js 16 or higher.
```bash theme={null}
npm install @latitude-data/sdk
# or
yarn add @latitude-data/sdk
# or
pnpm add @latitude-data/sdk
```
## Authentication and Initialization
Import the SDK and initialize it with your API key. You can generate API keys in your Latitude project settings under "API Access".
```typescript theme={null}
import { Latitude } from '@latitude-data/sdk'
const latitude = new Latitude(process.env.LATITUDE_API_KEY)
```
You can also provide additional options during initialization:
```typescript theme={null}
const latitude = new Latitude(process.env.LATITUDE_API_KEY, {
projectId: 123, // Your Latitude project ID
versionUuid: 'version-uuid', // Optional version UUID
}) // Keep your API key secure and avoid committing it directly into your codebase.
```
> Both `projectId` and `versionUuid` options can be overridden on a per-method basis when needed.
## Examples
Check out our [Examples](/examples) section for more examples of how to use the Latitude SDK.
## SDK Structure
The Latitude SDK is organized into several namespaces:
* `prompts`: Methods for managing and running prompts
* `runs`: Methods for managing active runs
* `logs`: Methods for pushing logs to Latitude
* `evaluations`: Methods for pushing evaluation results to Latitude
* `projects`: Methods for managing projects
* `versions`: Methods for managing project versions
## Prompt Management
### Get a Prompt
To retrieve a specific prompt by its path:
```typescript theme={null}
const prompt = await latitude.prompts.get('prompt-path')
```
### Get All Prompts
To retrieve all prompts in your project:
```typescript theme={null}
const prompts = await latitude.prompts.getAll()
```
### Get or Create a Prompt
To get an existing prompt or create a new one if it doesn't exist:
```typescript theme={null}
const prompt = await latitude.prompts.getOrCreate('prompt-path')
```
You can also provide the content when creating a new prompt:
```typescript theme={null}
const prompt = await latitude.prompts.getOrCreate('prompt-path', {
prompt: 'This is the content of my new prompt',
})
```
### Delete a Prompt
To delete a prompt from a draft version:
```typescript theme={null}
const result = await latitude.prompts.delete('prompt-path')
```
This soft-deletes the document. The deletion only works on draft (non-merged) commits.
## Version Management
### Get All Versions
To retrieve all versions from a project:
```typescript theme={null}
const versions = await latitude.versions.getAll()
```
You can also specify a different project ID:
```typescript theme={null}
const versions = await latitude.versions.getAll(123)
```
## Running Prompts
### Non-Streaming Run
Execute a prompt and get the complete response once generation is finished:
```typescript theme={null}
const result = await latitude.prompts.run('prompt-path', {
parameters: {
productName: 'CloudSync Pro',
audience: 'Small Business Owners',
},
// Disable streaming (enabled by default)
stream: false,
// Optional: Provide a custom identifier for this run
customIdentifier: 'email-campaign-2023',
})
console.log('Conversation UUID:', result.uuid)
console.log('Response:', result.response.text)
```
### Handling Streaming Responses
For real-time applications (like chatbots), use streaming to get response chunks as they are generated:
```typescript theme={null}
await latitude.prompts.run('prompt-path', {
parameters: {
productName: 'CloudSync Pro',
audience: 'Small Business Owners',
},
// Enable streaming
stream: true,
// Provide callbacks for events
onFinished: (result) => {
// Execution succeeded
console.log("Stream completed:", result.response.text);
},
onError: (error) => {
// Execution failed
console.log("Error:", error);
},
onEvent: ({ event, data }) => {
// All events can be individually handled here
if (event === StreamEventTypes.Provider && data.type === 'text-delta') {
console.log(data.textDelta)
} else if (
event === StreamEventTypes.Latitude &&
data.type === 'chain-completed'
) {
console.log('Conversation UUID:', data.uuid)
console.log('Conversation messages:', data.messages)
}
},
})
```
### Using Tools with Prompts
You can provide tool handlers that the model can call during execution:
```typescript theme={null}
await latitude.prompts.run('prompt-path', {
parameters: {
query: 'What is the weather in San Francisco?',
},
// Define the tools the model can use
tools: {
getWeather: async (args, details) => {
// `args` contains the arguments passed by the model
// `details` contains context like tool id, name, messages...
// The result can be anything JSON serializable
console.log('Getting weather for:', args.location)
return { temperature: '72°F', conditions: 'Sunny' }
},
},
})
```
### Prompts that return structured outputs
By default the sdk assumes your prompt return text. If you expect your prompt
to return structured output you can type it in the `prompts.run` method:
```typescript theme={null}
const result = await latitude.prompts.run<{ expected: 'property' }>(
'structured-output-prompt',
)
console.log(result.object) // outputs { expected: 'property' }
```
### Chat with a Prompt
Follow the conversation of a runned prompt:
```typescript theme={null}
const messages = [
{
role: 'user',
content: 'Hello, how can you help me today?',
},
]
const result = await latitude.prompts.chat('conversation-uuid', messages, {
// Chat options are similar to the run method
onFinished: (result) => {
console.log('Chat completed:', result.uuid)
},
onError: (error) => {
console.error('Chat error:', error.message)
},
})
console.log('Conversation UUID:', result.uuid)
console.log('Conversation messages:', result.conversation)
```
Messages follow the [PromptL](/promptl/syntax/messages) format. If you're
using a different method to run your prompts, you'll need to format your
messages accordingly.
### Running a Prompt in the Background
For long-running prompts, such as large Agent systems, that you don't need to wait for, use background runs:
```typescript theme={null}
const job = await latitude.prompts.run('prompt-path', {
parameters: {
productName: 'CloudSync Pro',
audience: 'Small Business Owners',
},
// Enable background processing
background: true,
})
console.log('Job UUID:', job.uuid)
// The request returns immediately with a conversation UUID
// You can use this UUID to attach to the run later to check its status or stop it programmatically
const result = await latitude.runs.attach(job.uuid, {
stream: true,
onEvent: ({ event, data }) => {
console.log('Received event:', event, data)
},
})
console.log('Conversation UUID:', result.uuid)
console.log('Conversation messages:', result.conversation)
```
## Run Management
### Stop a Run
Stop an active conversation that is currently running:
```typescript theme={null}
await latitude.runs.stop('conversation-uuid')
console.log('Run stopped successfully')
```
### Attach to a Run
Attach to an active conversation to receive its ongoing output:
```typescript theme={null}
const result = await latitude.runs.attach('conversation-uuid', {
// Optional: Enable streaming for real-time updates
stream: true,
// Optional: Provide callbacks for events
onEvent: ({ event, data }) => {
if (event === StreamEventTypes.Provider && data.type === 'text-delta') {
console.log(data.textDelta)
}
},
onFinished: (result) => {
console.log('Attach completed:', result.uuid)
},
onError: (error) => {
console.error('Attach error:', error.message)
},
})
console.log('Conversation UUID:', result.uuid)
console.log('Conversation messages:', result.conversation)
```
## Rendering Prompts
### Prompt Rendering
Render a prompt locally without running it:
```typescript theme={null}
const result = await latitude.prompts.render({
prompt: {
content: 'Your prompt content here with {{ parameters }}',
},
parameters: {
topic: 'Artificial Intelligence',
tone: 'Professional',
},
// Optional: Specify a provider adapter
adapter: Adapters.OpenAI,
})
console.log('Rendered config:', result.config)
console.log('Rendered messages:', result.messages)
```
### Chain Rendering
Render a chain of prompts locally:
```typescript theme={null}
const result = await latitude.prompts.renderChain({
prompt: {
path: 'prompt-path',
content: 'Your prompt content here with {{ parameters }}',
provider: 'openai',
},
parameters: {
topic: 'Machine Learning',
complexity: 'Advanced',
},
// Required: Process each step in the chain
onStep: async ({ config, messages }) => {
// Process each step in the chain
console.log('Processing step with messages:', messages)
// Return a string or a message object
return 'Step response'
},
// Optional: Specify a provider adapter
adapter: Adapters.OpenAI,
// Optional: Log responses to Latitude
logResponses: true,
// Optional: Define tools for the chain
tools: {
getExample: async (args, details) => {
return { example: 'This is an example response' }
},
},
})
console.log('Rendered config:', result.config)
console.log('Rendered messages:', result.messages)
```
### Agent Rendering
Render an agent prompt locally (similar to `renderChain` but with a final agent result):
```typescript theme={null}
const result = await latitude.prompts.renderAgent({
prompt: {
path: 'prompt-path',
content: 'Agent prompt content with {{ parameters }}',
provider: 'openai',
},
parameters: {
task: 'Research quantum computing',
depth: 'Detailed',
},
// Required: Process each agent step
onStep: async ({ config, messages }) => {
// Process each step in the agent execution
console.log('Processing agent step:', messages)
// Return a string or a message object
return 'Agent step response'
},
// Optional: Log responses to Latitude
logResponses: true,
// Optional: Define tools for the agent
tools: {
search: async (args, details) => {
console.log('Agent using search tool with args:', args)
return { results: ['Result 1', 'Result 2'] }
},
},
})
console.log('Rendered config:', result.config)
console.log('Rendered messages:', result.messages)
// Agent final response is available in result.result
console.log('Agent final response:', result.result)
```
Make sure to provide the `config.tools` parameter to the LLM provider in your
`onStep` handler, otherwise the AI won't be able to stop the Agent loop!
## Logging
### Creating Logs
Push a log to Latitude manually for a prompt:
```typescript theme={null}
const messages = [
{
role: 'user',
content: 'Hello, how can you help me today?',
},
]
const log = await latitude.logs.create('prompt-path', messages, {
response: 'I can help you with anything!',
})
```
## Evaluations
### Annotate a log
Push an evaluation result (annotate) to Latitude:
```typescript theme={null}
const result = await latitude.evaluations.annotate(
'conversation-uuid',
100,
'evaluation-uuid',
{
reason: 'I liked it!',
},
)
```
## Complete Method Reference
### Initialization
```typescript theme={null}
// SDK initialization
new Latitude(
apiKey: string,
options: {
projectId?: number,
versionUuid?: string,
__internal?: {
gateway?: GatewayApiConfig,
source?: LogSources,
retryMs?: number
}
}
)
```
### Prompts Namespace
```typescript theme={null}
// Get a prompt
latitude.prompts.get(
path: string,
options?: {
projectId?: number,
versionUuid?: string
}
): Promise
// Get all prompts
latitude.prompts.getAll(
options?: {
projectId?: number,
versionUuid?: string
}
): Promise
// Create a prompt
latitude.prompts.create(
path: string,
options?: {
projectId?: number,
versionUuid?: string,
prompt?: string
}
): Promise
// Get or create a prompt
latitude.prompts.getOrCreate(
path: string,
options?: {
projectId?: number,
versionUuid?: string,
prompt?: string
}
): Promise
// Delete a prompt
latitude.prompts.delete(
path: string,
options?: {
projectId?: number,
versionUuid?: string
}
): Promise<{ documentUuid: string, path: string }>
// Run a prompt
latitude.prompts.run(
path: string,
options: {
projectId?: number,
versionUuid?: string,
customIdentifier?: string,
parameters?: Record,
stream?: boolean,
background?: Background,
tools?: ToolCalledFn,
signal?: AbortSignal,
messages?: Message[], // Messages to append after the compiled prompt
userMessage?: string, // Deprecated: use `messages` instead
mcpHeaders?: Record>,
onEvent?: ({ event, data }: { event: StreamEventTypes, data: ChainEventDto }) => void,
onFinished?: (data: GenerationResponse) => void,
onError?: (error: LatitudeApiError) => void
}
): Promise | undefined>
// Chat with a prompt
latitude.prompts.chat(
uuid: string,
messages: Message[],
options?: {
stream?: boolean,
tools?: ToolCalledFn,
signal?: AbortSignal,
onEvent?: ({ event, data }: { event: StreamEventTypes, data: ChainEventDto }) => void,
onFinished?: (data: GenerationResponse) => void,
onError?: (error: LatitudeApiError) => void
}
): Promise | undefined>
// Render a prompt
latitude.prompts.render(
options: {
prompt: { content: string },
parameters: Record,
adapter?: ProviderAdapter
}
): Promise<{ config: Config, messages: M[] }>
// Render a chain
latitude.prompts.renderChain(
options: {
prompt: Prompt,
parameters: Record,
adapter?: ProviderAdapter,
onStep: (args: { config: Config, messages: M[] }) => Promise>,
tools?: RenderToolCalledFn,
logResponses?: boolean
}
): Promise<{ config: Config, messages: M[] }>
```
### Runs Namespace
```typescript theme={null}
// Attach to a run
latitude.runs.attach(
uuid: string,
options?: {
stream?: boolean,
tools?: ToolCalledFn,
signal?: AbortSignal,
onEvent?: ({ event, data }: { event: StreamEventTypes, data: ChainEventDto }) => void,
onFinished?: (data: GenerationResponse) => void,
onError?: (error: LatitudeApiError) => void
}
): Promise | undefined>
// Stop a run
latitude.runs.stop(uuid: string): Promise
```
### Projects Namespace
```typescript theme={null}
// Get all projects
latitude.projects.getAll(): Promise
// Create a project
latitude.projects.create(
name: string
): Promise<{
project: Project
version: Version
}>
```
### Logs Namespace
```typescript theme={null}
// Create a log
latitude.logs.create(
path: string,
messages: Message[],
options?: {
response?: string,
projectId?: number,
versionUuid?: string
}
): Promise
```
### Evaluations Namespace
```typescript theme={null}
// Annotate a log
latitude.evaluations.annotate(
uuid: string,
score: number,
evaluationUuid: string,
options?: {
reason?: string,
versionUuid?: string
}
): Promise
```
### Versions Namespace
```typescript theme={null}
// Get a version
latitude.versions.get(
projectId: number,
commitUuid: string
): Promise
// Get all versions
latitude.versions.getAll(projectId?: number): Promise
// Create a version
latitude.versions.create(
name: string,
options?: {
projectId?: number
}
): Promise
// Push version changes
latitude.versions.push(
projectId: number,
baseCommitUuid: string,
changes: Array<{
path: string
content: string
status: 'added' | 'modified' | 'deleted' | 'unchanged'
contentHash?: string
}>
): Promise<{ commitUuid: string }>
```
## Error Handling
The SDK throws `LatitudeApiError` instances when API requests fail. You can catch and handle these errors:
```typescript theme={null}
import { LatitudeApiError } from '@latitude-data/sdk'
async function handleErrors() {
try {
const prompt = await latitude.prompts.get('non-existent-prompt')
} catch (error) {
if (error instanceof LatitudeApiError) {
console.error('API Error:', error.message)
console.error('Error Code:', error.errorCode)
console.error('Status:', error.status)
} else {
console.error('Unexpected error:', error)
}
}
}
```
## Logging Features
* **Automatic Logging**: All runs through `latitude.prompts.run()` are automatically logged in Latitude, capturing inputs, outputs, performance metrics, and trace information.
* **Custom Identifiers**: Use the optional `customIdentifier` parameter to tag runs for easier filtering and analysis in the Latitude dashboard.
* **Response Identification**: Each response includes identifying information like `uuid` that can be used to reference the specific run later.
## Further Information
* [HTTP API Reference](/guides/api/reference)
* [API Access and Authentication](/guides/api/api-access)
* [Streaming Event Details](/guides/api/streaming-events)
# Development Setup
Source: https://docs-v1.latitude.so/guides/self-hosted/development-setup
Set up a local Latitude instance using Docker for development and testing.
# Development Setup
This guide explains how to set up a local instance of Latitude on your machine for development, testing, or contributing to the project. We primarily use Docker Compose to manage the necessary services.
Due to potential performance limitations with Next.js in Docker volume mounts
on some systems, parts of the development setup might involve running services
directly on the host. This guide focuses on the Docker Compose approach.
## Prerequisites
* **Git**: To clone the repository.
* **Docker & Docker Compose**: Ensure Docker Desktop or equivalent is installed and running.
* **Node.js & pnpm**: Required for building packages and potentially running some services locally. Install [pnpm](https://pnpm.io/installation).
* **Tmux & Tmuxinator (Optional but Recommended)**: Useful for managing multiple services in the terminal. Install [Tmuxinator](https://github.com/tmuxinator/tmuxinator) (e.g., `brew install tmuxinator` on macOS).
## Setup Steps
1. **Clone the Repository**:
```bash theme={null}
git clone https://github.com/latitude-dev/latitude-llm.git
cd latitude-llm
```
2. **Install Dependencies**:
```bash theme={null}
pnpm install
```
3. **Build Shared Packages**:
Build the core packages that other services depend on.
```bash theme={null}
pnpm build --filter='./packages/**'
```
4. **Configure Environment (if needed)**:
* Copy any example environment files (e.g., `.env.example` to `.env`) and adjust necessary variables. For a standard local setup, defaults are often sufficient.
5. **Start Services with Docker Compose**:
This command starts the database, message queue, and other background services defined in `docker-compose.yml`.
```bash theme={null}
docker compose up -d
```
The Docker Compose setup pulls pre-built images from GitHub Container Registry (ghcr.io/latitude-dev).
To build local images after making code changes, add the `--build` flag (e.g., `docker compose up -d --build`).
*Alternatively, if using Tmuxinator (recommended)*:
```bash theme={null}
tmuxinator start
```
This will typically start Docker Compose in one pane and potentially other services (like the web app) in other panes according to the `.tmuxinator.yml` config.
6. **Run Database Migrations**:
Once the database container is running (check `docker compose ps` or the tmuxinator output), apply the necessary database schema migrations.
```bash theme={null}
cd packages/core
pnpm db:migrate
cd ../..
```
7. **Run the Web Application (if not using Tmuxinator)**:
If you didn't use tmuxinator, you might need to start the Next.js web application manually:
```bash theme={null}
pnpm dev --filter web
```
## Accessing Local Latitude
* **Web UI**: Open your browser to `http://localhost:3000` (or the configured port).
* **Sign Up**: Create your first user account using any email address.
* **Email Confirmation**: Access the local MailHog instance (usually at `http://localhost:8025`) to find the confirmation email and complete the signup.
* **API**: The local API will be available (check `docker-compose.yml` or app configuration for the port, often proxied through the web UI).
## Configuration
* **Docker Compose**: Modify `docker-compose.yml` to adjust service configurations, ports, or volumes.
* **Environment Variables**: Core settings (database URLs, API keys for testing, etc.) are typically managed via `.env` files within the respective app/package directories.
* **Application Config**: Check specific application directories (e.g., `apps/web/config`) for further configuration options.
## Stopping the Environment
* **Docker Compose**: `docker compose down`
* **Tmuxinator**: Stop the session (e.g., `tmux kill-session -t latitude-llm`) or stop individual services in their panes.
Now you have a local Latitude instance running for development!
## Using the REPL
You can run a JavaScript REPL to interact with the local instance of Latitude.
From the root of the repository, start the REPL with `pnpm console`. Try:
```javascript theme={null}
await database.select().from(users).limit(10)
```
## Next Steps
* Learn about deploying to [Production](/guides/self-hosted/production-setup)
* Explore the [Project Structure](/meta/project-structure)
# Production Deployment
Source: https://docs-v1.latitude.so/guides/self-hosted/production-setup
Deploying and managing a self-hosted Latitude instance for production use.
## Infrastructure Components
A typical production deployment involves several key services:
1. **Web Application**: The main Next.js frontend (e.g., `app` service).
2. **API Gateway**: Handles incoming API requests (e.g., `gateway` service).
3. **Workers**: Processes background jobs like evaluations and dataset generation (e.g., `worker` service).
4. **WebSockets**: Manages real-time connections (e.g., `websockets` service).
5. **Database**: PostgreSQL for storing persistent data.
6. **Cache/Queue**: Redis for caching and message queuing.
7. **Reverse Proxy/Load Balancer**: (e.g., Traefik, Nginx, Caddy, Cloud Load Balancer) Handles incoming traffic, SSL termination, and routing to services.
8. **Object Storage (Recommended)**: S3-compatible storage for files like uploaded datasets or images.
## Installation Methods
While Latitude can be run using `docker compose` on a single machine, production deployments often benefit from more robust methods:
### 1. Docker Compose (Single Server)
* **Pros**: Simple setup for smaller deployments or initial testing.
* **Cons**: Limited scalability and fault tolerance.
* **Setup**: Follow the Docker Compose instructions, ensuring production-ready configuration in your `.env` file (strong passwords, S3 storage, proper domain/SSL setup via Traefik or another proxy).
```bash theme={null}
# Ensure .env is configured for production
cp .env.example .env
# Edit .env with production values (DB creds, S3, Traefik email, domain, etc.)
docker network create web # If using Traefik default config
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
We only provide amd64 images, if you are using Mac OS with Apple Sillicon, you
will need to enable amd64 emulation and download the amd64 images.
### 2. Container Orchestration (Kubernetes, ECS, etc.)
* **Pros**: Scalability, high availability, automated management.
* **Cons**: More complex initial setup.
* **Setup**: Requires creating deployment manifests (e.g., Kubernetes YAML, ECS Task Definitions) for each Latitude service (web, gateway, worker, websockets). You'll need to manage database and Redis instances separately (e.g., using managed cloud services like RDS/ElastiCache or deploying them within the orchestrator).
* Configure ingress controllers (like Nginx Ingress or Traefik Ingress) for routing.
* Manage secrets securely (e.g., Kubernetes Secrets, AWS Secrets Manager).
* Set up persistent volumes for the database.
### 3. Platform-as-a-Service (PaaS)
* Services like Heroku, Render, or Fly.io might be suitable, often requiring Docker container deployments.
* You'll need to configure buildpacks or Dockerfiles and manage addons for databases and Redis.
### 4. Helm Chart (OCI)
The chart is automatically published to GitHub Container Registry by the workflow
`.github/workflows/publish-helm-chart.yml` on tags matching `helm-chart-v*`. Install it with:
```bash theme={null}
helm install latitude oci://ghcr.io//latitude \
--version \
-n latitude --create-namespace \
-f charts/latitude/values.yaml \
-f charts/latitude/values.secrets.yaml
```
## Configuration for Production
Regardless of the method, ensure these are configured correctly:
* **`.env` File**: Set strong `POSTGRES_PASSWORD`, configure `APP_DOMAIN` and `APP_URL`, set up `MAIL_TRANSPORT` (e.g., SMTP, Mailgun) with valid credentials, configure `DRIVE_DISK` (recommend `s3`).
* **Storage**: Use S3-compatible object storage (`DRIVE_DISK=s3`) for scalability and persistence. Configure bucket names and IAM roles or access keys securely.
* **Database & Redis**: Use managed services (RDS, ElastiCache, Memorystore) or ensure your self-hosted instances are properly configured for performance, backups, and security.
* **Latitude Analytics**: We collect [anonymous usage data](https://github.com/latitude-dev/latitude-llm/blob/main/packages/core/src/lib/analytics/collectors/OpenSource.ts) to improve the product. You can opt-out by setting `OPT_OUT_ANALYTICS=true` in your `.env` file.
* **Reverse Proxy/SSL**: Configure your reverse proxy (Traefik, Nginx, etc.) to handle SSL termination (e.g., using Let's Encrypt) and route traffic correctly to the Latitude services.
* **Resource Allocation**: Ensure sufficient CPU, memory, and disk space for each service, especially the database and workers.
## Connecting Provider API Keys
In a self-hosted setup, Latitude needs access to your AI provider API keys (OpenAI, Anthropic, etc.) to function.
* **Configuration Method**: Add these keys securely to the environment where the Latitude services (specifically the gateway and potentially workers/API) run.
* **Docker Compose**: Add them to your `.env` file (e.g., `OPENAI_API_KEY=sk-...`).
* **Kubernetes**: Store them as Kubernetes Secrets and mount them as environment variables in your deployments.
* **ECS**: Use AWS Secrets Manager or Parameter Store integrated with Task Definitions.
* **Latitude UI**: Once configured in the environment, you might still need to "register" the provider within the Latitude Admin UI (Settings > Providers), but you typically won't need to paste the key directly into the UI if it's available in the environment.
## Scaling and Updating
* **Scaling**: With orchestrators, you can scale services horizontally by increasing replica counts (especially for `web`, `gateway`, `worker`, `websockets`). Scale database and Redis resources vertically or use managed services that scale.
* **Updating**: Pull the latest official Docker images from the [Latitude GitHub Container Registry](https://github.com/orgs/latitude-dev/packages) (or build your own from the latest source) and update your deployments (e.g., `docker compose pull && docker compose up -d`, `kubectl rollout restart deployment`, update ECS service).
* **Migrations**: Ensure database migrations run automatically on startup (the default `docker-compose.yml` includes a migrations service) or run them manually before updating application services (`cd packages/core && pnpm db:migrate`).
## Security and Backups
* **Security**: Restrict access to database and Redis ports, use strong passwords, keep dependencies updated, configure firewall rules, secure API keys and secrets.
* **Backups**: Implement regular backups for your PostgreSQL database. If using local storage (`DRIVE_DISK=local`), ensure the storage volume is backed up.
## Self-Hosting MCP Integrations
If using [Third-Party Integrations via MCP](/guides/integration/mcp-integrations), you'll need to deploy and manage those MCP servers within your infrastructure as well.
## Monitoring and Logging
* **Logging**: Configure Docker log drivers or orchestrator logging to aggregate logs from all services (web, gateway, worker, etc.).
* **Monitoring**: Use tools like Prometheus/Grafana, Datadog, or cloud provider monitoring services to track resource usage (CPU, memory), error rates, request latency, and queue lengths.
* **Health Checks**: Configure health check endpoints in your load balancer or orchestrator to monitor service availability.
* **Optional Integrations**: Configure Sentry (`SENTRY_DSN`) or PostHog in your `.env` for enhanced error tracking and analytics.
## Next Steps
* Review the [Local Development Setup](/guides/self-hosted/development-setup)
* Consult specific documentation for your chosen orchestrator or PaaS.
# Chains and Steps
Source: https://docs-v1.latitude.so/promptl/advanced/chains
Chains and Steps are used to create multi-step prompts that can interact with the AI model in stages.
## Overview
Chains in PromptL allow you to break complex workflows into smaller, manageable steps. Each step generates a response, which can then be used in subsequent steps. This approach improves the model's ability to perform complex tasks and provides greater control over dynamic conversations.
With Chains, you can:
* Process tasks incrementally to guide the model step-by-step.
* Store and reuse intermediate results dynamically.
* Customize step-specific configurations for more efficient execution.
* Isolate steps to minimize context overhead or confusion.
***
## Syntax
Define steps in your prompt using the `` tag. The engine pauses after each step, waits for the model's response, and adds it as an assistant message before continuing.
### Basic Syntax
```xml theme={null}
Analyze the following text and identify the main idea:
"The quick brown fox jumps over the lazy dog."
Now, summarize the identified main idea in one sentence.
```
### Step with Custom Configuration
Override the default configuration by adding attributes to the `` tag:
```tsx theme={null}
Rewrite the following paragraph in simpler language:
"Quantum mechanics explains the behavior of particles on a microscopic scale."
```
***
## Advanced Features
### Storing Step Responses
You can store the text response of a step in a variable using the `as` attribute. This allows you to reuse the response in later steps or logic blocks.
```xml theme={null}
Is this statement correct? {{ statement }}
Respond only with "correct" or "incorrect".
{{ if analysis == "correct" }}
Provide additional details about why the statement is correct.
{{ else }}
Explain why the statement is incorrect and provide the correct answer.
{{ endif }}
```
#### Parse Step Responses as JSON
The response of a step will be automatically parsed as JSON if the JSON output schema is defined.
```xml theme={null}
Is this statement correct? {{ statement }}
Respond only with "correct: true" or "correct: false" in a JSON object.
{{ if analysis.correct }}
Provide additional details about why the statement is correct.
{{ else }}
Explain why the statement is incorrect and provide the correct answer.
{{ endif }}
```
[Learn more about JSON Output](/guides/prompt-manager/json-output).
### Storing Raw Messages
Some providers will return additional metadata along with the response. To store the entire message object, instead of just the text response (e.g., role, content, and additional metadata), use the `raw` attribute:
```xml theme={null}
Summarize the following text:
{{ text }}
```
The raw response will return an object with the full message details, which contains the `role`, `content`, and other metadata provided by the model.
The `content` attribute will always be defined as an array of content objects, which can include text, images, tool calls and any other types of content returned by the LLM.
```json theme={null}
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "This is a summary of the text."
},
{
"type": "tool-call",
"toolCallId": "tool-123",
"toolName": "store-summary",
"toolArguments": {
"summary": "This is a summary of the text."
}
}
]
}
```
***
### Isolating Steps
Use the `isolated` attribute to prevent a step from inheriting context from previous steps. This can reduce unnecessary costs or confusion for the model.
```xml theme={null}
Summarize the following text:
{{ text1 }}
Summarize the following text:
{{ text2 }}
Compare these summaries and provide a conclusion:
{{ summary1 }}
{{ summary2 }}
```
In this example, the final step does not need to conside the full texts used in previous steps, so isolating the first two steps can help reduce context overhead, resulting in cheaper and more efficient processing.
***
### Limiting the number of steps
This feature is only available on the Latitude platform.
Latitude automatically applies a `maxSteps` limit of 20 to all prompts with configuration. This helps prevent infinite loops or excessive processing in long chains when creating complex workflows with steps within loops.
You can customize this limit by explicitly setting the `maxSteps` attribute on the main configuration section:
```xml theme={null}
---
maxSteps: 5
---
{{ for item in list }}
...
{{ endfor }}
```
Read more about this configuration in the [Latitude Prompt Configuration](/guides/prompt-manager/configuration#maxsteps) guide.
***
## Real-World Use Cases
### Multi-Step Workflow
Chains are ideal for breaking down tasks like:
1. Analyzing data.
2. Generating intermediate results.
3. Combining results for a final output.
```xml theme={null}
Analyze the following data and provide key insights:
{{ data }}
Based on the insights:
{{ analysis }}
Provide recommendations for improvement.
```
### Decision Trees
Use logic to adapt workflows based on intermediate results:
```xml theme={null}
Classify this document into one of the following categories: A, B, or C.
{{ if classification == "A" }}
Generate detailed content for Category A.
{{ else if classification == "B" }}
Generate detailed content for Category B.
{{ else }}
Generate detailed content for Category C.
{{ endif }}
```
***
## Implementation
To execute chains, use the `Chain` class. The chain evaluates the prompt step-by-step, waiting for the model's response at each step.
To run a step, execute the `step` method of the chain instance. The first time `step` is called, it should not include any arguments. Subsequent calls must always pass the model response message from the previous step.
### Example: Using the Chain Class
```javascript theme={null}
import { Chain } from 'promptl-ai';
import OpenAI from 'openai';
// Initialize the OpenAI client
const client = new OpenAI();
// Generate a response based on step messages and configuration
async function generateResponse({ config, messages }) {
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
...config,
messages,
});
return response.choices[0].message;
}
// Create a new chain
const chain = new Chain({
prompt: '...', // Your PromptL prompt as a string
parameters: {...} // Your prompt parameters
});
// Process the chain step-by-step
let result = await chain.step(); // The first step does not require any arguments
let lastResponse;
while (!result.completed) {
lastResponse = await generateResponse(result);
result = await chain.step(lastResponse); // Pass the model response to the next step
}
console.log('Final Output:', lastResponse.content);
```
***
## Debugging Chains
1. **Log Intermediate Steps**:
* Use the `raw` attribute to inspect full responses for debugging.
2. **Handle Errors Gracefully**:
* Implement fallback logic for unexpected responses or failures.
3. **Test Edge Cases**:
* Ensure your chains handle empty inputs, invalid configurations, or incomplete data.
***
## Summary
Chains and Steps provide powerful tools for breaking complex tasks into manageable parts. With features like custom configurations, variable storage, and step isolation, you can design robust, dynamic workflows tailored to any use case.
# Mocking
Source: https://docs-v1.latitude.so/promptl/advanced/mocking
Simulate assistant and tool responses in PromptL for testing and advanced development
## Overview
PromptL enables you to craft a controlled interaction history so that the model can be conditioned on arbitrary prior outputs. In advanced usage, there are two principle mechanisms of interest:
1. **Mocking Roles**\
Pretend that the assistant has already replied with specified content.
2. **Mocking Tool Calls**\
Emulate a tool invocation and its response so that PromptL sees it as if a real tool were executed.
***
## 1. Mocking Roles
Mocking a role means inserting an assistant response directly into the prompt. The model will behave as though this response actually occurred. This is useful when you want to:
* Seed the conversation with example assistant behavior
* Use few-shot prompting techniques
* Test how the model continues from a predetermined reply
### Syntax
Wrap the desired assistant reply between `` tags:
```xml theme={null}
Good morning!
Hello, I’m your weather assistant. How can I help you today?
```
Some models do not allow assistant messages as the last item.
## 2. Mocking Tool Calls
Mocking a tool call allows you to simulate both the *invocation* of an external function and its *response*—as if the model had called a real tool and received structured output. This gives you fine-grained control over how the model interprets prior tool usage and is especially useful in logic-heavy prompt flows that rely on external data.
### When to Use
* **Testing prompt branches** that depend on tool results
* **Simulating APIs** without triggering real network requests
* **Validating error-handling logic** by crafting edge-case tool responses
### Syntax
A mocked tool call consists of two components:
1. A `` element inside an `` block, which mimics the model making a function call.
2. A corresponding `` block with the same `id` and `name`, which contains the tool’s response.
```xml theme={null}
It's 17 °C in Barcelona.
```
# Prompt References
Source: https://docs-v1.latitude.so/promptl/advanced/snippets
Learn how to reference other prompts in PromptL
## Overview
Prompt references (Snippets) allow you to modularize your prompts by referencing other prompts within your project. This feature is particularly useful for:
* Managing large projects with reusable prompt components.
* Reducing duplication by reusing common sections (e.g., policies, instructions).
* Simplifying maintenance by centralizing updates to shared prompts.
***
## Syntax
To reference another prompt, use the `` tag. The `path` attribute specifies the relative or absolute path to the referenced prompt.
### Basic Usage
Referenced prompts are isolated from the parent prompt by default, meaning they don’t inherit variables. However, you can pass variables explicitly using attributes in the `` tag.
```tsx parent.promptl theme={null}
{{ user_question }}
```
```plaintext policies.promptl theme={null}
You are {{ assistant_name }}, an AI assistant created to help users.
Before answering any questions, follow these rules:
- Be respectful.
- Avoid sharing personal information.
- Use appropriate language.
```
In this example:
1. The parent prompt references `policies.promptl` and passes the `assistant_name` variable.
2. The `assistant_name` variable is interpolated in the referenced prompt.
***
## Setup
Prompt references are not enabled by default. Since PropmtL does not know how your prompts are structured, you must provide a `referenceFn` function to define how PromptL should locate and load referenced prompts.
You can structure your prompts in any way you like, as long as your `referenceFn` can find and load them. Some examples include:
* Storing prompts in a file system.
* Using a database to store prompts.
* Fetching prompts from an API.
### Basic `referenceFn`
Create a function that retrieves a prompt based on its path:
```javascript theme={null}
import fs from 'fs';
function getPrompt(path) {
return fs.readFileSync(path, 'utf8');
}
```
### Supporting Relative Paths
To resolve paths relative to the current prompt, you can define a second argument with the current prompt’s full path:
```javascript theme={null}
import path from 'path';
import fs from 'fs';
function getPrompt(relativePath, currentPath) {
const fullPath = path.resolve(path.dirname(currentPath), relativePath);
return fs.readFileSync(fullPath, 'utf8');
}
```
### Using the `referenceFn`
Pass your `referenceFn` to the `render` function:
```javascript theme={null}
import { render } from 'promptl-ai';
import getPrompt from './getPrompt'; // Import your custom resolve function
const { messages, config } = await render({
prompt: mainPrompt, // Your main PromptL prompt as a string
referenceFn: getPrompt, // Provide the resolve function
});
```
**Tip**: Prompts can be stored in files, a database, or any structured format. Adapt `referenceFn` to fit your storage solution.
***
## Real-World Examples
### Modular Prompts for Instructions
```xml main.promptl theme={null}
What can you do for me?
```
```xml instructions.promptl theme={null}
You are {{ assistant_name }}, an assistant trained to provide technical support.
Capabilities:
- Debugging code.
- Explaining concepts.
- Recommending resources.
```
***
### Nested References
Prompts can reference other prompts, enabling complex workflows.
```xml main.promptl theme={null}
```
```xml policies.promptl theme={null}
What are your rules?
```
***
## Best Practices
1. **Use Descriptive Paths**:
* Organize prompts logically (e.g., `prompts/policies.promptl`).
2. **Centralize Shared Logic**:
* Store common instructions, rules, or templates in reusable prompts.
3. **Pass Variables Explicitly**:
* Always pass required variables to avoid missing or mismatched data.
4. **Avoid Circular References**:
* Ensure prompts don’t reference each other in loops.
***
## Debugging Tips
If your prompt references aren’t working as expected:
1. **Check File Paths**:
* Ensure `path` in the `` tag matches the actual file structure.
2. **Log Resolved Prompts**:
* Add a `console.log()` in `referenceFn` to verify the correct prompt is being loaded.
3. **Handle Missing Prompts**:
* Add error handling in `referenceFn` to handle missing or unreadable prompts gracefully:
```javascript theme={null}
function getPrompt(path) {
try {
return fs.readFileSync(path, 'utf8');
} catch (error) {
console.error(`Error loading prompt: ${path}`);
throw error;
}
}
```
4. **Inspect Variables**:
* Confirm that required variables are being passed correctly.
***
## Summary
Prompt references enable modular and maintainable prompt structures by allowing you to reuse and manage shared sections across projects. With features like variable passing, nested references, and customizable resolution logic, PromptL makes it easy to handle even the largest and most complex prompt configurations.
# Introduction
Source: https://docs-v1.latitude.so/promptl/getting-started/introduction
Get started with PromptL
## What is PromptL?
[PromptL](https://promptl.ai/) is a versatile, user-friendly language that simplifies defining and managing dynamic prompts for LLMs. Whether you’re a developer or a non-technical user, PromptL offers a human-readable format that doesn’t compromise on power or flexibility.
## Why PromptL?
While LLMs are becoming more powerful and popular by the day, defining prompts for them can be a daunting task. All main LLM providers, despite their differences, have adopted a similar structure for their prompting. It consists of a conversation between the user and assistant, which is defined by a list of messages and a series of configuration options. In response, it will return an assistant message as a reply.
This structure looks something like this:
```json theme={null}
{
"model": "",
"temperature": 0.6,
"messages": [
{
"type": "system",
"content": "You are an AI assistant that writes personalized birthday messages."
},
{
"type": "user",
"content": "Write a birthday message for Tom, who loves programming!"
}
]
}
```
This structure, while straightforward, presents challenges:
* **Difficult to Write**: Non-technical users find JSON hard to write and understand.
* **Static and Rigid**: Simple structures aren't ideal for dynamic, user-driven conversations.
* **Code Overhead**: Customizing prompts requires repetitive, often messy code.
**PromptL** solves these issues by offering:
**Readable and Maintainable Syntax**: Easily write and manage prompts.
**Dynamic Flexibility**: Add dynamic variables to adapt prompts to different scenarios.
**Powerful Logic in a Single File**: Incorporate logic to handle complex workflows with ease.
Here's the same prompt using PromptL:
```markdown theme={null}
---
model:
temperature: 0.6
---
You are an AI assistant that writes personalized birthday messages.
Write a birthday message for {{ name }} who loves {{ hobby }}!
```
In this case, not only the syntax is way more readable and maintainable, but it also allows for dynamic generation of prompts by using variables like `{{ name }}` and `{{ hobby }}`!
## What’s Next?
This is just a small example of what PromptL can do. It’s a powerful tool to help you define smarter, more dynamic prompts for your LLMs. Ready to learn more? Let’s dive in!
* [Learn about PromptL syntax](/promptl/syntax/structure)
* [Try PromptL in your project](/promptl/usage/quick-start)
# Conditional Statements
Source: https://docs-v1.latitude.so/promptl/syntax/conditionals
Learn how to add content based on conditions in your prompts
## Overview
Conditional statements in PromptL enable dynamic and adaptive prompts. By incorporating logic into your prompts, you can:
* Tailor responses based on user input or context.
* Control the flow of conversations dynamically.
* Generate content conditionally for more personalized or complex interactions.
Conditionals are evaluated at runtime, ensuring that your prompts adapt seamlessly to the data provided.
***
## Syntax
Conditional blocks in PromptL use the `if`, `else`, and `endif` keywords, wrapped in `{{ }}`. The content within the block is processed only if the condition evaluates to `true`.
### Basic Syntax
```tsx theme={null}
{{ if condition }}
Content to display if the condition is true
{{ else }}
Content to display if the condition is false
{{ endif }}
```
### Example: Simple Conditional
```tsx theme={null}
{{ if age < 18 }}
The user is under the minimum required age. Respond with a kind message explaining this limitation.
{{ else }}
{{ question }}
{{ endif }}
```
***
## Advanced Usage
### Checking Variable Existence
You can use conditionals to check if a variable is defined before using it.
```plaintext theme={null}
{{ if last_name }}
{{ name = name + " " + last_name }}
{{ endif }}
Hi! My name is {{ name }}.
```
### Nested Conditions
Conditionals can be nested to handle more complex logic.
```tsx theme={null}
{{ if role == "admin" }}
The user is an admin, and has full access to all data.
{{ if feature_enabled }}
In addition, the special feature is enabled for this user.
{{ endif }}
{{ else }}
The user has a standard role and does not have access to admin features.
{{ endif }}
```
### Using Expressions in Conditions
Conditions can include complex expressions, such as combining variables or performing calculations.
```tex theme={null}
{{ if items_in_cart > 0 && user_logged_in }}
You have {{ items_in_cart }} items in your cart. Ready to checkout?
{{ else }}
Your cart is empty. Start shopping to add items!
{{ endif }}
```
***
## Best Practices
1. **Keep It Simple**: Avoid deeply nested conditionals. Break complex logic into smaller, reusable components.
2. **Define Default Values**: Ensure variables have defaults (`||`) to prevent unexpected errors.
3. **Test Edge Cases**: Check how your logic handles undefined variables or null values.
4. **Use Readable Conditions**: Use descriptive variable names and straightforward logic to improve maintainability.
* ✅ Good: `{{ if user_logged_in && has_permission }}`
* ❌ Bad: `{{ if x > 0 || y == 1 }}`
***
## Debugging Conditionals
If your conditional logic isn't behaving as expected:
* **Verify Variable Values**: Check if the variables used in your condition are defined and contain the expected data.
* **Simplify Conditions**: Break down complex expressions into smaller, testable conditions.
* **Add Debug Statements**: Temporarily output variable values for troubleshooting:
```tsx theme={null}
Debug: {{ user_logged_in && has_permission }}
```
***
## Advanced Example
Here’s a real-world example of a conditional block for a personalized travel assistant:
```tsx theme={null}
You are a travel assistant.
{{ if destination }}
Show me popular attractions in {{ destination }}.
{{ else }}
I’d like some travel recommendations.
{{ endif }}
```
***
## Summary
Conditional statements are a powerful tool in PromptL, enabling dynamic and personalized prompts that adapt to user input and context. By combining them with variables and expressions, you can build highly responsive and flexible conversations.
> Next: Learn about [Loops and Iteration](/promptl/syntax/loops) for even more dynamic capabilities.
# Configuration
Source: https://docs-v1.latitude.so/promptl/syntax/configuration
Learn how to configure your PromptL prompts
## Overview
The configuration section of a PromptL prompt is an optional yet powerful way to define how your LLM will behave. It allows you to set key parameters like the model, temperature, and other options specific to your LLM provider. This section is enclosed between three dashes (`---`) and written in YAML format:
```yaml theme={null}
---
model: gpt-4o
temperature: 0.6
top_p: 0.9
---
```
PromptL doesn't impose restrictions on what you include in the config section, so you can add any key-value pairs supported by your LLM provider.
If you're using the Latitude platform, check out the [Latitude Prompt Configuration](/guides/prompt-manager/configuration) guide for a more detailed guide on what you can include in the configuration section.
***
## Structure of the Config Section
### Key Characteristics:
1. **YAML Format**: The config section uses YAML, making it intuitive to write and easy to read.
2. **Flexibility**: Add as many or as few key-value pairs as needed.
3. **Placement**: Always appears at the top of the PromptL file.
Here’s another example with additional parameters:
```yaml theme={null}
---
model: gpt-3.5-turbo
temperature: 0.7
max_tokens: 500
stop: ["\n"]
presence_penalty: 0.2
frequency_penalty: 0.5
---
```
### Common Configuration Options:
While the specific keys you can use depend on your LLM provider, here are some commonly used options:
* `model`: Specifies the model to use (e.g., `gpt-4`, `gpt-3.5-turbo`).
* `temperature`: Controls randomness in responses (higher = more creative, lower = more deterministic).
* `top_p`: Adjusts the nucleus sampling probability for token generation.
* `max_tokens`: Limits the number of tokens in the response.
* `stop`: Defines one or more sequences where the assistant will stop generating tokens.
* `presence_penalty`: Encourages or discourages mentioning new topics.
* `frequency_penalty`: Penalizes repeated phrases for more diverse responses.
Refer to your LLM provider's documentation for a complete list of supported configuration options.
***
## Best Practices for Configurations
To make the most of your config section, consider these tips:
1. **Be Specific**: Define parameters explicitly to avoid unexpected behavior from your LLM.
* Example: Specify temperature and top\_p to control response variability.
2. **Experiment and Iterate**: LLM performance can vary based on your configuration. Adjust parameters like temperature or frequency\_penalty to fine-tune results.
3. **Reuse Configurations**: If you frequently use the same settings, consider creating reusable templates to streamline your workflow.
## Debugging Configurations
If your prompt isn't behaving as expected:
* **Check Your Parameters**: Ensure all keys and values are supported by your LLM provider.
* **Validate YAML Syntax**: Incorrect YAML formatting can cause errors.
* **Test with Minimal Configs**: Start with a simple configuration and build up to identify problematic settings.
# Loops and Iteration
Source: https://docs-v1.latitude.so/promptl/syntax/loops
Learn how to add multiple messages based on loop conditions in your prompts
## Overview
Loops in PromptL allow you to dynamically generate content or messages by iterating over lists or arrays. This is particularly useful for creating adaptive prompts based on user input or contextual data.
With loops, you can:
* Repeat sections of your prompt for each item in a list.
* Access each item's index for numbered output or additional logic.
* Handle empty lists gracefully using the `else` clause.
***
## Syntax
### Basic Loop
A loop is defined using the `for` and `endfor` keywords, wrapped in `{{ }}`. The content inside the loop is repeated for each item in the list.
```tsx theme={null}
{{ for item in list }}
- {{ item }}
{{ endfor }}
```
### Loop with Index
You can include an `index` parameter to track the iteration count, starting from `0`.
```tsx theme={null}
{{ for item, index in list }}
{{ index }}: {{ item }}
{{ endfor }}
```
### Loop with `else`
The `else` clause runs when the list is empty. Place it before the `endfor` keyword.
```tsx theme={null}
{{ for item in list }}
- {{ item }}
{{ else }}
No items to display.
{{ endfor }}
```
***
## Examples
### Basic Example: Listing Items
```tsx theme={null}
{{ for fruit in ["apple", "banana", "cherry"] }}
- {{ fruit }}
{{ endfor }}
```
**Output**:
```
- apple
- banana
- cherry
```
### Example with Index
```tsx theme={null}
{{ for fruit, index in ["apple", "banana", "cherry"] }}
{{ index + 1 }}. {{ fruit }}
{{ endfor }}
```
**Output**:
```
1. apple
2. banana
3. cherry
```
### Handling Empty Lists
```tsx theme={null}
{{ for item in [] }}
- {{ item }}
{{ else }}
The list is empty. No items to display.
{{ endfor }}
```
**Output**:
```
The list is empty. No items to display.
```
***
## Advanced Usage
### Iterating Over Objects
Loops can handle more complex data structures, such as arrays of objects.
```tsx theme={null}
{{ for user in users }}
- Name: {{ user.name }}, Age: {{ user.age }}
{{ endfor }}
```
For the input:
```tsx theme={null}
users = [
{ name: "Alice", age: 30 },
{ name: "Bob", age: 25 }
]
```
**Output**:
```
- Name: Alice, Age: 30
- Name: Bob, Age: 25
```
***
### Nested Loops
You can use nested loops for iterating over multi-dimensional data.
```tsx theme={null}
{{ for group, groupIndex in groups }}
Group {{ groupIndex + 1 }}:
{{ for member in group.members }}
- {{ member }}
{{ endfor }}
{{ endfor }}
```
For the input:
```tsx theme={null}
groups = [
{ members: ["Alice", "Bob"] },
{ members: ["Charlie", "Dana"] }
]
```
**Output**:
```
Group 1:
- Alice
- Bob
Group 2:
- Charlie
- Dana
```
***
## Best Practices
1. **Keep Loops Simple**:
* Avoid deeply nested loops unless necessary. Complex loops can make your prompts harder to read and maintain.
2. **Use Default Values**:
* Provide defaults for variables to prevent errors when lists are empty or data is incomplete.
* Example: `{{ item || "Unknown" }}`
3. **Combine with Conditionals**:
* Use `if` statements inside loops for conditional logic.
```tsx theme={null}
{{ for user in users }}
{{ if user.active }}
- {{ user.name }} (Active)
{{ else }}
- {{ user.name }} (Inactive)
{{ endif }}
{{ endfor }}
```
4. **Debugging**:
* Temporarily output the list and its elements to ensure the loop is iterating as expected.
***
## Debugging Tips
If your loop isn’t working as expected:
* **Verify Data**: Print the list you’re iterating over to ensure it contains the expected data.
* **Check Syntax**: Ensure `endfor` is present and properly matched with `for`.
* **Use `else` for Debugging**: Add an `else` clause to confirm whether the list is empty.
```tsx theme={null}
{{ for item in list }}
- {{ item }}
{{ else }}
Debug: The list is empty or not defined.
{{ endfor }}
```
***
## Summary
Loops in PromptL enable you to iterate over lists dynamically, generate repeated content, and handle complex data structures. By combining loops with variables and conditionals, you can create powerful, adaptive prompts tailored to any use case.
> Ready for more dynamic control? Explore [Conditional Statements](/promptl/syntax/conditionals) to complement your loops.
# Messages
Source: https://docs-v1.latitude.so/promptl/syntax/messages
Learn how to define messages in PromptL
## Overview
Messages are the building blocks of PromptL prompts. They define the flow of conversations between the user, the assistant, and other entities like tools. Each message is associated with a **role** that determines its purpose in the conversation.
PromptL supports the following message roles:
* **System**: Sets the context and rules for the assistant.
* **User**: Represents user input in the conversation.
* **Assistant**: Captures assistant responses or provides context for the LLM's output.
* **Tool**: Represents interactions with external tools or APIs.
***
## Roles
### System Messages
System messages provide high-level instructions and context for the assistant.
```xml theme={null}
You are a personal finance advisor. Provide actionable insights in a friendly tone.
```
### User Messages
User messages simulate user input in the conversation.
```xml theme={null}
How can I save more money each month?
```
***
## Tags
### The `` Tag
Messages can be defined using the `` tag with a `role` attribute:
```xml theme={null}
This is a system message.
```
### Shortcut Tags
For convenience, you can use specific tags for each role:
* ``: Equivalent to ``.
* ``: Equivalent to ``.
* ``: Equivalent to ``.
* ``: Equivalent to ``.
```xml theme={null}
You are a friendly chatbot.
What’s the weather today in Barcelona?
Let me check for you!
23ºC, sunny.
```
***
## Content Types
Messages can contain different types of content. By default, all plain text is considered `text` content, but you can specify other types using the `` tag or its shorthand equivalents.
* **Text (default)**: `` or ``.
* **Image**: `` or ``. Add the image URL or base64-encoded string – depending on your provider's requirements – as the content inside the tag.
* **File**: `` or ``. Add the file URL or base64-encoded string – depending on your provider's requirements – as the content inside the tag. Requires a MIME type `mime` attribute to specify the file type.
* **Tool Call**: `` or ``. Represents a tool invocation with attributes like `id`, `name`, and `arguments` (optional). Only allowed inside assistant messages.
Not all providers and all models will support all content types. Your provider may support files but not all types of files. Check your provider's documentation for compatibility.
### Examples
```tsx theme={null}
Take a look at this image:
[image url]
And this PDF:
[file url]
```
***
## Best Practices
* **Use System Messages for Context**: Define clear rules for the assistant to guide its behavior.
* **Leverage User Messages for Dynamic Input**: Use uncontrolled user variables inside user messages to define a clear difference authority between the user and the system rules.
***
## Advanced Examples
Here’s how multiple message roles and content types can work together:
```tsx theme={null}
You are a helpful assistant for travel planning.
Find me a good restaurant in {{ city }}.
Let me check for you.
The best-rated restaurant in {{ city }} is Gourmet Central.
```
# Structure
Source: https://docs-v1.latitude.so/promptl/syntax/structure
Learn about the general structure of a PromptL prompt
## Overview
The structure of a PromptL prompt is designed for clarity and flexibility, making it easy to define both global configurations and the messages that drive your LLM conversations. A PromptL prompt is divided into two main sections:
1. The **Config Section**, where you define global options for the prompt.
2. The **Messages**, which define the conversational flow between the user, assistant, and other roles.
***
## Config Section
The config section is an optional part of a PromptL prompt, defined at the very beginning. It allows you to specify global settings for your LLM, such as the `model`, `temperature`, or any other configuration supported by your provider.
This section is enclosed between triple dashes (`---`) and uses YAML format for key-value pairs:
```yaml theme={null}
---
model: gpt-4o
temperature: 0.6
top_p: 0.9
---
```
PromptL will not modify the config section, ensuring full compatibility with any LLM provider. This flexibility allows you to include custom configurations tailored to your needs.
***
## Messages
The messages section defines the conversational flow of your prompt. Messages are structured in a chat-based format and can represent one of the following types:
* **System**: Sets the context or rules for the conversation.
* **User**: Represents messages from the user to the assistant.
* **Assistant**: Represents messages from the assistant to the user.
* **Tool**: Used for interactions with external tools or APIs.
[Learn more about the configuration section](/promptl/syntax/config).
```xml theme={null}
You are a playful AI assistant that knows a lot about animals. Respond to the user's questions with fun facts about animals.
What is the largest mammal in the world?
```
In the example above:
* The first line is a system message that establishes the assistant's behavior.
* The `` block defines a user message.
[Learn more about Messages](/promptl/syntax/messages).
***
## Bringing it together
This two-part structure —**Config** for global settings and **Messages** for conversational flow— keeps your prompts organized and easy to understand. Whether you're crafting simple interactions or complex, dynamic workflows, PromptL's structure makes it straightforward.
# Variables
Source: https://docs-v1.latitude.so/promptl/syntax/variables
Learn how to define, interpret, and interpolate variables in PromptL
## Overview
Variables in PromptL allow you to store and reuse dynamic values across your prompt. They make your prompts more maintainable and adaptable by eliminating repetitive text and enabling dynamic content generation.
Variables are defined using double curly braces (`{{ }}`) and can be used directly in text or as part of logic expressions.
***
## Defining Variables
You can define a variable by assigning a value inside `{{ }}`:
```tsx theme={null}
{{ name = 'Alice' }}
Hi! My name is {{ name }}.
```
In this example:
* The variable `name` is assigned the value `"Alice"`.
* The variable is interpolated into the text when the prompt is processed.
***
## Interpolating Variables
Variables can be used anywhere in your prompt. When interpolated, their value replaces the placeholder in the text.
```tsx theme={null}
{{ city = "Barcelona" }}
Welcome to {{ city }}! Let me guide you through the best attractions here.
```
***
## Input Parameters
Variables don’t have to be defined in the prompt. Instead, their values can be provided as dynamic input parameters when the prompt is executed.
```tsx theme={null}
Hi! My name is {{ name }}.
```
In this example, the `name` variable can be defined in the code when running this prompt, allowing you to dynamically customize the output without modifying the prompt.
***
## Built-in Variables
PromptL provides special built-in variables that are automatically available in your prompts without needing to be defined.
### Current Date and Time
The `$now` variable returns the current date and time in ISO format:
```tsx theme={null}
Today is {{ $now }}.
```
This will output something like: `Today is 2024-01-15T10:30:45.123Z.`
You can also use `$now` in expressions:
```tsx theme={null}
{{ timestamp = $now.getTime() }}
The current timestamp is {{ timestamp }}.
```
`$now` is an instance of Javascript Date class, and you can call any of its
methods and properties.
***
## Default Values
You can define default values for variables using the `||` operator. If the variable is not defined in the prompt or provided as an input parameter, the default value will be used.
```tsx theme={null}
Hi! My name is {{ name || "Alice" }}.
```
Here, the `name` variable will default to `"Alice"` if no value is provided.
**Tip**: Use default values to ensure your prompts remain functional even if
some parameters are missing.
***
## Expressions
PromptL supports logic expressions, enabling you to perform calculations or transformations directly in the prompt.
### Simple Expressions
```tsx theme={null}
{{ age = 30 }}
{{ ageInMonths = age * 12 }}
I am {{ age }} years old, which is {{ ageInMonths }} months.
```
### Conditional Expressions
You can use ternary-like conditions to modify variable values dynamically:
```tsx theme={null}
{{ isAdmin = role == "admin" ? "Yes" : "No" }}
Is this user an admin? {{ isAdmin }}.
```
***
## Advanced Examples
### Combining Input Parameters, Defaults, and Expressions
```tsx theme={null}
{{ city = input.city || "Unknown" }}
{{ weather = input.weather || "clear" }}
The weather in {{ city }} is currently {{ weather }}.
```
### Nested or Structured Variables
Variables can represent objects or arrays, making it easy to handle structured data:
```tsx theme={null}
{{ user = { name: "Alice", age: 30, hobbies: ["reading", "cycling"] } }}
Hello, {{ user.name }}! You are {{ user.age }} years old and enjoy {{ user.hobbies[0] }}.
```
***
## Best Practices
* **Use Clear Variable Names**: Choose descriptive names to make your prompts easy to understand.
* Good: `user.name`, `weather.currentTemp`
* Bad: `x`, `temp`
* **Avoid Redefinition**: Define variables once and reuse them instead of redefining them unnecessarily.
* **Default Values for Robustness**: Use default values to handle missing inputs gracefully.
* **Combine with Logic**: Use expressions to preprocess data directly in the prompt, reducing complexity elsewhere.
***
## Debugging Variables
If a variable isn’t working as expected:
* **Check Scope**: Ensure the variable is defined or provided as an input parameter.
* **Validate Expressions**: Double-check calculations or transformations.
* **Use Defaults**: Add default values to catch undefined variables.
***
## Summary
Variables in PromptL enable dynamic and reusable prompts. By combining definitions, input parameters, defaults, and expressions, you can create highly flexible and maintainable conversations tailored to any use case.
# Anthropic
Source: https://docs-v1.latitude.so/promptl/usage/adapters/anthropic
Learn how to use PromptL with Anthropic
## Overview
PromptL integrates seamlessly with Anthropic’s API by using the `Anthropic` adapter. This ensures prompts are formatted correctly for their API and Node.js SDK, allowing you to generate dynamic prompts with ease.
**System Message Limitations**: Anthropic does not support system messages in the messages array. Instead:
* PropmtL will automatically move the first system message to the `config` object.
* System messages are not allowed after messages from other roles, and it will throw an error.
* Since system messages are moved to the configuration section, Anthropic will fail if there are no other user or assistant messages in the conversation.
***
## Basic Example
Here’s how to use PromptL with Anthropic’s API:
```typescript theme={null}
import Anthropic from '@anthropic-ai/sdk'
import { Adapters, render } from 'promptl-ai'
const prompt = `
---
model: claude-3-opus-20240229
max_tokens: 1024
---
Generate a joke about {{ topic }}.
`
const { messages, config } = await render({
prompt,
parameters: { topic: 'chickens' },
adapter: Adapters.anthropic, // Specify the Anthropic adapter
})
const client = new Anthropic({ apiKey: YOUR_ANTHROPIC_API_KEY })
const response = await client.messages.create({
...config,
messages,
})
console.log(response.content[0].text)
```
***
## Key Features
1. **Adapter-Specific Behavior**:
* System messages are extracted and placed in the `config` object.
* The `messages` array must contain non-system messages.
2. **Formatting**: PromptL formats messages in the format expected by Anthropic, ensuring compatibility.
3. **Support for Claude Models**: Works seamlessly with Anthropic’s Claude family of models.
***
## Troubleshooting
1. **Empty Messages Array**:
* Ensure your prompt contains non-system messages. Anthropic’s API does not allow an empty `messages` array.
2. **Check Configuration**:
* Anthropic always requires to define at least a `model` and `max_tokens` configuration.
3. **Error Handling**:
```typescript theme={null}
try {
const response = await client.messages.create({
...config,
messages,
})
console.log(response.content[0].text)
} catch (error) {
console.error('Error with Anthropic API:', error)
}
```
***
## Next Steps
* [Learn More About Anthropic’s API](https://docs.anthropic.com/en/api/messages)
* Explore advanced PromptL features:
* [Chains and Steps](syntax/chains)
* [Prompt References](syntax/prompt-references)
# OpenAI
Source: https://docs-v1.latitude.so/promptl/usage/adapters/openai
Learn how to use PromptL with OpenAI
## Overview
PromptL seamlessly integrates with OpenAI's API. By default, PromptL formats prompts in the structure required by OpenAI, so you can use the output directly without additional processing.
***
## Basic Example
Here’s how to generate a response from OpenAI using PromptL:
```typescript theme={null}
import { render } from 'promptl-ai'
import OpenAI from 'openai'
const prompt = `
---
model: gpt-4o
temperature: 0.6
---
Generate a joke about {{ topic }}.
`
const { messages, config } = await render({
prompt,
parameters: { topic: 'chickens' },
})
const client = new OpenAI({ apiKey: YOUR_OPENAI_API_KEY })
const response = await client.chat.completions.create({
...config,
messages,
})
console.log(response.choices[0].message.content)
```
***
## Key Features
1. **Default Adapter**: PromptL automatically uses the OpenAI adapter for correct formatting.
2. **Role-Based Messages**: OpenAI expects a `role` field (`system`, `user`, `assistant`) in messages, which PromptL handles for you.
3. **Configuration Pass-Through**: Configuration options (e.g., `temperature`, `model`) are passed directly to OpenAI’s API.
***
## Error Handling
When working with OpenAI, ensure you handle potential API errors gracefully:
```typescript theme={null}
try {
const response = await client.chat.completions.create({
...config,
messages,
})
console.log(response.choices[0].message.content)
} catch (error) {
console.error('Error with OpenAI API:', error)
}
```
***
## Next Steps
* [Learn More About OpenAI’s API](https://platform.openai.com/docs/api-reference/chat)
* Explore advanced PromptL features:
* [Chains and Steps](syntax/chains)
* [Prompt References](syntax/prompt-references)
# Use PromptL with your Provider
Source: https://docs-v1.latitude.so/promptl/usage/adapters/overview
Learn how to use PromptL with different LLM providers
## Overview
While most LLM providers follow a similar chat-like structure, there are subtle differences in how prompts are formatted and processed. PromptL addresses these differences by providing **Adapters** for each major provider, ensuring that your prompts are correctly formatted and seamlessly integrated.
Currently, PromptL supports:
* [OpenAI](/promptl/usage/adapters/openai)
* [Anthropic](/promptl/usage/adapters/anthropic)
More providers will be supported in the future, and you can even create your own custom adapters for unsupported platforms.
***
## Why Adapters?
Adapters handle provider-specific definition differences, such as:
* Message structure: OpenAI uses `role`-based messages, while Anthropic uses a `user/assistant` prefix.
* API integration: Adapters ensure compatibility with the provider’s API.
By using an adapter, you don’t need to worry about these differences—PromptL handles them for you.
***
## Getting Started with Adapters
Here’s how to use an adapter in your project. For this example, we’ll use OpenAI:
```javascript theme={null}
import { render, Adapter } from 'promptl-ai';
const { messages, config } = await render({
prompt,
adapter: Adapters.openai, // Specify the adapter
});
```
***
## Supported Providers
### OpenAI (default)
The OpenAI Adapter, which is selected by default, formats prompts to match OpenAI’s chat-completion API, including support for models like `gpt-4` and `gpt-3.5`.
* [Learn more about the OpenAI Adapter](/promptl/usage/adapters/openai)
### Anthropic
The Anthropic Adapter ensures compatibility with Anthropic’s API.
* [Learn more about the Anthropic Adapter](/promptl/usage/adapters/anthropic)
Additional providers will be supported in the future. Check back for updates!
***
## Extending Adapters
If you’re working with an unsupported provider, you can create your own adapter. Adapters are simple functions that transform PromptL’s `messages` and `config` into the format required by your provider.
An adapter is defined as an object with two functions: `{ fromPromptl, toPromptl }`.
Each function takes an object with `messages` and `config` properties and returns the same object with transformed data.
To see the structure of `messages` used in PromptL, check out the [GitHub PromptL Repository](https://github.com/latitude-dev/promptl/blob/main/src/types/message.ts)
### Example: Custom Adapter
```javascript theme={null}
const CustomAdapter = {
fromPromptl: ({ messages, config }) => {
// Transform PromptL messages to your provider's format
const formattedMessages = messages.map((msg) => ({
role: msg.type,
content: msg.text,
}));
return { messages: formattedMessages, config };
},
toPromptl: ({ messages, config }) => {
// Transform your provider's messages to PromptL format
const formattedMessages = messages.map((msg) => ({
type: msg.role,
text: msg.content,
}));
return { messages: formattedMessages, config };
},
};
```
Pass your custom adapter to the `render` function:
```javascript theme={null}
const { messages, config } = await render({
prompt,
adapter: CustomAdapter,
});
```
***
## Contribute or Request Support
We’re constantly working to support more providers. If you’d like to request a specific provider or contribute an adapter, check out our [GitHub repository](https://github.com/latitude-dev/promptl).
***
## Summary
Adapters make it easy to use PromptL with different LLM providers by handling provider-specific formatting and configuration. Whether you’re using OpenAI, Anthropic, or another platform, PromptL ensures seamless integration. Get started with a supported adapter or build your own for maximum flexibility.
# Quick Start
Source: https://docs-v1.latitude.so/promptl/usage/quick-start
Learn how to install and use PromptL in your project
## Overview
PromptL simplifies the process of creating and managing prompts for large language models (LLMs). This quick start guide will show you how to set up PromptL in your project and generate dynamic prompts with minimal effort.
> **Prerequisites**: Ensure you have Node.js installed and access to an LLM provider like OpenAI or Anthropic.
***
## Installation
Install PromptL via npm:
```bash theme={null}
$ npm install promptl-ai
```
You’ll also need the library for your LLM provider.
```bash OpenAI theme={null}
$ npm install openai
```
```bash Anthropic theme={null}
$ npm install @anthropic-ai/sdk
```
***
## Basic Usage
Here’s how to use PromptL to generate a dynamic prompt and interact with an LLM:
Different providers will require a different setup and structure. Check out the [Adapters](/promptl/usage/adapters/overview) section for more information on how to integrate with your provider.
### Example Code
```javascript theme={null}
import { render } from 'promptl-ai'
import OpenAI from 'openai'
// Define your PromptL prompt
const prompt = `
---
model: gpt-4o
temperature: 0.6
---
Generate a joke about {{ topic }}.
`
// Render the prompt with dynamic parameters
const { messages, config } = await render({
prompt,
parameters: { topic: 'chickens' },
})
// Initialize your LLM client
const client = new OpenAI()
const response = await client.chat.completions.create({
...config,
messages,
})
// Output the response
console.log(response.choices[0].message.content)
```
### How It Works:
1. **Prompt Definition**: The `prompt` variable defines the PromptL prompt, including configuration and template syntax.
2. **Dynamic Parameters**: The `parameters` object passes the value `topic: 'chickens'` to replace `{{ topic }}` in the prompt.
3. **Rendering**: The `render` function processes the prompt and generates the `messages` array and `config` object for your LLM provider.
4. **LLM Interaction**: The OpenAI client sends the `messages` and `config` to the model, generating a response.
***
## Advanced Example: Error Handling
For production environments, add error handling to manage unexpected issues:
```javascript theme={null}
import { render } from 'promptl-ai'
import OpenAI from 'openai'
async function generateResponse(prompt, parameters) {
try {
const { messages, config } = await render({ prompt, parameters })
const client = new OpenAI()
const response = await client.chat.completions.create({
...config,
messages,
})
return response.choices[0].message.content
} catch (error) {
console.error('Error generating response:', error)
return 'An error occurred while generating the response.'
}
}
// Example usage
const prompt = `
---
model: gpt-4o
temperature: 0.6
---
Generate a joke about {{ topic }}.
`
const joke = await generateResponse(prompt, { topic: 'chickens' })
console.log(joke)
```
***
## Next Steps
Once you’ve set up PromptL, explore its advanced features:
* [Syntax and Variables](syntax/variables): Learn how to define dynamic variables in your prompts.
* [Chains and Steps](syntax/chains): Break down complex tasks into multi-step prompts.
* [Prompt References](syntax/prompt-references): Reuse common prompt components across your project.
***
## Summary
PromptL makes it easy to create and manage dynamic prompts for LLMs. By following this guide, you’ve set up PromptL, generated a dynamic prompt, and integrated it with an LLM provider. Now, you’re ready to explore its full potential.