> ## Documentation Index
> Fetch the complete documentation index at: https://docs-v1.latitude.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Annotate log (HITL)

> Learn how to annotate log data with the Latitude SDK to perform HITL evaluations

This guide explains how to perform Human-in-the-Loop (HITL) evaluations of your prompt’s performance.

## Prompt

In this example, we have a simple prompt that asks the LLM to generate a joke. We want users to be able to evaluate the quality of the joke and provide feedback.

<CodeGroup>
  ```markdown example theme={null}
  ---
  provider: Latitude
  model: gpt-4o-mini
  temperature: 0.7
  ---

  Please tell me a joke about cats.
  ```
</CodeGroup>

## How does this work?

In this scenario, we use OpenAI’s API directly to run the prompt defined in Latitude. We retrieve the prompt using Latitude’s SDK, display the messages, and send them to the OpenAI API.

Once the model completes its response, we upload a log to our prompt in Latitude, which you can view in the prompt’s logs section.

Finally, we annotate the log with the feedback received from the user. In this case, the user rates the joke on a scale from 1 to 5 and provides a reason for their rating.

You can learn more about [HITL (Human-in-the-Loop) evaluations in our documentation](/guides/evaluations/humans-in-the-loop).

### Code examples

<CodeGroup>
  ```typescript Typescript theme={null}
  import { Latitude, Adapters } from '@latitude-data/sdk'
  import OpenAI from 'openai'

  // To run this example you need to create a evaluation on the prompt: `annontate-log/example`
  // Info: https://docs.latitude.so/guides/evaluations/overview
  const EVALUATION_UUID = 'YOUR_EVALUATION_UUID'

  async function run() {
    const sdk = new Latitude(process.env.LATITUDE_API_KEY, {
      projectId: Number(process.env.PROJECT_ID),
      versionUuid: 'live',
    })
    const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

    // Get the prompt from Latitude
    const prompt = await sdk.prompts.get('annotate-log/example')

    // Generate messages from the Latitude prompt
    // These messages are valid OpenAI messages. Note that we passed the Adapters.openai
    const { config, messages } = await sdk.prompts.render({
      prompt: { content: prompt.content },
      parameters: {},
      adapter: Adapters.openai,
    })

    // Call OpenAI
    const llmResponse = await openai.chat.completions.create({
      // @ts-ignore
      messages,
      model: config.model as string,
    })

    const { uuid } = await sdk.logs.create('annotate-log/example', messages, {
      response: llmResponse.choices[0].message.content,
    })

    // Score from 1 to 5 because the evaluation we created is of type `
    // More info: https://docs.latitude.so/guides/evaluations/humans-in-the-loop
    const result = await sdk.evaluations.annotate(uuid, 5, EVALUATION_UUID, {
      reason: 'This is a good joke!',
    })

    console.log('Result:', JSON.stringify(result, null, 2))
  }

  run()
  ```

  ```python Python theme={null}
  import asyncio
  import os

  from devtools import pprint
  from latitude_sdk import (
      AnnotateEvaluationOptions,
      CreateLogOptions,
      RenderPromptOptions,
      Latitude,
      LatitudeOptions,
  )
  from openai import AsyncOpenAI
  from promptl_ai import Adapter


  # To run this example you need to create a evaluation on the prompt: `annontate-log/example`
  # Info: https://docs.latitude.so/guides/evaluations/overview
  EVALUATION_UUID = "YOUR_EVALUATION_UUID"


  async def run():
      api_key = os.getenv("LATITUDE_API_KEY")
      sdk_options = LatitudeOptions(
          project_id=int(os.getenv("PROJECT_ID")),
          version_uuid="live",
      )
      sdk = Latitude(api_key, sdk_options)
      openai = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))

      # Get the prompt from Latitude
      prompt = await sdk.prompts.get("annotate-log/example")

      # Render the messages from the Latitude prompt
      render = await sdk.prompts.render(prompt.content, RenderPromptOptions(adapter=Adapter.OpenAI))

      # Call OpenAI with the messages from the prompt
      llm_result = await openai.chat.completions.create(
          model=render.config["model"],
          temperature=render.config["temperature"],
          messages=[message.model_dump() for message in render.messages],
      )

      llm_response = llm_result.choices[0].message.content
      latitude_render = await sdk.prompts.render(
          prompt.content,
          RenderPromptOptions(
              adapter=Adapter.Default,
          ),
      )

      log_result = await sdk.logs.create(
          "annotate-log/example",
          latitude_render.messages,
          CreateLogOptions(response=llm_response),
      )

      result = await sdk.evaluations.annotate(
          log_result.uuid, 1, EVALUATION_UUID, AnnotateEvaluationOptions(reason="This is a bad joke!")
      )

      pprint(result)


  asyncio.run(run())
  ```
</CodeGroup>