Search by text, image, multimodal query, filters, multi-run scope, SQL, and agentic chat

Text search

Use this guide when implementing video vector embedding search, hybrid vector and metadata search, SQL media search, or agentic media search on top of the same indexed media and prompt-run outputs.

Use text search first when the user can describe the desired content in natural language.

results = client.search.text(
    index_id="idx_archive",
    query="reporter speaking outside a station entrance",
    top_k=10,
    search_fields=["summary", "scene.location"],
)

Image and multimodal search

Use image or multimodal search when the user has a visual reference.

results = client.search.multimodal(
    index_id="idx_archive",
    text_query="red emergency vehicle",
    image_data=encoded_image,
    text_weight=0.7,
    image_weight=0.3,
    top_k=20,
)

Filter search with nested fields

Use filter search when the workflow needs exact or constrained matching.

{
  "conditions": [
    { "field": "scene.people[].emotion", "operator": "equals", "value": "happy", "type": "string" },
    { "field": "scene.people[].confidence", "operator": "greater_equal", "value": 0.8, "type": "number" }
  ],
  "page_size": 50
}

That pattern relies on canonical dotted field paths and [] for repeated objects.

Search by selected runs

When an index contains several prompt runs, pass run_ids deliberately instead of assuming the default search scope matches the workflow you need.

Use SQL search for analyst workflows

Use SQL search when the operator needs:

• selected columns
• aggregations
• analyst-style filtering
• repeatable saved-query behavior

Typical flow:

1. Get the SQL catalog for an index.
2. Inspect the available run-backed tables and columns.
3. Execute SQL or generate a draft query from natural language.

curl -X POST https://playground-api-stg-udk7d32fva-uc.a.run.app/api/v2/search/sql/idx_archive/catalog \
  -H "Authorization: Bearer <token-or-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"run_ids":["run_123"]}'

Use chat sessions for agentic search

Use chat sessions when the operator needs a search assistant rather than one fixed retrieval request.

Typical fit:

• an analyst wants to ask follow-up questions over the same search scope
• the client needs scoped conversational retrieval across specific indexes or prompt runs
• the product needs streaming assistant responses and tool-trace visibility

curl -X POST https://playground-api-stg-udk7d32fva-uc.a.run.app/api/v2/chat/sessions/session_123/turns/stream \
  -H "X-API-Key: sk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Find scenes where the same reporter appears outside multiple station entrances and summarize the differences.",
    "scope": {
      "index_ids": ["idx_archive"],
      "prompt_run_ids": ["run_people_v2", "run_locations_v1"]
    },
    "idempotency_key": "agent-search-session-123-turn-4"
  }'