Ask

/query finds segments; /ask answers questions, which usually needs more than segments.

application/json.

Body

When analyzer is unset, core picks the first present on the video, in the order default_video, people, diarization, transcript, object_detection, ocr.

Example

curl -X POST http://127.0.0.1:8077/ask \
  -H "X-Core-Token: $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What did the woman in the light grey shirt do?",
    "video_ids": ["6b9ca6e1a1b2c3d4"]
  }'

{
  "question": "What did the woman in the light grey shirt do?",
  "answer": "She worked the right-side checkout register for most of the recording, scanning items and handling payments [6b9ca6e1a1b2c3d4 0-24].",
  "sources": {
    "6b9ca6e1a1b2c3d4": { "routed_to": ["entities"], "entities": 4, "sections": 4, "chunks": 6 }
  },
  "results": [ { "…": "supporting segments, shaped as SearchResult at detail=standard" } ]
}

Routing

The question is embedded against a description of what each aggregate answers, and the ones that stand out are pulled into context.

"what did the woman in the light gray shirt do"  ->  entities
"which segments are unusual"                     ->  novelty
"how busy was the store"                         ->  stats
"what brands are visible"                        ->  ner
"who was with the man in yellow shorts"          ->  cooccurrence, entities

What goes into the answer

Context is assembled from four sources, per video:

1. The routed aggregates — the cross-segment conclusions that retrieval structurally cannot recover.
2. The video summary.
3. The summary sections nearest the question.
4. The retrieved segments — limit of them, from the chosen analyzer.

Answers cite [video_id start-end]. sources reports what was consulted per video, including which aggregates it routed to and how many of each element were used — worth surfacing, because it explains why an answer knows something no single segment says.

Why this over /query

A search returns the five moments most similar to a phrase. That is the right tool for "find the part where…" and the wrong one for "how busy was it" — which needs every chunk counted, not the five that best match the word "busy".

Aggregates already contain that reasoning. Routing to them is cheaper and more accurate than retrieving ten moments and asking a model to reason over them.

Empty cases

{ "question": "…", "answer": null, "sources": {}, "results": [], "error": "No videos ingested." }

Both come back as 200 with answer: null — they are answers about the corpus, not request failures.

Errors