Quickstart

Start core

cd core

pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
    --index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt

Create core/.env:

OPENAI_API_KEY=sk-...
HF_TOKEN=hf_...                       # diarization only
SUPABASE_URL=https://xxx.supabase.co
SUPABASE_SERVICE_ROLE_KEY=eyJ...

Create the public storage bucket once per project:

from supabase import create_client
create_client(URL, SERVICE_ROLE_KEY).storage.create_bucket("videos", options={"public": True})

Then run it:

python serve.py               # http://127.0.0.1:8077

Verify with GET /health — status is ok and storage.ok is true when the bucket is reachable. Interactive API docs live at /docs.

Set up the database

Run frontend/lib/supabase/migrations/schema.sql in the Supabase SQL editor. It creates projects, conversations, messages, video_core (the application's view of a video analysed by core), the project-assets bucket, and the RLS policies.

Safe to re-run: everything is IF NOT EXISTS / DROP … IF EXISTS.

Start the frontend

cd frontend
npm install
cp env.example .env.local

Fill in .env.local:

NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
NEXT_PUBLIC_SUPABASE_ADMIN=eyJ...          # service-role key

OPENAI_API_KEY=sk-...                      # plus any other provider you want in the model picker

CORE_API_URL=http://127.0.0.1:8077
CORE_API_TOKEN=                            # must match VIDEOMIND_API_TOKEN in core/.env

npm run dev                                # http://localhost:3000

Create a project and upload a video

Sign in, go to /projects, and create one. In the workspace, click Upload and either drop a file in or paste a direct video URL.

The upload dialog asks two things that matter:

• Which analyzers to run. default_video (scene description) plus diarization (speech with speakers) is a good default. See Analyzers.
• How to chunk. Leave it on the Audio + video preset unless you know your footage is carried by one signal. See Chunking.

Wait for indexing

Each card shows a status pill moving through Queued → Analyzing → Ready. The grid polls, and each poll reconciles the row against core's job.

Indexing is genuinely slow — a five-minute video needs roughly 110 seconds for the boundary detectors alone, before a single vision call. Only Ready videos are searchable.

Ask something

Type into the prompt box, or click a suggestion card. Use the Videos control next to the model selector to tag exactly which videos to search; tagged videos appear as chips above the input.

Try:

• "What happens in this video?" — the agent calls ask_video
• "Find the moment where someone approaches the counter" — search_moments
• "Show me clips of that" — show_clips, which opens the artifact panel

Clips open in the right-hand panel: a player on top, the clip list below. Click any row to jump to that moment; Play all plays them back to back.