@miraiclip/assistant

npm install @miraiclip/assistant

FunctionsCopy Anchor Link

applyCheckedCopy Anchor Link

Apply a batch as one all-or-nothing step, with two checks core doesn't do: fields a command would silently ignore are errors, and commands that changed nothing are reported, so the model never claims an edit that didn't happen.

function applyChecked(project: Project, commands: Command[]): ToolOutput

Guides: Editing assistant

applyCommandsSchemaCopy Anchor Link

apply_commands' input schema: type is an enum of the catalog's commands.

function applyCommandsSchema(catalog: Record<string, unknown>): Record<string, unknown>

Guides: Editing assistant

commitForkCopy Anchor Link

Replay a fork's commands onto the real project as one undoable step.

function commitFork(project: Project, fork: RecordingProject, label?: string): CommitResult

Guides: Editing assistant

createAssistantCopy Anchor Link

function createAssistant(options: AssistantOptions): Assistant

Guides: Editing assistant

createChatHandlerCopy Anchor Link

function createChatHandler(model: ChatModel, options?: ChatHandlerOptions): (request: Request) => Promise<Response | null>

Guides: Editing assistant

defineChatModelCopy Anchor Link

Check an adapter's shape once, at registration.

function defineChatModel<M extends ChatModel>(model: M): M

Guides: Editing assistant

defineToolCopy Anchor Link

function defineTool(tool: AssistantTool): AssistantTool

Guides: Editing assistant

describeCommandsCopy Anchor Link

One line per command, for a "what changed" list (keyframe edits fold into one line per clip).

function describeCommands(commands: readonly Command[]): string[]

Guides: Editing assistant

describeForAssistantCopy Anchor Link

The project as the model sees it: core's describeProject, plus what a model needs to make RELATIVE edits ("bigger", "louder", "keep the entrance") — track names, every clip's position, size and opacity, text styling, volume, effects and its animation recipe.

function describeForAssistant(doc: ProjectDocument): string

Guides: Editing assistant

editorToolsCopy Anchor Link

The tools that edit a Miraiclip project: read state, look up a command's schema, apply command batches, and two high-level tools (transitions, animation presets) that use the same logic as the editor's panels.

function editorTools(options?: EditorToolsOptions): AssistantTool[]

Guides: Editing assistant

forkProjectCopy Anchor Link

function forkProject(project: Project, options?: ForkOptions): RecordingProject

Guides: Editing assistant

openAIChatModelCopy Anchor Link

function openAIChatModel(options: OpenAIChatModelOptions): ChatModel

Guides: Editing assistant

parseToolArgumentsCopy Anchor Link

Parse a tool call's JSON arguments; unparseable JSON is reported to the model as a tool error.

function parseToolArguments(raw: string): Record<string, unknown>

Guides: Editing assistant

rateLimitedChatModelCopy Anchor Link

Wrap a model so calls wait for room under your provider's per-minute limits instead of failing with 429s — for batch jobs (evals, bulk edits) and for a server sharing one key. Each call reserves an estimate, which is replaced by the real usage when the response reports it.

function rateLimitedChatModel(model: ChatModel, options: RateLimitOptions): ChatModel

Guides: Editing assistant

remoteChatModelCopy Anchor Link

The model a backend serves, as a browser-side ChatModel.

function remoteChatModel(endpoint: string, options?: RemoteChatModelOptions): Promise<ChatModel>

Guides: Editing assistant

retryDelayMsCopy Anchor Link

How long the server asked us to wait (Retry-After header, or "try again in 1.2s / 350ms"), else exponential backoff.

function retryDelayMs(res: Pick<Response, "headers">, message: string, attempt: number): number

Guides: Editing assistant

scriptedChatModelCopy Anchor Link

A model that answers from a script: for tests, demos and offline development. Each complete call takes the next step; a string is a final text answer, an object can call tools. Requests are kept in requests.

function scriptedChatModel(steps: ScriptStep[], options?: {
    id?: string;
}): ChatModel & {
    requests: ChatRequest[];
}

Guides: Editing assistant

toolsFromDefinitionsCopy Anchor Link

Adopt tools that come as definitions plus one runner — e.g. @miraiclip/audio-sources' audioToolDefinitions + runAudioTool. Accepts { name, description, input_schema } or OpenAI function tools.

function toolsFromDefinitions(definitions: readonly unknown[], run: (name: string, input: Record<string, unknown>, ctx: ToolContext) => Promise<{
    ok: boolean;
    result?: unknown;
    error?: unknown;
}>, options?: {
    changes?: (name: string, input: Record<string, unknown>, result: unknown) => string[] | undefined;
}): AssistantTool[]

Guides: Editing assistant

toOpenAIMessagesCopy Anchor Link

Neutral messages → Chat Completions messages.

function toOpenAIMessages(messages: readonly ChatMessage[]): OpenAIMessage[]

Guides: Editing assistant

unknownFieldsCopy Anchor Link

Unknown payload fields for a command, judged against the best-matching schema branch.

function unknownFields(schema: unknown, payload: unknown): UnknownFields | null

Guides: Editing assistant

withExplicitIdsCopy Anchor Link

Built-in commands that would otherwise get a random id inside the engine.

function withExplicitIds(command: Command): Command

Guides: Editing assistant

ClassesCopy Anchor Link

ChatModelErrorCopy Anchor Link

An error from a model vendor, carrying its HTTP status when there was one.

Extends Error.

new ChatModelError(provider: string, message: string, status?: number | undefined)

Guides: Editing assistant

ConstantsCopy Anchor Link

BACKGROUND_TEMPLATECopy Anchor Link

The html template set_background uses: one {{color}} fill.

const BACKGROUND_TEMPLATE: "<div style=\"width:100%;height:100%;background:{{color}}\"></div>"

DEFAULT_INSTRUCTIONSCopy Anchor Link

const DEFAULT_INSTRUCTIONS: "You are the editing assistant inside a video editor. You change the user's project by calling tools; every change you make lands as ONE undo step when you finish, so act confidently.\n\nHow to work:\n- The current project is summarized below. Clip, track and asset ids there are what tools expect. Call get_state to ...

Guides: Editing assistant

InterfacesCopy Anchor Link

AssistantCopy Anchor Link

AssistantOptionsCopy Anchor Link

AssistantToolCopy Anchor Link

Extends ToolSpec.

AssistantTurnCopy Anchor Link

ChatHandlerOptionsCopy Anchor Link

Keys stay on the server: run the real model adapter in your backend behind createChatHandler, and give the browser remoteChatModel(endpoint) — a proxy with the same ChatModel shape. The agent loop and the tools run in the browser, next to the project; only model calls cross the network.

GET {base}/ → { id, label, models } POST {base}/chat { request: ChatRequest } → application/x-ndjson, one JSON object per line: { "type": "text", "delta": "…" } (as it streams) { "type": "done", "response": ChatResponse } { "type": "error", "error": "…", "status": 401 } errors before streaming → { error } with 400 / 404 / 413, or the vendor's status

Fetch-API handler (Request → Response): mounts in Node 18+, edge runtimes, Next, Hono, Express (via an adapter).

ChatModelCopy Anchor Link

Extends ChatModelInfo.

Guides: Editing assistant

ChatModelInfoCopy Anchor Link

ChatOptionsCopy Anchor Link

Guides: Editing assistant

ChatRequestCopy Anchor Link

Guides: Editing assistant

ChatResponseCopy Anchor Link

Guides: Editing assistant

ChatUsageCopy Anchor Link

EditorToolsOptionsCopy Anchor Link

ForkOptionsCopy Anchor Link

OpenAIChatModelOptionsCopy Anchor Link

OpenAI adapter, over the Chat Completions API with function calling.

Chat Completions is also what most other servers speak (Azure OpenAI, OpenRouter, Groq, Together, vLLM, Ollama, LM Studio…), so baseUrl points this one adapter at any of them.

RateLimitOptionsCopy Anchor Link

RecordingProjectCopy Anchor Link

A working copy of a project that records every command that sticks.

The assistant edits the copy while the model thinks (async, many steps); the real project only changes when the turn commits, by replaying the recorded commands in one transaction — one undo step, and nothing half-done if the request fails. Commands inside a transaction that rolls back are dropped, exactly as the copy drops their effects.

Replay must produce the same ids the model saw, so commands whose ids are optional get explicit ones before they run.

Extends Project.

RemoteChatModelOptionsCopy Anchor Link

ToolCallCopy Anchor Link

A tool call the model made. arguments is already parsed JSON.

ToolContextCopy Anchor Link

What a tool's run gets. project is the turn's working copy.

ToolSpecCopy Anchor Link

A tool as the model sees it: name, description and a JSON Schema for its input.

TurnOptionsCopy Anchor Link

UnknownFieldsCopy Anchor Link

TypesCopy Anchor Link

AssistantEventCopy Anchor Link

type AssistantEvent = {
    type: "text";
    delta: string;
} | {
    type: "tool-start";
    call: ToolCall;
} | {
    type: "tool-end";
    call: ToolCall;
    ok: boolean;
    changes?: string[];
    error?: string;
} | {
    type: "step";
    index: number;
};

ChatMessageCopy Anchor Link

type ChatMessage = {
    role: "system";
    content: string;
} | {
    role: "user";
    content: string;
} | {
    role: "assistant";
    content: string;
    toolCalls?: ToolCall[];
} | {
    role: "tool";
    toolCallId: string;
    name: string;
    content: string;
};

Guides: Editing assistant

CommitResultCopy Anchor Link

type CommitResult = {
    ok: true;
    applied: number;
} | {
    ok: false;
    error: string;
};

FetchLikeCopy Anchor Link

type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;

ScriptStepCopy Anchor Link

type ScriptStep = string | {
    text?: string;
    toolCalls?: (Omit<ToolCall, "id"> & {
        id?: string;
    })[];
} | ((request: ChatRequest, index: number) => ScriptStep);

ToolOutputCopy Anchor Link

A tool's answer. changes are human-readable lines for the app's "what changed" list.

type ToolOutput = {
    ok: true;
    result?: unknown;
    changes?: string[];
} | {
    ok: false;
    error: unknown;
};