Custom clip kinds

A custom clip kind has two halves. Core registers its props schema and the tracks it may go on; the renderer gets a factory that builds the clip's scene node.

npm install zod

1. Register the kind

import { registerClipKind } from "@miraiclip/core";
import { z } from "zod";

export const countdownProps = z.object({
  from: z.number().int().positive(),
  color: z.string().default("#ffffff"),
});

registerClipKind("countdown", { propsSchema: countdownProps, trackKinds: ["video"] });

project.dispatch({
  type: "clip/add",
  payload: { kind: "countdown", id: "count", trackId: "titles", startUs: 0, durationUs: 3_000_000, props: { from: 3 } },
});

The clip's own data lives under props, validated by your schema with defaults filled in. Everything else works like any clip: transform, keyframes, effects, clip/move, clip/trim, clip/split, undo and serialization.

`ClipKindRegistration` field	Meaning
`propsSchema`	Zod schema for `props`
`trackKinds`	Track kinds that accept the clip: `["video"]`, `["audio"]` or both

Rejection on `clip/add`	Meaning
`unknown-kind`	No clip kind with that name is registered
`invalid-props`	`props` don't match the schema
`kind-mismatch`	The track's kind isn't in `trackKinds`

registerClipKind throws for a built-in name (video, audio, image, text, caption, html) or a kind that's already registered. Check with clipKindRegistration(kind) before registering again, for example under hot module reload.

2. Write a node factory

A factory turns a clip into a SceneNode. It gets the clip and { backend, assets }. The simplest approach is to build on a node the backend already makes, here a text node:

import type { Clip, TextClip } from "@miraiclip/core";
import type { NodeFactory } from "@miraiclip/renderer";

interface CountdownProps {
  from: number;
  color: string;
}

// props were validated (and defaults filled) by the schema on clip/add
const propsOf = (clip: Clip) => ("props" in clip ? clip.props : {}) as unknown as CountdownProps;

const asText = (clip: Clip, text: string): TextClip => ({
  kind: "text",
  id: clip.id,
  trackId: clip.trackId,
  startUs: clip.startUs,
  durationUs: clip.durationUs,
  transform: clip.transform,
  text,
  fontFamily: "monospace",
  fontSizePx: 160,
  color: propsOf(clip).color,
});

export const factories: Record<string, NodeFactory> = {
  countdown: (clip, { backend }) => {
    const inner = backend.createText(asText(clip, String(propsOf(clip).from)));
    let shown = "";
    return {
      setPlacement: (placement) => inner.setPlacement(placement),
      setVisible: (visible) => inner.setVisible(visible),
      setZ: (z) => inner.setZ(z),
      setEffects: (effects) => inner.setEffects?.(effects),
      getLocalBounds: () => inner.getLocalBounds?.() ?? null,
      update: (next) => {
        shown = "";
        inner.update(asText(next, String(propsOf(next).from)));
      },
      // Every rendered frame while visible. timeUs is the timeline position.
      tick: (current, timeUs) => {
        const elapsedS = (timeUs - current.startUs) / 1_000_000;
        const text = String(Math.max(0, Math.ceil(propsOf(current).from - elapsedS)));
        if (text !== shown) {
          shown = text;
          inner.update(asText(current, text));
        }
      },
      destroy: () => inner.destroy(),
    };
  },
};

What the factory's node draws must depend only on the clip and the time passed to tick. Then preview, export and stills match frame for frame.

3. Pass the factories everywhere

Pass the same factories object to the player, to exports and to stills:

import { createPlayer, exportProject, renderProjectStill, type CreatePlayerOptions, type NodeFactory } from "@miraiclip/renderer";

declare const playerOptions: Omit<CreatePlayerOptions, "factories">;
declare const factories: Record<string, NodeFactory>;

const player = createPlayer(project, { ...playerOptions, factories });
const bytes = await exportProject(project, { format: "mp4", factories });
const poster = await renderProjectStill(project, { timeUs: 1_000_000, factories });

A clip kind without a factory draws nothing. The "video" key is reserved.

SceneNode

Member	Required	Called when
`setPlacement(placement)`	yes	Every frame: position in pixels, scale, rotation in radians, opacity (keyframes already applied). The object may be reused: copy it if you keep it
`setVisible(visible)`	yes	The clip enters or leaves the playhead
`setZ(z)`	yes	Stacking order changes. Higher draws on top
`update(clip)`	yes	After creation, and whenever the clip changes in the document
`destroy()`	yes	The clip is removed or the renderer shuts down
`tick(clip, timeUs)`	no	Every rendered frame while visible
`setEffects(effects)`	no	The effect stack changes. Without it, effects don't apply
`getLocalBounds()`	no	Selection boxes and `hitTest`. Return `null` while nothing is drawn
`whenReady()`	no	Exports and stills wait for it before drawing. Use it for async content
`isPending()`	no	Return `true` while content for the last `tick` is still loading; exports then wait and draw the frame again
`setReveal(fraction, direction)`	no	Wipe transitions. Without it, wipes show the clip whole

NodeFactoryContext gives you backend (createText, createImage, createVideo, and optionally createCaption, createHtml, createSolid) and the document's assets.

Editing props

clip/set-property has no props field. Register a command for your kind so edits validate, undo and sync like built-in ones:

import { CommandRejectedError } from "@miraiclip/core";
import { z } from "zod";

declare const countdownProps: z.ZodObject<{ from: z.ZodNumber; color: z.ZodDefault<z.ZodString> }>;

project.registerCommand({
  type: "countdown/set-props",
  schema: z.object({ clipId: z.string(), props: countdownProps.partial() }),
  handler: (doc, payload) => {
    const clip = doc.clips[payload.clipId];
    if (!clip || clip.kind !== "countdown" || !("props" in clip)) {
      throw new CommandRejectedError("countdown/set-props", "not-countdown", `no countdown clip "${payload.clipId}"`);
    }
    clip.props = countdownProps.parse({ ...clip.props, ...payload.props });
  },
});

project.dispatch({ type: "countdown/set-props", payload: { clipId: "count", props: { from: 5 } } });

See Commands for registerCommand.

Where custom clips render

Target	Custom clip kinds
Preview, `exportProject`, `renderProjectStill`	Yes, with `factories`
Worker export (`exportProjectInWorker`, `exportViaWorker`)	No: the clip draws nothing
Server export	No: the clip draws nothing

Factories are functions, so they can't be sent to a worker or a server. For an overlay that must render everywhere, use an HTML clip: it is plain data.

Notes

Keyframes on x, y, scale, rotation and opacity work on custom clips with no extra code: they arrive through setPlacement.
Transitions can join custom clips. findCuts doesn't list them; dispatch transition/add directly.
The registry is global to the page. Register at startup, before any command uses the kind.

1. Register the kindCopy Anchor Link

2. Write a node factoryCopy Anchor Link

3. Pass the factories everywhereCopy Anchor Link

SceneNodeCopy Anchor Link

Editing propsCopy Anchor Link

Where custom clips renderCopy Anchor Link

NotesCopy Anchor Link