Build it into your editor

Each step is a separate package that returns ordinary commands. Apply them with applyCommands and every result is on the timeline, editable and undoable as one step. Use all of them or only the ones you need.

RecordCopy Anchor Link

import { startRecording, sessionCommands } from "@miraiclip/capture";

const recording = await startRecording({
  screen: { video: true, preferCurrentTab: true } as DisplayMediaStreamOptions,
  microphone: true,
  camera: true,
  cursor: "never", // a smooth cursor is drawn later
});
recording.mark("Open settings", { kind: "step" }); // a step
const { screen, camera, session } = await recording.stop();

// A project from it: the recording on its own track, the camera as a
// picture-in-picture, a marker per step.
applyCommands(project, sessionCommands(session, { screenSrc, cameraSrc }));

Media is encoded with WebCodecs while you record (WebM by default, MP4 with format: "mp4") and can stream to a file (screenTarget, cameraTarget), so long recordings don't fill memory. microphone and camera also take a device id or a MediaStream you opened yourself, which lets you skip a missing device instead of failing.

Pass screen as DisplayMediaStreamOptions to steer the share picker (displaySurface: "browser" | "window" | "monitor", preferCurrentTab) and to record the screen's own sound (audio: true); with a microphone too, both are mixed into one track.

The session is plain JSON: the viewport, the media, and every interaction, in microseconds from the start. session.surface says what was recorded, and session.cursorInVideo whether the real pointer is in the video (Chrome: never in a tab, always in a window or screen; it ignores the cursor request). The log only sees the page it runs in. For a tab that page is the whole frame. If the tab is resized or zoomed while recording, the video keeps its first frame size and shows the resized page scaled to fit, centred; events after the resize are moved to match (fitResizedEvents), so session.viewport is the size the video shows. For a window or a screen, viewportInVideo places the page in the frame by estimate (estimateViewportInVideo, from the window's position and toolbars when recording starts); it's left out when the frame can't be matched, such as a screen other than the window's. The Capture extension is a small example.

The log never stores what was typed. See Private by design.

Logging several pagesCopy Anchor Link

For a window or screen recording, log every page and send the events to the recorder. In each page, createEventLogger({ onEvent }) streams events as they happen; eventToFrame(event, pageInFrame(surface, window)) moves them into the frame's coordinates; ignore leaves out your own controls. In the recorder, recording.addEvent({ ...event, tUs: recording.clockAt(epochMs) }) puts each one on the recording's clock (pass events: false to startRecording so it doesn't log its own page twice), and the session's viewport becomes the frame. The Capture extension does this through its background worker.

The cameraCopy Anchor Link

sessionCommands places the camera in a corner. Move it with placeCamera, and shape it with the frameShape effect:

import { placeCamera } from "@miraiclip/capture";

const shape = { square: 1, radius: 0.5, border: 0.004, borderColor: "#ffffff" }; // a circle with a thin white border
const transform = placeCamera({ frame: doc.settings, camera: { width: 1280, height: 720 }, corner: "top-left", widthFrac: 0.2, square: shape.square });
applyCommands(project, [
  { type: "effect/add", payload: { clipId: "camera", kind: "frameShape", params: shape } },
  { type: "clip/set-property", payload: { clipId: "camera", transform } },
], { label: "camera" });

frameShape crops toward a centred square (square, 0–1; the picture is cropped, not squashed), rounds the corners (radius, a fraction of the short side; 0.5 on a square is a circle) and draws a border inside the edge (border, a fraction of the composition's height, and borderColor). Clip bounds and hit tests follow the crop, so a selection box fits the visible shape. placeCamera keeps the visible part a margin from the corner and widthFrac of the frame wide.

Clean up the speechCopy Anchor Link

import { openAiAsr, planCleanup, removeWords, captionCommands } from "@miraiclip/transcript";

const transcript = { ...(await openAiAsr({ apiKey }).transcribe(audioFile)), assetId: "screen" };
const plan = planCleanup(project.getState().doc, transcript, { linkedClipIds: ["camera"] });
applyCommands(project, plan.commands, { label: "clean up speech" });

planCleanup removes filler words (per language), repeated words and long pauses as one ripple cut: the media, its linked camera and later markers move together. removeWords cuts words you pick by index, and editByText takes the transcript's text with words deleted. Transcripts stay in the media's own time, so they remain valid through later trims, cuts and speed changes. transcriptOnTimeline tells you what the timeline says now, and captionCommands makes word-timed captions from it.

Speech-to-text goes through an AsrProvider contract, with OpenAI and Deepgram adapters. Keep keys on your server with createAsrHandler, and use remoteAsr(endpoint, id) in the browser:

// server
import { createAsrHandler, openAiAsr } from "@miraiclip/transcript";
const handle = createAsrHandler([openAiAsr({ apiKey: process.env.OPENAI_API_KEY! })], { basePath: "/api/transcribe" });
// handle(request) → Response, or null for paths it doesn't own

// browser
const asr = remoteAsr("/api/transcribe", "openai");

PolishCopy Anchor Link

import { planScreencast } from "@miraiclip/screencast";

const plan = planScreencast(project.getState().doc, session, { clipId: "screen", callouts: true });
applyCommands(project, plan.commands, { label: "polish recording" });

planScreencast runs them in the right order on a scratch copy and returns the commands per planner (plan.steps), so a UI can offer each one as an option. Every planner respects trims and speed changes on the screen clip.

planCursor takes a style: "arrow" (default), "highlight" (the arrow with a soft coloured circle), "hand", "halo", "dot" or "ring", each a registry block with its hot spot at the centre (CURSOR_STYLES lists them, with pointer: false for the ones that only mark the spot). Use those over a recording whose cursorInVideo is true: they follow the real pointer exactly (no smoothing). Every style fades out after a leave event (the pointer left the logged pages) and back in at the next move (pointerGaps). size scales it, fill and stroke colour the arrow and hand, and highlight colours the circle, dot or ring. planClickEffects takes a ripple color.

When a recording has been cut or split into pieces, plan each piece (pass each clipId in turn, against the result of the previous one). When you change the timing later, with a cut or a voiceover, remove the old polish and plan it again on the new timeline. Apply both in one applyCommands call to keep them one undo step.

VoiceoverCopy Anchor Link

import { scriptFromMarkers, rewriteScript, synthesizeNarration, alignNarration } from "@miraiclip/narration";
import { openAiSpeechGenerator } from "@miraiclip/audio-sources";

let script = scriptFromMarkers(doc, { transcript }); // a line per step, from what was said
script = await rewriteScript(chatModel, doc, script, { style: "friendly, second person" });
const takes = await synthesizeNarration(openAiSpeechGenerator({ apiKey }), script, { store });
const { commands, placements } = alignNarration(doc, script, takes, { clipIds: ["screen"], linkedClipIds: ["camera"] });

Each line gets its step's length as a word budget, so a rewrite mostly fits; speech before the first step goes into the first line. alignNarration fits every line to its step, the same way every time. When a line runs long, the step's last frame is held (a freeze-frame clip), or with fit: "slow" the step plays slower. With speedUp, a step with little to say plays faster. placements reports what happened to each line.

Any chat model works for the rewrite (the assistant package's models fit), and any @miraiclip/audio-sources voice generator works for speech. Lengths are read from the audio itself, and unchanged lines are reused when you regenerate (reuse).

Written guideCopy Anchor Link

import { buildGuide, addStills, toHtml } from "@miraiclip/guides";
import { renderProjectStill } from "@miraiclip/renderer";

let guide = buildGuide(doc, { title: "Turn on dark mode", session, clipIds, script });
guide = await addStills(guide, doc, (d, o) => renderProjectStill(createProject(d), o), { annotate: true, store });
const html = toHtml(guide); // or toMarkdown(guide), toJson(guide)

Steps come from step markers (or from every click), with titles from the marker or the action ("Click “Save”", "Press Enter"). The text comes from the script or from what was said, with filler words dropped. Pass every piece of the recording as clipIds once it has been cut or split, so each action is found on the piece that shows it.

Screenshots are taken just before each action. With annotate, the control is outlined, the rest dimmed, and the step number shown beside it; private fields are never outlined. toHtml makes a standalone page that works in light and dark mode.

TranslateCopy Anchor Link

import { chatTranslator, localizeMany } from "@miraiclip/localize";

const variants = await localizeMany(doc, chatTranslator(chatModel), ["fr", "de", "hi"], { script, keep: ["Acme"] });
// variants.fr.doc, variants.fr.script, …

Each variant translates on-screen text, captions (keeping word timing when the word count matches), text in HTML blocks, step names and the project name, and returns the translated script. Re-voice it with synthesizeNarration and alignNarration on the variant, starting from the project before narration. Terms in keep and {{placeholders}} stay as written. Translators are a small contract: a chat model (chatTranslator), DeepL (deeplTranslator), the browser's built-in translator, or your own.

BrandCopy Anchor Link

import { applyBrand, parseBrandKit } from "@miraiclip/brand";

const kit = parseBrandKit(kitJson);
applyCommands(project, applyBrand(project.getState().doc, kit).commands, { label: "apply brand kit" });

A brand kit loads its fonts and uses them on text, captions and blocks, recolours block colour fields by role, adds a logo watermark, and adds an intro and outro (the content moves to make room). Applying it twice doesn't add a second intro or logo. Turn parts off with { intro: false, outro: false, watermark: false }.

Keys and costCopy Anchor Link

Every vendor call (speech-to-text, translation, chat, voice) goes through a small contract with your own key, and the packages include server handlers so keys never reach the browser. Exports run in the browser for free, or on your own servers with server export.