Timeline
Drag, trim and split clips on a timeline.
A timeline is a view of doc.tracks, doc.trackOrder and doc.clips, plus pointer handlers that dispatch clip/move, clip/trim and clip/split. The core has no timeline UI; this page shows the pieces.
Tracks and clips
import type { Clip } from "@miraiclip/core";
const { doc } = project.getState();
// trackOrder[0] is the bottom layer. Timelines usually draw the top layer first.
const rows = [...doc.trackOrder].reverse().map((trackId) => ({
track: doc.tracks[trackId]!,
clips: Object.values(doc.clips)
.filter((clip: Clip) => clip.trackId === trackId)
.sort((a, b) => a.startUs - b.startUs),
}));| Field | Use in the timeline |
|---|---|
track.kind | video or audio; decides which clips a row accepts (TRACK_ACCEPTS) |
track.locked, muted, solo, hidden | Row header toggles (track/set-property) |
clip.startUs, clip.durationUs | Left edge and width |
clip.trimStartUs | Video and audio only: where in the source the clip starts |
Time and pixels
Store zoom as pixels per second. Commands take whole microseconds, so round when converting from pixels, then snap to a frame.
import { projectFrameRate, snapToFrame } from "@miraiclip/core";
let pxPerSecond = 100; // zoom
const usToPx = (us: number) => (us / 1_000_000) * pxPerSecond;
const pxToUs = (px: number) => Math.round((px / pxPerSecond) * 1_000_000);
const rate = projectFrameRate(project.getState().doc.settings);
const snapped = snapToFrame(pxToUs(437), rate);Times must be integers
startUs: 1.5 fails validation (CommandValidationError). snapToFrame and frameToUs return integers.
A track row
A row that draws its clips, selects on click and moves a clip by dragging. The drag moves the element only; one clip/move is dispatched on release, so a drag is one undo step.
import { project } from "../shared/project";
import { clipsOnTrack, dragClip, usToPx } from "./timeline";
const row = document.querySelector<HTMLElement>("#track-main")!; // position: relative
const trackId = "main";
const pxPerSecond = 100;
// Rebuild the clip elements when the document changes.
function renderClips() {
row.replaceChildren(
...clipsOnTrack(project.getState().doc, trackId).map((clip) => {
const el = document.createElement("div");
el.className = "clip";
el.dataset.clipId = clip.id;
el.textContent = clip.id;
el.style.cssText = `position:absolute;left:${usToPx(clip.startUs, pxPerSecond)}px;width:${usToPx(clip.durationUs, pxPerSecond)}px`;
el.addEventListener("pointerdown", (e) => {
dragClip(project, clip.id, el, e, pxPerSecond);
project.setSelection([clip.id]);
});
return el;
}),
);
renderSelection();
}
// Selection only toggles an attribute, so a drag in progress keeps its element.
function renderSelection() {
const { selection } = project.getState();
for (const el of row.querySelectorAll<HTMLElement>(".clip")) {
el.dataset.selected = String(selection.includes(el.dataset.clipId!));
}
}
renderClips();
const stops = [
project.subscribe((s) => s.doc.clips, renderClips),
project.subscribe((s) => s.selection, renderSelection),
];
// When the view goes away:
// stops.forEach((stop) => stop());import { useCallback, useSyncExternalStore } from "react";
import type { Project, ProjectState } from "@miraiclip/core";
import { clipsOnTrack, dragClip, usToPx } from "./timeline";
function useProjectValue<T>(project: Project, select: (s: ProjectState) => T): T {
const subscribe = useCallback(
(onChange: () => void) => project.subscribe(select, onChange),
[project, select],
);
const read = () => select(project.getState());
return useSyncExternalStore(subscribe, read, read);
}
const selectDoc = (s: ProjectState) => s.doc;
const selectSelection = (s: ProjectState) => s.selection;
export function TimelineTrack({ project, trackId, pxPerSecond }: { project: Project; trackId: string; pxPerSecond: number }) {
const doc = useProjectValue(project, selectDoc);
const selection = useProjectValue(project, selectSelection);
return (
<div className="track" style={{ position: "relative", height: 48 }}>
{clipsOnTrack(doc, trackId).map((clip) => (
<div
key={clip.id}
className="clip"
data-selected={selection.includes(clip.id)}
style={{ position: "absolute", left: usToPx(clip.startUs, pxPerSecond), width: usToPx(clip.durationUs, pxPerSecond) }}
onPointerDown={(e) => {
project.setSelection([clip.id]);
dragClip(project, clip.id, e.currentTarget, e, pxPerSecond);
}}
>
{clip.id}
</div>
))}
</div>
);
}<script setup lang="ts">
import { computed, onScopeDispose, shallowRef } from "vue";
import type { Project } from "@miraiclip/core";
import { clipsOnTrack, dragClip, usToPx } from "./timeline";
const { project, trackId, pxPerSecond } = defineProps<{ project: Project; trackId: string; pxPerSecond: number }>();
const doc = shallowRef(project.getState().doc);
const selection = shallowRef(project.getState().selection);
const stops = [
project.subscribe((s) => s.doc, (v) => (doc.value = v)),
project.subscribe((s) => s.selection, (v) => (selection.value = v)),
];
onScopeDispose(() => stops.forEach((stop) => stop()));
const clips = computed(() => clipsOnTrack(doc.value, trackId));
function onPointerDown(clipId: string, e: PointerEvent) {
project.setSelection([clipId]);
dragClip(project, clipId, e.currentTarget as HTMLElement, e, pxPerSecond);
}
</script>
<template>
<div class="track" style="position: relative; height: 48px">
<div
v-for="clip in clips"
:key="clip.id"
class="clip"
:data-selected="selection.includes(clip.id)"
:style="{ position: 'absolute', left: `${usToPx(clip.startUs, pxPerSecond)}px`, width: `${usToPx(clip.durationUs, pxPerSecond)}px` }"
@pointerdown="onPointerDown(clip.id, $event)"
>
{{ clip.id }}
</div>
</div>
</template><script lang="ts">
import type { Project, ProjectDocument } from "@miraiclip/core";
import { clipsOnTrack, dragClip, usToPx } from "./timeline";
let { project, trackId, pxPerSecond }: { project: Project; trackId: string; pxPerSecond: number } = $props();
let doc = $state.raw<ProjectDocument>();
let selection = $state.raw<string[]>([]);
$effect(() => {
({ doc, selection } = project.getState());
const stops = [
project.subscribe((s) => s.doc, (v) => (doc = v)),
project.subscribe((s) => s.selection, (v) => (selection = v)),
];
return () => stops.forEach((stop) => stop());
});
const clips = $derived(doc ? clipsOnTrack(doc, trackId) : []);
function onPointerDown(clipId: string, e: PointerEvent) {
project.setSelection([clipId]);
dragClip(project, clipId, e.currentTarget as HTMLElement, e, pxPerSecond);
}
</script>
<div class="track" style="position: relative; height: 48px">
{#each clips as clip (clip.id)}
<div
class="clip"
role="button"
tabindex="-1"
data-selected={selection.includes(clip.id)}
style:position="absolute"
style:left="{usToPx(clip.startUs, pxPerSecond)}px"
style:width="{usToPx(clip.durationUs, pxPerSecond)}px"
onpointerdown={(e) => onPointerDown(clip.id, e)}
>
{clip.id}
</div>
{/each}
</div>The shared helpers, with no framework code:
import { projectFrameRate, snapToFrame, type Clip, type Project, type ProjectDocument } from "@miraiclip/core";
// Zoom is pixels per second of timeline.
export const usToPx = (us: number, pxPerSecond: number) => (us / 1_000_000) * pxPerSecond;
export const pxToUs = (px: number, pxPerSecond: number) => Math.round((px / pxPerSecond) * 1_000_000);
/** The clips on one track, in time order. */
export function clipsOnTrack(doc: ProjectDocument, trackId: string): Clip[] {
return Object.values(doc.clips)
.filter((clip) => clip.trackId === trackId)
.sort((a, b) => a.startUs - b.startUs);
}
/** True when [startUs, startUs + durationUs) overlaps no clip on the track except `ignoreId`. */
export function isFree(doc: ProjectDocument, trackId: string, startUs: number, durationUs: number, ignoreId?: string) {
return clipsOnTrack(doc, trackId).every(
(c) => c.id === ignoreId || c.startUs >= startUs + durationUs || c.startUs + c.durationUs <= startUs,
);
}
/** Drag a clip along its track. The DOM follows the pointer; one `clip/move` is dispatched on release. */
export function dragClip(
project: Project,
clipId: string,
el: HTMLElement,
down: { pointerId: number; clientX: number },
pxPerSecond: number,
) {
const clip = project.getState().doc.clips[clipId];
if (!clip) return;
const rate = projectFrameRate(project.getState().doc.settings);
let startUs = clip.startUs;
const onMove = (e: PointerEvent) => {
const deltaUs = pxToUs(e.clientX - down.clientX, pxPerSecond);
startUs = Math.max(0, snapToFrame(clip.startUs + deltaUs, rate));
el.style.translate = `${usToPx(startUs - clip.startUs, pxPerSecond)}px 0`;
};
const onUp = () => {
el.removeEventListener("pointermove", onMove);
el.style.translate = "";
const doc = project.getState().doc;
if (startUs !== clip.startUs && isFree(doc, clip.trackId, startUs, clip.durationUs, clipId)) {
project.dispatch({ type: "clip/move", payload: { clipId, startUs } });
}
};
el.setPointerCapture(down.pointerId);
el.addEventListener("pointermove", onMove);
el.addEventListener("pointerup", onUp, { once: true });
el.addEventListener("pointercancel", onUp, { once: true });
}Playhead and scrubbing
Draw the playhead from s.playheadUs. It changes every frame during playback, so update the element directly instead of re-rendering the timeline.
import { projectFrameRate, snapToFrame } from "@miraiclip/core";
import type { Player } from "@miraiclip/renderer";
declare const player: Player;
const pxPerSecond = 100;
const ruler = document.querySelector<HTMLElement>("#ruler")!;
const playheadLine = document.querySelector<HTMLElement>("#playhead")!;
project.subscribe(
(s) => s.playheadUs,
(us) => {
playheadLine.style.translate = `${(us / 1_000_000) * pxPerSecond}px 0`;
},
);
function seekToPointer(e: PointerEvent) {
const x = e.clientX - ruler.getBoundingClientRect().left;
const us = Math.max(0, Math.round((x / pxPerSecond) * 1_000_000));
player.seek(snapToFrame(us, projectFrameRate(project.getState().doc.settings)));
}
ruler.addEventListener("pointerdown", (e) => {
ruler.setPointerCapture(e.pointerId);
seekToPointer(e);
ruler.addEventListener("pointermove", seekToPointer);
ruler.addEventListener("pointerup", () => ruler.removeEventListener("pointermove", seekToPointer), { once: true });
});| Call | Effect |
|---|---|
player.seek(us) | Moves the preview and audio. The player then writes the time to s.playheadUs |
project.setPlayhead(us) | Writes s.playheadUs only. The player doesn't read it, so use this when no player exists (tests, headless tools) |
Moving clips
clip/move takes startUs, trackId, or both. Moving to a track of the wrong kind throws CommandRejectedError with code kind-mismatch.
project.dispatch({ type: "clip/move", payload: { clipId: "intro", startUs: 2_000_000, trackId: "overlay" } });Snapping to edges
Snap to the nearest clip edge or the playhead when it's within a few pixels, otherwise to the nearest frame.
import { projectFrameRate, snapToFrame } from "@miraiclip/core";
function snapStart(proposedUs: number, clipId: string, thresholdUs: number): number {
const { doc, playheadUs } = project.getState();
const clip = doc.clips[clipId]!;
const targets = [0, playheadUs];
for (const other of Object.values(doc.clips)) {
if (other.id === clipId) continue;
targets.push(other.startUs, other.startUs + other.durationUs);
}
let best = snapToFrame(proposedUs, projectFrameRate(doc.settings));
let bestDistance = thresholdUs;
for (const t of targets) {
// Snap the clip's start or its end to the target.
for (const candidate of [t, t - clip.durationUs]) {
const distance = Math.abs(candidate - proposedUs);
if (candidate >= 0 && distance < bestDistance) {
best = candidate;
bestDistance = distance;
}
}
}
return best;
}
const pxPerSecond = 100;
const startUs = snapStart(1_480_000, "intro", Math.round((8 / pxPerSecond) * 1_000_000)); // 8pxRules the core leaves to you
| Rule | Core behaviour | What to do |
|---|---|---|
| Overlap | Clips on one track may overlap; the later-starting clip draws on top | Check the target range is free before dispatching (isFree above), or use another track |
| Locked tracks | track.locked is stored, not enforced | Skip locked tracks in drag, trim, split and delete handlers |
| Transitions | clip/move, clip/trim, clip/split and clip/remove delete transitions that touch the clip | Re-add them if your UI keeps them across edits; see Transitions |
Trimming
A trim handle changes startUs, durationUs and, for video and audio, trimStartUs. Clamp the drag so the clip stays at least one frame long and doesn't run past its source media. clipHeadroomUs returns the unused source on each side, or Infinity for clips without source media (text, image, caption, html).
import { clipHeadroomUs, frameDurationUs, projectFrameRate } from "@miraiclip/core";
/** Drag the left edge by deltaUs (negative = extend left). */
function trimLeft(clipId: string, deltaUs: number) {
const { doc } = project.getState();
const clip = doc.clips[clipId]!;
const minUs = Math.ceil(frameDurationUs(projectFrameRate(doc.settings)));
const d = Math.min(
Math.max(deltaUs, -clipHeadroomUs(doc, clip, "in"), -clip.startUs),
clip.durationUs - minUs,
);
if (d === 0) return;
project.dispatch({
type: "clip/trim",
payload: {
clipId,
startUs: clip.startUs + d,
durationUs: clip.durationUs - d,
// Only video and audio clips have a source offset.
...("trimStartUs" in clip ? { trimStartUs: clip.trimStartUs + d } : {}),
},
});
}
/** Drag the right edge by deltaUs (positive = extend right). */
function trimRight(clipId: string, deltaUs: number) {
const { doc } = project.getState();
const clip = doc.clips[clipId]!;
const minUs = Math.ceil(frameDurationUs(projectFrameRate(doc.settings)));
const durationUs = Math.max(minUs, clip.durationUs + Math.min(deltaUs, clipHeadroomUs(doc, clip, "out")));
project.dispatch({ type: "clip/trim", payload: { clipId, durationUs } });
}| Payload field | Meaning |
|---|---|
startUs | New timeline start |
durationUs | New length (greater than 0) |
trimStartUs | New offset into the source. Rejected with code not-trimmable on clips without source media |
The core doesn't check durationUs against asset.durationUs. clipHeadroomUs reads asset.durationUs, so set it on asset/add; without it, the out-side headroom is Infinity.
Keyframes are measured from the clip's visible start (or end, for end-anchored ones), so a left trim shifts where start-anchored keyframes land on the timeline. See Keyframes.
Splitting at the playhead
clip/split rejects a time at or outside the clip's edges (out-of-range), so split only clips that strictly contain the playhead. Pass newClipId so you know the right half's id.
function splitAtPlayhead() {
const { doc, playheadUs, selection } = project.getState();
const targets = Object.values(doc.clips).filter(
(clip) =>
(selection.length === 0 || selection.includes(clip.id)) &&
!doc.tracks[clip.trackId]?.locked &&
clip.startUs < playheadUs &&
playheadUs < clip.startUs + clip.durationUs,
);
if (targets.length === 0) return;
project.transaction(() => {
for (const clip of targets) {
project.dispatch({
type: "clip/split",
payload: { clipId: clip.id, atUs: playheadUs, newClipId: `${clip.id}-${playheadUs}` },
});
}
}, "Split");
}The right half keeps every property, with trimStartUs advanced. The fade-in stays on the left half and the fade-out moves to the right.
Ripple edits
A ripple edit is several commands. Run them in a transaction so they undo together and roll back together if one fails.
/** Remove a clip and close the gap on its track. */
function rippleDelete(clipId: string) {
const { doc } = project.getState();
const removed = doc.clips[clipId];
if (!removed) return;
const endUs = removed.startUs + removed.durationUs;
const later = Object.values(doc.clips)
.filter((c) => c.trackId === removed.trackId && c.startUs >= endUs)
.sort((a, b) => a.startUs - b.startUs);
project.transaction(() => {
project.dispatch({ type: "clip/remove", payload: { clipId } });
for (const clip of later) {
project.dispatch({ type: "clip/move", payload: { clipId: clip.id, startUs: clip.startUs - removed.durationUs } });
}
}, "Ripple delete");
}Waveforms
The renderer computes peaks for an audio or video asset and downsamples them for a clip's visible range. Compute once per asset and cache the result.
import type { AudioClip } from "@miraiclip/core";
import { computeWaveformPeaks, openMediabunnyAudio, peaksForRange, type WaveformPeaks } from "@miraiclip/renderer";
const waveforms = new Map<string, Promise<WaveformPeaks | null>>();
function waveformFor(assetId: string): Promise<WaveformPeaks | null> {
let cached = waveforms.get(assetId);
if (!cached) {
const asset = project.getState().doc.assets[assetId]!;
cached = openMediabunnyAudio(assetId, asset.src).then(async (source) => {
if (!source || asset.durationUs === undefined) return null; // no audio track, or unknown length
try {
return await computeWaveformPeaks(source, { durationUs: asset.durationUs });
} finally {
source.dispose();
}
});
waveforms.set(assetId, cached);
}
return cached;
}
async function drawWaveform(canvas: HTMLCanvasElement, clip: AudioClip) {
const waveform = await waveformFor(clip.assetId);
if (!waveform) return;
const columns = peaksForRange(waveform, clip.trimStartUs, clip.trimStartUs + clip.durationUs, canvas.width);
const ctx = canvas.getContext("2d")!;
const mid = canvas.height / 2;
ctx.clearRect(0, 0, canvas.width, canvas.height);
columns.forEach((peak, x) => ctx.fillRect(x, mid - peak * mid, 1, Math.max(1, peak * canvas.height)));
}| Function | Returns |
|---|---|
openMediabunnyAudio(assetId, src) | An AudioTrackSource, or null when the file has no audio track |
computeWaveformPeaks(source, { durationUs, bucketsPerSecond?, maxBuckets?, signal? }) | { peaks, bucketUs, durationUs }: max amplitude 0–1 per bucket. Default 50 buckets per second, at most 20 000 |
peaksForRange(waveform, fromMediaUs, toMediaUs, width) | width values, the max per column |
Pass source-media times (trimStartUs onward), not timeline times. Peaks depend only on the asset, so trims and moves only need peaksForRange again.
Helpers
| Helper | Package | Use |
|---|---|---|
snapToFrame(us, rate) | core | Nearest frame start |
usToFrame(us, rate), frameToUs(frame, rate) | core | Frame-exact stepping, also at 29.97 |
frameDurationUs(rate) | core | One frame in µs (fractional at NTSC rates) |
usToTimecode(us, rate, { dropFrame? }) | core | HH:MM:SS:FF ruler labels |
projectFrameRate(settings) | core | The exact rate to pass to the helpers above |
clipHeadroomUs(doc, clip, "in" | "out") | core | Unused source media on each side |
findCuts(doc) | core | Every place a transition can go, with maxTransitionUs per cut |
compositionEnd(doc) | renderer | End of the last clip: the timeline's length |
See Project & time for frame rates and Clips for every clip command.