Audio
Stock and AI-generated audio with licenses.
project.transaction(() => {
project.dispatch({ type: "track/add", payload: { id: "music", kind: "audio", name: "Music" } });
project.dispatch({
type: "asset/add",
payload: { id: "song", kind: "audio", src: "/media/song.mp3", durationUs: 92_000_000 },
});
project.dispatch({
type: "clip/add",
payload: {
kind: "audio",
id: "bed",
trackId: "music",
assetId: "song",
startUs: 0,
durationUs: 30_000_000,
volume: 0.35,
fadeInUs: 1_000_000,
fadeOutUs: 2_000_000,
},
});
});Audio clips go on audio tracks. Video clips play their own sound too, with the same volume and fade fields.
Volume and fades
| Field | Default | Meaning |
|---|---|---|
volume | 1 | Gain, ≥ 0. Values above 1 amplify |
fadeInUs, fadeOutUs | absent | Linear fades at the clip's start and end. If together they are longer than the clip, both shrink proportionally |
trimStartUs | 0 | Where in the file the clip starts playing |
project.dispatch({ type: "clip/set-property", payload: { clipId: "bed", volume: 0.5, fadeOutUs: null } }); // null removes the fadeDuck under a voiceover
Keyframe volume like any other property. See Keyframes & animation.
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 2_000_000, value: 0.35 } });
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 2_400_000, value: 0.1 } });
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 8_000_000, value: 0.1 } });
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 8_400_000, value: 0.35 } });Volume keyframes, fades and clip volume multiply together. Preview and export produce the same mix.
Mute and solo
project.dispatch({ type: "track/set-property", payload: { trackId: "music", muted: true } });
project.dispatch({ type: "track/set-property", payload: { trackId: "voice", solo: true } });| Track field | Effect |
|---|---|
muted | The track's clips make no sound |
solo | When any track is soloed, only soloed tracks are heard |
Mute and solo apply to video tracks as well, and to exports. hidden on a video track hides its picture but keeps its sound.
Stock audio: @miraiclip/audio-sources
npm install @miraiclip/audio-sourcesThe package searches audio libraries through one interface and adds results to a project with their license. It makes requests with fetch; point each provider at your own backend to keep keys off the client.
import { createAudioLibrary, importAudio, openverseProvider, staticProvider } from "@miraiclip/audio-sources";
declare function copyToMyStorage(url: string): Promise<string>;
const library = createAudioLibrary([
staticProvider({
id: "house",
label: "Our library",
entries: [
{
id: "calm",
title: "Calm Piano",
src: "https://cdn.example.com/calm.mp3",
kind: "music",
durationUs: 92_000_000,
creator: "Jane Doe",
license: { id: "CC-BY-4.0", commercial: true, attributionRequired: true },
attribution: "“Calm Piano” by Jane Doe, CC BY 4.0",
},
],
}),
openverseProvider({ baseUrl: "/api/openverse" }), // your proxy to https://api.openverse.org
]);
const results = await library.searchAll({ query: "calm piano", kind: "music", commercialOnly: true });
const item = results.find((r) => r.result?.items.length)?.result?.items[0];
if (item) {
const resolved = await library.resolve(item);
const src = await copyToMyStorage(resolved.src); // stable URL with Range and CORS support
importAudio(project, { ...resolved, src }, { atUs: 0, volume: 0.35, fadeInUs: 1_000_000 });
}Providers
| Provider | Source | Keys |
|---|---|---|
staticProvider({ id, label, entries }) | Files you host, with the license you set per entry | none |
openverseProvider({ baseUrl?, accessToken?, sources? }) | Openverse: openly licensed music, sound effects and more | none; anonymous use is rate-limited |
freesoundProvider({ token?, baseUrl? }) | Freesound: sound effects and recordings | API key, or a proxy that adds it |
httpProvider({ id, label, endpoint, kinds?, headers? }) | Your backend, through the HTTP contract below | yours |
The network providers also take an optional fetch, so you can route requests your own way.
Searching
AudioQuery field | Meaning |
|---|---|
query | Search text |
kind | "music", "sfx" or "voice" |
commercialOnly | Only items whose license allows commercial use |
minDurationS, maxDurationS | Length filter in seconds |
page, pageSize | Paging (page starts at 1) |
signal | Cancel |
library.search(providerId, query) searches one provider. library.searchAll(query) searches all of them and returns { provider, result?, error? } per provider, so one failing source doesn't fail the rest.
Each AudioItem has title, kind, durationUs, creator, license (null when unknown), attribution, landingUrl, tags, and a previewUrl that a plain <audio> element can play.
Importing
importAudio(project, resolved, options) dispatches asset/add (with name, source, license and attribution) and clip/add as one undo step. It returns { assetId, clipId, trackId }.
| Option | Default |
|---|---|
atUs | The playhead |
trackId | The first audio track free for the clip's span, else a new track named after the kind ("Music", "Sound effects", "Voice") |
durationUs | The file's duration, else 10 s |
volume, fadeInUs, fadeOutUs | Clip fields |
assetId, clipId, trackName | Generated |
assetOnly | false. true adds the asset without a clip |
Copy remote files into your own storage before importing and pass the new src. The renderer needs a URL that supports Range requests and CORS, and won't break if the provider removes the file.
HTTP contract for httpProvider
GET {endpoint}/search?q=&kind=&commercialOnly=&minDurationS=&maxDurationS=&page=&pageSize= → AudioSearchResult
GET {endpoint}/items/{id} → AudioItem | 404
POST {endpoint}/resolve { item } → ResolvedAudioAI-generated audio
Generators produce sound effects, music or voice. The package includes elevenLabsGenerator and toneGenerator (an offline test tone). Write an adapter for any other service with defineGenerator.
import { createAudioLibrary, elevenLabsGenerator, importAudio } from "@miraiclip/audio-sources";
declare function storeFile(file: Blob, name: string): Promise<string>;
const library = createAudioLibrary([], {
generators: [elevenLabsGenerator({ baseUrl: "/api/elevenlabs", plan: "paid" })],
});
const job = library.generate(
"elevenlabs",
{ kind: "sfx", prompt: "glass shattering", durationS: 2 },
{ storeFile },
);
job.onChange((j) => console.log(j.status, j.progress));
const { resolved } = await job.result;
importAudio(project, resolved, { volume: 0.9 });| Request | Fields |
|---|---|
{ kind: "sfx" } | prompt, durationS? |
{ kind: "music" } | prompt, durationS?, instrumental?, lyrics? |
{ kind: "voice" } | text, voice?, language?, style? |
The job checks the request against the generator's kinds and limits, stores the file with storeFile (without it, the result uses an object URL or a data: URL), and records the generator's terms as the asset's license.
To keep vendor keys on the server, serve generators with createGeneratorHandler(generators, { basePath }) (a Fetch API handler) and load them in the browser with remoteGenerators(endpoint), then library.addGenerator(g) for each.
elevenLabsGenerator's plan sets the recorded terms: "free" (the default) is recorded as non-commercial with a credit line, "paid" as commercial. Confirm the vendor's current terms for your use.
Licenses and credits
Every item and generated file carries an AssetLicense: { id, url?, commercial, attributionRequired }.
import { creditsFor, licenseReport } from "@miraiclip/core";
const doc = project.getState().doc;
creditsFor(doc); // credit lines for an end card or description
licenseReport(doc, { commercial: true }); // non-commercial, unknown or uncredited assets- Filter with
commercialOnly: truefor commercial work. - Set
attributiononstaticProviderentries whose license requires credit. Without it,licenseReportreportsmissing-attribution. - API terms are separate from item licenses. Freesound's API is free for non-commercial use only; commercial use needs an agreement with Freesound, even for CC0 sounds.
parseCreativeCommons(code, version?)andcreditLine(title, creator, license)help when you write your own provider.
This is not legal advice. Check each source's terms for your use.
More on provenance fields: Assets. Audio search and generation as AI tools (audioToolDefinitions, runAudioTool): AI tools.