Miraiclip SDK
Creative

Audio

Stock and AI-generated audio with licenses.

project.transaction(() => {
  project.dispatch({ type: "track/add", payload: { id: "music", kind: "audio", name: "Music" } });
  project.dispatch({
    type: "asset/add",
    payload: { id: "song", kind: "audio", src: "/media/song.mp3", durationUs: 92_000_000 },
  });
  project.dispatch({
    type: "clip/add",
    payload: {
      kind: "audio",
      id: "bed",
      trackId: "music",
      assetId: "song",
      startUs: 0,
      durationUs: 30_000_000,
      volume: 0.35,
      fadeInUs: 1_000_000,
      fadeOutUs: 2_000_000,
    },
  });
});

Audio clips go on audio tracks. Video clips play their own sound too, with the same volume and fade fields.

Volume and fades

FieldDefaultMeaning
volume1Gain, ≥ 0. Values above 1 amplify
fadeInUs, fadeOutUsabsentLinear fades at the clip's start and end. If together they are longer than the clip, both shrink proportionally
trimStartUs0Where in the file the clip starts playing
project.dispatch({ type: "clip/set-property", payload: { clipId: "bed", volume: 0.5, fadeOutUs: null } }); // null removes the fade

Duck under a voiceover

Keyframe volume like any other property. See Keyframes & animation.

project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 2_000_000, value: 0.35 } });
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 2_400_000, value: 0.1 } });
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 8_000_000, value: 0.1 } });
project.dispatch({ type: "keyframe/set", payload: { clipId: "bed", property: "volume", timeUs: 8_400_000, value: 0.35 } });

Volume keyframes, fades and clip volume multiply together. Preview and export produce the same mix.

Mute and solo

project.dispatch({ type: "track/set-property", payload: { trackId: "music", muted: true } });
project.dispatch({ type: "track/set-property", payload: { trackId: "voice", solo: true } });
Track fieldEffect
mutedThe track's clips make no sound
soloWhen any track is soloed, only soloed tracks are heard

Mute and solo apply to video tracks as well, and to exports. hidden on a video track hides its picture but keeps its sound.

Stock audio: @miraiclip/audio-sources

npm install @miraiclip/audio-sources

The package searches audio libraries through one interface and adds results to a project with their license. It makes requests with fetch; point each provider at your own backend to keep keys off the client.

import { createAudioLibrary, importAudio, openverseProvider, staticProvider } from "@miraiclip/audio-sources";

declare function copyToMyStorage(url: string): Promise<string>;

const library = createAudioLibrary([
  staticProvider({
    id: "house",
    label: "Our library",
    entries: [
      {
        id: "calm",
        title: "Calm Piano",
        src: "https://cdn.example.com/calm.mp3",
        kind: "music",
        durationUs: 92_000_000,
        creator: "Jane Doe",
        license: { id: "CC-BY-4.0", commercial: true, attributionRequired: true },
        attribution: "“Calm Piano” by Jane Doe, CC BY 4.0",
      },
    ],
  }),
  openverseProvider({ baseUrl: "/api/openverse" }), // your proxy to https://api.openverse.org
]);

const results = await library.searchAll({ query: "calm piano", kind: "music", commercialOnly: true });
const item = results.find((r) => r.result?.items.length)?.result?.items[0];

if (item) {
  const resolved = await library.resolve(item);
  const src = await copyToMyStorage(resolved.src); // stable URL with Range and CORS support
  importAudio(project, { ...resolved, src }, { atUs: 0, volume: 0.35, fadeInUs: 1_000_000 });
}

Providers

ProviderSourceKeys
staticProvider({ id, label, entries })Files you host, with the license you set per entrynone
openverseProvider({ baseUrl?, accessToken?, sources? })Openverse: openly licensed music, sound effects and morenone; anonymous use is rate-limited
freesoundProvider({ token?, baseUrl? })Freesound: sound effects and recordingsAPI key, or a proxy that adds it
httpProvider({ id, label, endpoint, kinds?, headers? })Your backend, through the HTTP contract belowyours

The network providers also take an optional fetch, so you can route requests your own way.

Searching

AudioQuery fieldMeaning
querySearch text
kind"music", "sfx" or "voice"
commercialOnlyOnly items whose license allows commercial use
minDurationS, maxDurationSLength filter in seconds
page, pageSizePaging (page starts at 1)
signalCancel

library.search(providerId, query) searches one provider. library.searchAll(query) searches all of them and returns { provider, result?, error? } per provider, so one failing source doesn't fail the rest.

Each AudioItem has title, kind, durationUs, creator, license (null when unknown), attribution, landingUrl, tags, and a previewUrl that a plain <audio> element can play.

Importing

importAudio(project, resolved, options) dispatches asset/add (with name, source, license and attribution) and clip/add as one undo step. It returns { assetId, clipId, trackId }.

OptionDefault
atUsThe playhead
trackIdThe first audio track free for the clip's span, else a new track named after the kind ("Music", "Sound effects", "Voice")
durationUsThe file's duration, else 10 s
volume, fadeInUs, fadeOutUsClip fields
assetId, clipId, trackNameGenerated
assetOnlyfalse. true adds the asset without a clip

Copy remote files into your own storage before importing and pass the new src. The renderer needs a URL that supports Range requests and CORS, and won't break if the provider removes the file.

HTTP contract for httpProvider

GET  {endpoint}/search?q=&kind=&commercialOnly=&minDurationS=&maxDurationS=&page=&pageSize=  → AudioSearchResult
GET  {endpoint}/items/{id}                                                                  → AudioItem | 404
POST {endpoint}/resolve   { item }                                                          → ResolvedAudio

AI-generated audio

Generators produce sound effects, music or voice. The package includes elevenLabsGenerator and toneGenerator (an offline test tone). Write an adapter for any other service with defineGenerator.

import { createAudioLibrary, elevenLabsGenerator, importAudio } from "@miraiclip/audio-sources";

declare function storeFile(file: Blob, name: string): Promise<string>;

const library = createAudioLibrary([], {
  generators: [elevenLabsGenerator({ baseUrl: "/api/elevenlabs", plan: "paid" })],
});

const job = library.generate(
  "elevenlabs",
  { kind: "sfx", prompt: "glass shattering", durationS: 2 },
  { storeFile },
);
job.onChange((j) => console.log(j.status, j.progress));

const { resolved } = await job.result;
importAudio(project, resolved, { volume: 0.9 });
RequestFields
{ kind: "sfx" }prompt, durationS?
{ kind: "music" }prompt, durationS?, instrumental?, lyrics?
{ kind: "voice" }text, voice?, language?, style?

The job checks the request against the generator's kinds and limits, stores the file with storeFile (without it, the result uses an object URL or a data: URL), and records the generator's terms as the asset's license.

To keep vendor keys on the server, serve generators with createGeneratorHandler(generators, { basePath }) (a Fetch API handler) and load them in the browser with remoteGenerators(endpoint), then library.addGenerator(g) for each.

elevenLabsGenerator's plan sets the recorded terms: "free" (the default) is recorded as non-commercial with a credit line, "paid" as commercial. Confirm the vendor's current terms for your use.

Licenses and credits

Every item and generated file carries an AssetLicense: { id, url?, commercial, attributionRequired }.

import { creditsFor, licenseReport } from "@miraiclip/core";

const doc = project.getState().doc;
creditsFor(doc); // credit lines for an end card or description
licenseReport(doc, { commercial: true }); // non-commercial, unknown or uncredited assets
  • Filter with commercialOnly: true for commercial work.
  • Set attribution on staticProvider entries whose license requires credit. Without it, licenseReport reports missing-attribution.
  • API terms are separate from item licenses. Freesound's API is free for non-commercial use only; commercial use needs an agreement with Freesound, even for CC0 sounds.
  • parseCreativeCommons(code, version?) and creditLine(title, creator, license) help when you write your own provider.

This is not legal advice. Check each source's terms for your use.

More on provenance fields: Assets. Audio search and generation as AI tools (audioToolDefinitions, runAudioTool): AI tools.

On this page