Miraiclip SDK
API reference

@miraiclip/audio-sources

Stock and generated audio for Miraiclip: one AudioProvider contract (search, resolve, generate) with Openverse, Freesound, static-catalog and HTTP-backend adapters, a one-transaction importAudio, and LLM tool definitions.

npm install @miraiclip/audio-sources
Version0.2.1
Runs inBrowser or Node
Exports70
GuidesAudio

Functions

audioAssetPayload

The asset/add payload for a resolved file, provenance included.

function audioAssetPayload(resolved: ResolvedAudio, id: string): {
    attribution?: string;
    license?: _miraiclip_core.AssetLicense;
    durationUs?: number;
    id: string;
    kind: "audio";
    src: string;
    name: string;
    source: _miraiclip_core.AssetSource;
}

audioToolDefinitions

function audioToolDefinitions(library: AudioLibrary, options?: AudioToolDefinitionsOptions): unknown[]

Guides: Editing assistant, Audio

createAudioLibrary

function createAudioLibrary(providers: AudioProvider[], options?: CreateAudioLibraryOptions): AudioLibrary

Guides: Audio

createGeneratorHandler

Returns a Fetch-API handler; resolves null for paths it doesn't own.

function createGeneratorHandler(generators: AudioGenerator[], options?: GeneratorHandlerOptions): (request: Request) => Promise<Response | null>

Guides: Audio

creditLine

A conventional credit line: “Title” by Creator (License).

function creditLine(title: string, creator: string | undefined, license: AssetLicense | null, landingUrl?: string): string

Guides: Audio

defineGenerator

Check an adapter's shape once, at registration (typos fail early).

function defineGenerator<G extends AudioGenerator>(generator: G): G

Guides: Audio

describeGenerator

The serializable description of a generator (what a backend advertises).

function describeGenerator(generator: AudioGenerator): GeneratorInfo

elevenLabsGenerator

function elevenLabsGenerator(options?: ElevenLabsGeneratorOptions): AudioGenerator

Guides: Audio

freesoundItem

function freesoundItem(raw: FreesoundSound, provider?: string): AudioItem

freesoundProvider

function freesoundProvider(options?: FreesoundProviderOptions): AudioProvider

Guides: Audio

httpProvider

function httpProvider(options: HttpProviderOptions): AudioProvider

Guides: Audio

importAudio

Add a resolved file to a project in ONE transaction (one undo step): asset/add with source, license and attribution, then clip/add on an audio track. If the app copies remote files into its own storage, do that first and pass { ...resolved, src: storedUrl }.

function importAudio(project: Project, resolved: ResolvedAudio, options?: ImportAudioOptions): ImportAudioResult

Guides: Asset import, Collaboration, Audio

nameFor

A readable asset name from the request.

function nameFor(request: GenerateRequest): string

openverseItem

function openverseItem(raw: OpenverseAudio, provider?: string): AudioItem

openverseProvider

function openverseProvider(options?: OpenverseProviderOptions): AudioProvider

Guides: Audio

parseCreativeCommons

Creative Commons → AssetLicense. Accepts the forms providers use: codes ("by-nc", "cc0", "pdm"), names ("Attribution Noncommercial", "Creative Commons 0") and deed URLs ("https://creativecommons.org/licenses/by-sa/4.0/").

  • commercial: false for any NC variant.
  • attributionRequired: false for CC0 and the Public Domain Mark. Returns null for anything unrecognized — callers treat that as unlicensed.
function parseCreativeCommons(input: string | null | undefined, version?: string | null): AssetLicense | null

Guides: Audio

remoteGenerator

One remote generator from its advertised info.

function remoteGenerator(endpoint: string, info: GeneratorInfo, options?: RemoteGeneratorsOptions): AudioGenerator

remoteGenerators

The generators a backend serves, as browser-side AudioGenerators.

function remoteGenerators(endpoint: string, options?: RemoteGeneratorsOptions): Promise<AudioGenerator[]>

Guides: Audio

runAudioTool

function runAudioTool(name: string, input: unknown, ctx: RunAudioToolContext): Promise<AudioToolResult>

Guides: Editing assistant, Audio

startGeneration

Run a generator as a cancellable job and normalize its output into ResolvedAudio.

function startGeneration(generator: AudioGenerator, request: GenerateRequest, options?: StartGenerationOptions): GenerationJob

staticProvider

function staticProvider(options: StaticProviderOptions): AudioProvider

Guides: Audio

toneGenerator

function toneGenerator(options?: ToneGeneratorOptions): AudioGenerator

Guides: Audio

validateRequest

Reject requests a generator can't take, with messages fit for users and agents.

function validateRequest(generator: Pick<AudioGenerator, "id" | "kinds" | "limits">, request: GenerateRequest): string | null

Classes

AudioSourceError

A provider request failed (network, HTTP status, or an unexpected payload).

Extends Error.

new AudioSourceError(provider: string, message: string, status?: number | undefined)
MemberTypeDescription
providerstring
status?number | undefined

Constants

AUDIO_TOOL_NAMES

const AUDIO_TOOL_NAMES: readonly ["search_audio", "add_audio", "generate_audio", "list_voices"]

Interfaces

AudioGenerator

MemberTypeDescription
idstring
labelstring
kindsAudioKind[]
models?GeneratorModel[]
voices?(options?: { signal?: AbortSignal; }) => Promise<GeneratorVoice[]>Voices for kind: "voice".
limits?Partial<Record<AudioKind, GeneratorLimits>>
paramsSchema?Partial<Record<AudioKind, ParamsSchema>>
termsGeneratorTerms
generate(request: GenerateRequest, options: GenerateOptions) => Promise<GeneratedAudio>Produce audio. Sync vendors return the file; job-based vendors poll or stream here and report through onProgress. Must honor signal.

AudioItem

One search result: enough to list, preview and credit it.

MemberTypeDescription
providerstringProvider id this item came from.
idstringThe provider's id for the item (stable; used by resolve / getItem).
titlestring
kind?AudioKind
durationUs?Us
creator?string
previewUrl?stringA URL a plain <audio> element can play for previewing (no CORS needed).
landingUrl?stringWhere a person can see the item and its license on the provider's site.
licenseAssetLicense | nullnull when the provider gives no license (treat as not safe to use).
attribution?stringThe credit line the license asks for, when it asks for one.
tags?string[]

Guides: Audio

AudioLibrary

The set of providers (search) and generators (AI) an app offers. UIs, the AI tools and the MCP server all read from one library, so adding a source or swapping a generation vendor never touches them. Items from searches are remembered, so an agent can add by id.

MemberTypeDescription
providers() => AudioProvider[]
get(id: string) => AudioProvider | undefined
searchable(kind?: AudioQuery["kind"]) => AudioProvider[]Providers that can search, optionally for one kind.
search(providerId: string, query: AudioQuery) => Promise<AudioSearchResult>
searchAll(query: AudioQuery, providerIds?: string[]) => Promise<{ provider: string; result?: AudioSearchResult; error?: string; }[]>Search several providers at once; failures are reported per provider.
item(providerId: string, itemId: string) => Promise<AudioItem | null>A previously seen item, or a lookup through the provider's getItem.
resolve(item: AudioItem, options?: { signal?: AbortSignal; }) => Promise<ResolvedAudio>
generators(kind?: GenerateRequest["kind"]) => AudioGenerator[]Generators, optionally those that can make one kind.
generator(id: string) => AudioGenerator | undefined
addGenerator(generator: AudioGenerator) => voidRegister a generator later (e.g. after remoteGenerators() resolves). Replaces one with the same id.
generate(generatorId: string, request: GenerateRequest, options?: StartGenerationOptions) => GenerationJobStart a generation job (validated against the generator's kinds and limits).
onGeneratorsChange(listener: () => void) => () => voidSubscribe to generator registrations.

AudioProvider

One searchable audio source: a stock library, an in-app catalog, your backend. (Generation services are AudioGenerators — see generate/types.) Network-facing adapters take fetch and a base URL so apps can route them through their own backend (keys stay server-side, CORS is a non-issue).

MemberTypeDescription
idstring
labelstring
capabilitiesAudioProviderCapabilities
notice?stringShort note shown to users, e.g. terms of use.
search?(query: AudioQuery) => Promise<AudioSearchResult>
getItem?(id: string, options?: { signal?: AbortSignal; }) => Promise<AudioItem | null>Look up one item by id (lets an agent add by id after a search).
resolve(item: AudioItem, options?: { signal?: AbortSignal; }) => Promise<ResolvedAudio>Turn an item into a playable file with its provenance.

AudioProviderCapabilities

MemberTypeDescription
search?AudioKind[]Kinds search can return. Absent: the provider can't search.

AudioQuery

A search. Filters a provider can't apply server-side are applied to the page it returns.

MemberTypeDescription
querystring
kind?AudioKind
commercialOnly?booleanOnly items whose license allows commercial use.
minDurationS?number
maxDurationS?number
page?number1-based.
pageSize?number
signal?AbortSignal

Guides: Audio

AudioSearchResult

MemberTypeDescription
itemsAudioItem[]
pagenumber
hasMoreboolean
total?number

AudioToolDefinitionsOptions

The AI surface: audio search, import and generation as LLM tools, the same shape as core's toToolDefinitions. runAudioTool executes a tool call against a library and a project; results are small JSON objects meant to go straight back to the model. Adding audio still lands as ordinary core commands (one transaction), so undo, history and collaboration behave as usual.

MemberTypeDescription
style?"anthropic" | "openai""anthropic" (default) → { name, description, input_schema }; "openai" → function tools.

CreateAudioLibraryOptions

MemberTypeDescription
generators?AudioGenerator[]

ElevenLabsGeneratorOptions

MemberTypeDescription
apiKey?stringAPI key. Omit when baseUrl is a proxy that authenticates for you.
baseUrl?stringDefault "https://api.elevenlabs.io".
plan?"free" | "paid"Plan, for the output terms recorded on assets. Default "free".
defaultVoiceId?stringVoice used when a voice request names none (default: the first voice in the account).
outputFormat?stringElevenLabs output format (default "mp3_44100_128").
fetch?FetchLike
id?string
label?string

FreesoundProviderOptions

MemberTypeDescription
token?stringAPI key. Omit when baseUrl is a proxy that authenticates for you.
baseUrl?stringDefault "https://freesound.org/apiv2".
fetch?FetchLike
id?string
label?string

FreesoundSound

MemberTypeDescription
idnumber
name?string
username?string
license?string
duration?number
url?string
tags?string[]
previews?Record<string, string>

GeneratedAudio

The bytes (or a fetchable URL) a generator produced, plus what it is.

MemberTypeDescription
dataBlob | ArrayBuffer | Uint8Array | { url: string; }The audio. A { url } must be fetchable by whoever stores it.
mimeTypestring
durationUs?number
id?stringThe vendor's id for this generation (request id, song id, …).
model?stringThe model that produced it.
license?AssetLicenseOverrides the generator's terms for this output (e.g. per-model terms).
attribution?string

GeneratedResult

MemberTypeDescription
resolvedResolvedAudioA playable file with provenance, ready for importAudio.
file?BlobThe bytes, when the generator returned bytes (or they were fetched to store them).

GenerateOptions

MemberTypeDescription
model?stringA model id from generator.models (default: the adapter's default for the kind).
seed?number
format?"mp3" | "wav" | "ogg" | "webm"Container/codec preference; adapters map it onto their own format names.
params?Record<string, unknown>Vendor-specific settings, validated by nothing here and passed through untouched.
signal?AbortSignal
onProgress?(progress: GenerateProgress) => void

GenerateProgress

MemberTypeDescription
progress?number0..1 when the vendor reports it.
message?string

GenerationJob

A generation in progress. One shape for sync and job-based vendors.

MemberTypeDescription
idstring
generatorstring
requestGenerateRequest
statusGenerationStatus
progressnumber | undefined
messagestring | undefined
errorstring | undefined
resultPromise<GeneratedResult>
cancel() => void
onChange(listener: (job: GenerationJob) => void) => () => voidCalled on every status/progress change. Returns an unsubscribe.

GeneratorHandlerOptions

MemberTypeDescription
basePath?stringPath prefix the handler is mounted at, e.g. "/api/generate".

GeneratorLimits

MemberTypeDescription
minDurationS?number
maxDurationS?number
maxPromptChars?number
maxTextChars?number

GeneratorModel

MemberTypeDescription
idstring
labelstring
kindsAudioKind[]

GeneratorTerms

MemberTypeDescription
licenseAssetLicenseWhat the output may be used for, as recorded on the asset.
attribution?stringCredit line the terms ask for, if any.
notice?stringShort note shown to users.

GeneratorVoice

MemberTypeDescription
idstring
namestring
language?string
gender?string
description?string
previewUrl?string

HttpProviderOptions

MemberTypeDescription
idstring
labelstring
endpointstring
kinds?AudioKind[]
headers?Record<string, string>
fetch?FetchLike
notice?string

ImportAudioOptions

MemberTypeDescription
atUs?UsTimeline position for the clip (default: the playhead).
trackId?stringTarget track. Default: the first audio track free for the clip's span, else a new one.
durationUs?UsClip length (default: the file's duration, then 10 s).
volume?number
fadeInUs?Us
fadeOutUs?Us
assetId?string
clipId?string
trackName?stringName for a newly created track (default "Music" / "Sound effects" / "Voice" / "Audio").
assetOnly?booleanAdd the asset only (no clip).

ImportAudioResult

MemberTypeDescription
assetIdstring
clipId?string
trackId?string

MusicRequest

MemberTypeDescription
kind"music"
promptstring
durationS?number
instrumental?boolean
lyrics?stringLyrics to sing, when the model supports vocals.

OpenverseAudio

The parts of an Openverse audio result this adapter reads.

MemberTypeDescription
idstring
title?string | null
url?string | null
foreign_landing_url?string | null
creator?string | null
license?string | null
license_version?string | null
license_url?string | null
attribution?string | null
category?string | null
duration?number | null
filetype?string | null
tags?{ name: string; }[] | null
source?string | null

OpenverseProviderOptions

MemberTypeDescription
baseUrl?stringDefault "https://api.openverse.org". Point at your backend proxy to avoid CORS/rate limits.
accessToken?stringOAuth access token for registered apps (higher rate limits).
sources?string[]Restrict to these upstream sources, e.g. ["freesound", "jamendo", "wikimedia_audio"].
fetch?FetchLike
id?string
label?string

RemoteGeneratorsOptions

MemberTypeDescription
fetch?FetchLike
headers?Record<string, string>

ResolvedAudio

A playable file plus everything asset/add records about it. src is a URL the renderer can fetch (Range + CORS) — apps that copy remote files into their own storage replace it before importing (see importAudio).

MemberTypeDescription
srcstring
mimeType?string
durationUs?Us
namestring
sourceAssetSource
license?AssetLicense
attribution?string
kind?AudioKind

RunAudioToolContext

MemberTypeDescription
libraryAudioLibrary
projectProject
store?(resolved: ResolvedAudio) => Promise<ResolvedAudio>Prepare a resolved file before it's added — typically copy it into your own storage and return { ...resolved, src: storedUrl }.
storeFile?(file: Blob, name: string) => Promise<string>Persist a generated file and return its durable URL (see StartGenerationOptions.storeFile).
onGeneration?(job: GenerationJob) => voidObserve generation progress (e.g. to show it in a chat UI).
signal?AbortSignal

SfxRequest

The vendor-neutral generation contract. Nothing here names a service: an ElevenLabs, OpenAI, Google, ByteDance or self-hosted adapter is a small object implementing AudioGenerator. Common intent goes in the request; anything vendor-specific goes in options.params, described by the adapter's paramsSchema so UIs and AI tools can offer it without code.

MemberTypeDescription
kind"sfx"
promptstring
durationS?number

StartGenerationOptions

Extends Omit<GenerateOptions, "signal" \| "onProgress">.

MemberTypeDescription
signal?AbortSignal
storeFile?(file: Blob, name: string) => Promise<string>Persist the file and return its durable URL (your storage). Without it, the result uses an object URL (browsers: session-only) or a data: URL.
name?stringAsset name (default: derived from the prompt/text).
probeDurationUs?(file: Blob) => Promise<number | undefined>Measure duration when the vendor doesn't report it (default: Web Audio decode when available).

StaticAudioEntry

MemberTypeDescription
idstring
titlestring
srcstring
kind?AudioKind
durationUs?number
creator?string
license?AssetLicense | null
attribution?string
tags?string[]
landingUrl?string
mimeType?string

StaticProviderOptions

MemberTypeDescription
idstring
labelstring
entriesStaticAudioEntry[]
notice?string

ToneGeneratorOptions

MemberTypeDescription
id?string
label?string
latencyMs?numberSimulated latency in ms (progress is reported along the way). Default 0.
sampleRate?number

VoiceRequest

MemberTypeDescription
kind"voice"
textstringWhat to say.
voice?stringA voice id from generator.voices(). Default: the adapter's default voice.
language?stringBCP-47 / ISO 639-1 language hint, e.g. "en", "hi".
style?stringFree-form delivery direction ("warm, slow, smiling"), for models that take one.

Types

AudioKind

What a sound is for. Providers map their own categories onto these.

type AudioKind = "music" | "sfx" | "voice";

AudioToolName

type AudioToolName = (typeof AUDIO_TOOL_NAMES)[number];

AudioToolResult

type AudioToolResult = {
    ok: true;
    result: unknown;
} | {
    ok: false;
    error: string;
};

FetchLike

type FetchLike = (input: string, init?: RequestInit) => Promise<Response>;

GenerateRequest

type GenerateRequest = SfxRequest | MusicRequest | VoiceRequest;

GenerationStatus

type GenerationStatus = "running" | "done" | "failed" | "canceled";

GeneratorInfo

Everything about a generator except generate — what a backend advertises.

type GeneratorInfo = Omit<AudioGenerator, "generate" | "voices"> & {
    hasVoices: boolean;
};

ParamsSchema

A JSON Schema object describing options.params for one kind.

type ParamsSchema = {
    type: "object";
    properties: Record<string, Record<string, unknown>>;
    required?: string[];
    additionalProperties?: boolean;
};

On this page