@miraiclip/audio-sources
Stock and generated audio for Miraiclip: one AudioProvider contract (search, resolve, generate) with Openverse, Freesound, static-catalog and HTTP-backend adapters, a one-transaction importAudio, and LLM tool definitions.
npm install @miraiclip/audio-sources| Version | 0.2.1 |
| Runs in | Browser or Node |
| Exports | 70 |
| Guides | Audio |
Functions
audioAssetPayload
The asset/add payload for a resolved file, provenance included.
function audioAssetPayload(resolved: ResolvedAudio, id: string): {
attribution?: string;
license?: _miraiclip_core.AssetLicense;
durationUs?: number;
id: string;
kind: "audio";
src: string;
name: string;
source: _miraiclip_core.AssetSource;
}audioToolDefinitions
function audioToolDefinitions(library: AudioLibrary, options?: AudioToolDefinitionsOptions): unknown[]Guides: Editing assistant, Audio
createAudioLibrary
function createAudioLibrary(providers: AudioProvider[], options?: CreateAudioLibraryOptions): AudioLibraryGuides: Audio
createGeneratorHandler
Returns a Fetch-API handler; resolves null for paths it doesn't own.
function createGeneratorHandler(generators: AudioGenerator[], options?: GeneratorHandlerOptions): (request: Request) => Promise<Response | null>Guides: Audio
creditLine
A conventional credit line: “Title” by Creator (License).
function creditLine(title: string, creator: string | undefined, license: AssetLicense | null, landingUrl?: string): stringGuides: Audio
defineGenerator
Check an adapter's shape once, at registration (typos fail early).
function defineGenerator<G extends AudioGenerator>(generator: G): GGuides: Audio
describeGenerator
The serializable description of a generator (what a backend advertises).
function describeGenerator(generator: AudioGenerator): GeneratorInfoelevenLabsGenerator
function elevenLabsGenerator(options?: ElevenLabsGeneratorOptions): AudioGeneratorGuides: Audio
freesoundItem
function freesoundItem(raw: FreesoundSound, provider?: string): AudioItemfreesoundProvider
function freesoundProvider(options?: FreesoundProviderOptions): AudioProviderGuides: Audio
httpProvider
function httpProvider(options: HttpProviderOptions): AudioProviderGuides: Audio
importAudio
Add a resolved file to a project in ONE transaction (one undo step):
asset/add with source, license and attribution, then clip/add on an
audio track. If the app copies remote files into its own storage, do that
first and pass { ...resolved, src: storedUrl }.
function importAudio(project: Project, resolved: ResolvedAudio, options?: ImportAudioOptions): ImportAudioResultGuides: Asset import, Collaboration, Audio
nameFor
A readable asset name from the request.
function nameFor(request: GenerateRequest): stringopenverseItem
function openverseItem(raw: OpenverseAudio, provider?: string): AudioItemopenverseProvider
function openverseProvider(options?: OpenverseProviderOptions): AudioProviderGuides: Audio
parseCreativeCommons
Creative Commons → AssetLicense. Accepts the forms providers use:
codes ("by-nc", "cc0", "pdm"), names ("Attribution Noncommercial",
"Creative Commons 0") and deed URLs
("https://creativecommons.org/licenses/by-sa/4.0/").
commercial: false for any NC variant.attributionRequired: false for CC0 and the Public Domain Mark. Returns null for anything unrecognized — callers treat that as unlicensed.
function parseCreativeCommons(input: string | null | undefined, version?: string | null): AssetLicense | nullGuides: Audio
remoteGenerator
One remote generator from its advertised info.
function remoteGenerator(endpoint: string, info: GeneratorInfo, options?: RemoteGeneratorsOptions): AudioGeneratorremoteGenerators
The generators a backend serves, as browser-side AudioGenerators.
function remoteGenerators(endpoint: string, options?: RemoteGeneratorsOptions): Promise<AudioGenerator[]>Guides: Audio
runAudioTool
function runAudioTool(name: string, input: unknown, ctx: RunAudioToolContext): Promise<AudioToolResult>Guides: Editing assistant, Audio
startGeneration
Run a generator as a cancellable job and normalize its output into ResolvedAudio.
function startGeneration(generator: AudioGenerator, request: GenerateRequest, options?: StartGenerationOptions): GenerationJobstaticProvider
function staticProvider(options: StaticProviderOptions): AudioProviderGuides: Audio
toneGenerator
function toneGenerator(options?: ToneGeneratorOptions): AudioGeneratorGuides: Audio
validateRequest
Reject requests a generator can't take, with messages fit for users and agents.
function validateRequest(generator: Pick<AudioGenerator, "id" | "kinds" | "limits">, request: GenerateRequest): string | nullClasses
AudioSourceError
A provider request failed (network, HTTP status, or an unexpected payload).
Extends Error.
new AudioSourceError(provider: string, message: string, status?: number | undefined)| Member | Type | Description |
|---|---|---|
provider | string | |
status? | number | undefined |
Constants
AUDIO_TOOL_NAMES
const AUDIO_TOOL_NAMES: readonly ["search_audio", "add_audio", "generate_audio", "list_voices"]Interfaces
AudioGenerator
| Member | Type | Description |
|---|---|---|
id | string | |
label | string | |
kinds | AudioKind[] | |
models? | GeneratorModel[] | |
voices? | (options?: { signal?: AbortSignal; }) => Promise<GeneratorVoice[]> | Voices for kind: "voice". |
limits? | Partial<Record<AudioKind, GeneratorLimits>> | |
paramsSchema? | Partial<Record<AudioKind, ParamsSchema>> | |
terms | GeneratorTerms | |
generate | (request: GenerateRequest, options: GenerateOptions) => Promise<GeneratedAudio> | Produce audio. Sync vendors return the file; job-based vendors poll or stream here and report through onProgress. Must honor signal. |
AudioItem
One search result: enough to list, preview and credit it.
| Member | Type | Description |
|---|---|---|
provider | string | Provider id this item came from. |
id | string | The provider's id for the item (stable; used by resolve / getItem). |
title | string | |
kind? | AudioKind | |
durationUs? | Us | |
creator? | string | |
previewUrl? | string | A URL a plain <audio> element can play for previewing (no CORS needed). |
landingUrl? | string | Where a person can see the item and its license on the provider's site. |
license | AssetLicense | null | null when the provider gives no license (treat as not safe to use). |
attribution? | string | The credit line the license asks for, when it asks for one. |
tags? | string[] |
Guides: Audio
AudioLibrary
The set of providers (search) and generators (AI) an app offers. UIs, the AI tools and the MCP server all read from one library, so adding a source or swapping a generation vendor never touches them. Items from searches are remembered, so an agent can add by id.
| Member | Type | Description |
|---|---|---|
providers | () => AudioProvider[] | |
get | (id: string) => AudioProvider | undefined | |
searchable | (kind?: AudioQuery["kind"]) => AudioProvider[] | Providers that can search, optionally for one kind. |
search | (providerId: string, query: AudioQuery) => Promise<AudioSearchResult> | |
searchAll | (query: AudioQuery, providerIds?: string[]) => Promise<{ provider: string; result?: AudioSearchResult; error?: string; }[]> | Search several providers at once; failures are reported per provider. |
item | (providerId: string, itemId: string) => Promise<AudioItem | null> | A previously seen item, or a lookup through the provider's getItem. |
resolve | (item: AudioItem, options?: { signal?: AbortSignal; }) => Promise<ResolvedAudio> | |
generators | (kind?: GenerateRequest["kind"]) => AudioGenerator[] | Generators, optionally those that can make one kind. |
generator | (id: string) => AudioGenerator | undefined | |
addGenerator | (generator: AudioGenerator) => void | Register a generator later (e.g. after remoteGenerators() resolves). Replaces one with the same id. |
generate | (generatorId: string, request: GenerateRequest, options?: StartGenerationOptions) => GenerationJob | Start a generation job (validated against the generator's kinds and limits). |
onGeneratorsChange | (listener: () => void) => () => void | Subscribe to generator registrations. |
AudioProvider
One searchable audio source: a stock library, an in-app catalog, your
backend. (Generation services are AudioGenerators — see generate/types.)
Network-facing adapters take fetch and a base URL so apps can route them
through their own backend (keys stay server-side, CORS is a non-issue).
| Member | Type | Description |
|---|---|---|
id | string | |
label | string | |
capabilities | AudioProviderCapabilities | |
notice? | string | Short note shown to users, e.g. terms of use. |
search? | (query: AudioQuery) => Promise<AudioSearchResult> | |
getItem? | (id: string, options?: { signal?: AbortSignal; }) => Promise<AudioItem | null> | Look up one item by id (lets an agent add by id after a search). |
resolve | (item: AudioItem, options?: { signal?: AbortSignal; }) => Promise<ResolvedAudio> | Turn an item into a playable file with its provenance. |
AudioProviderCapabilities
| Member | Type | Description |
|---|---|---|
search? | AudioKind[] | Kinds search can return. Absent: the provider can't search. |
AudioQuery
A search. Filters a provider can't apply server-side are applied to the page it returns.
| Member | Type | Description |
|---|---|---|
query | string | |
kind? | AudioKind | |
commercialOnly? | boolean | Only items whose license allows commercial use. |
minDurationS? | number | |
maxDurationS? | number | |
page? | number | 1-based. |
pageSize? | number | |
signal? | AbortSignal |
Guides: Audio
AudioSearchResult
| Member | Type | Description |
|---|---|---|
items | AudioItem[] | |
page | number | |
hasMore | boolean | |
total? | number |
AudioToolDefinitionsOptions
The AI surface: audio search, import and generation as LLM tools, the same shape as
core's toToolDefinitions. runAudioTool executes a tool call against a
library and a project; results are small JSON objects meant to go straight
back to the model. Adding audio still lands as ordinary core commands
(one transaction), so undo, history and collaboration behave as usual.
| Member | Type | Description |
|---|---|---|
style? | "anthropic" | "openai" | "anthropic" (default) → { name, description, input_schema }; "openai" → function tools. |
CreateAudioLibraryOptions
| Member | Type | Description |
|---|---|---|
generators? | AudioGenerator[] |
ElevenLabsGeneratorOptions
| Member | Type | Description |
|---|---|---|
apiKey? | string | API key. Omit when baseUrl is a proxy that authenticates for you. |
baseUrl? | string | Default "https://api.elevenlabs.io". |
plan? | "free" | "paid" | Plan, for the output terms recorded on assets. Default "free". |
defaultVoiceId? | string | Voice used when a voice request names none (default: the first voice in the account). |
outputFormat? | string | ElevenLabs output format (default "mp3_44100_128"). |
fetch? | FetchLike | |
id? | string | |
label? | string |
FreesoundProviderOptions
| Member | Type | Description |
|---|---|---|
token? | string | API key. Omit when baseUrl is a proxy that authenticates for you. |
baseUrl? | string | Default "https://freesound.org/apiv2". |
fetch? | FetchLike | |
id? | string | |
label? | string |
FreesoundSound
| Member | Type | Description |
|---|---|---|
id | number | |
name? | string | |
username? | string | |
license? | string | |
duration? | number | |
url? | string | |
tags? | string[] | |
previews? | Record<string, string> |
GeneratedAudio
The bytes (or a fetchable URL) a generator produced, plus what it is.
| Member | Type | Description |
|---|---|---|
data | Blob | ArrayBuffer | Uint8Array | { url: string; } | The audio. A { url } must be fetchable by whoever stores it. |
mimeType | string | |
durationUs? | number | |
id? | string | The vendor's id for this generation (request id, song id, …). |
model? | string | The model that produced it. |
license? | AssetLicense | Overrides the generator's terms for this output (e.g. per-model terms). |
attribution? | string |
GeneratedResult
| Member | Type | Description |
|---|---|---|
resolved | ResolvedAudio | A playable file with provenance, ready for importAudio. |
file? | Blob | The bytes, when the generator returned bytes (or they were fetched to store them). |
GenerateOptions
| Member | Type | Description |
|---|---|---|
model? | string | A model id from generator.models (default: the adapter's default for the kind). |
seed? | number | |
format? | "mp3" | "wav" | "ogg" | "webm" | Container/codec preference; adapters map it onto their own format names. |
params? | Record<string, unknown> | Vendor-specific settings, validated by nothing here and passed through untouched. |
signal? | AbortSignal | |
onProgress? | (progress: GenerateProgress) => void |
GenerateProgress
| Member | Type | Description |
|---|---|---|
progress? | number | 0..1 when the vendor reports it. |
message? | string |
GenerationJob
A generation in progress. One shape for sync and job-based vendors.
| Member | Type | Description |
|---|---|---|
id | string | |
generator | string | |
request | GenerateRequest | |
status | GenerationStatus | |
progress | number | undefined | |
message | string | undefined | |
error | string | undefined | |
result | Promise<GeneratedResult> | |
cancel | () => void | |
onChange | (listener: (job: GenerationJob) => void) => () => void | Called on every status/progress change. Returns an unsubscribe. |
GeneratorHandlerOptions
| Member | Type | Description |
|---|---|---|
basePath? | string | Path prefix the handler is mounted at, e.g. "/api/generate". |
GeneratorLimits
| Member | Type | Description |
|---|---|---|
minDurationS? | number | |
maxDurationS? | number | |
maxPromptChars? | number | |
maxTextChars? | number |
GeneratorModel
| Member | Type | Description |
|---|---|---|
id | string | |
label | string | |
kinds | AudioKind[] |
GeneratorTerms
| Member | Type | Description |
|---|---|---|
license | AssetLicense | What the output may be used for, as recorded on the asset. |
attribution? | string | Credit line the terms ask for, if any. |
notice? | string | Short note shown to users. |
GeneratorVoice
| Member | Type | Description |
|---|---|---|
id | string | |
name | string | |
language? | string | |
gender? | string | |
description? | string | |
previewUrl? | string |
HttpProviderOptions
| Member | Type | Description |
|---|---|---|
id | string | |
label | string | |
endpoint | string | |
kinds? | AudioKind[] | |
headers? | Record<string, string> | |
fetch? | FetchLike | |
notice? | string |
ImportAudioOptions
| Member | Type | Description |
|---|---|---|
atUs? | Us | Timeline position for the clip (default: the playhead). |
trackId? | string | Target track. Default: the first audio track free for the clip's span, else a new one. |
durationUs? | Us | Clip length (default: the file's duration, then 10 s). |
volume? | number | |
fadeInUs? | Us | |
fadeOutUs? | Us | |
assetId? | string | |
clipId? | string | |
trackName? | string | Name for a newly created track (default "Music" / "Sound effects" / "Voice" / "Audio"). |
assetOnly? | boolean | Add the asset only (no clip). |
ImportAudioResult
| Member | Type | Description |
|---|---|---|
assetId | string | |
clipId? | string | |
trackId? | string |
MusicRequest
| Member | Type | Description |
|---|---|---|
kind | "music" | |
prompt | string | |
durationS? | number | |
instrumental? | boolean | |
lyrics? | string | Lyrics to sing, when the model supports vocals. |
OpenverseAudio
The parts of an Openverse audio result this adapter reads.
| Member | Type | Description |
|---|---|---|
id | string | |
title? | string | null | |
url? | string | null | |
foreign_landing_url? | string | null | |
creator? | string | null | |
license? | string | null | |
license_version? | string | null | |
license_url? | string | null | |
attribution? | string | null | |
category? | string | null | |
duration? | number | null | |
filetype? | string | null | |
tags? | { name: string; }[] | null | |
source? | string | null |
OpenverseProviderOptions
| Member | Type | Description |
|---|---|---|
baseUrl? | string | Default "https://api.openverse.org". Point at your backend proxy to avoid CORS/rate limits. |
accessToken? | string | OAuth access token for registered apps (higher rate limits). |
sources? | string[] | Restrict to these upstream sources, e.g. ["freesound", "jamendo", "wikimedia_audio"]. |
fetch? | FetchLike | |
id? | string | |
label? | string |
RemoteGeneratorsOptions
| Member | Type | Description |
|---|---|---|
fetch? | FetchLike | |
headers? | Record<string, string> |
ResolvedAudio
A playable file plus everything asset/add records about it. src is a
URL the renderer can fetch (Range + CORS) — apps that copy remote files
into their own storage replace it before importing (see importAudio).
| Member | Type | Description |
|---|---|---|
src | string | |
mimeType? | string | |
durationUs? | Us | |
name | string | |
source | AssetSource | |
license? | AssetLicense | |
attribution? | string | |
kind? | AudioKind |
RunAudioToolContext
| Member | Type | Description |
|---|---|---|
library | AudioLibrary | |
project | Project | |
store? | (resolved: ResolvedAudio) => Promise<ResolvedAudio> | Prepare a resolved file before it's added — typically copy it into your own storage and return { ...resolved, src: storedUrl }. |
storeFile? | (file: Blob, name: string) => Promise<string> | Persist a generated file and return its durable URL (see StartGenerationOptions.storeFile). |
onGeneration? | (job: GenerationJob) => void | Observe generation progress (e.g. to show it in a chat UI). |
signal? | AbortSignal |
SfxRequest
The vendor-neutral generation contract. Nothing here names a service: an
ElevenLabs, OpenAI, Google, ByteDance or self-hosted adapter is a small
object implementing AudioGenerator. Common intent goes in the request;
anything vendor-specific goes in options.params, described by the
adapter's paramsSchema so UIs and AI tools can offer it without code.
| Member | Type | Description |
|---|---|---|
kind | "sfx" | |
prompt | string | |
durationS? | number |
StartGenerationOptions
Extends Omit<GenerateOptions, "signal" \| "onProgress">.
| Member | Type | Description |
|---|---|---|
signal? | AbortSignal | |
storeFile? | (file: Blob, name: string) => Promise<string> | Persist the file and return its durable URL (your storage). Without it, the result uses an object URL (browsers: session-only) or a data: URL. |
name? | string | Asset name (default: derived from the prompt/text). |
probeDurationUs? | (file: Blob) => Promise<number | undefined> | Measure duration when the vendor doesn't report it (default: Web Audio decode when available). |
StaticAudioEntry
| Member | Type | Description |
|---|---|---|
id | string | |
title | string | |
src | string | |
kind? | AudioKind | |
durationUs? | number | |
creator? | string | |
license? | AssetLicense | null | |
attribution? | string | |
tags? | string[] | |
landingUrl? | string | |
mimeType? | string |
StaticProviderOptions
| Member | Type | Description |
|---|---|---|
id | string | |
label | string | |
entries | StaticAudioEntry[] | |
notice? | string |
ToneGeneratorOptions
| Member | Type | Description |
|---|---|---|
id? | string | |
label? | string | |
latencyMs? | number | Simulated latency in ms (progress is reported along the way). Default 0. |
sampleRate? | number |
VoiceRequest
| Member | Type | Description |
|---|---|---|
kind | "voice" | |
text | string | What to say. |
voice? | string | A voice id from generator.voices(). Default: the adapter's default voice. |
language? | string | BCP-47 / ISO 639-1 language hint, e.g. "en", "hi". |
style? | string | Free-form delivery direction ("warm, slow, smiling"), for models that take one. |
Types
AudioKind
What a sound is for. Providers map their own categories onto these.
type AudioKind = "music" | "sfx" | "voice";AudioToolName
type AudioToolName = (typeof AUDIO_TOOL_NAMES)[number];AudioToolResult
type AudioToolResult = {
ok: true;
result: unknown;
} | {
ok: false;
error: string;
};FetchLike
type FetchLike = (input: string, init?: RequestInit) => Promise<Response>;GenerateRequest
type GenerateRequest = SfxRequest | MusicRequest | VoiceRequest;GenerationStatus
type GenerationStatus = "running" | "done" | "failed" | "canceled";GeneratorInfo
Everything about a generator except generate — what a backend advertises.
type GeneratorInfo = Omit<AudioGenerator, "generate" | "voices"> & {
hasVoices: boolean;
};ParamsSchema
A JSON Schema object describing options.params for one kind.
type ParamsSchema = {
type: "object";
properties: Record<string, Record<string, unknown>>;
required?: string[];
additionalProperties?: boolean;
};