Miraiclip SDK

AWS

Run the render worker on ECS with Fargate, on Lambda, or on an EC2 instance.

All three options run the render worker image. Build it first, and push it to Amazon ECR.

OptionPick it for
ECS on FargateThe default. Containers behind a load balancer, no servers to manage
LambdaBursts of short distributed chunks, scaling to zero
EC2One machine with a fixed cost, or GPUs

The commands below use the AWS CLI v2. Set your region and account once:

export AWS_REGION=us-east-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export IMAGE=$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/miraiclip-render-worker:latest

Push the image to ECR

aws ecr create-repository --repository-name miraiclip-render-worker

aws ecr get-login-password | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com

docker build --platform linux/amd64 -t $IMAGE .
docker push $IMAGE

The image is x86-64 only (Google Chrome has no Linux Arm build), so use x86-64 everywhere below, not Graviton.

Store the worker's token in Secrets Manager:

aws secretsmanager create-secret --name miraiclip/render-token --secret-string "$(openssl rand -hex 32)"

ECS on Fargate

You run a service of worker tasks behind an Application Load Balancer, and point httpRunner (or your own backend) at the load balancer's URL.

1. Cluster and logs

aws ecs create-cluster --cluster-name miraiclip
aws logs create-log-group --log-group-name /ecs/miraiclip-render-worker

2. Task definition

The task's execution role needs the AmazonECSTaskExecutionRolePolicy managed policy, plus secretsmanager:GetSecretValue on the token secret.

task-definition.json
{
  "family": "miraiclip-render-worker",
  "requiresCompatibilities": ["FARGATE"],
  "networkMode": "awsvpc",
  "cpu": "2048",
  "memory": "4096",
  "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" },
  "executionRoleArn": "arn:aws:iam::<account-id>:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "name": "worker",
      "image": "<account-id>.dkr.ecr.<region>.amazonaws.com/miraiclip-render-worker:latest",
      "essential": true,
      "portMappings": [{ "containerPort": 8080, "protocol": "tcp" }],
      "secrets": [
        { "name": "RENDER_TOKEN", "valueFrom": "arn:aws:secretsmanager:<region>:<account-id>:secret:miraiclip/render-token" }
      ],
      "healthCheck": {
        "command": ["CMD-SHELL", "wget -qO- http://localhost:8080/healthz || exit 1"],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 30
      },
      "stopTimeout": 120,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/miraiclip-render-worker",
          "awslogs-region": "<region>",
          "awslogs-stream-prefix": "worker"
        }
      }
    }
  ]
}
aws ecs register-task-definition --cli-input-json file://task-definition.json

For 4K output, use "cpu": "4096" and "memory": "8192". Fargate gives each task 20 GiB of temporary disk, which is enough for most outputs; raise it with ephemeralStorage if not.

3. Load balancer

Create an Application Load Balancer and a target group in the EC2 console (or with aws elbv2), with these settings:

SettingValue
Target typeIP addresses
Protocol and portHTTP, 8080
Health check path/healthz
ListenerHTTPS 443 with an ACM certificate

Then change two defaults that matter for render jobs:

# A job sends nothing back until the file is done: wait up to 30 minutes, not 60 seconds.
aws elbv2 modify-load-balancer-attributes --load-balancer-arn <alb-arn> \
  --attributes Key=idle_timeout.timeout_seconds,Value=1800

# Each task renders one job at a time: send new jobs to the least busy task.
aws elbv2 modify-target-group-attributes --target-group-arn <target-group-arn> \
  --attributes Key=load_balancing.algorithm.type,Value=least_outstanding_requests

Set the idle timeout above your longest job. The maximum is 4000 seconds; for longer exports, split them with renderDistributed.

4. Service

aws ecs create-service \
  --cluster miraiclip \
  --service-name render-worker \
  --task-definition miraiclip-render-worker \
  --desired-count 2 \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[<subnet-a>,<subnet-b>],securityGroups=[<sg>],assignPublicIp=ENABLED}" \
  --load-balancers "targetGroupArn=<target-group-arn>,containerName=worker,containerPort=8080" \
  --health-check-grace-period-seconds 60

The tasks' security group only needs port 8080 open to the load balancer's security group. Tasks pull the image and fetch assets over the internet: use public subnets with assignPublicIp=ENABLED, or private subnets with a NAT gateway.

5. Autoscaling

Scale on CPU: an idle worker uses almost none, a rendering one uses all it has.

aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --resource-id service/miraiclip/render-worker \
  --scalable-dimension ecs:service:DesiredCount \
  --min-capacity 1 --max-capacity 20

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --resource-id service/miraiclip/render-worker \
  --scalable-dimension ecs:service:DesiredCount \
  --policy-name render-cpu \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{"TargetValue":60,"PredefinedMetricSpecification":{"PredefinedMetricType":"ECSServiceAverageCPUUtilization"}}'

6. Render

import { httpRunner, renderDistributed } from "@miraiclip/server-export";

const runner = httpRunner({
  endpoints: "https://render.example.com", // the load balancer's HTTPS listener
  headers: { authorization: `Bearer ${process.env.RENDER_TOKEN}` },
  concurrency: 8, // at most the number of running tasks
});

await renderDistributed(doc, { format: "mp4", out: "final.mp4", runner });

Or send single jobs with fetch, as in Send a render job.

Lambda

Lambda runs one request per instance, scales out by itself and costs nothing when idle. Its limits shape how you use it:

LimitValueWhat it means
Timeout15 minutesKeep each job short: use renderDistributed with enough chunks
Request body6 MBThe job JSON (the document). Keep media out of the document as data: URLs
Response body200 MB, with response streamingTurn streaming on, as below. Without it, the limit is 6 MB
Architecturex86-64Chrome has no Linux Arm build

1. Add the Lambda Web Adapter

The Lambda Web Adapter runs the worker's HTTP server unchanged inside Lambda. Add it on top of the worker image:

Dockerfile.lambda
FROM <account-id>.dkr.ecr.<region>.amazonaws.com/miraiclip-render-worker:latest
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.9.1 /lambda-adapter /opt/extensions/lambda-adapter
ENV AWS_LWA_INVOKE_MODE=response_stream \
    AWS_LWA_READINESS_CHECK_PATH=/healthz \
    HOME=/tmp

HOME=/tmp gives Chrome a writable home folder: everything else in a Lambda container is read-only.

export LAMBDA_IMAGE=$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/miraiclip-render-worker:lambda
docker build --platform linux/amd64 --provenance=false -f Dockerfile.lambda -t $LAMBDA_IMAGE .
docker push $LAMBDA_IMAGE

2. Create the function and its URL

The function's role needs the AWSLambdaBasicExecutionRole managed policy.

aws lambda create-function \
  --function-name miraiclip-render-worker \
  --package-type Image \
  --code ImageUri=$LAMBDA_IMAGE \
  --role arn:aws:iam::$ACCOUNT_ID:role/miraiclip-render-lambda \
  --architectures x86_64 \
  --memory-size 4096 \
  --timeout 900 \
  --ephemeral-storage Size=4096 \
  --environment "Variables={RENDER_TOKEN=<token>}"

aws lambda create-function-url-config \
  --function-name miraiclip-render-worker \
  --auth-type NONE \
  --invoke-mode RESPONSE_STREAM

Lambda gives CPU in proportion to memory: 4096 MB is a little over 2 vCPUs. Raise --memory-size (up to 10240) for faster chunks. --ephemeral-storage is the /tmp space each job writes its output to.

A function URL with --auth-type NONE also needs a resource policy that allows public calls. The Lambda console adds it when you create the URL there; from the CLI, see Control access to function URLs. The worker still checks RENDER_TOKEN on every job.

3. Render

const runner = httpRunner({
  endpoints: "https://<url-id>.lambda-url.<region>.on.aws/",
  headers: { authorization: `Bearer ${process.env.RENDER_TOKEN}` },
  concurrency: 32, // Lambda starts an instance per job in flight
});

await renderDistributed(doc, { format: "mp4", out: "final.mp4", runner, chunks: 32 });

The first job on a new instance also starts Chrome, which adds a few seconds. Many short chunks spread over many instances is where Lambda pays off.

EC2

One instance running the container. Use a compute-optimized x86-64 type, such as c7i.xlarge (4 vCPU, 8 GB), with Amazon Linux 2023:

sudo dnf install -y docker
sudo systemctl enable --now docker

aws ecr get-login-password | sudo docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com

sudo docker run -d --name render-worker --restart unless-stopped \
  -p 8080:8080 \
  -e RENDER_TOKEN=<token> \
  $IMAGE

The instance role needs AmazonEC2ContainerRegistryReadOnly to pull from ECR. Put the instance behind a load balancer or a reverse proxy for HTTPS rather than opening port 8080 to the internet. To use more cores on one machine, run several containers on different ports, one per 2 vCPUs, and list them all in httpRunner's endpoints.

Docker

Build the render worker image, run it, and send it render jobs.

Google Cloud

Run the render worker on Cloud Run or a Compute Engine VM.

On this page

Push the image to ECRECS on Fargate1. Cluster and logs2. Task definition3. Load balancer4. Service5. Autoscaling6. RenderLambda1. Add the Lambda Web Adapter2. Create the function and its URL3. RenderEC2