TechOverflow Logo
TechOverflow

Whishper self-hosted GPU transcription setup with Docker & Traefik

First run the init script to create the necessary directories and files:

#!/bin/sh
# Libretranslate
mkdir -p libretranslate_data libretranslate_cache
chown -R 1032:nogroup libretranslate_data libretranslate_cache
# MongoDB
mkdir -p mongodb_data mongodb_logs
chown -R 999:999 mongodb_data mongodb_logs

Now create the docker-compose.yml file:

services:
  mongo:
    image: mongo
    env_file:
      - .env
    restart: unless-stopped
    volumes:
      - ./mongodb_data:/data/db
      - ./mongodb_logs:/var/log/mongodb/
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${DB_USER:-whishper}
      MONGO_INITDB_ROOT_PASSWORD: ${DB_PASS:-whishper}
    expose:
      - 27017
    command: ['--logpath', '/var/log/mongodb/mongod.log']

  translate:
    container_name: whisper-libretranslate
    image: libretranslate/libretranslate:latest-cuda
    restart: unless-stopped
    volumes:
      - ./libretranslate_data:/home/libretranslate/.local/share
      - ./libretranslate_cache:/home/libretranslate/.local/cache
    env_file:
      - .env
    tty: true
    environment:
      LT_DISABLE_WEB_UI: False
      LT_UPDATE_MODELS: True
      LT_LOAD_ONLY: en,de
    expose:
      - 5000
    networks:
      default:
        aliases:
          - translate
    healthcheck:
      test: ['CMD-SHELL', './venv/bin/python scripts/healthcheck.py']
      interval: 2s
      timeout: 3s
      retries: 5
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

  whishper:
    pull_policy: always
    image: pluja/whishper:${WHISHPER_VERSION:-latest-gpu}
    env_file:
      - .env
    volumes:
      - ./whishper_uploads:/app/uploads
      - ./whishper_logs:/var/log/whishper
    container_name: whishper
    restart: unless-stopped
    networks:
      default:
        aliases:
          - whishper
    ports:
      - 8082:80
    depends_on:
      - mongo
      - translate
    environment:
      PUBLIC_INTERNAL_API_HOST: "http://127.0.0.1:80"
      PUBLIC_TRANSLATION_API_HOST: ""
      PUBLIC_API_HOST: ${WHISHPER_HOST:-}
      LT_LOAD_ONLY: de,en
      WHISPER_MODELS: ${WHISPER_MODELS}
      PUBLIC_WHISHPER_PROFILE: gpu
      WHISPER_MODELS_DIR: /app/models
      UPLOAD_DIR: /app/uploads
      CPU_THREADS: 4
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

Now create the .env file:

# Whisper Configuration
WHISPER_MODELS: small,medium.en,large-v3
WHISHPER_HOST=https://whishper.mydomain.com

# Database Configuration
DB_USER=whishper
DB_PASS=Ra1aeJoh5ahngaed6xuShaeReil3ae

Now create the Traefik configuration file traefik.yml. Remember to adjust the TLS settings to your needs: See Simple Traefik docker-compose setup with Lets Encrypt Cloudflare DNS-01 & TLS-ALPN-01 & HTTP-01 challenges for my setup.

[http.routers]
  [http.routers.whishper-mydomain]
    rule = "Host(`whishper.mydomain.com`)"
    entryPoints = ["websecure"]
    service = "whishper-mydomain"
    middlewares = ["whishper-mydomain-basicauth"]
    [http.routers.whishper-mydomain.tls]
      certResolver = "alpn-ec384"


[http.services]
  [http.services.whishper-mydomain.loadBalancer]
    [[http.services.whishper-mydomain.loadBalancer.servers]]
      url = "http://10.1.2.3:8082"

[http.middlewares]
  [http.middlewares.whishper-mydomain-basicauth.basicAuth]
    users = [
      "myuser:$apr1$KwZDqK6Q$Sr6/.MLMv5JUuFPSNY30L/"
    ]
    realm = "whishper-mydomain"

After that, you can start the containers with:

docker-compose up -d

and install the autorestart using our script

curl -fsSL https://techoverflow.net/scripts/create-docker-compose-service.sh | sudo bash /dev/stdin

See Create a systemd service for your docker-compose project in 10 seconds for more information about this script.

Now you can access the Whishper web interface at https://whishper.mydomain.com and start transcribing audio files.