TechOverflow Logo
TechOverflow

MinIO/S3-backed static hosting using Traefik reverse proxy + nginx-s3-gateway

The following docker-compose.yml uses nginx-s3-gateway to proxy HTTP requests to a configured S3-compatible storage backend (in this case, MinIO). This provides production-ready performance supported by Nginx, Inc., plus high configurability since internally, nginx-s3-gateway is using nginx, so it can flexibly be configured.

See Simple Traefik docker-compose setup with Lets Encrypt Cloudflare DNS-01 & TLS-ALPN-01 & HTTP-01 challenges for details on how we setup Traefik to work with these container labels.

services:
  s3-gateway:
    image: nginxinc/nginx-s3-gateway:latest
    environment:
      # Required (see repo docs)
      S3_BUCKET_NAME: "mydomain-app"
      AWS_ACCESS_KEY_ID: "mydomain-app-reader"
      AWS_SECRET_ACCESS_KEY: "Ooch7rooD8yo8ooW6ohghahNgaeshu"
      S3_SERVER: "minio.mydomain.com"
      S3_REGION: "eu-central-1"
      S3_STYLE: "path"
      S3_SERVER_PORT: "443"
      S3_SERVER_PROTO: "https"
      AWS_SIGS_VERSION: "4"
      PROVIDE_INDEX_PAGE: true
      ALLOW_DIRECTORY_LIST: "false"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app-mydomain.rule=Host(`app.mydomain.com`)"
      - "traefik.http.routers.app-mydomain.entrypoints=websecure"
      - "traefik.http.routers.app-mydomain.tls.certresolver=cloudflare-ec384"
      - "traefik.http.routers.app-mydomain.tls.domains[0].main=mydomain.com"
      - "traefik.http.routers.app-mydomain.tls.domains[0].sans=*.mydomain.com"

Benchmark results

Using wrk to benchmark this. The server is running on a AMD Ryzen 5 3600 6-Core Processor with exclusively HDDs (no SSDs), with MinIO serving the S3 bucket over HTTPS, leading . The client is connected over a routed network with 0.3ms ping latency.

$ wrk -t20 -c1000 -d30s --latency https://my.domain.com
Running 30s test @ https://my.domain.com/
  20 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    55.16ms   77.11ms   1.92s    96.45%
    Req/Sec   289.80     88.91   630.00     72.58%
  Latency Distribution
     50%   41.14ms
     75%   53.31ms
     90%   86.89ms
     99%  439.90ms
  171457 requests in 30.09s, 1.01GB read
  Socket errors: connect 0, read 0, write 0, timeout 850
  Non-2xx or 3xx responses: 6

As can be seen from the results, the setup is capable of handling a extreme load with reasonable latency and low error rate, even given the suboptimal HDD-only configuration plus additional TLS-to-MinIO overhead, making it suitable for production use cases.