Use Cloudflare with Caddy to Scrub Traffic

Lately, I've been constantly hit by botnet crawlers. It's getting unbearable. Time to switch to Cloudflare!

Setting up Cloudflare is actually quite simple. It only takes one step to get done. And it's completely free.

• Register for Cloudflare
• Let Cloudflare manage your domain
• Point your service domain to the actual IP address
• Remove rate limiting from Caddy. Let Cloudflare handle rate limiting instead.
• Configure Caddy to trust Cloudflare's IPs as proxies. Use the dev plugin to download Cloudflare's IP list and trust them.
• Download Cloudflare's origin server certificate to Caddy, and have Caddy serve this certificate. This way, if an external request accidentally connects directly to my Caddy instance, it will fail.
• In the Cloudflare TLS Overview section, change the Current encryption mode to Full (strict). This forces Cloudflare to only trust this specific certificate.
• Add Caddy rules to reject any requests that do not come from Cloudflare. Download Cloudflare's public key and require mTLS verification. Enable: Authenticated Origin Pulls. This ensures hackers cannot request the server even if they try to override the hostname and IP.
• When Caddy forwards requests to the application, extract the client's IP from Cloudflare's headers and send it to the application. Log this IP properly. Handle Cloudflare's IP gracefully.
• Update py_syncer to consume the client IP instead of remote_ip, so that logs and statistics reflect the real IP addresses.

graph TD
    %% 定义样式
    classDef good fill:#d4edda,stroke:#28a745,stroke-width:2px;
    classDef bad fill:#f8d7da,stroke:#dc3545,stroke-width:2px;
    classDef cf fill:#fff3cd,stroke:#ffc107,stroke-width:2px;
    classDef server fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;

    %% 角色
    Hacker[🔴 黑客/扫描器]:::bad
    User[🟢 真实用户]:::good
    
    %% Cloudflare 层
    subgraph Cloudflare_Edge [Cloudflare 边缘网络]
        CF_WAF[🛡️ WAF & DDoS 防护]:::cf
        CF_Cert[📜 客户端证书 mTLS Key]:::cf
    end

    %% 你的服务器层
    subgraph Anduin_Server [AnduinOS Swarm Server]
        direction TB
        FW[🔥 防火墙 443 Port]:::server
        
        subgraph Caddy_Container [Caddy Container]
            Caddy_TLS[🔒 mTLS 验证]:::server
            Caddy_Trust[🤝 Trusted Proxies IP还原]:::server
            Origin_Cert[📄 Origin CA 证书]:::server
        end
        
        subgraph App_Layer [业务应用层]
            WebApp[ASP.NET Core App]:::good
            PySyncer[🐍 PySyncer]:::good
        end
        
        DB[(ClickHouse)]:::server
    end

    %% 流量路径 - 正常用户
    User -->|HTTPS| CF_WAF
    CF_WAF -->|携带 Client Cert| FW
    FW --> Caddy_TLS
    Caddy_TLS -- "验证通过 (有证书)" --> Caddy_Trust
    Caddy_Trust -- "解析出 Client IP: 4.145.x.x" --> WebApp
    Caddy_Trust -- "JSON Log (Client IP)" --> PySyncer
    PySyncer -->|写入真实IP| DB

    %% 流量路径 - 黑客
    Hacker -.->|直连 IP 无证书| FW
    FW -.-> Caddy_TLS
    Caddy_TLS -- "❌ 拒绝连接 (Handshake Fail)" --> Hacker
    
    %% 补充说明
    note1[Cloudflare 负责限流 & 挡住第一波攻击] --- CF_WAF
    note2[Caddy 负责物理阻断非 CF 流量] --- Caddy_TLS
    note3[全链路加密 Full Strict] --- Origin_Cert

    linkStyle 6,7,8 stroke:#28a745,stroke-width:2px;
    linkStyle 9,10,11 stroke:#dc3545,stroke-width:2px,stroke-dasharray: 5 5;

Cloudflare settings to manage:

• DNS -> Settings -> DNS SEC -> On
• Security -> Settings -> Bot Fight Mode -> On
• Security -> Settings -> AI Labyrinth -> On
• Security -> Settings -> Browser integrity check -> On
• SSL/TLS -> Current encryption mode -> Full Strict
• SSL/TLS -> Edge Certificates -> HSTS -> On
• SSL/TLS -> Edge Certificates -> Minimum TLS Version -> 1.2
• SSL/TLS -> Edge Certificates -> Always Use HTTPS -> On
• SSL/TLS -> Origin Server -> Authenticated Origin Pulls -> On
• Speed -> Settings -> Protocol Optimization -> HTTP3 -> On
• Speed -> Settings -> Protocol Optimization -> 0-RTT Connection Resumption -> On
• Speed -> Settings -> Content Optimization -> Rocket Loader -> Off
• Account -> Analytics & Logs -> Web analytics -> Manage site -> Advanced -> Delete
• Caching -> Tiered Cache -> Smart
• Caching -> Cache Rules -> Create rule -> Cache default file extensions -> Deploy
• Caching -> Configuration -> Crawler Hints
• Network -> IPv6 Compatibility -> On
• Network -> WebSockets -> On
• Network -> IP Geolocation -> On
• Scrape Shield -> Email Address Obfuscation -> On

While adjusting the Cloudflare settings, we also need to configure Caddy accordingly.

Caddy Configuration

When first compiling Caddy, the general process is as follows:

• Compile the binary first
• Then generate the configuration file. The configuration file generation is divided into three steps:
  • Cloudflare's IP list
  • Base zone
  • Business code zone
• Then generate a dummy certificate to facilitate Caddy's verification.
• Finally, distribute the real certificate (issued by Cloudflare) to Caddy.

Caddy abandons handling HTTPS certificates!! However, it still enables HTTPS, using Cloudflare's certificate!

Caddy cannot be accessed by non-Cloudflare clients! Caddy enforces client mTLS! Caddy verifies Cloudflare's certificate on every request!

Caddy only trusts Cloudflare as a front-end proxy!

# ============================
# Prepare caddy Environment
FROM localhost:8080/public_mirror/caddy:builder AS caddy-build-env

RUN xcaddy build \
    --with github.com/ueffel/caddy-brotli \
    --with github.com/caddyserver/transform-encoder

# ============================
# Prepare Caddyfile build Environment
FROM localhost:8080/box_starting/local_ubuntu AS config-build-env
WORKDIR /app

# Install curl for fetching Cloudflare IPs and openssl for generating dummy certs
# Also download Cloudflare Origin Pull CA certificate for mTLS verification
RUN apt-get update && \
    apt-get install -y curl openssl ca-certificates && \
    mkdir -p /app/Dist/certs && \
    curl -fsSL -o /app/Dist/certs/origin-pull-ca.pem https://developers.cloudflare.com/ssl/static/authenticated_origin_pull_ca.pem && \
    rm -rf /var/lib/apt/lists/*

COPY . .

# Outputs to /app/Dist/Caddyfile
RUN chmod +x /app/build_proxy.sh
RUN /bin/bash /app/build_proxy.sh

# Generate dummy certificates for build-time validation
# These will be replaced by real certificates at runtime via Docker volumes
RUN openssl req -x509 -newkey rsa:2048 -nodes \
    -keyout /app/Dist/certs/anduinos.key \
    -out /app/Dist/certs/anduinos.pem \
    -days 1 -subj "/CN=localhost"

# ============================
# Prepare Runtime Environment
FROM localhost:8080/public_mirror/caddy:latest

WORKDIR /app

EXPOSE 80 443

COPY --from=caddy-build-env /usr/bin/caddy /usr/bin/caddy
COPY --from=config-build-env /app/Dist/Caddyfile /etc/caddy/Caddyfile

# Copy dummy certificates to expected location for validation
# Note: These will be replaced by real certificates at runtime via Docker volumes
COPY --from=config-build-env /app/Dist/certs/anduinos.pem /data/caddy/certs/anduinos.pem
COPY --from=config-build-env /app/Dist/certs/anduinos.key /data/caddy/certs/anduinos.key

# Copy Cloudflare Origin Pull CA certificate to /etc/caddy (safe from volume mount)
COPY --from=config-build-env /app/Dist/certs/origin-pull-ca.pem /etc/caddy/origin-pull-ca.pem

# Now we can safely validate the Caddyfile with dummy certificates in place
RUN caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile && \
    mkdir -p /var/log/caddy /data/caddy/logs && \
    touch /data/caddy/logs/web.log

ENTRYPOINT ["sh", "-c", "caddy run --config /etc/caddy/Caddyfile --adapter caddyfile & tail -f /data/caddy/logs/web.log & wait"]

Don't compile the Dockerfile above just yet. There's a lot I haven't explained.

Obviously,

apt-get update && \
apt-get install -y curl openssl ca-certificates && \
mkdir -p /app/Dist/certs && \
curl -fsSL -o /app/Dist/certs/origin-pull-ca.pem https://developers.cloudflare.com/ssl/static/authenticated_origin_pull_ca.pem && \
rm -rf /var/lib/apt/lists/*

This is to download Cloudflare's certificate. This is a public key; each time Cloudflare makes a request, it encrypts the data using its private key. This allows Caddy to be forced to act as a puppet of Cloudflare.

Process: /app/build_proxy.sh is very complex and consists of three internal steps.

#!/bin/bash
set -e

echo "Building Caddyfile at $(pwd)..."
mkdir -p ./Dist

echo "Fetching Cloudflare IP ranges..."
chmod +x ./fetch_cloudflare_ips.sh
./fetch_cloudflare_ips.sh

echo "Adding empty lines to the end of files without a newline..."
find . -type f -name '*.conf' ! -name 'cloudflare_ips.conf' | while read -r file; do
    last_line=$(tail -n 1 "$file")
    if [[ -n "$last_line" ]]; then
        echo "" >> "$file"
        echo "修复文件结尾：$file"
    fi
done

echo "Building sites under $(pwd)..."
find . -type f -name "*.conf" ! -name "cloudflare_ips.conf" | while read -r file; do cat "$file"; echo -e "\n\n"; done | tee ./Dist/Sites.temp > /dev/null

echo "Appending cloudflare ips, baseline and business sites into final Caddyfile..."
(cat ./cloudflare_ips.conf; echo -e "\n\n"; cat ./baseline; echo -e "\n\n"; cat ./Dist/Sites.temp) | tee ./Dist/Caddyfile > /dev/null

echo "Caddyfile built."
ls ./Dist/ -ashl

It will first download Cloudflare's IP list, then concatenate the configuration files for each service into a complete Caddyfile. The download process is as follows:

#!/bin/bash
set -e

echo "Fetching Cloudflare IP ranges..."

# Fetch IPv4 ranges
echo "Fetching IPv4 ranges from https://www.cloudflare.com/ips-v4"
IPV4_RANGES=$(curl -s https://www.cloudflare.com/ips-v4 | tr '\n' ' ')

# Fetch IPv6 ranges
echo "Fetching IPv6 ranges from https://www.cloudflare.com/ips-v6"
IPV6_RANGES=$(curl -s https://www.cloudflare.com/ips-v6 | tr '\n' ' ')

# Combine all ranges
ALL_RANGES="$IPV4_RANGES $IPV6_RANGES"

echo "Generating cloudflare_ips.conf..."

# Generate the configuration file
cat > ./cloudflare_ips.conf << EOF
# Auto-generated Cloudflare Configuration
# Generated at: $(date -u +"%Y-%m-%d %H:%M:%S UTC")

# 1. Trust proxy configuration
(cloudflare_trust) {
    trusted_proxies static $ALL_RANGES
}

# 2. Security & TLS Configuration
# This snippet handles both: certificate loading + mTLS verification
(limit_to_cloudflare) {
    tls /data/caddy/certs/anduinos.pem /data/caddy/certs/anduinos.key {
        client_auth {
            mode require_and_verify
            trust_pool file /etc/caddy/origin-pull-ca.pem
        }
    }
}
EOF

echo "✓ Cloudflare IP configuration generated successfully"
echo "  IPv4 ranges: $(echo $IPV4_RANGES | wc -w)"
echo "  IPv6 ranges: $(echo $IPV6_RANGES | wc -w)"
echo "  Total ranges: $(echo $ALL_RANGES | wc -w)"

The above code generates a cloudflare_ips.conf file with content roughly as follows:

# Auto-generated Cloudflare Configuration
# Generated at: 2026-01-10 16:50:43 UTC

# 1. Trust proxy configuration
(cloudflare_trust) {
    trusted_proxies static 173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 141.101.64.0/18 108.162.192.0/18 190.93.240.0/20 188.114.96.0/20 197.234.240.0/22 198.41.128.0/17 162.158.0.0/15 104.16.0.0/13 104.24.0.0/14 172.64.0.0/13 131.0.72.0/22 2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32
}

# 2. Security & TLS Configuration
# This snippet handles both: certificate loading + mTLS verification
(limit_to_cloudflare) {
    tls /data/caddy/certs/anduinos.pem /data/caddy/certs/anduinos.key {
        client_auth {
            mode require_and_verify
            trust_pool file /etc/caddy/origin-pull-ca.pem
        }
    }
}

Its purpose is to be referenced later in the baseline, where (limit_to_cloudflare) includes:

• Enforcing mTLS verification (validating the Cloudflare certificate, so only Cloudflare can initiate requests)
• Responding with the Cloudflare certificate, thereby allowing Cloudflare to trust Caddy.

Once mutual verification is complete, the strictest Full Strict mode can be enabled.

After that, /app/build_proxy.sh will assemble the baseline file. The baseline file is the basic configuration for Caddy, for example:

{
    log {
        format json
        output file /data/caddy/logs/web.log {
            roll_size 1gb
            roll_uncompressed
        }
        level debug
    }

    servers :443 {
        import cloudflare_trust
        
        listener_wrappers {
            http_redirect
            tls
        }
    }
}

(hsts) {
    header Strict-Transport-Security max-age=63072000
}

Among these, cloudflare_trust allows Caddy to trust Cloudflare's IPs as proxies, thereby correctly restoring the real client IP.

Afterwards, /app/build_proxy.sh concatenates the configuration files for each service into a complete Caddyfile. A service configuration looks like this:

tracer.anduinos.com {
    log
    import limit_to_cloudflare
    reverse_proxy http://tracer_app:5000
}

download.anduinos.com {
    log
    import hsts
    import limit_to_cloudflare
    encode br gzip
    reverse_proxy http://download_web:5000
}

These files may be scattered across various directories. /app/build_proxy.sh will locate all of them and concatenate them. Specifically, it will find all *.conf files in the current directory (excluding cloudflare_ips.conf).

Here is how I organize these directories:

anduin@ultra:~/Source/Repos/Anduin/bash-app/AnduinOS-Swarm$ tree
.
├── stage2
│   ├── images
│   │   └── sites
│   │       ├── baseline
│   │       ├── build_proxy.sh
│   │       ├── cloudflare_ips.conf
│   │       ├── Dockerfile
│   │       └── fetch_cloudflare_ips.sh
│   └── stacks
│       └── incoming
│           ├── docker-compose.yml
│           └── test.conf
└── stage4
    └── stacks
        ├── anduinos
        │   ├── anduinos.conf
        │   └── docker-compose.yml
        ├── clickhouse
        │   ├── clickhouse.conf
        │   ├── config_override.xml
        │   ├── docker-compose.yml
        │   └── users_override.xml
        ├── download
        │   ├── docker-compose.yml
        │   └── download.conf
        ├── grafana
        │   ├── docker-compose.yml
        │   └── grafana.conf
        ├── news
        │   ├── docker-compose.yml
        │   └── news.conf
        ├── shepherd
        │   └── docker-compose.yml
        └── tracer
            ├── docker-compose.yml
            └── tracer.conf

Its advantage is that it allows configuration files for each business to be kept separate, making them easier to manage and maintain. Each time you build the sites image, remember to write a script that copies all *.conf files from the stage4/stacks directory to the stage2/images/sites directory.

rm -rf ./stage2/images/sites/discovered
mkdir -p ./stage2/images/sites/discovered && \
    cp ./stage2/stacks/**/*.conf ./stage2/images/sites/discovered && \
    cp ./stage4/stacks/**/*.conf ./stage2/images/sites/discovered

The fully assembled final Caddyfile will be placed at /app/Dist/Caddyfile and then copied into the final Caddy image. It may look something like this:

# Auto-generated Cloudflare Configuration
# Generated at: 2026-01-10 15:13:24 UTC

# 1. Trust proxy configuration
(cloudflare_trust) {
    trusted_proxies static 173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 141.101.64.0/18 108.162.192.0/18 190.93.240.0/20 188.114.96.0/20 197.234.240.0/22 198.41.128.0/17 162.158.0.0/15 104.16.0.0/13 104.24.0.0/14 172.64.0.0/13 131.0.72.0/22 2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32
}

# 2. Security & TLS Configuration
# This snippet handles both: certificate loading + mTLS verification
(limit_to_cloudflare) {
    tls /data/caddy/certs/anduinos.pem /data/caddy/certs/anduinos.key {
        client_auth {
            mode require_and_verify
            trust_pool file /etc/caddy/origin-pull-ca.pem
        }
    }
}

{
    log {
        format json
        output file /data/caddy/logs/web.log {
            roll_size 1gb
            roll_uncompressed
        }
        level debug
    }

    servers :443 {
        import cloudflare_trust
        
        listener_wrappers {
            http_redirect
            tls
        }
    }
}

(hsts) {
    header Strict-Transport-Security max-age=63072000
}

grafana.anduinos.com {
    log
    import hsts
    import limit_to_cloudflare
    encode br gzip
    reverse_proxy http://grafana_grafana:3000
}

download.anduinos.com {
    log
    import hsts
    import limit_to_cloudflare

    encode br gzip

    reverse_proxy http://download_web:5000
}

It contains three important parts: the one from cloudflare_ips.conf, the baseline, and the configuration files for each service.

However, it still doesn't work because Caddy doesn't have a certificate yet. We generated a dummy certificate in the Dockerfile above. In production, we must provide Caddy with the real Cloudflare certificate. This is done using a Docker Volume:

My docker-compose.yml file looks like this:

version: '3.9'

services:
  sites:
    image: localhost:8080/box_starting/local_sites
    ports:
      # These ports are for internal use. For external, FRP will handle it.
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    networks:
      - proxy_app
    volumes:
      - sites-data:/data
    stop_grace_period: 60s
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 16G
      update_config:
        order: stop-first
        delay: 60s

volumes:
  sites-data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /swarm-vol/sites-data

networks:
  proxy_app:
    external: true

The directory structure of /swarm-vol/sites-data is as follows:

anduin@anduinos-pl:/swarm-vol/sites-data$ tree
.
└── caddy
    ├── acme
    │   └── acme-v02.api.letsencrypt.org-directory
    │       ├── challenge_tokens
    │       └── users
    │           └── [email protected]
    │               ├── anduin.json
    │               └── anduin.key
    ├── certs
    │   ├── anduinos.key
    │   └── anduinos.pem
    ├── instance.uuid
    ├── last_clean.json
    ├── locks
    ├── logs  [error opening dir]
    └── ocsp

Here, certs/anduinos.pem and certs/anduinos.key are the actual Cloudflare certificates. This allows Caddy to operate using the real certificates once it starts.

Please protect the certificates carefully. Otherwise, a hacker who obtains the certificates could impersonate the server and send responses to Cloudflare.

At this point, we have completed a three-layer configuration:

• Caddy uses Cloudflare's certificates
• Caddy validates Cloudflare's mTLS certificate on every request
• Caddy only trusts Cloudflare as the reverse proxy

In actual production, this setup is transparent to users. They will still receive the real IP address in the X-Forwarded-For header.