Zero downtime deployment with SSE and long startup times

Published on

Generated by Gemini

Intro: Zero-Downtime Deployment for Spring Boot Apps with Docker Compose

Deploying updates to a live application is a critical process. For applications with long startup times and persistent connections like Server-Sent Events (SSE), the challenge is to deploy without interrupting active users. A standard restart is not an option—it would mean minutes of downtime and abruptly terminated user sessions.

This tutorial provides a complete, step-by-step guide to achieving true zero-downtime deployments for a Spring Boot application using Docker Compose. We will tackle two specific, real-world challenges:

Our solution combines the Blue/Green deployment strategy for seamless traffic switching with graceful connection draining to protect long-lived user connections.

The Core Concepts

1. Blue/Green Deployment

This strategy eliminates downtime by running two identical production environments, which we call “Blue” and “Green.”

The deployment process is simple and safe:

  1. Deploy the new “Green” version alongside the live “Blue” version.
  2. Wait for “Green” to fully start up and pass its health checks. During this time, it receives no user traffic.
  3. Once “Green” is healthy, a reverse proxy (like Nginx) instantly switches all traffic from “Blue” to “Green.”
  4. The old “Blue” environment is kept running for a while to handle existing connections and can be shut down later.

This method ensures a healthy instance is always available to handle requests, making the switch instantaneous from the user’s perspective.

2. Graceful Shutdown & Connection Draining

For applications with long-lived connections (like SSE, WebSockets, or long file uploads), simply stopping the old container is dangerous. A standard docker stop command would terminate all active SSE connections.

Graceful shutdown solves this. The process is:

  1. The reverse proxy stops sending new requests to the old container.
  2. The application is signaled to shut down gracefully. It stops accepting new connections but allows existing ones to continue until they terminate naturally.
  3. The container waits until the last connection is closed before it finally shuts down.

This ensures that a deployment does not interrupt or disconnect active users.

The Tutorial: A Step-by-Step Guide

Prerequisites

Step 1: Enable Graceful Shutdown in Spring Boot

First, configure your application to handle connection draining. Add the following properties to your src/main/resources/application.properties (or application.yml):

# Enable graceful shutdown mode. The server will stop accepting new requests
# but wait for active ones to complete.
server.shutdown=graceful

# Set the maximum time to wait. This MUST be longer than your max SSE
# connection time (15m). We'll use 16 minutes to be safe.
spring.lifecycle.timeout-per-shutdown-phase=16m

Crucially, you must rebuild your application’s Docker image after adding these properties.

Step 2: Create the docker-compose.yml File

This file defines our three services: the Nginx reverse proxy and the Blue/Green instances of our app. Notice the healthcheck for handling the slow startup and the stop_grace_period to allow for connection draining.

# docker-compose.yml

version: '3.8'

services:
  # The Reverse Proxy that directs traffic
  nginx:
    image: nginx:1.25-alpine
    container_name: zdd_nginx
    volumes:
      # Mounts our dynamically generated Nginx config
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "80:80"
    depends_on:
      app_blue:
        condition: service_started
      app_green:
        condition: service_started

  # "Blue" instance of your application
  app_blue:
    image: my-spring-boot-app:latest # ⬅️ Replace with your app image
    container_name: app_blue
    environment:
      - SPRING_PROFILES_ACTIVE=blue
    expose:
      - "8080"
    # This tells Docker to wait 16 mins before force-killing the container.
    # It allows our Spring Boot graceful shutdown to complete.
    stop_grace_period: 16m
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/actuator/health"]
      interval: 10s
      timeout: 5s
      retries: 5
      # Grace period for the 1.5 min startup before checks begin
      start_period: 90s

  # "Green" instance of your application
  app_green:
    image: my-spring-boot-app:latest # ⬅️ Replace with your app image
    container_name: app_green
    environment:
      - SPRING_PROFILES_ACTIVE=green
    expose:
      - "8080"
    stop_grace_period: 16m # Must also have the grace period
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/actuator/health"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 90s

Step 3: Configure Nginx

Our Nginx configuration needs to be dynamic so our script can easily switch traffic. We’ll use a template that the script will populate.

  1. Create a directory named nginx.
  2. Inside it, create a file named nginx.conf.template:
# nginx/nginx.conf.template

events {}

http {
    # Defines a group of servers. Our script will point this to the
    # active container (e.g., app_blue or app_green).
    upstream spring_app {
        server ${TARGET_HOST}:8080;
    }

    server {
        listen 80;

        location / {
            # Route all traffic to our upstream group
            proxy_pass http://spring_app;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            # Headers needed for SSE to work through a proxy
            proxy_set_header Connection '';
            proxy_http_version 1.1;
            chunked_transfer_encoding off;
            proxy_buffering off;
            proxy_cache off;
        }
    }
}

Note: The final proxy headers are essential for ensuring SSE connections are not buffered or prematurely closed by Nginx.

Step 4: The Automated Deployment Script

This bash script automates the entire Blue/Green deployment and graceful shutdown process. Save it as deploy.sh in your project’s root directory.

#!/bin/bash
set -e

# This function polls the Docker health status of a container
wait_for_health() {
    local container_name=$1
    echo "--> Waiting for '$container_name' to be healthy..."
    
    # Loop until the health check status is "healthy"
    until [ "$(docker inspect -f {{.State.Health.Status}} $container_name)" == "healthy" ]; do
        echo -n "."
        sleep 3
    done
    echo "" # Newline for cleaner output
    echo "--> '$container_name' is healthy!"
}

# 1. Determine which color is current and which is next
if docker ps --filter "name=app_blue" --filter "status=running" | grep -q 'app_blue'; then
    CURRENT_COLOR="blue"
    NEXT_COLOR="green"
else
    CURRENT_COLOR="green"
    NEXT_COLOR="blue"
fi

echo "✅ Current active version is: $CURRENT_COLOR"
echo "🚀 Deploying new version: $NEXT_COLOR"

# 2. Start the new container without touching the others
echo "--> Starting the '$NEXT_COLOR' container..."
# --no-deps prevents restarting Nginx or the other app
# --build ensures we use the latest code
docker-compose up -d --build --no-deps app_$NEXT_COLOR

# 3. Wait for the new container to pass its health check
wait_for_health app_$NEXT_COLOR

# 4. Update Nginx to route traffic to the new container
echo "--> Switching Nginx traffic to '$NEXT_COLOR'..."
export TARGET_HOST="app_$NEXT_COLOR"
envsubst '$TARGET_HOST' < nginx/nginx.conf.template > nginx/nginx.conf

# 5. Gracefully reload Nginx to apply the new config
docker-compose exec nginx nginx -s reload
echo "--> Traffic switched successfully!"

# 6. Stop the old container gracefully
echo "--> Sending graceful shutdown signal to '$CURRENT_COLOR' container..."
echo "--> It will remain alive to drain active SSE connections (up to 16 mins)."
docker-compose stop app_$CURRENT_COLOR

echo "✅ Zero-downtime deployment complete! The old container will exit once all connections are closed."

Finally, make the script executable: chmod +x deploy.sh.

Deployment Workflow

Your deployment process is now fully automated.

First-Time Setup

On the very first deployment, you need to establish a “Blue” version and configure Nginx to point to it.

# 1. Start only the blue container initially
docker-compose up -d --build app_blue

# 2. Run the deployment script to configure Nginx and bring up green
./deploy.sh

Subsequent Deployments

For all future updates, simply run the script:

./deploy.sh

The script will handle everything: it detects that “Green” is now the live version, brings up a new “Blue” instance, waits for it to be healthy, switches traffic, and gracefully shuts down the old “Green” instance, protecting all active users.