feat: add env-variable based uvicorn app timeout (Unstructured-IO#401)

Modify app-start.sh so that if MAX_LIFETIME_SECONDS environmental
variable is set, app will be gracefully (and later forcefully if it
fails) shutdown after set amount of seconds.

Co-authored-by: Filip Knefel <[email protected]>

Author: ds-filipknefel

CHANGELOG.md

@@ -1,6 +1,7 @@
-## 0.0.66-dev0
+## 0.0.66-dev1
 
 * Add support for `unique_element_ids` parameter.
+* Add max lifetime, via MAX_LIFETIME_SECONDS env-var, to API containers
 
 ## 0.0.65
 

Makefile

@@ -69,6 +69,7 @@ docker-start-api:
 	docker run -p 8000:8000 \
 	-it --rm  \
 	--mount type=bind,source=$(realpath .),target=/home/notebook-user/local \
+	$(if $(MAX_LIFETIME_SECONDS),-e MAX_LIFETIME_SECONDS=$(MAX_LIFETIME_SECONDS)) \
 	pipeline-family-${PIPELINE_FAMILY}-dev:latest scripts/app-start.sh
 
 .PHONY: docker-start-bash

README.md

@@ -397,6 +397,15 @@ You may also set the optional `UNSTRUCTURED_API_KEY` env variable to enable requ
 #### Controlling Server Load
 Some documents will use a lot of memory as they're being processed. To mitigate OOM errors, the server will return a 503 if the host's available memory drops below 2GB. This is configured with the environment variable `UNSTRUCTURED_MEMORY_FREE_MINIMUM_MB`, which defaults to 2048. You can lower this value to reduce these messages, that is, allow the server to use more memory. Otherwise, you can set to 0 to fully remove this check.
 
+#### Controlling server life time
+By default server will run for indefinitely. To change that the `MAX_LIFETIME_SECONDS` environmental variable can be set. If server is run with this variable set, it will enter a graceful shutdown period after `MAX_LIFETIME_SECONDS` from its initialization. Graceful shutdown period lasts for up to 3600 seconds and during it:
+- server denies any new requests - they're met with an empty response,
+- server continues processing active requests and shuts down (ending graceful period) if all of them are processed.
+
+After the graceful period is over if server is still running, it is shutdown forcefully, cancelling all active requests and sending empty responses to each of them.
+
+*Max lifetime requires gnu [timeout](https://www.gnu.org/software/coreutils/manual/html_node/timeout-invocation.html#timeout-invocation) to be installed, available by default on most linux systems. Downloadable on MacOS as gtimeout with gnu coreutils.*
+
 ## :dizzy: Instructions for using the Docker image
 
 The following instructions are intended to help you get up and running using Docker to interact with `unstructured-api`.

scripts/app-start.sh

@@ -1,5 +1,28 @@
 #!/usr/bin/env bash
 
-uvicorn prepline_general.api.app:app \
-	--log-config logger_config.yaml \
+NUMREGEX="^[0-9]+$"
+GRACEFUL_SHUTDOWN_PERIOD_SECONDS=3600
+TIMEOUT_COMMAND='timeout'
+OPTIONAL_TIMEOUT=''
+
+if [[ -n $MAX_LIFETIME_SECONDS ]]; then
+    if ! command -v $TIMEOUT_COMMAND &> /dev/null; then
+        TIMEOUT_COMMAND='gtimeout'
+        echo "Warning! 'timeout' command is required but not available. Checking for gtimeout."
+    elif ! command -v $TIMEOUT_COMMAND &> /dev/null; then
+        echo "Warning! 'gtimeout' command is required but not available. Running without max lifetime."
+    elif [[ $MAX_LIFETIME_SECONDS =~ $NUMREGEX ]]; then
+        OPTIONAL_TIMEOUT="timeout --preserve-status --foreground --kill-after ${GRACEFUL_SHUTDOWN_PERIOD_SECONDS} ${MAX_LIFETIME_SECONDS}"
+        echo "Server's lifetime set to ${MAX_LIFETIME_SECONDS} seconds."
+    else
+        echo "Warning! MAX_LIFETIME_SECONDS was not properly set, an integer was expected, got ${MAX_LIFETIME_SECONDS}. Running without max lifetime."
+    fi
+fi
+
+${OPTIONAL_TIMEOUT} \
+    uvicorn prepline_general.api.app:app \
+	    --log-config logger_config.yaml \
         --host 0.0.0.0
+
+echo "Server was shutdown"
+[ -n "$MAX_LIFETIME_SECONDS" ] && echo "Reached timeout of $MAX_LIFETIME_SECONDS seconds"