You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The self-hosting guide comes in two parts. The first part is a simple setup where you run everything on one server. In the second part, the webapp and worker components are split on two separate machines.
14
+
The self-hosting guide covers two alternative setups. The first options uses a simple setup where you run everything on one server. With the second option, the webapp and worker components are split on two separate machines.
15
15
16
16
You're going to need at least one Debian (or derivative) machine with Docker and Docker Compose installed. We'll also use Ngrok to expose the webapp to the internet.
17
17
@@ -55,7 +55,7 @@ Should the burden ever get too much, we'd be happy to see you on [Trigger.dev cl
55
55
56
56
You will also need a way to expose the webapp to the internet. This can be done with a reverse proxy, or with a service like Ngrok. We will be using the latter in this guide.
57
57
58
-
## Part 1: Single server
58
+
## Option 1: Single server
59
59
60
60
This is the simplest setup. You run everything on one server. It's a good option if you have spare capacity on an existing machine, and have no need to independently scale worker capacity.
3. Log in to Docker Hub both locally and your server. For the split setup, this will be the worker machine. You may want to create an [access token](https://hub.docker.com/settings/security) for this.
2. Run the start script with the `worker` argument
218
+
2.**Startup.**Run the start script with the `worker` argument
211
219
212
220
```bash
213
221
./start.sh worker
214
222
```
215
223
216
-
2. Tunnelling is _not_ required for the worker components.
224
+
3.**Tunnelling.** This is _not_ required for the worker components.
225
+
226
+
4.**Registry setup.** Follow the [registry setup](/open-source-self-hosting#registry-setup) section but run the last command on the worker machine - note the container name is different:
By default, payloads over 512KB will be offloaded to S3-compatible storage. If you don't provide the required env vars, runs with payloads larger than this will fail.
OBJECT_STORE_ACCESS_KEY_ID="<r2 access key with read/write access to bucket>"
245
+
OBJECT_STORE_SECRET_ACCESS_KEY="<r2 secret key>"
246
+
```
247
+
248
+
Alternatively, you can increase the threshold:
217
249
218
-
## Checkpoint support
250
+
```bash
251
+
# size in bytes, example with 5MB threshold
252
+
TASK_PAYLOAD_OFFLOAD_THRESHOLD=5242880
253
+
```
254
+
255
+
### Version locking
256
+
257
+
There are several reasons to lock the version of your Docker images:
258
+
-**Backwards compatibility.** We try our best to maintain compatibility with older CLI versions, but it's not always possible. If you don't want to update your CLI, you can lock your Docker images to that specific version.
259
+
-**Ensuring full feature support.** Sometimes, new CLI releases will also require new or updated platform features. Running unlocked images can make any issues difficult to debug. Using a specific tag can help here as well.
260
+
261
+
By default, the images will point at the latest versioned release via the `v3` tag. You can override this by specifying a different tag in your `.env` file. For example:
262
+
263
+
```bash
264
+
TRIGGER_IMAGE_TAG=v3.0.4
265
+
```
266
+
267
+
### Checkpoint support
219
268
220
269
<Warning>
221
270
This requires an _experimental Docker feature_. Successfully checkpointing a task today, does not
Checkpointing allows you to save the state of a running container to disk and restore it later. This can be useful for
226
275
long-running tasks that need to be paused and resumed without losing state. Think fan-out and fan-in, or long waits in email campaigns.
227
276
228
-
The checkpoints will be pushed to the same registry as the deployed images. Please see the [Registry setup](#registry-setup) section for more information.
277
+
The checkpoints will be pushed to the same registry as the deployed images. Please see the [registry setup](#registry-setup) section for more information.
229
278
230
-
### Requirements
279
+
####Requirements
231
280
232
281
- Debian, **NOT** a derivative like Ubuntu
233
282
- Additional storage space for the checkpointed containers
234
283
235
-
### Setup
284
+
####Setup
236
285
237
286
Underneath the hood this uses Checkpoint and Restore in Userspace, or [CRIU](https://github.com/checkpoint-restore/criu) in short. We'll have to do a few things to get this working:
By default, payloads over 512KB will be offloaded to S3-compatible storage. If you don't provide the required env vars, runs with payloads larger than this will fail.
OBJECT_STORE_ACCESS_KEY_ID="<r2 access key with read/write access to bucket>"
301
-
OBJECT_STORE_SECRET_ACCESS_KEY="<r2 secret key>"
302
-
```
303
-
304
-
Alternatively, you can increase the threshold:
305
-
306
-
```bash
307
-
# size in bytes, example with 5MB threshold
308
-
TASK_PAYLOAD_OFFLOAD_THRESHOLD=5242880
309
-
```
310
-
311
341
## Updating
312
342
313
343
Once you have everything set up, you will periodically want to update your Docker images. You can easily do this by running the update script and restarting your services:
@@ -354,18 +384,6 @@ git stash pop
354
384
./stop.sh && ./start.sh
355
385
```
356
386
357
-
## Version locking
358
-
359
-
There are several reasons to lock the version of your Docker images:
360
-
-**Backwards compatibility.** We try our best to maintain compatibility with older CLI versions, but it's not always possible. If you don't want to update your CLI, you can lock your Docker images to that specific version.
361
-
-**Ensuring full feature support.** Sometimes, new CLI releases will also require new or updated platform features. Running unlocked images can make any issues difficult to debug. Using a specific tag can help here as well.
362
-
363
-
By default, the images will point at the latest versioned release via the `v3` tag. You can override this by specifying a different tag in your `.env` file. For example:
364
-
365
-
```bash
366
-
TRIGGER_IMAGE_TAG=v3.0.4
367
-
```
368
-
369
387
## CLI usage
370
388
371
389
This section highlights some of the CLI commands and options that are useful when self-hosting. Please check the [CLI reference](/cli-introduction) for more in-depth documentation.
0 commit comments