|
7 | 7 | **Note #2:** NVidia GPU CuBLAS support requires a NVidia GPU with sufficient VRAM (approximately as much as the size above) and Docker NVidia support (see [container-toolkit/install-guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html))
|
8 | 8 |
|
9 | 9 | # Simple Dockerfiles for building the llama-cpp-python server with external model bin files
|
10 |
| -- `./openblas_simple/Dockerfile` - a simple Dockerfile for non-GPU OpenBLAS, where the model is located outside the Docker image |
11 |
| - - `cd ./openblas_simple` |
12 |
| - - `docker build -t openblas_simple .` |
13 |
| - - `docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t openblas_simple` |
14 |
| - where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. |
15 |
| -- `./cuda_simple/Dockerfile` - a simple Dockerfile for CUDA accelerated CuBLAS, where the model is located outside the Docker image |
16 |
| - - `cd ./cuda_simple` |
17 |
| - - `docker build -t cuda_simple .` |
18 |
| - - `docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t cuda_simple` |
19 |
| - where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. |
| 10 | +## openblas_simple - a simple Dockerfile for non-GPU OpenBLAS, where the model is located outside the Docker image |
| 11 | +``` |
| 12 | +cd ./openblas_simple |
| 13 | +docker build -t openblas_simple . |
| 14 | +docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t openblas_simple |
| 15 | +``` |
| 16 | +where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. |
| 17 | + |
| 18 | +## cuda_simple - a simple Dockerfile for CUDA accelerated CuBLAS, where the model is located outside the Docker image |
| 19 | +``` |
| 20 | +cd ./cuda_simple |
| 21 | +docker build -t cuda_simple . |
| 22 | +docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t cuda_simple |
| 23 | +``` |
| 24 | +where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. |
20 | 25 |
|
21 | 26 | # "Open-Llama-in-a-box" - Download a MIT licensed Open Llama model and install into a Docker image that runs an OpenBLAS-enabled llama-cpp-python server
|
22 | 27 | ```
|
|
0 commit comments