Skip to content

Commit 213760e

Browse files
parenteparente
authored
Merge pull request #961 from parente/sudo-recipe
Add sudo recipe
2 parents 45f07a1 + 0b2ced3 commit 213760e

File tree

1 file changed

+105
-41
lines changed

1 file changed

+105
-41
lines changed

‎docs/using/recipes.md

Lines changed: 105 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,32 @@
11
# Contributed Recipes
22

3-
Users sometimes share interesting ways of using the Jupyter Docker Stacks. We encourage users to [contribute these recipes](../contributing/recipes.md) to the documentation in case they prove useful to other members of the community by submitting a pull request to `docs/using/recipes.md`. The sections below capture this knowledge.
3+
Users sometimes share interesting ways of using the Jupyter Docker Stacks. We encourage users to
4+
[contribute these recipes](../contributing/recipes.md) to the documentation in case they prove
5+
useful to other members of the community by submitting a pull request to `docs/using/recipes.md`.
6+
The sections below capture this knowledge.
7+
8+
## Using `sudo` within a container
9+
10+
Password authentication is disabled for the `NB_USER` (e.g., `jovyan`). This choice was made to
11+
avoid distributing images with a weak default password that users ~might~ will forget to change
12+
before running a container on a publicly accessible host.
13+
14+
You can grant the within-container `NB_USER` passwordless `sudo` access by adding
15+
`-e GRANT_SUDO=yes` and `--user root` to your Docker command line or appropriate container
16+
orchestrator config.
17+
18+
For example:
19+
20+
```
21+
docker run -it -e GRANT_SUDO=yes --user root jupyter/minimal-notebook
22+
```
23+
24+
**You should only enable `sudo` if you trust the user and/or if the container is running on an
25+
isolated host.**
426

527
## Using `pip install` or `conda install` in a Child Docker image
628

7-
Create a new Dockerfile like the one shown below.
29+
Create a new Dockerfile like the one shown below.
830

931
```dockerfile
1032
# Start from a core stack version
@@ -19,8 +41,8 @@ Then build a new image.
1941
docker build --rm -t jupyter/my-datascience-notebook .
2042
```
2143

22-
To use a requirements.txt file, first create your `requirements.txt` file
23-
with the listing of packages desired. Next, create a new Dockerfile like the one shown below.
44+
To use a requirements.txt file, first create your `requirements.txt` file with the listing of
45+
packages desired. Next, create a new Dockerfile like the one shown below.
2446

2547
```dockerfile
2648
# Start from a core stack version
@@ -44,11 +66,14 @@ RUN conda install --yes --file /tmp/requirements.txt && \
4466
fix-permissions /home/$NB_USER
4567
```
4668

47-
Ref: [docker-stacks/commit/79169618d571506304934a7b29039085e77db78c](https://github.com/jupyter/docker-stacks/commit/79169618d571506304934a7b29039085e77db78c#commitcomment-15960081)
69+
Ref:
70+
[docker-stacks/commit/79169618d571506304934a7b29039085e77db78c](https://github.com/jupyter/docker-stacks/commit/79169618d571506304934a7b29039085e77db78c#commitcomment-15960081)
4871

4972
## Add a Python 2.x environment
5073

51-
Python 2.x was removed from all images on August 10th, 2017, starting in tag `cc9feab481f7`. You can add a Python 2.x environment by defining your own Dockerfile inheriting from one of the images like so:
74+
Python 2.x was removed from all images on August 10th, 2017, starting in tag `cc9feab481f7`. You can
75+
add a Python 2.x environment by defining your own Dockerfile inheriting from one of the images like
76+
so:
5277

5378
```
5479
# Choose your desired base image
@@ -70,33 +95,43 @@ $CONDA_DIR/envs/python2/bin/kernda -o -y /usr/local/share/jupyter/kernels/python
7095
USER $NB_USER
7196
```
7297

73-
Ref: [https://github.com/jupyter/docker-stacks/issues/440](https://github.com/jupyter/docker-stacks/issues/440)
98+
Ref:
99+
[https://github.com/jupyter/docker-stacks/issues/440](https://github.com/jupyter/docker-stacks/issues/440)
74100

75101
## Run JupyterLab
76102

77-
JupyterLab is preinstalled as a notebook extension starting in tag [c33a7dc0eece](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history).
103+
JupyterLab is preinstalled as a notebook extension starting in tag
104+
[c33a7dc0eece](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history).
78105

79-
Run jupyterlab using a command such as `docker run -it --rm -p 8888:8888 jupyter/datascience-notebook start.sh jupyter lab`
106+
Run jupyterlab using a command such as
107+
`docker run -it --rm -p 8888:8888 jupyter/datascience-notebook start.sh jupyter lab`
80108

81109
## Let's Encrypt a Notebook server
82110

83-
See the README for the simple automation here [https://github.com/jupyter/docker-stacks/tree/master/examples/make-deploy](https://github.com/jupyter/docker-stacks/tree/master/examples/make-deploy) which includes steps for requesting and renewing a Let's Encrypt certificate.
111+
See the README for the simple automation here
112+
[https://github.com/jupyter/docker-stacks/tree/master/examples/make-deploy](https://github.com/jupyter/docker-stacks/tree/master/examples/make-deploy)
113+
which includes steps for requesting and renewing a Let's Encrypt certificate.
84114

85-
Ref: [https://github.com/jupyter/docker-stacks/issues/78](https://github.com/jupyter/docker-stacks/issues/78)
115+
Ref:
116+
[https://github.com/jupyter/docker-stacks/issues/78](https://github.com/jupyter/docker-stacks/issues/78)
86117

87118
## Slideshows with Jupyter and RISE
88119

89-
[RISE](https://github.com/damianavila/RISE) allows via extension to create live slideshows of your notebooks, with no conversion, adding javascript Reveal.js:
120+
[RISE](https://github.com/damianavila/RISE) allows via extension to create live slideshows of your
121+
notebooks, with no conversion, adding javascript Reveal.js:
90122

91123
```
92124
# Add Live slideshows with RISE
93125
RUN conda install -c damianavila82 rise
94126
```
95-
Credit: [Paolo D.](https://github.com/pdonorio) based on [docker-stacks/issues/43](https://github.com/jupyter/docker-stacks/issues/43)
127+
128+
Credit: [Paolo D.](https://github.com/pdonorio) based on
129+
[docker-stacks/issues/43](https://github.com/jupyter/docker-stacks/issues/43)
96130

97131
## xgboost
98132

99-
You need to install conda's gcc for Python xgboost to work properly. Otherwise, you'll get an exception about libgomp.so.1 missing GOMP_4.0.
133+
You need to install conda's gcc for Python xgboost to work properly. Otherwise, you'll get an
134+
exception about libgomp.so.1 missing GOMP_4.0.
100135

101136
```
102137
%%bash
@@ -110,20 +145,31 @@ import xgboost
110145

111146
Sometimes it is useful to run the Jupyter instance behind a nginx proxy, for instance:
112147

113-
- you would prefer to access the notebook at a server URL with a path (`https://example.com/jupyter`) rather than a port (`https://example.com:8888`)
114-
- you may have many different services in addition to Jupyter running on the same server, and want to nginx to help improve server performance in manage the connections
148+
- you would prefer to access the notebook at a server URL with a path
149+
(`https://example.com/jupyter`) rather than a port (`https://example.com:8888`)
150+
- you may have many different services in addition to Jupyter running on the same server, and want
151+
to nginx to help improve server performance in manage the connections
115152

116-
Here is a [quick example NGINX configuration](https://gist.github.com/cboettig/8643341bd3c93b62b5c2) to get started. You'll need a server, a `.crt` and `.key` file for your server, and `docker` & `docker-compose` installed. Then just download the files at that gist and run `docker-compose up -d` to test it out. Customize the `nginx.conf` file to set the desired paths and add other services.
153+
Here is a [quick example NGINX configuration](https://gist.github.com/cboettig/8643341bd3c93b62b5c2)
154+
to get started. You'll need a server, a `.crt` and `.key` file for your server, and `docker` &
155+
`docker-compose` installed. Then just download the files at that gist and run `docker-compose up -d`
156+
to test it out. Customize the `nginx.conf` file to set the desired paths and add other services.
117157

118158
## Host volume mounts and notebook errors
119159

120-
If you are mounting a host directory as `/home/jovyan/work` in your container and you receive permission errors or connection errors when you create a notebook, be sure that the `jovyan` user (UID=1000 by default) has read/write access to the directory on the host. Alternatively, specify the UID of the `jovyan` user on container startup using the `-e NB_UID` option described in the [Common Features, Docker Options section](../using/common.html#Docker-Options)
160+
If you are mounting a host directory as `/home/jovyan/work` in your container and you receive
161+
permission errors or connection errors when you create a notebook, be sure that the `jovyan` user
162+
(UID=1000 by default) has read/write access to the directory on the host. Alternatively, specify the
163+
UID of the `jovyan` user on container startup using the `-e NB_UID` option described in the
164+
[Common Features, Docker Options section](../using/common.html#Docker-Options)
121165

122-
Ref: [https://github.com/jupyter/docker-stacks/issues/199](https://github.com/jupyter/docker-stacks/issues/199)
166+
Ref:
167+
[https://github.com/jupyter/docker-stacks/issues/199](https://github.com/jupyter/docker-stacks/issues/199)
123168

124169
## Manpage installation
125170

126-
Most containers, including our Ubuntu base image, ship without manpages installed to save space. You can use the following dockerfile to inherit from one of our images to enable manpages:
171+
Most containers, including our Ubuntu base image, ship without manpages installed to save space. You
172+
can use the following dockerfile to inherit from one of our images to enable manpages:
127173

128174
```dockerfile
129175
# Choose your desired base image
@@ -148,7 +194,10 @@ RUN echo "MANPATH_MAP ${CONDA_DIR}/bin ${CONDA_DIR}/man" >> /etc/manpath.config
148194
USER $NB_UID
149195
```
150196

151-
Adding the documentation on top of an existing singleuser image wastes a lot of space and requires reinstalling every system package, which can take additional time and bandwidth; the `datascience-notebook` image has been shown to grow by almost 3GB when adding manapages in this way. Enabling manpages in the base Ubuntu layer prevents this container bloat:
197+
Adding the documentation on top of an existing singleuser image wastes a lot of space and requires
198+
reinstalling every system package, which can take additional time and bandwidth; the
199+
`datascience-notebook` image has been shown to grow by almost 3GB when adding manapages in this way.
200+
Enabling manpages in the base Ubuntu layer prevents this container bloat:
152201

153202
```Dockerfile
154203
# Ubuntu 18.04 (bionic) from 2018-05-26
@@ -178,18 +227,24 @@ We also have contributed recipes for using JupyterHub.
178227

179228
### Use JupyterHub's dockerspawner
180229

181-
In most cases for use with DockerSpawner, given any image that already has a notebook stack set up, you would only need to add:
230+
In most cases for use with DockerSpawner, given any image that already has a notebook stack set up,
231+
you would only need to add:
182232

183-
1. install the jupyterhub-singleuser script (for the right Python)
233+
1. install the jupyterhub-singleuser script (for the right Python)
184234
2. change the command to launch the single-user server
185235

186-
Swapping out the `FROM` line in the `jupyterhub/singleuser` Dockerfile should be enough for most cases.
236+
Swapping out the `FROM` line in the `jupyterhub/singleuser` Dockerfile should be enough for most
237+
cases.
187238

188-
Credit: [Justin Tyberg](https://github.com/jtyberg), [quanghoc](https://github.com/quanghoc), and [Min RK](https://github.com/minrk) based on [docker-stacks/issues/124](https://github.com/jupyter/docker-stacks/issues/124) and [docker-stacks/pull/185](https://github.com/jupyter/docker-stacks/pull/185)
239+
Credit: [Justin Tyberg](https://github.com/jtyberg), [quanghoc](https://github.com/quanghoc), and
240+
[Min RK](https://github.com/minrk) based on
241+
[docker-stacks/issues/124](https://github.com/jupyter/docker-stacks/issues/124) and
242+
[docker-stacks/pull/185](https://github.com/jupyter/docker-stacks/pull/185)
189243

190244
### Containers with a specific version of JupyterHub
191245

192-
To use a specific version of JupyterHub, the version of `jupyterhub` in your image should match the version in the Hub itself.
246+
To use a specific version of JupyterHub, the version of `jupyterhub` in your image should match the
247+
version in the Hub itself.
193248

194249
```
195250
FROM jupyter/base-notebook:5ded1de07260
@@ -198,8 +253,8 @@ RUN pip install jupyterhub==0.8.0b1
198253

199254
Credit: [MinRK](https://github.com/jupyter/docker-stacks/issues/423#issuecomment-322767742)
200255

201-
202-
Ref: [https://github.com/jupyter/docker-stacks/issues/177](https://github.com/jupyter/docker-stacks/issues/177)
256+
Ref:
257+
[https://github.com/jupyter/docker-stacks/issues/177](https://github.com/jupyter/docker-stacks/issues/177)
203258

204259
## Spark
205260

@@ -215,7 +270,7 @@ import os
215270
os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages "org.apache.hadoop:hadoop-aws:2.7.3" pyspark-shell'
216271

217272
import pyspark
218-
myAccessKey = input()
273+
myAccessKey = input()
219274
mySecretKey = input()
220275

221276
spark = pyspark.sql.SparkSession.builder \
@@ -240,7 +295,7 @@ from pyspark.sql import SQLContext
240295
sqlContext = SQLContext(sc)
241296

242297
hadoopConf = sc._jsc.hadoopConfiguration()
243-
myAccessKey = input()
298+
myAccessKey = input()
244299
mySecretKey = input()
245300
hadoopConf.set("fs.s3.impl", "org.apache.hadoop.fs.s3native.NativeS3FileSystem")
246301
hadoopConf.set("fs.s3.awsAccessKeyId", myAccessKey)
@@ -249,7 +304,8 @@ hadoopConf.set("fs.s3.awsSecretAccessKey", mySecretKey)
249304
df = sqlContext.read.parquet("s3://myBucket/myKey")
250305
```
251306

252-
Ref: [https://github.com/jupyter/docker-stacks/issues/127](https://github.com/jupyter/docker-stacks/issues/127)
307+
Ref:
308+
[https://github.com/jupyter/docker-stacks/issues/127](https://github.com/jupyter/docker-stacks/issues/127)
253309

254310
### Using Local Spark JARs
255311

@@ -267,13 +323,18 @@ directKafkaStream.pprint()
267323
ssc.start()
268324
```
269325

270-
Ref: [https://github.com/jupyter/docker-stacks/issues/154](https://github.com/jupyter/docker-stacks/issues/154)
326+
Ref:
327+
[https://github.com/jupyter/docker-stacks/issues/154](https://github.com/jupyter/docker-stacks/issues/154)
271328

272329
### Using spark-packages.org
273330

274-
If you'd like to use packages from [spark-packages.org](https://spark-packages.org/), see [https://gist.github.com/parente/c95fdaba5a9a066efaab](https://gist.github.com/parente/c95fdaba5a9a066efaab) for an example of how to specify the package identifier in the environment before creating a SparkContext.
331+
If you'd like to use packages from [spark-packages.org](https://spark-packages.org/), see
332+
[https://gist.github.com/parente/c95fdaba5a9a066efaab](https://gist.github.com/parente/c95fdaba5a9a066efaab)
333+
for an example of how to specify the package identifier in the environment before creating a
334+
SparkContext.
275335

276-
Ref: [https://github.com/jupyter/docker-stacks/issues/43](https://github.com/jupyter/docker-stacks/issues/43)
336+
Ref:
337+
[https://github.com/jupyter/docker-stacks/issues/43](https://github.com/jupyter/docker-stacks/issues/43)
277338

278339
### Use jupyter/all-spark-notebooks with an existing Spark/YARN cluster
279340

@@ -339,17 +400,17 @@ RUN jupyter toree install --sys-prefix --spark_opts="--master yarn --deploy-mode
339400
USER $NB_USER
340401
```
341402

342-
Credit: [britishbadger](https://github.com/britishbadger) from [docker-stacks/issues/369](https://github.com/jupyter/docker-stacks/issues/369)
403+
Credit: [britishbadger](https://github.com/britishbadger) from
404+
[docker-stacks/issues/369](https://github.com/jupyter/docker-stacks/issues/369)
343405

344406
## Run Jupyter Notebook/Lab inside an already secured environment (i.e., with no token)
345407

346408
(Adapted from [issue 728](https://github.com/jupyter/docker-stacks/issues/728))
347409

348-
The default security is very good. There are use cases, encouraged by
349-
containers, where the jupyter container and the system it runs within, lie
350-
inside the security boundary. In these use cases it is convenient to launch the
351-
server without a password or token. In this case, you should use the `start.sh`
352-
script to launch the server with no token:
410+
The default security is very good. There are use cases, encouraged by containers, where the jupyter
411+
container and the system it runs within, lie inside the security boundary. In these use cases it is
412+
convenient to launch the server without a password or token. In this case, you should use the
413+
`start.sh` script to launch the server with no token:
353414

354415
For jupyterlab:
355416

@@ -358,13 +419,15 @@ docker run jupyter/base-notebook:6d2a05346196 start.sh jupyter lab --LabApp.toke
358419
```
359420

360421
For jupyter classic:
422+
361423
```
362424
docker run jupyter/base-notebook:6d2a05346196 start.sh jupyter notebook --NotebookApp.token=''
363425
```
364426

365427
## Enable nbextension spellchecker for markdown (or any other nbextension)
366428

367429
NB: this works for classic notebooks only
430+
368431
```
369432
# Update with your base image of choice
370433
FROM jupyter/minimal-notebook:latest
@@ -377,4 +440,5 @@ RUN pip install jupyter_contrib_nbextensions && \
377440
jupyter nbextension enable spellchecker/main --user
378441
```
379442

380-
Ref: [https://github.com/jupyter/docker-stacks/issues/675](https://github.com/jupyter/docker-stacks/issues/675)
443+
Ref:
444+
[https://github.com/jupyter/docker-stacks/issues/675](https://github.com/jupyter/docker-stacks/issues/675)

0 commit comments

Comments
 (0)