add grpcio grpcio-status to support spark connect (#2017)

* add grpcio grpcio_status to support spark connect

* Sort install list

* Fix package name

* Update pyspark docs with new deps grpcio and grpcio-status

* set grpcio and grpcio-status version as 1.56

* exclude grpcio and grpcio-status in test_packages.py

* Update selecting.md

* Update test_packages.py

* Update Dockerfile

---------

Co-authored-by: Ayaz Salikhov <[email protected]>

Author: xieshuaihu

docs/using/selecting.md

@@ -209,7 +209,9 @@ It contains:
 
 - Everything in `jupyter/scipy-notebook` and its ancestor images
 - [Apache Spark](https://spark.apache.org/) with Hadoop binaries
-- [pyarrow](https://arrow.apache.org/docs/python/) library
+- [grpcio-status](https://github.com/grpc/grpc/tree/master/src/python/grpcio_status)
+- [grpcio](https://grpc.io/docs/languages/python/quickstart/)
+- [pyarrow](https://arrow.apache.org/docs/python/)
 
 ### jupyter/all-spark-notebook
 

images/pyspark-notebook/Dockerfile

@@ -74,6 +74,8 @@ USER ${NB_UID}
 # 1. Check out the Spark branch you are on.
 # 2. Find the pandas version in the file spark/dev/infra/Dockerfile.
 RUN mamba install --yes \
+    'grpcio-status' \
+    'grpcio' \
     'pandas=2.0.3' \
     'pyarrow' && \
     mamba clean --all -f -y && \

tests/base-notebook/test_packages.py

@@ -69,6 +69,8 @@
     "bzip2",
     "ca-certificates",
     "conda-forge::blas[build=openblas]",
+    "grpcio-status",
+    "grpcio",
     "hdf5",
     "jupyterlab-git",
     "openssl",