How to set up a Jupyter notebook sandbox on localhost

Sometimes it may be useful to set up a Jupyter “sandbox” in your own development environment with a copy of files from MI configuration.

As an example, it can be used for debugging and developing Python transformations, or interactive exploration of your data.

The script below prepares the same directory structure as a configuration in MI and assumes you have downloaded the input files.

First, install Jupyter notebook $ pip install jupyter

Install the script into ~/.local/bin/meiro-sandbox (or somewhere else on your $PATH).

Run $ chmod u+x ~/.local/bin/meiro-sandbox 

Use $ meiro-sandbox path/to/configuration_xy_input_files.zip

It will open up a browser with a notebook for you.

#/bin/sh
set -e
PATH_TO_ZIP=$1
if [[ -z "$PATH_TO_ZIP" ]]
then
   echo "Create jupyter notebook sandbox from meiro downloaded files"
   echo "    Usage:"
   echo "        $ meiro-sandbox path/to/configuration_xy_input_files.zip"
   exit 1
fi
TMP_DATADIR=/tmp/meiro_sandbox_$(date +%Y%m%d%H%M%S)
PATH_IPYNB=$TMP_DATADIR/sandbox.ipynb
echo "Creating sandbox ${TMP_DATADIR}"
mkdir -p $TMP_DATADIR/out/tables $TMP_DATADIR/out/files $TMP_DATADIR/in
echo '{"cells": [{"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4"}}, "nbformat": 4, "nbformat_minor": 4}' > $PATH_IPYNB
unzip $PATH_TO_ZIP -d $TMP_DATADIR/in
jupyter notebook $PATH_IPYNB
read -p "Do you want to keep the sandbox? [Yy] for yes; empty/Nn to delete" -n 1 -r
echo    # (optional) move to a new line
if [[ $REPLY =~ ^[Yy]$ ]]
then
   echo "Keeping sandbox at ${TMP_DATADIR}"
else
   echo "Deleting sandbox"
   rm -r $TMP_DATADIR
fi

After stopping the Jupyter server you are prompted to keep/delete the sandbox.

# pocin @ mbp in /tmp [12:51:59] C:1
$ meiro-sandbox ~/Downloads/configuration_183_input_files.zip
Create jupyter notebook sandbox from meiro downloaded files
   Usage:
       $ meiro-sandbox path/to/configuration_xy_input_files.zip
# pocin @ mbp in /tmp [12:52:20] C:130
$ meiro-sandbox ~/Downloads/configuration_183_input_files.zip
Creating sandbox /tmp/meiro_sandbox_20200122125220
Archive:  /Users/pocin/Downloads/configuration_183_input_files.zip
extracting: /tmp/meiro_sandbox_20200122125220/in/tables/crm_events_master.csv
extracting: /tmp/meiro_sandbox_20200122125220/in/tables/db_patron.csv
extracting: /tmp/meiro_sandbox_20200122125220/in/tables/transactions.csv
[I 12:52:22.811 NotebookApp] The port 8888 is already in use, trying another port.
[I 12:52:22.869 NotebookApp] JupyterLab extension loaded from /usr/local/var/pyenv/versions/3.7.4/lib/python3.7/site-packages/jupyterlab
[I 12:52:22.869 NotebookApp] JupyterLab application directory is /usr/local/var/pyenv/versions/3.7.4/share/jupyter/lab
[I 12:52:22.872 NotebookApp] Serving notebooks from local directory: /tmp/meiro_sandbox_20200122125220
[I 12:52:22.872 NotebookApp] The Jupyter Notebook is running at:
[I 12:52:22.872 NotebookApp] http://localhost:8889/?token=3d5810eccb77705549d7164ef6fc353611076aa9a2e12c9a
[I 12:52:22.872 NotebookApp]  or http://127.0.0.1:8889/?token=3d5810eccb77705549d7164ef6fc353611076aa9a2e12c9a
[I 12:52:22.872 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:52:22.893 NotebookApp]
   To access the notebook, open this file in a browser:
       file:///Users/pocin/Library/Jupyter/runtime/nbserver-70567-open.html
   Or copy and paste one of these URLs:
       http://localhost:8889/?token=3d5810eccb77705549d7164ef6fc353611076aa9a2e12c9a
    or http://127.0.0.1:8889/?token=3d5810eccb77705549d7164ef6fc353611076aa9a2e12c9a
^C[I 12:52:25.072 NotebookApp] interrupted
Serving notebooks from local directory: /tmp/meiro_sandbox_20200122125220
0 active kernels
The Jupyter Notebook is running at:
http://localhost:8889/?token=3d5810eccb77705549d7164ef6fc353611076aa9a2e12c9a
or http://127.0.0.1:8889/?token=3d5810eccb77705549d7164ef6fc353611076aa9a2e12c9a
Shutdown this notebook server (y/[n])? ^C[C 12:52:25.837 NotebookApp] received signal 2, stopping
[I 12:52:25.838 NotebookApp] Shutting down 0 kernels
Do you want to keep the sandbox? [Yy] for yes; empty/Nn to delete^C