In last post, we have set up a
jupyterhub in CEntOS, the same installation guide can be found for Ubuntu-like system.
In this post, we will discuss how to make
jupyterhub work with
Step 1: create a kernel directory in the user’s home folder
Step 2: create and edit kernel file
Create & Edit the file:
Put the script below into the file:
Please replace the
<spark_dir> with the location of the spark:
/opt/mapr/spark/spark-1.4.1 in my example.
Put lauching arguments in
PYSPARK_SUBMIT_ARGS, as shown here create a local
--master yarn-client pyspark-shell in my case for
Step 3: launch jupyterhub and create a spark notebook with kernel spark-*.*.*
Optional Step: copy the file to every spark user