java.lang.ClassNotFoundException: com.mysql.jdbc.Driver

Error:

java.lang.ClassNotFoundException: com.mysql.jdbc.Driver

Solution:
Both spark driver and executor need mysql driver on class path so specify

• spark.driver.extraClassPath = /usr/share/java/mysql-connector-java.jar 
• spark.executor.extraClassPath = /usr/share/java/mysql-connector-java.jar

#Initialize SparkSession and SparkContext

from pyspark.sql import SparkSession

from pyspark import SparkContext

#Create a Spark Session

SpSession = SparkSession \

    .builder \

    .master("local[2]") \

    .appName("prabhath") \

    .config("spark.executor.memory", "1g") \

    .config("spark.cores.max","2") \

    .config("spark.driver.extraClassPath", "/usr/share/java/mysql-connector-java.jar") \

    .config("spark.executor.extraClassPath", "/usr/share/java/mysql-connector-java.jar") \

    .config("spark.sql.warehouse.dir", "/Users/jlyang/Spark/spark-warehouse")\

    .getOrCreate()

#Get the Spark Context from Spark Session

SpContext = SpSession.sparkContext

demoDf = SpSession.read.format("jdbc").options(

    url="jdbc:mysql://localhost:3306/testpraba1",

    driver = "com.mysql.jdbc.Driver",

    dbtable = "users_userprofile",

    user="root",

    password="XXXXX").load()

demoDf.show()

ERROR MasterWebUI: Failed to bind MasterWebUI

Problem: 
While starting start-master.sh, getting ERROR MasterWebUI: Failed to bind MasterWebUI

Solution:
check SPARK_LOCAL_IP is set to correct IP Address
export SPARK_LOCAL_IP=192.168.254.122 

Error:
Spark Command: /opt/java/jdk1.8.0_201/bin/java -cp /opt/spark/conf/:/opt/spark/jars/* -Xmx1g org.apache.spark.deploy.master.Master --host 127.0.0.1 --port 7077 --webui-port 8080
========================================
19/03/12 14:03:50 ERROR MasterWebUI: Failed to bind MasterWebUI
java.net.BindException: Cannot assign requested address: Service 'MasterUI' failed after 16 retries (starting from 8080)! Consider explicitly setting the appropriate port for the service 'MasterUI' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:433)
    at sun.nio.ch.Net.bind(Net.java:425)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at org.spark_project.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:351)
    at org.spark_project.jetty.server.ServerConnector.open(ServerConnector.java:319)
    at org.spark_project.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
    at org.spark_project.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
    at org.spark_project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$newConnector$1(JettyUtils.scala:353)
    at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$httpConnect$1(JettyUtils.scala:380)
    at org.apache.spark.ui.JettyUtils$$anonfun$7.apply(JettyUtils.scala:383)
    at org.apache.spark.ui.JettyUtils$$anonfun$7.apply(JettyUtils.scala:383)
    at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2269)
    at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
    at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2261)
    at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:383)
    at org.apache.spark.ui.WebUI.bind(WebUI.scala:132)
    at org.apache.spark.deploy.master.Master.onStart(Master.scala:140)
    at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:122)
    at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:205)
    at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:101)
    at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:221)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)