Spark - Mysql 연결에러 (java.lang.ClassNotFoundException)

 

[ 에러 ]

Spark로 Mysql 연결시

java.lang.ClassNotFoundException: com.mysql.jdbc.Driver

에러가 뜬다면 

 

1. spark-shell 

spark-shell --jars mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar

: --jars옵션을 통해 스파크쉘을 실행한다

 

2. spark-submit

jars폴더 안에 mysql-connector파일을 넣어준다.

cp -r $HIVE_HOME/lib/mysql-connector-java-5.1.46-bin.jar $SPARK_HOME/jars/

* CDH환경일시 

cp ./mysql-connector-java-5.1.46-bin.jar /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/spark/jars