spark 本地调试

🧙 Questions

☄️ Ideas

如果本地无法启动gworker

/Users/ispong/isxcode/spark-yun/spark-yun-dist/spark-min/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://ispong-mac.local:7077

分开启动

bash start-master.sh
bash start-worker.sh --webui-port 8081 spark://ispong-mac.local:7077
bash stop-all.sh

本地启动spark

nohup bash ${agent_path}/spark-min/sbin/start-master.sh --host 0.0.0.0 --port 7077 --webui-port 8081 > /dev/null 2>&1 &
nohup bash ${agent_path}/spark-min/sbin/start-worker.sh --host 0.0.0.0 --webui-port 8082 spark://0.0.0.0:7077 > /dev/null 2>&1 &
bash stop-cluster

🔗 Links

• spark docs