Data Platform/spark

spark-submit deploy-mode option

태하팍 2016. 11. 2. 17:45
반응형


  • Client mode
    • Want to get a job result (dynamic analysis)
    • Easier for developping/debugging
    • Control where your Driver Program is running
    • Always up application: expose your Spark job launcher as REST service or a Web UI
  • Cluster mode
    • Easier for resource allocation (let the master decide): Fire and forget
    • Monitor your Driver Program from Master Web UI like other workers
    • Stop at the end: one job is finished, allocated resources a freed


spark-submit 관련 


반응형