Skip to content

Instantly share code, notes, and snippets.

@maxtomassi

maxtomassi/StopSparkStreaming.scala

Last active August 29, 2015 14:08
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save maxtomassi/cd3f5eb8e5c039fd9e8f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save maxtomassi/cd3f5eb8e5c039fd9e8f to your computer and use it in GitHub Desktop.
Download ZIP
Gracefully stopping a Spark Streaming application
Raw
StopSparkStreaming.scala
def main(args: Array[String]) {
// Prepare your environment
val ssc = new StreamingContext(conf, Seconds(batchDurationInSec))
// Do your processing
sys.ShutdownHookThread {
log.info("Gracefully stopping Spark Streaming Application")
ssc.stop(true, true)
log.info("Application stopped")
}
ssc.start
ssc.awaitTermination
}
@eatgrass
Copy link

eatgrass commented Apr 10, 2015

I tried the Java version on spark 1.3 like below, I don't know why it doesn't work？

Runtime.getRuntime().addShutdownHook( new Thread(){
    public void run(){
        sc.stop();
        System.out.println("stopped gracefully");
    }
});

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment