Skip to content

Instantly share code, notes, and snippets.

@SudhakarReddyPeddinti

SudhakarReddyPeddinti/Apache spark - For loop

Created September 10, 2016 22:56
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save SudhakarReddyPeddinti/c5e0cc61e09817aee38c5385ea705360 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save SudhakarReddyPeddinti/c5e0cc61e09817aee38c5385ea705360 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. SudhakarReddyPeddinti created this gist Sep 10, 2016.
    9 changes: 9 additions & 0 deletions Apache spark - For loop
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,9 @@
    var l = List(34, 32, 132, 352) // Scala List

    l.foreach{i =>
    val data:DataFrame = DataContainer.getDataFrame(i) // get DataFrame
    val x = new MyClass(data) // initialize MyClass with new Object
    x.setSettings(...)
    x.calcSomething()
    x.saveResults() // writes the Results into another Dataframe that is saved to HDFS
    }