Skip to content

Instantly share code, notes, and snippets.

@mmalohlava

mmalohlava/ZipPrediction.scala

Created October 6, 2015 17:57
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mmalohlava/6d52df29c1b1c58a434a to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mmalohlava/6d52df29c1b1c58a434a to your computer and use it in GitHub Desktop.
Download ZIP
Zip prediction with actual response
Raw
ZipPrediction.scala
//It depends what is your next processing step, i
// f you need to have data in (a) Spark DataFrame or you are happy with data in (b) H2OFrame.
// (a) We need data in DataFrame
val reconstructionError = dlModel.scoreAutoEncoder(train, Key.make)
val df: DataFrame = h2oContext.asDataFrame(reconstructionError)(sqlContext)
val joinedFrame = testData.zip(df) // This is Spark DataFrame
// Note: you need Spark 1.3, or 1.4
// (b) We need data in H2OFrame
val testDataH2OFrame: H2OFrame = h2oContext.asH2OFrame(testData);
val joinedH2OFrame = reconstructionError.add(testDataH2OFrame)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment