Skip to content

Instantly share code, notes, and snippets.

@dmateusp

dmateusp/FinalExampleTransform.scala

Created June 4, 2019 07:03
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save dmateusp/4cef1b6d28208e04a16235178189ec8e to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save dmateusp/4cef1b6d28208e04a16235178189ec8e to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. dmateusp created this gist Jun 4, 2019.
    16 changes: 16 additions & 0 deletions FinalExampleTransform.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,16 @@
    // chain transform calls
    dfTransactions
    .transform(extractPayerBeneficiary("details"))
    .transform(sumAmounts(date_trunc("day", col("ts")), col("details_beneficiary")))

    // andThen
    dfTransactions
    .transform(extractPayerBeneficiary("details") andThen sumAmounts(date_trunc("day", col("ts")), col("details_beneficiary")))

    // compose
    dfTransactions
    .transform(sumAmounts(date_trunc("day", col("ts")), col("details_beneficiary")) compose extractPayerBeneficiary("details"))

    // Function.chain
    dfTransactions
    .transform(Function.chain(List(extractPayerBeneficiary("details"), sumAmounts(date_trunc("day", col("ts")), col("details_beneficiary")))))