Skip to content

Instantly share code, notes, and snippets.

@yujikiriki

yujikiriki/Dockerfile

Created September 17, 2015 13:14
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save yujikiriki/0825037d72a5293e2008 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save yujikiriki/0825037d72a5293e2008 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. Yuji Kiriki created this gist Sep 17, 2015.
    25 changes: 25 additions & 0 deletions Dockerfile
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,25 @@
    FROM nimmis/java:openjdk-8-jdk
    MAINTAINER yujikiriki

    # expose spark ports
    EXPOSE 8080
    EXPOSE 7077
    EXPOSE 8081
    EXPOSE 6066

    WORKDIR /usr/local
    # add python
    RUN apt-get install -y python
    #add scala
    RUN wget http://www.scala-lang.org/files/archive/scala-2.11.7.deb
    RUN dpkg -i scala-2.11.7.deb
    RUN rm -f scala-2.11.7.deb
    # build spark
    RUN export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m"
    RUN curl -s http://mirror.metrocast.net/apache/spark/spark-1.4.1/spark-1.4.1.tgz| tar -xz ; ln -s spark-1.4.1 spark
    RUN cd spark ; sbt/sbt assembly
    # start spark standalone master with one slave
    RUN /usr/local/spark/sbin/start-master.sh
    RUN env
    RUN /usr/local/spark/sbin/start-slave.sh spark://$HOSTNAME:7077
    CMD /usr/local/spark/bin/spark-shell --master spark://$HOSTNAME:7077