###Tested with:
- Spark 2.0.0 pre-built for Hadoop 2.7
- Mac OS X 10.11
- Python 3.5.2
Use s3 within pyspark with minimal hassle.
Bikash Sharma bikashsharmabks
bikashsharmabks
/ setup-notes.md
Created
May 29, 2018 14:28
— forked from eddies/setup-notes.md
Spark 2.0.0 and Hadoop 2.7 with s3a setup
bikashsharmabks
/ rm-none-docker
Created
May 21, 2018 10:22
— forked from fojuth/rm-none-docker
Remove <none> Docker images
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| docker rmi $(docker images | grep "^<none>" | awk "{print $3}") |
bikashsharmabks
/ Indian-Female-Names.csv
Created
April 30, 2018 09:57
— forked from mbejda/Indian-Female-Names.csv
Dataset of ~14,000 Indian female names for NLP training and analysis. The names have been retrieved from public records. (name,gender,race)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name | gender | race | |
|---|---|---|---|
| shivani | f | indian | |
| isha | f | indian | |
| smt shyani devi | f | indian | |
| divya | f | indian | |
| mansi | f | indian | |
| mazida | f | indian | |
| pooja | f | indian | |
| kajal | f | indian | |
| meena | f | indian |
bikashsharmabks
/ kafka-ubuntu16.04-install.rst
Created
March 20, 2018 06:30
— forked from monkut/kafka-ubuntu16.04-install.rst
kafka installation on ubuntu 16.04
# referecing: # https://www.digitalocean.com/community/tutorials/how-to-install-apache-kafka-on-ubuntu-14-04 # https://chongyaorobin.wordpress.com/2015/07/08/step-by-step-of-install-apache-kafka-on-ubuntu-standalone-mode/
- Add 'kafka' user:
$ sudo useradd kafka -m
- Install Java:
bikashsharmabks
/ install.sh
Created
August 31, 2017 06:19
— forked from bikashPeople10/install.sh
Install OpenCV 3 with Python 2.7 on Ubuntu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Configuration | |
| HOME_DIR=/home/[user]/ | |
| VERSION=3.2.0 | |
| # Installation | |
| sudo apt-get update | |
| sudo apt-get upgrade | |
| sudo apt-get install -y build-essential cmake pkg-config | |
| sudo apt-get install -y libjpeg8-dev libtiff5-dev libjasper-dev libpng12-dev | |
| sudo apt-get install -y libavcodec-dev libavformat-dev libswscale-dev libv4l-dev |