Skip to content

Instantly share code, notes, and snippets.

View emptyr1's full-sized avatar

Mudit emptyr1

32 followers · 81 following
  • United States
View GitHub Profile
@bmaupin
bmaupin / free-database-hosting.md
Last active March 11, 2026 06:33
Free database hosting
@emptyr1
emptyr1 / SQL_Collections.md
Created October 6, 2018 17:34
SQL_Collections
@enricofoltran
enricofoltran / main.go
Last active December 30, 2025 18:52
A simple golang web server with basic logging, tracing, health check, graceful shutdown and zero dependencies
package main
import (
"context"
"flag"
"fmt"
"log"
"net/http"
"os"
"os/signal"
@claudinei-daitx
claudinei-daitx / SparkSessionS3.scala
Created December 15, 2017 13:02
Create a Spark session optimized to work with Amazon S3.
import org.apache.spark.sql.SparkSession
object SparkSessionS3 {
//create a spark session with optimizations to work with Amazon S3.
def getSparkSession: SparkSession = {
val spark = SparkSession
.builder
.appName("my spark application name")
.config("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
.config("spark.hadoop.fs.s3a.access.key", "my access key")
@isaacmg
isaacmg / run_jar.py
Created January 29, 2017 08:56
A simple example of using a DAG to run a jar file.
from airflow import DAG
from airflow.operators import BashOperator
from datetime import datetime
import os
import sys
args = {
'owner': 'airflow'
, 'start_date': datetime(2017, 1, 27)
, 'provide_context': True
@emptyr1
emptyr1 / spark_jupyter.sh
Last active January 6, 2017 06:05
spark + ec2 + jupyter
# start with this: https://gist.github.com/modqhx/c761020d987fd21929c64e69af27af2f
wget http://d3kbcqa49mib13.cloudfront.net/spark-2.1.0-bin-hadoop2.7.tgz | tar -xvzf spark-2.1.0-bin-hadoop2.7.tgz
mv spark-2.1.0-bin-hadoop2.7 /usr/local/spark2.1
echo 'SPARK_HOME=/usr/local/spark2.1' >> ~/.bashrc
echo 'PATH=$SPARK_HOME/bin:$PATH' >> ~/.bashrc
# Install java and set JAVA_HOME if not set already from here: https://www.digitalocean.com/community/tutorials/how-to-install-java-on-ubuntu-with-apt-get
sudo apt-get update
@rondy
rondy / Effective_Engineer.md
Last active March 16, 2026 13:08

FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.

Effective Engineer - Notes

What's an Effective Engineer?

@konradko
konradko / golang_on_rpi.md
Last active September 24, 2024 14:37
Install Golang 1.7 on Raspberry Pi 
wget https://storage.googleapis.com/golang/go1.7.linux-armv6l.tar.gz
tar -C /usr/local -xzf go1.7.linux-armv6l.tar.gz
export PATH=$PATH:/usr/local/go/bin
@pwpearson
pwpearson / Scala Spark Window Function Example.scala
Created July 22, 2016 14:18
Spark example of using row_number and rank.
// This example shows how to use row_number and rank to create
// a dataframe of precipitation values associated with a zip and date
// from the closest NOAA station
import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions._
// mocked NOAA weather station data
case class noaaData(zip:String,station:String,date:Long,value:String=null,distance:Int)
val t = Seq(
@gwillem
gwillem / ansible-bootstrap-ubuntu-16.04.yml
Created June 16, 2016 21:59
Get Ansible to work on bare Ubuntu 16.04 without python 2.7
# Add this snippet to the top of your playbook.
# It will install python2 if missing (but checks first so no expensive repeated apt updates)
# gwillem@gmail.com
- hosts: all
gather_facts: False
tasks:
- name: install python 2
raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)