Duy Nguyen duynguyenhoang

Code is clean if it can be understood easily – by everyone on the team. Clean code can be read and enhanced by a developer other than its original author. With understandability comes readability, changeability, extensibility and maintainability.

General rules

Follow standard conventions.
Keep it simple stupid. Simpler is always better. Reduce complexity as much as possible.
Boy scout rule. Leave the campground cleaner than you found it.
Always find root cause. Always look for the root cause of a problem.

Design rules

Finding

$ grep -rnw 'directory' -e "pattern"

-r is recursive, -n is line number and -w stands match the whole word. Along with these, --exclude or --include parameter could be used for efficient searching. Something like below:

$ grep --include={*.c,*.h} -rnw 'directory' -e "pattern"

This will only search through the files which have .c or .h extensions. Similarly a sample use of --exclude:

Standalone Spark 2.0.0 with s3

###Tested with:

Spark 2.0.0 pre-built for Hadoop 2.7
Mac OS X 10.11
Python 3.5.2

Goal

Use s3 within pyspark with minimal hassle.

	-- Create sample table
	CREATE TABLE users (id Int, name String, created_at DateTime default NOW())
	ENGINE = Memory;

	-- Insert data
	INSERT INTO users(id, name) VALUES(1, 'First 1')
	-- Stop for a while
	INSERT INTO users(id, name) VALUES(1, 'First 2')

	-- Dedup query

	#!/bin/bash
	# Stupid simple script to help you learn/take note daily.
	# Requires https://github.com/charmbracelet/glow to render markdown in terminal

	# IMPORTANT: Your learn files location
	export LEARN_HOME=$HOME/learn

	lhe() {
	# Simple help function to list all commands
	for i in {1..50}; do echo -n "="; done

	# .gitignore sample
	###################

	# Ignore all files in this dir...
	*

	# ... except for this one.
	!.gitignore

	#!/bin/bash
	# This script allows you to backup a single volume from a container
	# Data in given volume is saved in the current directory in a tar archive.
	CONTAINER_NAME=$1
	VOLUME_NAME=$2

	usage() {
	echo "Usage: $0 [container name] [volume name]"
	exit 1
	}

	// Demonstrate https://stackoverflow.com/questions/54781538/node-elasticsearch-bulk-index-fails-silently
	const elasticsearch = require('elasticsearch');
	const elastic = new elasticsearch.Client({
	host: `localhost:9200`
	});

	const inputArray = [
	{
	"index": {
	"_index": "test-2019.02.19"

	# NOTE, CHANGE BELOW VALUE
	# SSH user is 'deployer'
	# Server address is 'remote_server_address'
	# Server location is '/your/server/location/'
	before_script:
	##
	## Install ssh-agent if not already installed, it is required by Docker.
	## (change apt-get to yum if you use an RPM-based image)
	##
	- 'which ssh-agent \|\| ( apt-get update -y && apt-get install openssh-client rsync -y )'

	#!/bin/bash

	# for setup on MacOS: https://www.ibm.com/developerworks/community/blogs/jfp/entry/Installing_XGBoost_on_Mac_OSX?lang=en
	# http://stackoverflow.com/questions/40942893/how-to-install-xgboost-on-osx-with-multi-threading
	# MacOS's default c++ compiler (OpenMP) doesn't support multithreading, so to get the performance boost we need to
	# build xgboost from source with g++, and install it using devtools in R.

	# this script sets up xgboost on an Ubuntu AWS cluster. I'm using a c4.4xlarge at the moment.
	# RUN AS SUDO - TO INSTALL PACKAGES, R NEEDS ROOT ACCESS
	cd ~