Zeeshan Haider zeeshanalisyed

Spark - High availability

As a reminder, here are the components in play to run an application:

The cluster:
- Spark Master: coordinates the resources
- Spark Workers: offer resources to run the applications
The application:

	# pre
	yum -y install epel-release yum-plugin-priorities
	# install tripleo repos

	# This will pull the latest version RPM
	PKG=$(curl https://trunk.rdoproject.org/centos7/current/ \| grep python2-tripleo-repos- \| awk -F'"' '{print $8}')

	# This will install it
	yum install -y https://trunk.rdoproject.org/centos7/current/${PKG}

	--log_gc (Log heap samples on garbage collection for the hp2ps tool.)
	type: bool default: false
	--expose_gc (expose gc extension)
	type: bool default: false
	--max_new_space_size (max size of the new generation (in kBytes))
	type: int default: 0
	--max_old_space_size (max size of the old generation (in Mbytes))
	type: int default: 0
	--max_executable_size (max size of executable memory (in Mbytes))
	type: int default: 0