Feng Wang (Felix) hezila

Info

This guide sets up a non-clustered Nutch crawler, which stores its data via HBase. We will not learn how to setup Hadoop et al., but just the bare minimum to crawl and index websites on a single machine.

Terms

Nutch - the crawler (fetches and parses websites)
HBase - filesystem storage for Nutch (Hadoop component, basically)

Feature Learning

Learning Feature Representations with K-means by Adam Coates and Andrew Y. Ng
The devil is in the details: an evaluation of recent feature encoding methods by Chatfield et. al.
Emergence of Object-Selective Features in Unsupervised Feature Learning by Coates, Ng
Scaling Learning Algorithms towards AI Benjio & LeCun
A Theory of Feature Learning by Brendan van Rooyen, Robert C. Williamson

Deep Learning

Dropout: A Simple Way to Prevent Neural Networks from Overfitting by Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov
[Understanding

Flask-SQLAlchemy Caching

The following gist is an extract of the article Flask-SQLAlchemy Caching. It allows automated simple cache query and invalidation of cache relations through event among other features.

Usage

retrieve one object

# pulling one User object

user = User.query.get(1)

	try:
	from StringIO import StringIO as BytesIO
	except ImportError:
	from io import BytesIO

	import matplotlib.pyplot as plt


	def render_latex(formula, fontsize=12, dpi=300, format_='svg'):
	"""Renders LaTeX formula into image."""

	sudo apt-get update
	sudo apt-get install docker.io
	source /etc/bash_completion.d/docker.io
	sudo docker run -i -t ubuntu /bin/bash

	##########################################################################################
	# use case: training a classifier
	#
	# Many systems classify documents by assigning “tag” or “category” fields. Classifying
	# documents can be a tedious manual process and so in this example we will train a classifier
	# to automatically spot keywords in new documents that suggest a suitable category.

	curl -XGET "http://localhost:9200/products_fr/_search" -d'
	{
	"query": {

	#!/bin/bash

	### USAGE
	###
	### ./ElasticSearch.sh 1.7 will install Elasticsearch 1.7
	### ./ElasticSearch.sh will fail because no version was specified (exit code 1)
	###
	### CLI options Contributed by @janpieper
	### Check http://www.elasticsearch.org/download/ for latest version of ElasticSearch

	import com.mongodb.BasicDBObject;
	import com.mongodb.DBCollection;
	import com.mongodb.DBObject;
	import com.mongodb.MongoClient;
	import org.bson.BSON;
	import org.bson.Transformer;
	import org.joda.time.DateTime;

	import java.net.UnknownHostException;
	import java.util.Date;

	def get_count(q):
	count_q = q.statement.with_only_columns([func.count()]).order_by(None)
	count = q.session.execute(count_q).scalar()
	return count

	q = session.query(TestModel).filter(...).order_by(...)

	# Slow: SELECT COUNT(*) FROM (SELECT ... FROM TestModel WHERE ...) ...
	print q.count()

	<!DOCTYPE html>
	<html>
	<head>
	<script>
	window.MediaSource = window.MediaSource \|\| window.WebKitMediaSource;

	function testTypes(types) {
	for (var i = 0; i < types.length; ++i)
	console.log("MediaSource.isTypeSupported(" + types[i] + ") : " + MediaSource.isTypeSupported(types[i]));
	}