Roberto Panai rpanai

All Your Data Does Not Belong In a Database

Businesses are machines producing mountains of data about sales, usage, customer, costs, etc... Traditionally data processing is highly centralised with teams of staff and computer running hot a whirling ready to process. We can do better than moving the mountain of data into the corporate data machine - so long as that machinary is light enough to be moved to the data.

Don't move the mountain - Bring the processing to the data

We've had this problem; a huge directory of files in CSV format, conataining vital information for our business. But it's in CSV, requires analysis, and don't you don't feel like learning sed/grep/awk today - besides it's 2017 and no-one thinks those tools are easy to use.

Remove script for Gmail

function Intialize() {
  return;
}

function Install() {
  ScriptApp.newTrigger("purgeGmail")
 .timeBased().everyMinutes(10).create();

	import vaex
	import numpy as np
	import dask.dataframe as dd
	import dask
	import dask.distributed
	import json
	import os
	import time
	import argparse
	import multiprocessing