Scott Cruwys scruwys

Useful Pandas Snippets

A personal diary of DataFrame munging over the years.

Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)

Attention: the list was moved to

This page is not maintained anymore, please update your bookmarks.

	import logging.config
	import os
	from django.utils.log import DEFAULT_LOGGING

	# Disable Django's logging setup
	LOGGING_CONFIG = None

	LOGLEVEL = os.environ.get('LOGLEVEL', 'info').upper()

	logging.config.dictConfig({

	import os
	import unittest

	from airflow.models import DagBag

	class TestDags(unittest.TestCase):
	"""
	Generic tests that all DAGs in the repository should be able to pass.
	"""

	-- Table information like sortkeys, unsorted percentage
	-- see http://docs.aws.amazon.com/redshift/latest/dg/r_SVV_TABLE_INFO.html
	SELECT * FROM svv_table_info;

	-- Table sizes in GB
	SELECT t.name, COUNT(tbl) / 1000.0 AS gb
	FROM (
	SELECT DISTINCT datname, id, name
	FROM stv_tbl_perm
	JOIN pg_database ON pg_database.oid = db_id