Farid Uyar fuyar

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h or --help depending on your psql version):

-E: will describe the underlaying queries of the \ commands (cool for learning!)
-l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)

To see status of a Kafka Consumer Group's consumption status,

/bin/kafka-consumer-groups.sh --bootstrap-server server_ip:port --describe --group consumer_group_name | awk '{lag_sum += $5; total_sum += $4; done_sum += $3} END {print "Completed count: " done_sum ", Lag: " lag_sum ", Total number of records: " total_sum}'

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

	-- show running queries (pre 9.2)
	SELECT procpid, age(clock_timestamp(), query_start), usename, current_query
	FROM pg_stat_activity
	WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%'
	ORDER BY query_start desc;

	-- show running queries (9.2)
	SELECT pid, age(clock_timestamp(), query_start), usename, query
	FROM pg_stat_activity
	WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'

	#!/usr/bin/env bash
	#
	# File: kafka-move-leadership.sh
	#
	# Description
	# ===========
	#
	# Generates a Kafka partition reassignment JSON snippet to STDOUT to move the leadership
	# of any replicas away from the provided "source" broker to different, randomly selected
	# "target" brokers. Run this script with `-h` to show detailed usage instructions.

	#!/bin/bash -xe

	# Parse server.properties to get Zookeeper hosts
	ZK_HOST=$(grep 'zookeeper.connect=' /etc/kafka/server.properties \| cut -d'=' -f 2)

	# Find the newest version of Kafka in the /opt directory
	KAFKA_PATH=$(find /opt/ -maxdepth 1 -name kafka-2* \| sort \| tail -n1)

	# Manually configured list of brokers to rebalance across (zero-indexed)
	BROKER_LIST="0,1,2"