| import duckdb | |
| from google.cloud import bigquery | |
| bqclient = bigquery.Client() | |
| table = bigquery.TableReference.from_string( | |
| "bigquery-public-data.utility_us.country_code_iso" | |
| ) | |
| rows = bqclient.list_rows(table) | |
| country_code_iso = rows.to_arrow(create_bqstorage_client=True) | |
| cursor = duckdb.connect() |
The steps below bootstrap an instance of airflow, configured to use the kubernetes airflow executor, working within a minikube cluster.
This guide works with the airflow 1.10 release, however will likely break or have unnecessary extra steps in future releases (based on recent changes to the k8s related files in the airflow source).
| # ---- Base python ---- | |
| FROM python:3.6 AS base | |
| # Create app directory | |
| WORKDIR /app | |
| # ---- Dependencies ---- | |
| FROM base AS dependencies | |
| COPY gunicorn_app/requirements.txt ./ | |
| # install app dependencies | |
| RUN pip install -r requirements.txt |
Let's look at an example of how to launch a Kubernetes cluster from scratch on DigitalOcean, including kubeadm, an Nginx Ingress controller, and Letsencrypt certificates.
We'll be creating a four-node cluster (k8s-master, k8s-000...k8s-002), load balancer, and ssl certificates.
| #!/usr/bin/env bash | |
| # set -x | |
| if [[ $EUID -ne 0 ]]; then | |
| echo "You must be root to run this script" | |
| exit 1 | |
| fi | |
| # Returns all available interfaces, except "lo" and "veth*". |
| #!/usr/bin/env sh | |
| curl -X POST -H "Accept: application/json" -H "Content-type: application/json" \ | |
| -d "{\"value1\": \"$*\"}" \ | |
| 'https://maker.ifttt.com/trigger/{your event name}}/with/key/{insert your key}' \ | |
| >/dev/null 2>&1 |
| #!/usr/bin/env lua | |
| --[[ | |
| dsl-print [some.dsl] | |
| Example of a lua DSL language, that can use function call or assignment to define | |
| keys, and uses call sequence to implicitly define nesting. | |
| ]] | |
| function dsl(code, file) | |
| local loaded, estr = loadstring(code, file) |
| import io.Source | |
| import scala.util.control.Breaks._ | |
| /** | |
| * Scala TicTacToe game without any side effects | |
| * | |
| * Written in response to following post (which also contains task description): | |
| * http://blog.tmorris.net/scala-exercise-with-types-and-abstraction/ | |
| */ | |
| object TicTacToe { |
