Skip to content

Instantly share code, notes, and snippets.

View poteznyKrolik's full-sized avatar

mzajac poteznyKrolik

  • 03:15 (UTC -07:00)
View GitHub Profile
@poteznyKrolik
poteznyKrolik / timeout.py
Created February 27, 2023 23:53 — forked from TySkby/timeout.py
Timeout decorator/context manager using signals (for Python 3)
#!/usr/bin/env python3
"""Easily put time restrictions on things
Note: Requires Python 3.x
Usage as a context manager:
```
with timeout(10):
something_that_should_not_exceed_ten_seconds()
```
@poteznyKrolik
poteznyKrolik / DecodeLargeJSON.py
Created October 8, 2020 22:05 — forked from niccokunzmann/DecodeLargeJSON.py
Decode large json files
"""
Response to
http://stackoverflow.com/a/22904200/1320237
"""
import json.scanner
import json.decoder
from json.decoder import JSONDecoder
class FileString(object):
@poteznyKrolik
poteznyKrolik / zipr.py
Created September 30, 2020 18:53 — forked from bbengfort/zipr.py
Dealing with Zip archives and json data in Python
#!/usr/bin/env python3
import os
import json
import random
import zipfile
config = {
"color": "red",
"amount": 42.24,
@poteznyKrolik
poteznyKrolik / pil_s3.py
Created September 29, 2020 22:03 — forked from ghandic/pil_s3.py
Load image from S3 directly into memory as PIL image and write to S3 directly from memory from PIL image
import boto3
from PIL import Image
from io import BytesIO
import os
class S3ImagesInvalidExtension(Exception):
pass
class S3ImagesUploadFailed(Exception):
pass
@poteznyKrolik
poteznyKrolik / schemacrawler-sqlite-macos-howto.md
Created September 18, 2020 22:28 — forked from dannguyen/schemacrawler-sqlite-macos-howto.md
How to use schemacrawler to generate schema diagrams for SQLite from the commandline (Mac OS)
@poteznyKrolik
poteznyKrolik / code.py
Last active August 19, 2020 16:54 — forked from FBosler/code.py
functools.py
from functools import lru_cache
from datetime import datetime
@lru_cache(maxsize=None)
def fib_cache(n):
if n < 2:
return n
return fib_cache(n-1) + fib_cache(n-2)
def fib_no_cache(n):
@poteznyKrolik
poteznyKrolik / README.md
Created July 30, 2020 18:52 — forked from magnetikonline/README.md
AWS clone VPC route table and routes.

AWS clone VPC route table and routes

Python script to clone an existing VPC route table. Script output is a series of AWS CLI calls to create the route table and assign routes.

Update AWS_TARGET_REGION and SOURCE_ROUTE_TABLE_ID to suit.

Note: does not currently support NAT Gateways routes due to Boto 2 API limitation.

Example

$ ./clone-route-table.py
# https://hakibenita.com/fast-load-data-python-postgresql
from typing import Iterator, Dict, Any, Optional
from urllib.parse import urlencode
import datetime
#------------------------ Profile
import time
@poteznyKrolik
poteznyKrolik / gist:6ec7c2226dd7194f081d13252bba4de7
Created June 20, 2019 21:07 — forked from chanks/gist:7585810
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@poteznyKrolik
poteznyKrolik / darkify_slack.sh
Created June 20, 2019 20:18 — forked from ryanpcmcquen/darkify_slack.sh
Darkify your Slack.
#!/bin/sh
# Darkify Slack on Mac OS or Linux.
# curl https://gist.githubusercontent.com/ryanpcmcquen/8a7ddc72460eca0dc1f2dc389674dde1/raw/darkify_slack.sh | sh
if [ "`uname -s`" = "Darwin" ]; then
SLACK_INTEROP_JS="/Applications/Slack.app/Contents/Resources/app.asar.unpacked/src/static/ssb-interop.js"
else
SLACK_INTEROP_JS="/usr/lib/slack/resources/app.asar.unpacked/src/static/ssb-interop.js"
fi