Joor joor0x

Software Engineer Specialized in FinTech

joor0x / semchunk.py

Last active April 23, 2024 11:15

Different approaches to chunk text in Python

	import semchunk
	import tiktoken

	text = """Alan Mathison Turing OBE FRS (/ˈtjʊərɪŋ/; 23 June 1912 – 7 June 1954) was an English mathematician, computer scientist, logician, cryptanalyst, philosopher and theoretical biologist.[5] Turing was highly influential in the development of theoretical computer science, providing a formalisation of the concepts of algorithm and computation with the Turing machine, which can be considered a model of a general-purpose computer.[6][7][8] He is widely considered to be the father of theoretical computer science and artificial intelligence.[9]

	Born in London, Turing was raised in southern England. He graduated in maths from King's College, Cambridge, and in 1938, earned a maths PhD from Princeton University. During the Second World War, Turing worked for the Government Code and Cypher School at Bletchley Park, Britain's codebreaking centre that produced Ultra intelligence. He led Hut 8, the section responsible for German naval cryptanalysis. He devised techniques for speedin

joor0x / download_bars.py

Created October 25, 2023 14:47 — forked from wrighter/download_bars.py

A command line utility to download historical data from Interactive Brokers

	#!/usr/bin/env python

	import os
	import sys
	import argparse
	import logging
	from datetime import datetime, timedelta

	from typing import List, Optional
	from collections import defaultdict