Ian Chute ianchute

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

##VGG16 model for Keras

This is the Keras model of the 16-layer network used by the VGG team in the ILSVRC-2014 competition.

It has been obtained by directly converting the Caffe model provived by the authors.

Details about the network architecture can be found in the following arXiv paper:

Very Deep Convolutional Networks for Large-Scale Image Recognition

K. Simonyan, A. Zisserman

	import numpy as np
	import faiss


	class FaissKNeighbors:
	def __init__(self, k=5):
	self.index = None
	self.y = None
	self.k = k

	""" Implementation of OKapi BM25 with sklearn's TfidfVectorizer
	Distributed as CC-0 (https://creativecommons.org/publicdomain/zero/1.0/)
	"""

	import numpy as np
	from sklearn.feature_extraction.text import TfidfVectorizer
	from scipy import sparse


	class BM25(object):

	"""
	Example of a Streamlit app for an interactive Prodigy dataset viewer that also lets you
	run simple training experiments for NER and text classification.

	Requires the Prodigy annotation tool to be installed: https://prodi.gy
	See here for details on Streamlit: https://streamlit.io.
	"""
	import streamlit as st
	from prodigy.components.db import connect
	from prodigy.models.ner import EntityRecognizer, merge_spans, guess_batch_size

	__author__ = "lewis.r.liu@gmail.com"
	__copyright__ = "Copyright 2020, 2018, https://gist.github.com/ruxi/ff0e9255d74a3c187667627214e1f5fa"
	__license__ = "MIT"
	__version__ = "0.0.2"

	# update: June 13, 2020
	# created: Feb 19, 2018
	# desc: seaborn jointplot with 'hue'
	# prepared for issue: https://github.com/mwaskom/seaborn/issues/365
	# resolved (22 Aug 2020): https://github.com/mwaskom/seaborn/pull/2210

	# Taken from https://forums.aws.amazon.com/thread.jspa?messageID=332091

	sudo su -

	cd /usr/local/bin
	mkdir ffmpeg

	cd ffmpeg
	wget http://ffmpeg.gusari.org/static/64bit/ffmpeg.static.64bit.latest.tar.gz
	tar -xzf ffmpeg.static.64bit.latest.tar.gz

	"""
	helloevolve.py implements a genetic algorithm that starts with a base
	population of randomly generated strings, iterates over a certain number of
	generations while implementing 'natural selection', and prints out the most fit
	string.

	The parameters of the simulation can be changed by modifying one of the many
	global variables. To change the "most fit" string, modify OPTIMAL. POP_SIZE
	controls the size of each generation, and GENERATIONS is the amount of
	generations that the simulation will loop through before returning the fittest

	import pandas as pd

	def _map_to_pandas(rdds):
	""" Needs to be here due to pickling issues """
	return [pd.DataFrame(list(rdds))]

	def toPandas(df, n_partitions=None):
	"""
	Returns the contents of `df` as a local `pandas.DataFrame` in a speedy fashion. The DataFrame is
	repartitioned if `n_partitions` is passed.

	# -- coding: utf-8 --

	u"""
	Beta regression for modeling rates and proportions.

	References
	----------
	Grün, Bettina, Ioannis Kosmidis, and Achim Zeileis. Extended beta regression
	in R: Shaken, stirred, mixed, and partitioned. No. 2011-22. Working Papers in
	Economics and Statistics, 2011.