Jan Roudaut janroudaut

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

Random numbers in bash et al

There is a variable $RANDOM than you can read in bash or zsh to get a random number beteen 0 and 32767. Here in zsh on my (old) mac:

% echo $RANDOM
13757
% echo $RANDOM
16896

Genomics - A programmer's guide.

Andy Thomason is a Senior Programmer at Genomics PLC. He has been witing graphics systems, games and compilers since the '70s and specialises in code performance.

https://www.genomicsplc.com

	# This script will transcribe an audio file (mp3, wav, etc.) to text and then clean the text using a local LLM model via Ollama. Technically, this script will work with any LLM that supports the standard OpenAI bindings with minor adjustments.

	# GETTING STARTED:
	# 1. Install required python packages (pip install openai python-dotenv)
	# 2. Git clone a copy of ggerganov/whisper (https://github.com/ggerganov/whisper.cpp)
	# 3. Build the whisper binary (see the whisper.cpp README for instructions)
	# 4. Download one of the whisper models (largev2 is the most accurate for all languages, though the base model works reasonably well for English).
	# 5. Install ffmpeg (brew install ffmpeg on macOS, apt-get install ffmpeg)
	# 6. Install ollama (https://ollama.com/download)
	# 7. Download an LLM model (https://ollama.com/library)

	# ruby's git repo investigation zsh one-liner-ish thingy
	# (a starting point for investigating anomalous contributions in git repositories)

	echo "$(find . -type f ! -size 0 ! -path './.git*' -exec grep -IL . "{}" \;)" \| \
	sed -e "s/^\.\///g" \| \
	while read line; \
	do \
	echo ">>>>>>>>$line"; \
	echo "$(git log --follow --find-renames=40% --pretty=format:"%ad%x0A%h%x0A%an%x20<%ae>%x0A%s" -- "$line" \| head -n 4)"; \
	commitdates="$(git log --follow --find-renames=40% --pretty=format:"%ae" -- "$line" \| head -n 1 \| xargs -I {} git log --author={} --pretty=format:"%ad")"; \

	#!/usr/bin/env python

	import multiprocessing
	import random
	import time


	class Logger:
	def __init__(self, num_lines, last_output_per_process, terminal_lock):
	self.num_lines = num_lines

	MIT License

	Copyright (c) 2021 Daniel Ethridge

	Permission is hereby granted, free of charge, to any person obtaining a copy
	of this software and associated documentation files (the "Software"), to deal
	in the Software without restriction, including without limitation the rights
	to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
	copies of the Software, and to permit persons to whom the Software is
	furnished to do so, subject to the following conditions:

	#!/bin/bash
	###
	### my-script — does one thing well
	###
	### Usage:
	### my-script <input> <output>
	###
	### Options:
	### <input> Input file to read.
	### <output> Output file to write. Use '-' for stdout.

	#include <time.h> // Robert Nystrom
	#include <stdio.h> // @munificentbob
	#include <stdlib.h> // for Ginny
	#define r return // 2008-2019
	#define l(a, b, c, d) for (i y=a;y\
	<b; y++) for (int x = c; x < d; x++)
	typedef int i;const i H=40;const i W
	=80;i m[40][80];i g(i x){r rand()%x;
	}void cave(i s){i w=g(10)+5;i h=g(6)
	+3;i t=g(W-w-2)+1;i u=g(H-h-2)+1;l(u