Alex Espinoza aespinoza

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

macOS Internals

Understand your Mac and iPhone more deeply by tracing the evolution of Mac OS X from prelease to Swift. John Siracusa delivers the details.

Starting Points

How to use this gist

You've got two main options:

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

	{
	"permissions": {
	"allow": [
	"Bash(git status:*)",
	"Bash(git diff:*)",
	"Bash(git log:*)",
	"Bash(git add:*)",
	"Bash(git grep:*)",
	"Bash(poetry update:*)",
	"Bash(pip show:*)",

	// Claude Code is a Beta product per Anthropic's Commercial Terms of Service.
	// By using Claude Code, you agree that all code acceptance or rejection decisions you make,
	// and the associated conversations in context, constitute Feedback under Anthropic's Commercial Terms,
	// and may be used to improve Anthropic's products, including training models.
	// You are responsible for reviewing any code suggestions before use.

	// (c) Anthropic PBC. All rights reserved. Use is subject to Anthropic's Commercial Terms of Service (https://www.anthropic.com/legal/commercial-terms).

	// Version: 0.2.9

	# Follow Things To Do After Installing Ubuntu 24.04 LTS

	$ apt update -y
	$ apt upgrade -y
	$ apt update -y
	$ apt install build-essential checkinstall
	$ apt install ubuntu-restricted-extras
	$ apt install software-properties-common
	$ apt install apt-show-versions
	$ apt upgrade -o APT::Get::Show-Upgraded=true

	From self[at]sungpae.com Mon Nov 8 16:59:48 2021
	Date: Mon, 8 Nov 2021 16:59:48 -0600
	From: Sung Pae <self[at]sungpae.com>
	To: security@docker.com
	Subject: Permissive forwarding rule leads to unintentional exposure of
	containers to external hosts
	Message-ID: <YYmr4l1isfH9VQCn@SHANGRILA>
	MIME-Version: 1.0
	Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="QR1yLfEBO/zgxYVA"

-# Teams 2.0 (no work account support yet)
-winget uninstall MicrosoftTeams_8wekyb3d8bbwe
-# Your Phone
-winget uninstall Microsoft.YourPhone_8wekyb3d8bbwe
-# Widgets (Web Experience Pack)
-winget uninstall MicrosoftWindows.Client.WebExperience_cw5n1h2txyewy
-# Cortana

	# This is, in my opinion, the preferred way to solve this since you can have several context pointing to different accounts/environments. See: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html and https://github.com/linaro-its/aws2-wrap#use-the-credentials-via-awsconfig
	# Uses a credential_process profile in your .aws/config file. It's this profile you will use when setting up the docker compose ecs context as the credentials_process looks up the needed credentials.

	$ cat ~/.aws/config
	[profile account1]
	sso_start_url = https://d-123456789.awsapps.com/start#/
	sso_region = us-east-1
	sso_account_id = 123
	sso_role_name = AdministratorAccess
	region = eu-north-1