ErwinXu

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

The ffast and the Furious

This is a small and admittedly contrived demo showing how some weird but safe code could become vulnerable if run in an environment where some shared library has changed the FPU's FTZ/DAZ bits to force denormals to zero.

To run it:

# Create an empty file
$ touch gofast.c

Structured concurrency and Lua (part 1)

John Belmonte, 2022-Sep

I've started writing a toy structured concurrency implementation for the Lua programming language. Some motivations:

use it as a simple introduction to structured concurrency from the perspective of Lua (this article)
learn the fundamental properties of structured concurrency and how to implement them
share code that could become the starting point for a real Lua library and framework

So what is structured concurrency? For now, I'll just say that it's a programming paradigm that makes managing concurrency (arguably the hardest problem of computer science) an order of magnitude easier in many contexts. It achieves this in ways that seem subtle to us—clearly so, since its utility didn't reach critical mass until around 2018[^sc_birth] (just as control structures like functions, if, and while weren't introduced to languages until long after the first compu

You are doing JSON APIs wrong.

When you use JSON to call an API - not a REST API, but something like JSON-RPC - you will usually want to encode one of several possible messages.

Your request body looks like this:

{
 "type": "MessageWithA",

September 2022:

This has spread to a far wider audience than I had anticipated - probably my fault for using a title that is in hindsight catnip for link aggregators. I wrote this back in 2021 just as a bunch of personal thoughts of my experiences using Rust over the years (not always well thought through), and don't intend on trying to push them further, outside of personal experiments and projects.

Managing a living language is challenging and difficult work, and I am grateful for all the hard work that the Rust community and contributors put in given the difficult constraints they work within. Many of the things I listed below are not new, and there's been plenty of difficult discussions about many of them over the years, and some are being worked on or postponed, or rejected for various good reasons. For more thoughts, please see my comment below.

My personal list of Rust gr

Commit jupyter notebooks code to git and keep output locally

Add a filter to git config by running the following command in bash inside the repo:

git config filter.strip-notebook-output.clean 'jupyter nbconvert --ClearOutputPreprocessor.enabled=True --to=notebook --stdin --stdout --log-level=ERROR'

Create a .gitattributes file inside the directory with the notebooks
Add the following to that file:

Genomics - A programmer's guide.

Andy Thomason is a Senior Programmer at Genomics PLC. He has been witing graphics systems, games and compilers since the '70s and specialises in code performance.

https://www.genomicsplc.com

58 bytes of CSS to look great nearly everywhere

When making this website, i wanted a simple, reasonable way to make it look good on most displays. Not counting any minimization techniques, the following 58 bytes worked well for me:

main {
  max-width: 38rem;
  padding: 2rem;
  margin: auto;
}

Learning Rust

The following is a list of resources for learning Rust as well as tips and tricks for learning the language faster.

Warning

Rust is not C or C++ so the way your accustomed to do things in those languages might not work in Rust. The best way to learn Rust is to embrace its best practices and see where that takes you.

The generally recommended path is to start by reading the books, and doing small coding exercises until the rules around borrow checking become intuitive. Once this happens, then you can expand to more real world projects. If you find yourself struggling hard with the borrow checker, seek help. It very well could be that you're trying to solve your problem in a way that goes against how Rust wants you to work.

Moved to Shopify/graphql-design-tutorial