Last active
March 30, 2026 14:55
-
-
Save creationix/a2dc88dd963b2742592959440f23ba33 to your computer and use it in GitHub Desktop.
Revisions
-
creationix revised this gist
Mar 30, 2026 . 1 changed file with 99 additions and 58 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,6 +2,39 @@ Rex has no user-space type annotations. Types are inferred from domain interface files (`.rexd`), literals, operators, and type predicates. The type system exists purely for tooling — the compiler and interpreter are untyped. ## Type Summary ### Scalar types | Type | Description | |------|-------------| | `integer` | Integer value (`42`, `-1`, `0`) | | `number` | Any numeric value — integer or decimal (`3.14`, `1e10`) | | `boolean` | `true` or `false` | | `string` | String value (`"hello"`) | | `null` | The null value | | `none` | Absence of value — missing keys, failed lookups, deletion tombstones | | `"GET"` | String literal type — only this exact value | ### Container types | Type | Description | |------|-------------| | `[T]` | Array of `T` | | `{key: T, ...}` | Object with known fields | | `{*: T}` | Map — any string key, lookup returns `T \| none` | | `{key: T, *: U}` | Object with known fields and wildcard fallback | ### Meta types | Type | Description | |------|-------------| | `some` | A value exists but its type is opaque — must narrow before use | | `T \| U` | Union — value can be `T` or `U` | | `unknown` | Alias for `some \| none` | | `never` | Function does not return (throws or diverges) | | `Name` | Reference to a type alias defined in `.rexd` | --- ## Domain Interface Files (`.rexd`) @@ -17,7 +50,7 @@ A `.rexd` file declares the types of globals, functions, and type aliases availa Uppercase names define reusable types: ```rex Headers = {*: string | [string]} HttpMethod = "GET" | "POST" | "PUT" | "DELETE" | "PATCH" ``` @@ -31,8 +64,8 @@ req = { method: HttpMethod path: string | [string] headers: Headers query: {*: string | [string]} cookies: {*: string} ip: string body: string } @@ -46,7 +79,7 @@ mut res = { // Simple typed globals config: unknown secrets: {*: string} ``` #### Functions @@ -80,34 +113,42 @@ ip: string | `integer` | Integer only | | `boolean` | `true` or `false` | | `null` | The null value | | `some` | A value exists but type is opaque — must narrow before use. Does NOT include `none`. | | `none` | No value / tombstone. Navigation on `none` produces `none`. | | `unknown` | Alias for `some \| none` — a value might or might not exist. | | `never` | Function does not return (throws or diverges) | | `"GET"` | String literal type — only this exact string | | `[T]` | Array of `T` | | `{key: T, ...}` | Object with known fields | | `{*: T}` | Map — any string key accepted, lookup returns `T \| none` (key may not exist) | | `{key: T, *: U}` | Object with known fields (exact type) and a wildcard fallback (`U \| none`) for other keys | | `T \| U` | Union — value can be `T` or `U` | | `Name` | Reference to a type alias | ### `some`, `none`, and `unknown` Three primitive concepts for presence and absence: - **`some`** — a value exists, but we don't know its type. Must narrow with type predicates before using in operations. Navigation on `some` produces `some | none` (the key might not exist). - **`none`** — no value. The result of missing properties, failed comparisons, deletion tombstones. Navigation on `none` produces `none`. - **`unknown`** — alias for `some | none`. Might be a value, might be absent. Common for map lookups and opaque domain fields. ```rex config: some // definitely a value, type opaque config.timeout // some | none (key may not exist) cookies.session // string | none (map lookup) when cookies.session do // cookies.session: string (none removed — value exists) end data: unknown // same as some | none when data do // data: some (none removed — value exists) when number(data) do // data: number (narrowed further) data + 1 // valid end end ``` @@ -141,7 +182,7 @@ The type checker walks the program top-to-bottom, inferring a type for every exp | `"hello"` | `string` | | `true`, `false` | `boolean` | | `null` | `null` | | `none` | `none` | | `[1, 2, 3]` | `[integer]` | | `{a: 1, b: "x"}` | `{a: integer, b: string}` | @@ -175,17 +216,17 @@ x += 1 // still integer (integer + integer = integer) ### Comparison operators Comparisons return the left-hand value on success, `none` on failure: ```rex x > 10 // type: typeof(x) | none a == b // type: typeof(a) | none ``` This means: ```rex score = x > 10 // score: number | none (if x: number) ``` ### Logical operators @@ -195,38 +236,38 @@ Rex uses existence-based logic, not boolean logic: | Expression | Type | Semantics | |------------|--------------------------|------------------------------------| | `a or b` | `typeof(a) \| typeof(b)` | First defined value | | `a and b` | `typeof(b) \| none` | Second value if first is defined | | `a nor b` | `typeof(b) \| none` | Second value if first is none | The `or` pattern is commonly used for defaults: ```rex max = max or 100 // max: number (if max: number | none from domain) ``` ### Control flow **When / Unless:** ```rex when cond do body end // type: typeof(body) | none when cond do a else b end // type: typeof(a) | typeof(b) unless cond do body end // type: typeof(body) | none ``` **Loops:** ```rex for x in items do body end // type: typeof(body) (last iteration) while cond do body end // type: typeof(body) | none ``` **Comprehensions produce arrays:** ```rex [x * 2 for x in items] // type: [number] (if items: [number]) [self in 1..10] // type: [integer] {k: v for k, v in obj} // type: {*: typeof(v)} ``` ### Property access @@ -235,27 +276,27 @@ Accessing a property on a typed value narrows by lookup: ```rex req.method // HttpMethod → "GET" | "POST" | ... req.headers // Headers → {*: string | [string]} req.headers.host // string | [string] (from map value type) ``` For objects with known fields, accessing a known field returns its type. Accessing an unknown field is an **error** (but still types as `none`): ```rex point = {x: 1, y: 2} point.x // integer point.z // error: unknown property 'z' on {x: integer, y: integer} // type: none ``` For maps (`{*: V}`), any string key is accepted but the key might not exist, so the result is `V | none`: ```rex cookies = req.cookies // {*: string} cookies.session // string | none (key may not exist) when cookies.session do // cookies.session: string (narrowed — none removed) end ``` @@ -264,49 +305,49 @@ end When a type is a union, property access is resolved on each branch independently and the results are unioned: ```rex // Given: value: {a: number, b: string} | {*: boolean} value.a // Left branch: {a: number, b: string}.a → number // Right branch: {*: boolean}.a → boolean | none (map allows any key) // Combined: number | boolean | none ``` This means a union of a known-field object and a map is useful: known fields are accessed precisely, but the map branch provides a fallback for arbitrary keys. ```rex // A config that has known fields but also allows extensions Config = {timeout: number, retries: integer} | {*: unknown} config.timeout // number | unknown | none config.custom // error on left branch (unknown field), unknown | none on right // combined: unknown | none ``` An object with known fields unioned with a map does NOT suppress the error on unknown fields from the object branch — the error is still reported as a diagnostic, but the type includes the map branch's result so the program is valid. ### Navigation on none Accessing a property on `none` is not an error — it simply produces `none`. This means all property chains are implicitly optional, similar to `?.` in JavaScript: ```rex config.routing.timeout // if config: unknown, this is unknown | none // if config: {routing: {timeout: number}}, this is number // if config is none at runtime, this is none // no runtime error in any case ``` If a type includes `none` in a union, navigation propagates `none`: ```rex user = users.0 // user: {name: string} | none (array index) user.name // string | none (none propagates) when user do user.name // string (narrowed — user is defined) end ``` This means the type checker never reports a navigation chain as an error. Unknown properties on concrete types produce a warning and type `none`, but the program is still valid. ### Type narrowing @@ -328,13 +369,13 @@ end #### Via existence The `when` construct narrows `none` out of the type: ```rex name = req.query.name // string | [string] | none (map lookup) when name do // name: string | [string] (none removed) end ``` @@ -379,10 +420,10 @@ The type checker produces warnings and errors based on inferred types. The type system intentionally does not check: - Arithmetic overflow (numbers are arbitrary precision in Rex) - Array bounds (out of bounds reads are none) - Exhaustiveness of literal unions (the `else` branch handles unknown values) There is no `any` escape hatch. `some` requires narrowing before use — the type checker ensures all operations have compatible types. --- -
creationix revised this gist
Mar 29, 2026 . No changes.There are no files selected for viewing
-
creationix revised this gist
Mar 29, 2026 . 1 changed file with 792 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,792 @@ # Rex Edge Server (`rex-serve`) ## Context Rex has a working Rust compiler and interpreter (`rex-core`) with a `HostObject` trait, refs system, opcodes, and gas-bounded execution -- all the primitives needed to run untrusted Rex programs as HTTP handlers. The existing `samples/http-domain/` already demonstrates HTTP routing patterns in Rex. This plan designs the server that makes those patterns real: a filesystem-routed HTTP server where `.rex` files are edge functions. ## Architecture ### New crate: `crates/rex-serve/` Built on **axum** (hyper/tokio). On startup, scans a `routes/` directory, compiles `.rex` files to REXC bytecode, builds a route table, and dispatches incoming requests through middleware chains to handlers. ``` crates/rex-serve/ Cargo.toml src/ main.rs CLI entry, arg parsing, server startup server.rs axum app, graceful shutdown router.rs filesystem scan, route table, pattern matching handler.rs request → Context → run() → response middleware.rs _middleware.rex chain loading and execution refs.rs HostObject impls for req/res opcodes.rs built-in opcode registrations (db, json, log, crypto, time, http, fs, markdown, template) static_files.rs static file serving with content-type detection reload.rs filesystem watcher (notify crate), atomic table swap config.rs rex-serve.toml parsing ``` ### Dependencies - `rex-core` (workspace), `axum` 0.7+, `tokio`, `clap`, `notify` 7+, `rusqlite` + `r2d2`, `serde` + `serde_json`, `toml`, `pulldown-cmark` (markdown), `mime_guess` (content-type detection) --- ## Filesystem Routing ``` routes/ _middleware.rex → middleware for ALL routes index.rex → / health.rex → /health api/ _middleware.rex → middleware for /api/** keys.rex → /api/keys keys/ [id].rex → /api/keys/:id webhooks/ _middleware.rex → middleware for /api/webhooks/** [provider].rex → /api/webhooks/:provider [provider]/ events.rex → /api/webhooks/:provider/events events/ [event-id].rex → /api/webhooks/:provider/events/:event-id ``` ### Conventions - **One file per route, all methods in one file.** Branch on `req.method` inside the handler (matches existing Rex HTTP samples). - `index.rex` → maps to the directory path itself - `[param].rex` → dynamic segment, value available as `params.param` - `[...rest].rex` → catch-all, value available as `params.rest` (array) - `_middleware.rex` → middleware for directory + all descendants (never a route handler) - `_` prefixed dirs/files (other than `_middleware.rex`) are **private** -- never served or routed, but readable by handlers via `fs.read()`. Used for layouts, content, shared data: - `_layouts/` — HTML template files - `_content/` — markdown or data files - `_data/` — JSON seed data, config - **All other non-`.rex` files are static assets**, served directly with auto-detected content-type ### Static + Dynamic Mixing Any file in the route tree that isn't `.rex` and doesn't start with `_` is a static asset: ``` routes/ style.css → GET /style.css (static) favicon.ico → GET /favicon.ico (static) images/ logo.svg → GET /images/logo.svg (static) index.rex → GET / (dynamic) about.html → GET /about.html (static) api/ keys.rex → * /api/keys (dynamic) ``` **Resolution priority** for a given URL path: 1. `.rex` handler (dynamic) — highest priority 2. Static file — exact path match 3. `index.rex` in directory — if path is a directory 4. `index.html` in directory — static directory index 5. 404 **Middleware runs for static files too.** The full middleware chain executes before serving a static file. This lets middleware add security headers, check auth, do CORS, etc. for all assets. If middleware short-circuits (status >= 400), the static file is not served. ### Private `_` directories Directories and files starting with `_` are never routable or servable, but handlers can read them with `fs.read()`: ``` routes/ _layouts/ page.html ← not served, but handlers can fs.read("_layouts/page.html") admin.html _content/ blog/ hello-world.md ← not served, but handlers can read + render it getting-started.md _data/ nav.json ← shared navigation data blog/ [slug].rex → reads from _content/blog/, renders with _layouts/page.html ``` ### Matching Priority 1. Static segments beat dynamic (`/api/keys` > `/api/[id]`) 2. Named params beat catch-all (`/api/[id]` > `/api/[...rest]`) 3. Longer paths beat shorter paths --- ## HTTP Interface via Refs Uses the existing `.config.rex` short codes from `samples/http-domain/`: | Ref | Aliases | Type | Description | |-----|---------|------|-------------| | `H` | `req.headers`, `headers` | case-insensitive map (HostObject) | Request headers | | `M` | `req.method`, `method` | string | HTTP method | | `P` | `req.path`, `path` | string | URL path | | `Q` | `req.query`, `query` | string \| object | Query string | | `C` | `req.cookies`, `cookies` | object | Cookie map | | `I` | `req.ip`, `ip` | string | Client IP | | `B` | `req.body`, `body` | string | Request body | | `D` | `req.domain`, `domain` | string | Host header | | `S` | `res.status`, `status` | integer | Response status (default 200) | | `RH` | `res.headers` | object (HostObject) | Response headers | | `EC` | `config` | object (read-only HostObject) | Edge config | | `SC` | `secrets` | object (read-only HostObject) | Secrets | **New ref for the server:** | Ref | Aliases | Type | Description | |-----|---------|------|-------------| | `PA` | `req.params`, `params` | object | Route params from `[param]` segments | ### Response Convention - `res.status` and `res.headers` are set via ref mutation (HostObject `set()`) - The **return value** of the program is the response body - Object/Array return → JSON-serialized, `content-type: application/json` auto-set - String return → sent as-is with whatever content-type was set - Undefined return → empty body ### Middleware Data Passing Middleware sets **vars** (e.g., `principal = key-data`). These persist into the handler's context because the server chains `RunResult.vars` into the next program's `Context.vars`. ### Short-Circuit After each middleware runs, if `res.status >= 400`, the chain stops and the middleware's return value becomes the response body. --- ## Opcodes Registered on every `Context` before execution: ``` // Already established in .config.rex json.parse(text) → object json.stringify(value) → string log.info(msg) → undefined log.warning(msg) → undefined log.error(msg) → undefined res.redirect(url, status) → (sets status + Location header, halts) res.rewrite(url) → (internal rewrite, re-runs routing) res.proxy(url) → (proxies request) // New for the server — storage db.get(key) → string | undefined db.set(key, value) → true db.delete(key) → true db.list(prefix) → [{key, value}, ...] // Filesystem (sandboxed to project root, read-only) fs.read(path) → string | undefined (reads file relative to routes/ root) fs.glob(pattern) → [path, ...] (list matching files) fs.meta(path) → {size, modified} | undefined // Content transformation markdown.render(text) → string (HTML) template.render(tmpl, data) → string (mustache-style {{key}} substitution) // Utilities time.now() → unix timestamp ms (integer) time.uuid() → UUIDv7 string crypto.hash(algo, data) → hex string crypto.hmac(algo, key, data) → hex string crypto.random(bytes) → hex string http.fetch(url, options) → {status, headers, body} ``` ### Filesystem Access: `fs.read()` / `fs.glob()` Handlers can read any file in the project tree (including `_` prefixed private files). Paths are relative to the project root (where `rex-serve.toml` lives). The sandbox prevents path traversal -- `fs.read("../../etc/passwd")` resolves to the project root and fails. This is how dynamic handlers reference static content: ```rex /* routes/blog/[slug].rex — render a blog post */ content = fs.read("_content/blog/" + params.slug + ".md") unless content do res.status = 404 res.headers.content-type = "text/html" "<h1>Not Found</h1>" end when content do html-body = markdown.render(content) layout = fs.read("_layouts/page.html") res.headers.content-type = "text/html" template.render(layout, {title: params.slug, body: html-body}) end ``` ### Transformation Pipeline: `template.render()` Mustache-style substitution. `{{key}}` is replaced with the HTML-escaped value from the data object. `{{{key}}}` (triple braces) is unescaped (for injecting pre-rendered HTML like markdown output). ```html <!-- _layouts/page.html --> <!DOCTYPE html> <html> <head> <title>{{title}} — My Site</title> <link rel="stylesheet" href="/style.css"> </head> <body> <nav>{{nav}}</nav> <main>{{{body}}}</main> <footer>Built with Rex</footer> </body> </html> ``` ```rex /* Handler fills in the template */ template.render(layout, { title: "Hello World" nav: nav-html body: markdown.render(content) /* unescaped via {{{ }}} in template */ }) ``` For async opcodes (db, http.fetch): the Rex interpreter is synchronous. These block within `spawn_blocking`. Safe because Rex programs are gas-bounded and short-lived. --- ## Request Processing Pipeline ``` Request arrives for GET /articles/getting-started 1. Look up route: a. Check .rex handlers → routes/articles/[slug].rex matches, params={slug: "getting-started"} b. (If no .rex match, check static files → serve if found, skip to step 11) 2. Collect middleware: [routes/_middleware.rex] (articles/ has no _middleware.rex) 3. Build initial refs from HTTP request (H, M, P, Q, B, C, I, D, PA) 4. Create ResponseHostObject (status=200, headers={}) For each middleware: 5. Build Context { refs, vars (accumulated), host_objects, opcodes, gas_limit } 6. rex_core::interpret::run(middleware_bytecode, ctx) 7. Accumulate vars from RunResult 8. If ResponseHostObject.status >= 400 → return early with middleware return value as body 9. Build Context for handler (with accumulated vars from middleware) 10. rex_core::interpret::run(handler_bytecode, ctx) - Handler calls fs.read("_content/getting-started.md") → reads private file - Handler calls markdown.render(content) → HTML string - Handler calls fs.read("_layouts/page.html") → reads private layout - Handler calls template.render(layout, {title, body}) → final HTML - Handler returns the HTML string 11. Build HTTP response: - Status from ResponseHostObject - Headers from ResponseHostObject (including content-type: text/html set by handler) - Body from RunResult.value (string → sent as-is) Static file example: GET /style.css 1. No .rex match 2. Static file found: routes/style.css 3. Run middleware chain (same steps 2-8 above — security headers get added) 4. Serve file bytes with content-type: text/css (from mime_guess) ``` --- ## Concurrency Model - Each request → tokio task → `spawn_blocking` for Rex execution - Route table shared via `Arc<RwLock<RouteTable>>` (read lock for dispatch, write lock for hot reload) - DB connection pool via `r2d2` (SQLite) or `deadpool` (Redis) - Hot reload: `notify` watches `routes/`, recompiles changed files, atomic table swap --- ## Configuration: `rex-serve.toml` ```toml [server] host = "0.0.0.0" port = 3000 max_body_bytes = 1_048_576 gas_limit = 1_000_000 [routes] dir = "routes" [db] backend = "sqlite" # or "redis", "memory" path = "data.db" [secrets] env_prefix = "REX_SECRET_" ``` --- ## Domain Type File: `rex-serve.rexd` Every rex-serve project includes a `.rexd` file so the LSP provides completions, hover docs, and type checking for all server globals, functions, and types. This file ships with `rex-serve` and is copied into new projects (or referenced via config). ```rex // ── Types ────────────────────────────────────────────────────────────── Headers = {string | [string]} HttpMethod = "GET" | "POST" | "PUT" | "DELETE" | "PATCH" | "OPTIONS" | "HEAD" FileMeta = {size: integer, modified: integer} FetchOptions = {method: string, headers: {string}, body: string} FetchResult = {status: integer, headers: Headers, body: string} DbEntry = {key: string, value: string} // ── Request (read-only) ──────────────────────────────────────────────── // HTTP request data. Read-only — populated by the server before the handler runs. req = { // HTTP method (GET, POST, etc.) method: HttpMethod // URL path (e.g. "/api/users/123") path: string | [string] // Request headers as a case-insensitive map headers: Headers // Parsed query string parameters query: {string | [string]} // Parsed cookie values cookies: {string} // Client IP address ip: string // Request body as a string body: string // Host/domain from the Host header domain: string // Route parameters from [param] segments (e.g. {slug: "hello-world"}) params: {string | [string]} } // Short aliases for common request fields // HTTP method method: HttpMethod // URL path path: string | [string] // Request headers headers: Headers // Parsed query parameters query: {string | [string]} // Parsed cookies cookies: {string} // Request body body: string // Route parameters params: {string | [string]} // ── Response (mutable) ───────────────────────────────────────────────── // HTTP response. Mutable — set status, headers, and body to control the response. mut res = { // Response status code (default: 200) status: integer // Response headers headers: Headers } // Response status code (shorthand for res.status) mut status: integer // ── Edge Config (read-only) ──────────────────────────────────────────── // Host-managed project configuration. Read-only, set outside of Rex. config: unknown // Host-managed secrets store. Read-only. secrets: {string} // ── Logging ──────────────────────────────────────────────────────────── // Log an informational message log.info(message: unknown) // Log a warning log.warning(message: unknown) // Log an error log.error(message: unknown) // ── Response Control ─────────────────────────────────────────────────── // Internal rewrite to another path. Restarts routing. res.rewrite(url: string): never // HTTP redirect with status code (default 302) res.redirect(url: string, status: integer): never // Proxy the request to another URL res.proxy(url: string): unknown // ── JSON ─────────────────────────────────────────────────────────────── // Parse a JSON string into a value json.parse(text: string): unknown // Serialize a value to a JSON string json.stringify(value: unknown): string // ── Database (KV) ────────────────────────────────────────────────────── // Get a value by key. Returns undefined if not found. db.get(key: string): string // Set a key-value pair. Returns true. db.set(key: string, value: string): boolean // Delete a key. Returns true. db.delete(key: string): boolean // List entries matching a key prefix. db.list(prefix: string): [DbEntry] // ── Filesystem (read-only, sandboxed to project root) ────────────────── // Read a file's contents as a string. Path is relative to project root. fs.read(path: string): string // List files matching a glob pattern relative to project root. fs.glob(pattern: string): [string] // Get file metadata (size in bytes, modified as unix timestamp ms). fs.meta(path: string): FileMeta // ── Content Transformation ───────────────────────────────────────────── // Render markdown text to HTML markdown.render(text: string): string // Render a mustache-style template with data. {{key}} escapes HTML, {{{key}}} is raw. template.render(template: string, data: unknown): string // ── Utilities ────────────────────────────────────────────────────────── // Current time as unix timestamp in milliseconds time.now(): integer // Generate a UUIDv7 string time.uuid(): string // Hash data with the given algorithm (e.g. "sha256"). Returns hex string. crypto.hash(algorithm: string, data: string): string // HMAC with the given algorithm, key, and data. Returns hex string. crypto.hmac(algorithm: string, key: string, data: string): string // Generate random bytes. Returns hex string. crypto.random(bytes: integer): string // Make an HTTP request. Returns response with status, headers, and body. http.fetch(url: string, options: FetchOptions): FetchResult ``` This file lives at `knowledge-base/rex-serve.rexd` (or whichever project root). The LSP picks it up automatically and provides full IDE support for all server APIs. --- ## Example App: Knowledge Base A content site with an admin API. Exercises CRUD, auth middleware, dynamic routing, body parsing, JSON responses, static assets, markdown rendering, HTML templates, and multiple middleware layers. ### File Tree ``` knowledge-base/ rex-serve.toml routes/ _middleware.rex # Global: request-id, security headers _layouts/ page.html # HTML shell for content pages admin.html # HTML shell for admin pages _content/ getting-started.md # Seed article api-reference.md # Seed article style.css # Static: GET /style.css favicon.ico # Static: GET /favicon.ico index.rex # Homepage: list all articles articles/ [slug].rex # Render single article from _content/ or db api/ _middleware.rex # Auth: API key validation articles.rex # GET list / POST create articles/ [slug].rex # GET / PUT / DELETE single article health.rex # GET /health ``` ### `routes/_middleware.rex` — Global ```rex /* Security headers + request ID for all requests (including static files) */ request-id = headers.x-request-id or time.uuid() res.headers.x-request-id = request-id res.headers.x-content-type-options = "nosniff" res.headers.x-frame-options = "DENY" ``` ### `routes/_layouts/page.html` — Content template ```html <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>{{title}} — Knowledge Base</title> <link rel="stylesheet" href="/style.css"> </head> <body> <header><a href="/">Knowledge Base</a></header> <main>{{{body}}}</main> <footer>Powered by Rex</footer> </body> </html> ``` ### `routes/index.rex` — Homepage (dynamic + static content mix) ```rex /* List all articles: merge _content/ files with db-stored articles */ res.headers.content-type = "text/html" layout = fs.read("_layouts/page.html") // Articles from the filesystem file-slugs = fs.glob("_content/*.md") // Articles from the database db-articles = db.list("article:") db-items = [json.parse(a.value) for a in db-articles] // Build article list HTML list-html = "<ul>" for slug in file-slugs do name = slug.replace("_content/", "").replace(".md", "") list-html += "<li><a href=\"/articles/" + name + "\">" + name + "</a></li>" end for item in db-items do list-html += "<li><a href=\"/articles/" + item.slug + "\">" + item.title + "</a></li>" end list-html += "</ul>" template.render(layout, { title: "Home" body: "<h1>Articles</h1>" + list-html }) ``` ### `routes/articles/[slug].rex` — Render article (filesystem + db fallback) ```rex /* Render a single article from _content/ or from the database */ res.headers.content-type = "text/html" layout = fs.read("_layouts/page.html") slug = params.slug // Try filesystem first file-content = fs.read("_content/" + slug + ".md") // Fall back to database db-record = unless file-content do db.get("article:" + slug) end db-article = when db-record do json.parse(db-record) end content = file-content or (db-article and db-article.body) title = (db-article and db-article.title) or slug unless content do res.status = 404 template.render(layout, {title: "Not Found", body: "<h1>Article not found</h1>"}) end when content do html-body = markdown.render(content) template.render(layout, {title: title, body: html-body}) end ``` ### `routes/api/_middleware.rex` — API auth ```rex /* API key auth for all /api routes */ api-key = headers.authorization unless api-key do res.status = 401 {ok: false, error: "missing_api_key"} end when api-key do key-valid = db.get("keys:" + api-key) unless key-valid do res.status = 401 {ok: false, error: "invalid_api_key"} end principal = api-key end ``` ### `routes/api/articles.rex` — CRUD list + create ```rex /* Article management API */ when method == "GET" do articles = db.list("article:") items = [json.parse(a.value) for a in articles] {ok: true, articles: [{slug: a.slug, title: a.title, updated: a.updated} for a in items]} end when method == "POST" do input = json.parse(body) unless input and input.slug and input.title and input.body do res.status = 422 {ok: false, error: "slug_title_body_required"} end when input and input.slug and input.title and input.body do record = { slug: input.slug title: input.title body: input.body created: time.now() updated: time.now() } db.set("article:" + input.slug, json.stringify(record)) res.status = 201 {ok: true, slug: input.slug} end end unless method == "GET" or method == "POST" do res.status = 405 {ok: false, error: "method_not_allowed"} end ``` ### `routes/api/articles/[slug].rex` — CRUD single article ```rex /* Single article: GET, PUT, DELETE */ slug = params.slug when method == "GET" do record = db.get("article:" + slug) unless record do res.status = 404 {ok: false, error: "not_found"} end when record do {ok: true, article: json.parse(record)} end end when method == "PUT" do input = json.parse(body) existing = db.get("article:" + slug) unless existing do res.status = 404 {ok: false, error: "not_found"} end when existing do old = json.parse(existing) updated = { slug: slug title: input.title or old.title body: input.body or old.body created: old.created updated: time.now() } db.set("article:" + slug, json.stringify(updated)) {ok: true, slug: slug} end end when method == "DELETE" do db.delete("article:" + slug) {ok: true, deleted: slug} end unless method == "GET" or method == "PUT" or method == "DELETE" do res.status = 405 {ok: false, error: "method_not_allowed"} end ``` ### What This Exercises | Feature | Where | |---|---| | Static file serving | `style.css`, `favicon.ico` served directly | | Dynamic routing | `[slug].rex` in both `articles/` and `api/articles/` | | Private dirs (`_`) | `_layouts/`, `_content/` — not routable, read by handlers | | `fs.read()` + `fs.glob()` | Homepage lists `_content/*.md`, article page reads markdown | | `markdown.render()` | Article pages render `.md` → HTML | | `template.render()` | All HTML pages use `_layouts/page.html` | | Filesystem + DB fallback | Article page checks `_content/` first, then database | | CRUD API | `/api/articles` + `/api/articles/[slug]` | | Auth middleware | `/api/_middleware.rex` protects all API routes | | Middleware on static | Global middleware adds security headers to CSS/favicon too | | JSON responses | API routes return objects (auto-serialized) | | HTML responses | Content routes set `content-type: text/html`, return strings | --- ## Implementation Phases ### Phase 1: Static routing + basic handler dispatch - Create `crates/rex-serve/` with Cargo.toml - `router.rs`: scan directory, compile `.rex` → bytecode, build route table - `refs.rs`: `RequestHostObject`, `ResponseHostObject` implementing `HostObject` - `handler.rs`: match route → build Context → `rex_core::interpret::run()` → extract response - `server.rs`: axum fallback handler - Test with `health.rex` returning `{ok: true}` ### Phase 2: Static file serving - `static_files.rs`: serve non-`.rex` files with `mime_guess` content-type detection - Integrate into handler: `.rex` match → dynamic, else → static file, else → 404 - Skip `_` prefixed files/dirs in static serving - Test with `style.css` served at `/style.css` ### Phase 3: Middleware chain - `middleware.rs`: scan for `_middleware.rex`, sort root→leaf - Chain execution in handler: run each, check short-circuit, propagate vars - Middleware runs for both dynamic AND static requests - Test with global middleware adding headers to static files ### Phase 4: Dynamic routing - Parse `[param]` and `[...rest]` from filenames - Inject `params` ref (`PA`) - Route specificity sorting ### Phase 5: Core opcodes - `opcodes.rs`: register db.*, json.*, log.*, time.*, crypto.* - SQLite backend for db.* via rusqlite + r2d2 ### Phase 6: Content opcodes - `fs.read()`, `fs.glob()`, `fs.meta()` — sandboxed to project root - `markdown.render()` via pulldown-cmark - `template.render()` — mustache-style `{{key}}` / `{{{raw}}}` substitution - Test with a `.rex` handler that reads markdown + renders through a template ### Phase 7: Hot reload + hardening - `reload.rs`: notify watcher, atomic route table swap - Graceful shutdown, request logging, error recovery - `rex-serve.toml` config ### Phase 8: Type file + knowledge base example app - Write `rex-serve.rexd` — the domain type interface for IDE support - Write all the route files, layouts, content, and static assets shown above - End-to-end test: homepage lists articles, article page renders markdown, API CRUD works, static CSS served, middleware adds headers everywhere --- ## Key Files to Modify/Reference - `packages/rusty-rex/Cargo.toml` — add `rex-serve` to workspace members - `packages/rusty-rex/crates/rex-core/src/interpret.rs` — `Context`, `HostObject`, `run()`, `RunResult` - `packages/rusty-rex/crates/rex-core/src/lib.rs` — `compile()` function - `packages/rusty-rex/crates/rex-cli/src/main.rs` — reference for Context setup (line 247) - `samples/http-domain/.config.rex` — canonical ref definitions (short codes) - `rex-types.md` — `.rexd` type file specification ## Verification 1. `cargo build -p rex-serve` compiles 2. Start server: `cargo run -p rex-serve -- --dir knowledge-base/routes` 3. `curl localhost:3000/health` → `{"ok": true}` 4. `curl localhost:3000/style.css` → CSS content with `content-type: text/css` + security headers from middleware 5. `curl localhost:3000/` → HTML page listing articles from `_content/` directory 6. `curl localhost:3000/articles/getting-started` → rendered markdown article in HTML layout 7. `curl localhost:3000/api/articles` without auth → 401 8. `curl -X POST localhost:3000/api/articles -H "Authorization: <key>" -d '{"slug":"new","title":"New","body":"# New\nHello"}'` → 201 9. `curl localhost:3000/articles/new` → newly created article rendered from DB 10. Hot reload: edit `_layouts/page.html` → change reflected on next request -
creationix created this gist
Mar 29, 2026 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,400 @@ # Rex Type System Rex has no user-space type annotations. Types are inferred from domain interface files (`.rexd`), literals, operators, and type predicates. The type system exists purely for tooling — the compiler and interpreter are untyped. --- ## Domain Interface Files (`.rexd`) A `.rexd` file declares the types of globals, functions, and type aliases available to Rex programs in a given project. The LSP searches upward from the open file for `*.rexd` files. ### Syntax `.rexd` files use Rex-like syntax for familiarity but describe types, not executable code. #### Type aliases Uppercase names define reusable types: ```rex Headers = {string | [string]} HttpMethod = "GET" | "POST" | "PUT" | "DELETE" | "PATCH" ``` #### Globals Lowercase names declare values available to Rex programs: ```rex // Read-only (default) req = { method: HttpMethod path: string | [string] headers: Headers query: {string | [string]} cookies: {string} ip: string body: string } // Mutable — the program can assign to properties mut res = { status: number headers: Headers body: string } // Simple typed globals config: unknown secrets: {string} ``` #### Functions Dot-path names with typed arguments and optional return type: ```rex log.info(message: unknown) log.warning(message: unknown) json.parse(text: string): unknown json.stringify(value: unknown): string res.rewrite(url: string): never res.redirect(url: string, status: number): never ``` #### Comments as documentation `//` comments immediately above a declaration are extracted as hover documentation: ```rex // Client IP address from the connecting socket or X-Forwarded-For header ip: string ``` ### Type syntax | Syntax | Meaning | |-----------------|-----------------------------------------------------| | `string` | String value | | `number` | Any numeric value (integer or decimal) | | `integer` | Integer only | | `boolean` | `true` or `false` | | `null` | The null value | | `unknown` | Some defined value of unknown type — must narrow (via `number()`, `string()`, etc.) before use in operations. Does NOT include `undefined`. Use `unknown \| undefined` for "might be absent". | | `never` | Function does not return (throws or diverges) | | `"GET"` | String literal type — only this exact string | | `[T]` | Array of `T` | | `{key: T, ...}` | Object with known fields | | `{T}` | Map with string keys and value type `T` | | `T \| U` | Union — value can be `T` or `U` | | `Name` | Reference to a type alias | ### `unknown` vs `undefined` `unknown` and `undefined` are distinct: - **`unknown`** — a value exists, but we don't know its type. Must narrow with type predicates before using in operations. Navigation on `unknown` produces `unknown`. - **`undefined`** — no value. The result of missing properties, failed comparisons, etc. Navigation on `undefined` produces `undefined`. - **`unknown | undefined`** — might be a value, might be absent. Common for map lookups and optional domain fields. ```rex config: unknown // definitely a value, type unknown config.timeout // unknown (navigation on unknown → unknown) cookies.session // unknown | undefined (map lookup, key may not exist) when cookies.session do // cookies.session: unknown (narrowed — undefined removed, value exists) when number(cookies.session) do // cookies.session: number (narrowed further) cookies.session + 1 // valid end end ``` ### Mutability Globals are read-only by default. Use `mut` to allow property writes: ```rex mut res = { status: number, headers: Headers, body: string } ``` Inside Rex code, `res.status = 404` is valid. Without `mut`, the LSP reports an error on write attempts. ### No short codes `.rexd` files describe the developer-facing API. The names match what the programmer writes in Rex code (`req.headers`, `res.status`). The compiler maps these to internal short ref codes (`'H`, `'S`) via a separate host-provided mapping. The developer never sees or writes short codes. --- ## Type Inference The type checker walks the program top-to-bottom, inferring a type for every expression. ### Literals | Expression | Type | |------------------|---------------------------| | `42` | `integer` | | `3.14`, `314e-2` | `number` | | `"hello"` | `string` | | `true`, `false` | `boolean` | | `null` | `null` | | `undefined` | `undefined` | | `[1, 2, 3]` | `[integer]` | | `{a: 1, b: "x"}` | `{a: integer, b: string}` | ### Variables Assignment creates a variable with the type of the right-hand side: ```rex x = 42 // x: integer name = "Ada" // name: string items = [1 2 3] // items: [integer] ``` Compound assignment preserves type: ```rex x = 0 // x: integer x += 1 // still integer (integer + integer = integer) ``` ### Arithmetic operators | Expression | Type | |------------------------------------|------------------------------------------------------------------------------| | `integer + integer` | `integer` | | `number + number` | `number` | | `string + string` | `string` (concatenation) | | `number + string` | error: cannot add number and string | | `a - b`, `a * b`, `a / b`, `a % b` | `number` (or `integer` if both integer, except `/` which is always `number`) | | `-a` | same as `a` | ### Comparison operators Comparisons return the left-hand value on success, `undefined` on failure: ```rex x > 10 // type: typeof(x) | undefined a == b // type: typeof(a) | undefined ``` This means: ```rex score = x > 10 // score: number | undefined (if x: number) ``` ### Logical operators Rex uses existence-based logic, not boolean logic: | Expression | Type | Semantics | |------------|--------------------------|------------------------------------| | `a or b` | `typeof(a) \| typeof(b)` | First defined value | | `a and b` | `typeof(b) \| undefined` | Second value if first is defined | | `a nor b` | `typeof(b) \| undefined` | Second value if first is undefined | The `or` pattern is commonly used for defaults: ```rex max = max or 100 // max: number (if max: number | undefined from domain) ``` ### Control flow **When / Unless:** ```rex when cond do body end // type: typeof(body) | undefined when cond do a else b end // type: typeof(a) | typeof(b) unless cond do body end // type: typeof(body) | undefined ``` **Loops:** ```rex for x in items do body end // type: typeof(body) (last iteration) while cond do body end // type: typeof(body) | undefined ``` **Comprehensions produce arrays:** ```rex [x * 2 for x in items] // type: [number] (if items: [number]) [self in 1..10] // type: [integer] {k: v for k, v in obj} // type: {typeof(v)} ``` ### Property access Accessing a property on a typed value narrows by lookup: ```rex req.method // HttpMethod → "GET" | "POST" | ... req.headers // Headers → {string | [string]} req.headers.host // string | [string] (from map value type) ``` For objects with known fields, accessing a known field returns its type. Accessing an unknown field is an **error** (but still types as `undefined`): ```rex point = {x: 1, y: 2} point.x // integer point.z // error: unknown property 'z' on {x: integer, y: integer} // type: undefined ``` For maps (`{V}`), any string key is valid but the key might not exist, so the result is `V | undefined`: ```rex cookies = req.cookies // {string} cookies.session // string | undefined (key may not exist) when cookies.session do // cookies.session: string (narrowed — undefined removed) end ``` #### Property access on unions When a type is a union, property access is resolved on each branch independently and the results are unioned: ```rex // Given: value: {a: number, b: string} | {boolean} value.a // Left branch: {a: number, b: string}.a → number // Right branch: {boolean}.a → boolean | undefined (map allows any key) // Combined: number | boolean | undefined ``` This means a union of a known-field object and a map is useful: known fields are accessed precisely, but the map branch provides a fallback for arbitrary keys. ```rex // A config that has known fields but also allows extensions Config = {timeout: number, retries: integer} | {unknown} config.timeout // number | unknown | undefined config.custom // error on left branch (unknown field), unknown | undefined on right // combined: unknown | undefined ``` An object with known fields unioned with a map does NOT suppress the error on unknown fields from the object branch — the error is still reported as a diagnostic, but the type includes the map branch's result so the program is valid. ### Navigation on undefined Accessing a property on `undefined` is not an error — it simply produces `undefined`. This means all property chains are implicitly optional, similar to `?.` in JavaScript: ```rex config.routing.timeout // if config: unknown, this is unknown // if config: {routing: {timeout: number}}, this is number // if config: undefined at runtime, this is undefined // no runtime error in any case ``` If a type includes `undefined` in a union, the result of navigation adds `undefined` to the property type: ```rex user = users.0 // user: {name: string} | undefined (array index) user.name // string | undefined (undefined propagates) when user do user.name // string (narrowed — user is defined) end ``` This means the type checker never reports a navigation chain as an error. Unknown properties on concrete types produce a warning and type `undefined`, but the program is still valid. ### Type narrowing #### Via type predicates Rex's type predicates (`number()`, `string()`, `object()`, `array()`, `boolean()`) act as type guards: ```rex when number(value) do // value: number (narrowed from whatever it was) value + 1 // valid end when string(value) do // value: string value + " suffix" // valid end ``` #### Via existence The `when` construct narrows `undefined` out of the type: ```rex name = req.query.name // string | [string] | undefined (map lookup) when name do // name: string | [string] (undefined removed) end ``` #### Via comparison Comparisons narrow the type: ```rex when req.method == "GET" do // req.method: "GET" (narrowed from the union) end ``` --- ## Diagnostics The type checker produces warnings and errors based on inferred types. ### Errors (prevent correct execution) | Check | Example | Message | |---------------------------|----------------------------------------------------|---------------------------------------------------------------| | Type mismatch in operator | `"hello" - 1` | Cannot subtract from string | | Wrong argument type | `json.parse(42)` | Expected string for argument 'text' of json.parse, got number | | Wrong argument count | `json.parse(a, b)` | json.parse expects 1 argument, got 2 | | Missing required field | `{x: 1}` where `{x: number, y: number}` expected | Missing required field 'y' | | Field type mismatch | `{status: "ok"}` where `{status: number}` expected | Field 'status' has type string, expected number | | Assign to read-only | `req.method = "POST"` | Cannot assign to read-only property 'method' on 'req' | ### Warnings (likely mistakes) | Check | Example | Message | |--------------------|----------------------------------------------|--------------------------------------------------------| | Unknown property | `req.headrs` | Unknown property 'headrs'. Did you mean 'headers'? | | Unused variable | `x = 1` (never read) | Variable 'x' is assigned but never used | | Unreachable code | `break; x = 1` | Unreachable code after break | | Extra object field | `{x: 1, y: 2, z: 3}` where `{x, y}` expected | Unknown field 'z' (structural subtyping allows extras) | ### Not checked The type system intentionally does not check: - Arithmetic overflow (numbers are arbitrary precision in Rex) - Array bounds (out of bound reads are undefined) - Exhaustiveness of literal unions (the `else` branch handles unknown values) There is no `any` escape hatch. `unknown` requires narrowing before use — the type checker ensures all operations have compatible types. --- ## How it works together 1. Developer creates `http.rexd` in the project root describing the HTTP domain 2. LSP finds and parses the `.rexd` file on startup 3. When a `.rex` file is opened, the LSP: - Parses the file via the rowan CST parser - Seeds the type environment from the `.rexd` globals - Walks the CST, inferring types for each expression - Reports diagnostics (errors, warnings) - Provides completions, hover, go-to-definition from the inferred types 4. On each edit, the LSP incrementally re-parses and re-checks 5. No type annotations appear in `.rex` files — everything is inferred