Created
March 29, 2026 07:13
-
-
Save swyxio/31821c7abbb52ae49c4e62895e8eb6a1 to your computer and use it in GitHub Desktop.
Revisions
-
swyxio revised this gist
Mar 29, 2026 . 1 changed file with 211 additions and 196 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,229 +1,244 @@ # CloudForge — Dev Notes All instructions and decisions from the build sessions, grouped by topic. --- ## Vision & Core Requirements Build a fully-realized GitHub clone on Cloudflare infrastructure. Basic functionality: create accounts, create repos, push and pull git, create issues, create PRs, render the file tree for the current main branch, navigate branches, manage collaborators, store blobs. The main twist is that **everything runs on Cloudflare** — Workers, D1, Durable Objects, R2. Make all choices autonomously. Loop until done. "Done" means self-reviewed, fully working, tested, and deployable. A user should be able to create a new git repo locally and push it up to CloudForge. Use [ralph-claude-code](https://github.com/frankbria/ralph-claude-code) for initial scaffolding and autonomous dev loops. --- ## Architecture Decisions - **Backend**: Cloudflare Workers with Hono framework - **Database**: Cloudflare D1 (SQLite at the edge) for relational data — users, repos, refs, issues, PRs, collaborators - **Object Storage**: Cloudflare R2 for git objects — blobs, trees, commits, packfiles, keyed as `{repoId}/objects/{sha[0:2]}/{sha[2:]}` - **Concurrency**: Durable Objects for repo-level locks during push operations - **Auth**: JWT (HMAC-SHA256) + PBKDF2 password hashing via Web Crypto API - **Frontend**: React 18 SPA + Vite + TailwindCSS, served as static assets from the same Worker via `[assets]` in wrangler.toml (eliminates CORS) - **Monorepo**: npm workspaces — `packages/api`, `packages/web`, `packages/shared` - **Framework**: React Router v6 (no Next.js/Remix — they have issues on Cloudflare). Manual caching + hover prefetching is sufficient for SPA performance. --- ## Git Protocol Implement the [Git Smart HTTP Protocol](https://www.git-scm.com/docs/http-protocol) from scratch: - `GET /:owner/:repo.git/info/refs?service=git-upload-pack` — reference discovery for clone/fetch - `GET /:owner/:repo.git/info/refs?service=git-receive-pack` — reference discovery for push - `POST /:owner/:repo.git/git-receive-pack` — receive pushed objects - `POST /:owner/:repo.git/git-upload-pack` — send objects for clone/fetch Implementation includes: full packfile parsing with zlib decompression (RFC 1950), OFS_DELTA and REF_DELTA resolution, custom Huffman decoder for scanning deflate streams, packfile generation with SHA1 checksums, object graph walking for clone, pkt-line protocol encoding/decoding, HTTP Basic Auth for git operations. --- ## Test Accounts & Data - **User 1**: username `test`, email `test@test.com`, password `test` — prepopulate with repos - **User 2**: username `test2`, email `test2@test.com`, password `test2` — different repos, used for cross-user privacy testing - Verify: seeing a repo you don't own works for public repos; private repos return 404 (not 403) even if you guess the name --- ## Organizations & Teams - An account can start an org - An account has its own org and its own team - An org can have many teams - A team can have many accounts as members - Repos belong to orgs and can be edited by any team with permissions - Role hierarchy: owner > admin > member --- ## User Profile & Homepage - Don't use `/dashboard` — redirect logged-in users to `/{username}` so they can share that URL - The repo named `{username}/{username}` is special: its README displays on the profile page (like GitHub's profile README, making it a bit of social media) - Profile photos: allow upload within reasonable size limits, resize down for PFP use, generate gravatar or SVG with name/initials as fallback - **Don't show user emails publicly or leak them in traffic/source code** — only expose email to the user themselves --- ## Repository Features ### File Browser - VS Code-style sidebar tree navigation with inline file viewing (no full page reloads) - Show last commit messages and relative dates in file listing (not raw SHA hashes) - Handle `.git` suffix in URLs gracefully - Multi-segment path routing for deep file paths (e.g., `migrations/0001_initial.sql`) ### File Viewer - Filetype-aware viewer with read/edit mode (inspired by GitHub Next Blocks) - Different icons for folders vs file types — VS Code Seti-style filetype icons (inline SVG, no external deps) - Source view should be **directly editable** — no separate edit mode or edit button - Commit panel only appears when content is "dirty" (changed from original) - Commit options: commit to main or commit to new branch - Proper UTF-8 decoding (`atob` + `Uint8Array` + `TextDecoder`) — no mojibake - Preview modes: markdown rendering, JSON pretty-print, CSV table ### Markdown Rendering - Proper typography — styled headings, bullet points, code blocks - Table of contents sidebar (auto-generated from headings) - Hoverable anchor links on headings (like GitHub READMEs) - Collapsible H1/H2 sections with chevron toggle - Copy button on all code blocks (hover to reveal) - Scrollable code blocks (max-height with overflow) - Better table rendering: row hover effects, alternating rows, rounded corners - Fix Unicode box-drawing characters in architecture diagrams (proper monospace rendering) ### Navigation & Performance - **No full page reloads** — this is an SPA, edit URL and browser history without reloading - In-memory content cache (`useRef<Map>`) for instant back/forward navigation - Hover prefetching — load content on mouse hover so navigation feels instant - Stable layout — no jumping/jank when navigating between files and directories ### Stars - Star/unstar repos, find starred repos later with the date starred - Display star count on repos - Let people see who has starred a repo (stargazers list) ### Rename - Allow users to rename repos; everything should transfer and redirect properly ### Description - Allow users to tweak repo descriptions --- ## Issues & Pull Requests ### Issues - Create issues with title and body - Issue comments - Open/closed state with accurate counts when switching tabs - Auto-increment issue numbering ### Pull Requests - **Far more info** on PR pages — don't leave them bare-bones - Show diff between source and target branch - Show list of commits in the PR - Show changed files summary (files changed, additions, deletions) - Merge button for repo owners - Color-coded status: open=green, closed=red, merged=purple - Smart auto-title from branch name (e.g., `feature/add-dark-mode` → "Add dark mode") - Smart auto-description from commit messages (formatted as bullet list) - Preview diff before creating PR - Use AI/intelligence features where possible --- ## Theming & Accessibility - Light/dark/auto color theming in user preferences - Auto follows system via `prefers-color-scheme` media query - Persists to localStorage - Tailwind class-based dark mode (`darkMode: 'class'`) - **Accessibility settings**: high contrast mode, font size (small 14px / default 16px / large 18px), reduced motion - Full dark mode coverage across every page and component --- ## Design - Do a design pass to make everything look better - Look up frontend-design skill for reference - Condense repo header vertical space — move clone URL into a small dropdown button (like GitHub's green "Code" button) near the tab bar - Star button visible in repo header --- ## Security & Safety ### Rate Limiting - Propose per-user/org/client rate limits and guards against storing too much and pushing too often - Propose egress limits so CloudForge doesn't get used as a CDN - Implemented: unauthenticated 60/min per IP, authenticated 300/min per user, write ops 120/min, git ops 30/hr, repo creation 10/hr, storage quota 500MB warning ### Content Safety - Implement basic CSAM content checks and safety/privacy/antispam measures so we don't end up hosting illegal things - **Status: not yet implemented** — flagged as high priority ### Privacy - Private repos return 404 (not 403) to non-authorized users — privacy by obfuscation - Don't expose user emails in public API responses --- ## Testing - Write E2E Playwright tests for all pages with expects for test data - Robust test suite covering error cases — error nicely if poorly formatted - Unit tests for git internals (pkt-line, objects, delta, packfile) - API E2E tests - Privacy E2E tests - Browser-test every route after each deploy --- ## AI & Smart Features - Research Cloudflare's built-in/free LLM/AI Gateway offerings - Add smart AI features: auto-titling git commits when made in the web editor - Agentic/semantic search across one repo, all repos in account, or all repos user has access to - Research GitHub Linguist for filetype colors - Research GitHub Next Blocks for custom file renderers - **Status**: heuristic-based smart features implemented (branch name → title, commits → description). Full AI Gateway integration not yet done. --- ## Deployment - Build: `npm run build` (runs tsc + vite build in web package) - Deploy: copy `packages/web/dist` to `packages/api/public`, then `npx wrangler deploy` - Environment: `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` required - JWT_SECRET set via `wrangler secret put` - Frontend served on same domain via `[assets]` in wrangler.toml (no CORS issues) - Live at: https://cloudforge-api.shawnthe14483.workers.dev --- ## Implementation Status ### Completed - Full git protocol (push/pull/clone) - User auth (register/login/JWT) - Repository CRUD with file browser - Issues + comments + labels - Pull requests with diffs, commits, smart descriptions - Branch management - Organization/team system - Dark mode + accessibility settings - VS Code-style filetype icons - Inline editing with dirty-state commit panel - Content cache + hover prefetch (no-jank navigation) - Enhanced markdown renderer (TOC, anchors, collapsible, copy buttons) - Stars (backend + frontend) - Repo rename - Rate limiting - Email privacy - E2E Playwright tests - Comprehensive design pass ### Not Yet Implemented - CSAM/content safety checks (high priority) - Profile photo upload with resize and gravatar fallback - Cloudflare AI Gateway integration (semantic search, real AI commit titles) - Webhook notifications - CI/CD pipeline equivalent - Side-band-64k for large repos - Merge conflict detection --- ## Framework Considerations (Decided Against) Considered migrating to Remix, TanStack Router, or [Vinext](https://github.com/cloudflare/vinext) for better SSR/prefetching on Cloudflare. Decision: **not worth it**. React Router v6 with manual caching (in-memory Map + hover prefetch) gives us what we need without framework lock-in or Cloudflare compatibility headaches. -
Mar 29, 2026 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,229 @@ CloudForge — Dev Notes All instructions and decisions from the build sessions, grouped by topic. Vision & Core Requirements Build a fully-realized GitHub clone on Cloudflare infrastructure. Basic functionality: create accounts, create repos, push and pull git, create issues, create PRs, render the file tree for the current main branch, navigate branches, manage collaborators, store blobs. The main twist is that everything runs on Cloudflare — Workers, D1, Durable Objects, R2. Make all choices autonomously. Loop until done. "Done" means self-reviewed, fully working, tested, and deployable. A user should be able to create a new git repo locally and push it up to CloudForge. Use ralph-claude-code for initial scaffolding and autonomous dev loops. Architecture Decisions Backend: Cloudflare Workers with Hono framework Database: Cloudflare D1 (SQLite at the edge) for relational data — users, repos, refs, issues, PRs, collaborators Object Storage: Cloudflare R2 for git objects — blobs, trees, commits, packfiles, keyed as {repoId}/objects/{sha[0:2]}/{sha[2:]} Concurrency: Durable Objects for repo-level locks during push operations Auth: JWT (HMAC-SHA256) + PBKDF2 password hashing via Web Crypto API Frontend: React 18 SPA + Vite + TailwindCSS, served as static assets from the same Worker via [assets] in wrangler.toml (eliminates CORS) Monorepo: npm workspaces — packages/api, packages/web, packages/shared Framework: React Router v6 (no Next.js/Remix — they have issues on Cloudflare). Manual caching + hover prefetching is sufficient for SPA performance. Git Protocol Implement the Git Smart HTTP Protocol from scratch: GET /:owner/:repo.git/info/refs?service=git-upload-pack — reference discovery for clone/fetch GET /:owner/:repo.git/info/refs?service=git-receive-pack — reference discovery for push POST /:owner/:repo.git/git-receive-pack — receive pushed objects POST /:owner/:repo.git/git-upload-pack — send objects for clone/fetch Implementation includes: full packfile parsing with zlib decompression (RFC 1950), OFS_DELTA and REF_DELTA resolution, custom Huffman decoder for scanning deflate streams, packfile generation with SHA1 checksums, object graph walking for clone, pkt-line protocol encoding/decoding, HTTP Basic Auth for git operations. Test Accounts & Data User 1: username test, email test@test.com, password test — prepopulate with repos User 2: username test2, email test2@test.com, password test2 — different repos, used for cross-user privacy testing Verify: seeing a repo you don't own works for public repos; private repos return 404 (not 403) even if you guess the name Organizations & Teams An account can start an org An account has its own org and its own team An org can have many teams A team can have many accounts as members Repos belong to orgs and can be edited by any team with permissions Role hierarchy: owner > admin > member User Profile & Homepage Don't use /dashboard — redirect logged-in users to /{username} so they can share that URL The repo named {username}/{username} is special: its README displays on the profile page (like GitHub's profile README, making it a bit of social media) Profile photos: allow upload within reasonable size limits, resize down for PFP use, generate gravatar or SVG with name/initials as fallback Don't show user emails publicly or leak them in traffic/source code — only expose email to the user themselves Repository Features File Browser VS Code-style sidebar tree navigation with inline file viewing (no full page reloads) Show last commit messages and relative dates in file listing (not raw SHA hashes) Handle .git suffix in URLs gracefully Multi-segment path routing for deep file paths (e.g., migrations/0001_initial.sql) File Viewer Filetype-aware viewer with read/edit mode (inspired by GitHub Next Blocks) Different icons for folders vs file types — VS Code Seti-style filetype icons (inline SVG, no external deps) Source view should be directly editable — no separate edit mode or edit button Commit panel only appears when content is "dirty" (changed from original) Commit options: commit to main or commit to new branch Proper UTF-8 decoding (atob + Uint8Array + TextDecoder) — no mojibake Preview modes: markdown rendering, JSON pretty-print, CSV table Markdown Rendering Proper typography — styled headings, bullet points, code blocks Table of contents sidebar (auto-generated from headings) Hoverable anchor links on headings (like GitHub READMEs) Collapsible H1/H2 sections with chevron toggle Copy button on all code blocks (hover to reveal) Scrollable code blocks (max-height with overflow) Better table rendering: row hover effects, alternating rows, rounded corners Fix Unicode box-drawing characters in architecture diagrams (proper monospace rendering) Navigation & Performance No full page reloads — this is an SPA, edit URL and browser history without reloading In-memory content cache (useRef<Map>) for instant back/forward navigation Hover prefetching — load content on mouse hover so navigation feels instant Stable layout — no jumping/jank when navigating between files and directories Stars Star/unstar repos, find starred repos later with the date starred Display star count on repos Let people see who has starred a repo (stargazers list) Rename Allow users to rename repos; everything should transfer and redirect properly Description Allow users to tweak repo descriptions Issues & Pull Requests Issues Create issues with title and body Issue comments Open/closed state with accurate counts when switching tabs Auto-increment issue numbering Pull Requests Far more info on PR pages — don't leave them bare-bones Show diff between source and target branch Show list of commits in the PR Show changed files summary (files changed, additions, deletions) Merge button for repo owners Color-coded status: open=green, closed=red, merged=purple Smart auto-title from branch name (e.g., feature/add-dark-mode → "Add dark mode") Smart auto-description from commit messages (formatted as bullet list) Preview diff before creating PR Use AI/intelligence features where possible Theming & Accessibility Light/dark/auto color theming in user preferences Auto follows system via prefers-color-scheme media query Persists to localStorage Tailwind class-based dark mode (darkMode: 'class') Accessibility settings: high contrast mode, font size (small 14px / default 16px / large 18px), reduced motion Full dark mode coverage across every page and component Design Do a design pass to make everything look better Look up frontend-design skill for reference Condense repo header vertical space — move clone URL into a small dropdown button (like GitHub's green "Code" button) near the tab bar Star button visible in repo header Security & Safety Rate Limiting Propose per-user/org/client rate limits and guards against storing too much and pushing too often Propose egress limits so CloudForge doesn't get used as a CDN Implemented: unauthenticated 60/min per IP, authenticated 300/min per user, write ops 120/min, git ops 30/hr, repo creation 10/hr, storage quota 500MB warning Content Safety Implement basic CSAM content checks and safety/privacy/antispam measures so we don't end up hosting illegal things Status: not yet implemented — flagged as high priority Privacy Private repos return 404 (not 403) to non-authorized users — privacy by obfuscation Don't expose user emails in public API responses Testing Write E2E Playwright tests for all pages with expects for test data Robust test suite covering error cases — error nicely if poorly formatted Unit tests for git internals (pkt-line, objects, delta, packfile) API E2E tests Privacy E2E tests Browser-test every route after each deploy AI & Smart Features Research Cloudflare's built-in/free LLM/AI Gateway offerings Add smart AI features: auto-titling git commits when made in the web editor Agentic/semantic search across one repo, all repos in account, or all repos user has access to Research GitHub Linguist for filetype colors Research GitHub Next Blocks for custom file renderers Status: heuristic-based smart features implemented (branch name → title, commits → description). Full AI Gateway integration not yet done. Deployment Build: npm run build (runs tsc + vite build in web package) Deploy: copy packages/web/dist to packages/api/public, then npx wrangler deploy Environment: CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID required JWT_SECRET set via wrangler secret put Frontend served on same domain via [assets] in wrangler.toml (no CORS issues) Live at: https://cloudforge-api.shawnthe14483.workers.dev Implementation Status Completed Full git protocol (push/pull/clone) User auth (register/login/JWT) Repository CRUD with file browser Issues + comments + labels Pull requests with diffs, commits, smart descriptions Branch management Organization/team system Dark mode + accessibility settings VS Code-style filetype icons Inline editing with dirty-state commit panel Content cache + hover prefetch (no-jank navigation) Enhanced markdown renderer (TOC, anchors, collapsible, copy buttons) Stars (backend + frontend) Repo rename Rate limiting Email privacy E2E Playwright tests Comprehensive design pass Not Yet Implemented CSAM/content safety checks (high priority) Profile photo upload with resize and gravatar fallback Cloudflare AI Gateway integration (semantic search, real AI commit titles) Webhook notifications CI/CD pipeline equivalent Side-band-64k for large repos Merge conflict detection Framework Considerations (Decided Against) Considered migrating to Remix, TanStack Router, or Vinext for better SSR/prefetching on Cloudflare. Decision: not worth it. React Router v6 with manual caching (in-memory Map + hover prefetch) gives us what we need without framework lock-in or Cloudflare compatibility headaches.