Diversio Engineering · new engineer guide · Arch Linux

Diversio development on Arch Linux

Set up GitHub and the monolith, then run Django and both React apps. Use the same guide for the design system, just dev, local-ci, worktrees, and the team’s Pi and cmux workflow.

Begin with the primer Read How We Work See Systems Open Agentic Tools Watch Muhammad’s videos

1 anchor cloneclean code + canonical env source

4 servicesbackend · RQ · 2 frontends

2 productsDiversio and Optimo stay separate

1 commandjust dev controls the local loop

Guide checklist progress0 / 0

Anchor clone ruleUse the canonical monolith/ clone as the clean source for Git history, submodule pins, and ignored env files. For each task, run uv run scripts/create_worktree.py there. Install dependencies, run services, edit code, test, and commit in the generated sibling worktree.

The whole setup in plain English

Local development runs a private copy of Diversio on your laptop. You can change code without touching shared systems. This primer introduces each topic in the guide; the numbered sections provide the commands.

Anchor and worktree model

Keep the main clone as your clean source

Anchor clonethe clean master copy
stores canonical env files

→

Worktree scriptthe safe photocopier
uv run scripts/create_worktree.py

→

Working monolithyour task-specific lab bench
safe to edit and run

Keep the anchor clone clean. For each ticket, run the script to make a sibling worktree with the chosen code and copied local configuration. Make your changes in that worktree. After GitHub merges the pull request, delete the worktree.

Git vocabulary

Code and Git

Repository: a project folder plus its complete change history.
Clone: your local copy of a repository.
Submodule: a separate repository placed inside the monolith at an exact saved commit.
Worktree: another working folder sharing Git history with the anchor clone.
Branch: a named lane for one change.
Commit: a saved checkpoint with a message.
Pull request: a request for teammates and automation to review and merge your branch.

Application vocabulary

Browser to database

Frontend: React code running in the browser: pages, forms, buttons, and charts.
Backend/API: Django code that checks identity and tenants, enforces rules, and returns data.
Database: PostgreSQL stores durable product data.
Redis + RQ: a waiting line and workers for jobs that should not block the web request.
Design system: shared components and visual rules used by both frontends.
Port: a numbered local doorway, such as a backend URL on port 8000.

Tool vocabulary

Local tool belt

Terminal: a text remote control for your computer.
Env file: ignored local settings and secrets. Do not commit it.
Dependency: third-party code the project needs.
uv / pnpm: install and run the exact Python / JavaScript toolchains.
Docker: runs isolated services such as PostgreSQL and Redis.
just: memorable shortcuts for longer project commands.
tmux: keeps several terminal panes in one session.
CI: automated lint, type, test, build, and security checks.

Guide topics at a glance

Access

Your building pass

GitHub recognizes your account, and SSH proves that this laptop belongs to you. NPM_TOKEN opens private packages. The team secret manager holds app secrets and approved local data.

Arch setup

Your lab equipment

pacman installs the base tools. Docker runs PostgreSQL and Redis in containers. uv and pnpm select the project’s Python and Node versions.

Environment and data

Your local settings

Env files tell the apps where services live and provide secrets. An approved database dump gives you sample companies, surveys, and dashboard results. Treat both as confidential and keep them out of Git.

just dev

Your service conductor

just dev chooses a worktree, assigns ports, starts the backend and frontends, and shows their URLs. Use the same command to check status, read logs, and stop services.

Repositories

One campus, several departments

The monolith points to separate repositories for the backend, frontends, design system, infrastructure, website, and engineering tools. Each repository keeps its own branches and pull requests.

Git and review

Your draft and submission

A branch holds one task. A commit saves a checkpoint. A pull request asks the team to review the work, while local-ci checks it before submission.

Engineering references

Your team handbook

engineering.diversio.com explains current systems and working habits. The Drive guidelines add team and dashboard context. Repository docs own current commands.

Pi and cmux

The team’s agent workflow

Pi runs the coding session, and dev-workflow adds shared review stages. cmux gives Mac users agent panes and notifications. On Arch, Pi runs inside tmux and uses the same workflow commands in the current session.

Dashboard handoff

Follow one visible value

Start with a chart or number in React. Follow its API request into Django, then find the tenant-scoped query and test. You will see how the screen connects to stored data.

Troubleshooting

Read the state before changing code

Run just dev status, then inspect the relevant log. Check ports, authentication, environment files, Docker architecture, and database volumes before you reinstall tools.

Handoff and first week

Watch, trace, and ship

Read the team’s AI workflow posts and watch Muhammad’s recordings. Then get the stack running, trace one dashboard value, and ship one small tested pull request.

Product boundary

Two products share one backend

Django serves Diversio and Optimo. Diversio uses the main frontend and legacy backend apps; Optimo uses optimo-frontend and optimo_* apps. Keep their users and tenant rules separate.

Read before you runRead the explanation above a command and confirm your current directory. Ask a teammate when a step needs credentials or approved data.

A dashboard request from click to response

Browser

React frontend

Shows the page and sends an HTTPS API request.

→

Server

Django API

Authenticates the user, checks the tenant, and runs business rules.

→

Data + work

PostgreSQL / RQ

Reads or writes data; queues slow work, then returns a response.

React receives the response and updates the screen. Debug the same route: visible component → API call → Django endpoint → tenant-scoped query → test.

The six-move daily loop

Go to the clean anchor clone.
Create and enter a task worktree.
Start the app with just dev.
Edit a small behavior and run its focused test.
Commit, run local-ci, push, and open a pull request.
Remove the worktree after GitHub merges the pull request.

Checks are different kinds of feedback

Test: does the behavior still work?
Lint: does the code follow simple quality rules?
Type check: do values have the shapes the code expects?
Build: can the tools package the application?
Migration: does a database shape change have safe instructions?
Code review: can another engineer understand and trust the change?

Your first goalGet one worktree running and log in. Trace one dashboard number, then change one tested behavior. A complete loop shows you how the repositories connect.

Know the system before installing it

One Django project serves Diversio and Optimo. Each product has its own users, tenants, APIs, frontend, and business rules.

Diversio FrontendReact + Vite · frontend/

Diversio userdashboard, surveys, analytics

→

Django4Lyfebackend/ · Python 3.14 · Django

Diversio lane
dashboardapp, survey, pulse_iq, titan
Users: User + Company
APIs: /api/v1, /api/v2

Optimo lane
optimo_* apps
Users: OptimoUser + OptimoOrganization
API: /optimo/api/v1

PostgreSQL + Redis + RQdata, cache, background work

←

Optimo FrontendReact + Vite · optimo-frontend/

Optimo userpeople intelligence product

Shared UI

Design system

design-system/ publishes @diversioteam/diversio-ds for both frontends.

Local control plane

Monolith + just

The root checkout pins all repositories and gives worktree-aware start, stop, status, logs, ports, and URL aliases.

Public knowledge

Engineering website

engineering-website/ renders the public site and reads Agentic Tools content from agent-skills-marketplace/.

Hard boundaryDo not mix optimo_* business logic with dashboardapp, survey, pulse_iq, or titan. Similar-looking concepts can belong to different users and tenants.

Get access before debugging setup

Missing access or local files cause many day-one setup errors. Ask the team for these items before you debug code.

Team-provided

Access checklist

GitHub membership in DiversioTeam with access to private product repositories
GitHub organization SSO authorization completed
Access to the team secret manager / 1Password Technology vault
Approved local backend/.env values
Access to the team-approved Crafting database dump for dashboard work that needs realistic data
Crafting account, cs CLI access to dev-dump-upload, and the approved dump ZIP password
Your own GitHub token with read:packages for private design-system packages
Diversio Google Workspace access to Muhammad’s handoff folder
A Pi model provider through a subscription login or team-approved API key

Security boundary

Do not solve access by sharing secrets

Create your own token; do not copy another engineer’s personal access token.
Keep .env, Bruno environment files, database dumps, and tokens out of Git.
Use only the team-approved dump and treat its contents as confidential.
Keep deployment credentials such as CIRCLECI_TOKEN out of the first-day setup unless your role needs releases.

Ask a teammateRepository templates show the file shapes. The team supplies secrets and the Crafting dump password through private channels. Download the approved dataset with the monolith helper described below.

Install the Arch-native foundation

Use pacman for packages and Docker Engine for containers. Arch provides GNU date; use Linux clipboard and open commands in place of pbcopy and open.

Install host packages

We checked these package names against the official Arch repositories. The repository bootstrap installs the project’s Node and pnpm versions in a later step.

Arch Linux · host tools

sudo pacman -Syu --needed \
  base-devel git github-cli openssh curl wget jq just fzf tmux lsof \
  docker docker-compose xdg-utils wl-clipboard xclip mkcert nss \
  uv zip unzip coreutils procps-ng \
  glib2 cairo pango harfbuzz gobject-introspection fontconfig \
  gdk-pixbuf2 libffi libnotify

mkcert + nss support the Diversio frontend’s local HTTPS certificate. The graphics libraries cover host-run WeasyPrint; Docker mode already carries its own system dependencies.

Enable Docker Engine

The local backend, PostgreSQL, Redis, the optional full stack, and the .localhost URL router use Docker.

Docker service

sudo systemctl enable --now docker.service
sudo usermod -aG docker "$USER"

# Log out and back in so group membership takes effect, then:
docker version
docker compose version
docker run --rm hello-world

Docker group warningThe Docker group grants root-equivalent access. Rootless Docker may prevent the hostname router from binding host port 80 without extra configuration. Direct localhost URLs still work.

Replace macOS commands on Arch

Docs may say	Arch equivalent	Reason
`brew install …`	`sudo pacman -S …`	Native package management; no Linuxbrew required for core tools.
Docker Desktop	`docker` + `docker-compose` services	Native Linux daemon and Compose plugin.
`pbcopy`	`wl-copy` on Wayland; `xclip` on X11	The `just dev` picker detects these.
`open URL`	`xdg-open URL`	The picker and scripts detect `xdg-open`.
`gdate`	`date`	Arch already ships GNU coreutils.
cmux	`tmux` via `just dev mode=tmux`	cmux-specific Pi commands are optional; the dev runner has first-class tmux mode.
`DYLD_FALLBACK_LIBRARY_PATH`	Do not set it	macOS needs that workaround. Arch uses the Linux libraries above.
`launchd`	`systemd` / user services	Long-running helpers may need it. The normal app loop does not.

Authenticate GitHub and package access

Identity

SSH key

All submodule URLs in .gitmodules use this key.

→

GitHub CLI

`gh auth login`

Issues, PRs, releases, Actions, and local-ci fallback auth.

→

Packages

`NPM_TOKEN`

Private @diversioteam packages such as the design system.

GitHub setup

# Create a key if this machine has none.
test -f "$HOME/.ssh/id_ed25519" || \
  ssh-keygen -t ed25519 -C "your.name@diversio.com"

# Authenticate and upload the public key.
gh auth login --hostname github.com --git-protocol ssh --web
gh ssh-key add "$HOME/.ssh/id_ed25519.pub" --title "$(hostname)-arch"
gh auth status

# This command prints a success message but may exit non-zero because GitHub offers no shell.
ssh -T git@github.com

# Confirm access to private product repositories.
gh repo view DiversioTeam/Django4Lyfe --json nameWithOwner,viewerPermission
gh repo view DiversioTeam/Diversio-Frontend --json nameWithOwner,viewerPermission

Generate your own `NPM_TOKEN`

Do not copy another engineer’s token. GitHub Packages requires a personal access token (classic) for local npm registry access.

Step 1

Create the token

Open GitHub’s classic-token form, choose the team-approved expiration, and grant only read:packages. Generate the token and copy it when GitHub shows it once.

Step 2

Authorize the organization

Open GitHub Settings → Developer settings → Personal access tokens → Tokens (classic). Choose Configure SSO beside the token and authorize DiversioTeam. Ask the team if organization approval remains pending.

Step 3

Store your token

Save it in Aryaman’s private 1Password item. Do not put it in the shared env archive, a repository file, Slack, or shell history.

Configure and test GitHub Packages without placing the token in shell history

# Bash hides the pasted value. The token lives in this shell after export.
read -rsp "Paste your GitHub package token: " NPM_TOKEN
echo
export NPM_TOKEN

# The repo bootstrap uses the same user-level pnpm setting.
pnpm config set --location=user //npm.pkg.github.com/:_authToken "$NPM_TOKEN"

# This prints a package version, not the token.
pnpm view @diversioteam/diversio-ds version \
  --registry=https://npm.pkg.github.com

Official requirementGitHub’s npm registry documentation requires a personal access token (classic). The read:packages scope grants package installation; it does not grant package publishing.

Optional local-ci tokenlocal-ci checks LOCAL_CI_GITHUB_TOKEN first, then uses the authenticated gh session. Start with gh. Add a dedicated token if status posting needs it.

Create the permanent anchor clone

The anchor clone provides the shared Git object store, worktree factory, submodule source, and canonical home for ignored env files. Feature work happens in generated worktrees.

Clone the monolith and its submodules once

Clone

mkdir -p "$HOME/work/diversio"
cd "$HOME/work/diversio"
git clone --recursive git@github.com:DiversioTeam/monolith.git
cd monolith

git submodule status
git status --short

If a submodule directory is empty, recover with git submodule update --init --recursive. Keep this anchor clone on its normal branch with tracked files and gitlinks clean. Detached submodule HEADs are normal because the monolith pins commits.

Install `local-ci` without Homebrew

The public DiversioTeam/local-ci-runner release includes Linux amd64 and arm64 binaries. The command below installs the latest release through gh.

Arch-native local-ci install

mkdir -p "$HOME/.local/bin"
case "$(uname -m)" in
  x86_64) local_ci_arch="amd64" ;;
  aarch64|arm64) local_ci_arch="arm64" ;;
  *) echo "Unsupported architecture: $(uname -m)"; exit 1 ;;
esac

local_ci_tag="$(gh release view \
  --repo DiversioTeam/local-ci-runner \
  --json tagName --jq .tagName)"
local_ci_version="${local_ci_tag#v}"
temp_dir="$(mktemp -d)"

gh release download "$local_ci_tag" \
  --repo DiversioTeam/local-ci-runner \
  --pattern "local-ci_${local_ci_version}_linux_${local_ci_arch}.tar.gz" \
  --dir "$temp_dir"
tar -xzf "$temp_dir"/local-ci_*.tar.gz -C "$temp_dir"
install -Dm755 "$temp_dir/local-ci" "$HOME/.local/bin/local-ci"
rm -rf "$temp_dir"

export PATH="$HOME/.local/bin:$PATH"
local-ci version

Run the monolith toolchain bootstrap

It checks gh, local-ci, GitHub auth, NPM_TOKEN, installs pnpm 10.34.4, and preinstalls Node 24.18.0, 24.15.0, 22.23.1, and 20.20.2.

Bootstrap pinned JS tools

export PNPM_HOME="${PNPM_HOME:-$HOME/.pnpm}"
export PATH="$HOME/.local/bin:$PNPM_HOME:$PATH"
: "${NPM_TOKEN:?Generate and export your personal read:packages token first}"

./scripts/dev-bootstrap-pnpm.sh
./scripts/dev-bootstrap-pnpm.sh --check-only

pnpm --version
node --version
gh auth status
local-ci version

Expected Linux noteThe helper may report a Homebrew warning. Ignore it on Arch after you install gh and local-ci.

Store safe PATH configuration

Shell profile

cat >> "$HOME/.bashrc" <<'EOF'
export PNPM_HOME="${PNPM_HOME:-$HOME/.pnpm}"
export PATH="$HOME/.local/bin:$PNPM_HOME:$PATH"
EOF

# If you use zsh, append the same two lines to ~/.zshrc instead.
source "$HOME/.bashrc"

Keep the literal NPM_TOKEN out of committed files. Use your secret manager’s shell integration or export the token in shells that install packages.

Keep code work out of the anchorUse the anchor to store ignored configuration and create worktrees. Create feature branches, install dependencies, start services, edit product code, and commit from a generated worktree.

Seed env files in the anchor, then create a worktree

The anchor clone stores the source env files. The worktree script copies them into a new sibling checkout and writes frontend API URLs for that worktree.

Canonical monolith/tracked code untouched · ignored env files live here

→

uv run scripts/create_worktree.pyselects commits · copies env · syncs API URLs · installs Git hooks

→

monolith-<task>/edit · install · run · test · commit · delete when done

Anchor clone · setup once

Store canonical env files, then generate the real checkout

Run from the untouched anchor

cd "$HOME/work/diversio/monolith"

# Tracked files and submodule pins must remain clean.
git status --short

# Create canonical ignored env files from templates where available.
test -f backend/.env || cp backend/.env.template backend/.env
test -f frontend/.env.development || \
  cp frontend/.env.development.template frontend/.env.development
test -f design-system/.env || \
  cp design-system/.env.dist design-system/.env
test -f optimo-frontend/.env || \
  cp optimo-frontend/.env.example optimo-frontend/.env
chmod 600 backend/.env

# Add approved backend/bruno/.env from 1Password when API work needs it.
# Fill the canonical files with team-approved local values, then:
uv run scripts/create_worktree.py

# Enter the exact sibling path printed by the script.
cd ../monolith-<worktree-name>

The script copies any source env files that exist: backend/.env, backend/bruno/.env, frontend/.env.development, design-system/.env, and optimo-frontend/.env. The same script writes per-worktree .env.local API targets so copied files do not retain stale URLs.

Generated worktree

Install dependencies in the working monolith

Run after cd into the generated worktree

# Backend: Python 3.14, locked uv environment, hooks.
(cd backend && uv sync && just install-hooks)

# Diversio frontend: Node 24.15.0.
(cd frontend && \
  ./scripts/ci/install_pnpm_and_dependencies.sh && \
  ./scripts/doctor.sh)

# For Optimo work: Node 24.18.0.
(cd optimo-frontend && \
  ./scripts/ci/install_pnpm_and_dependencies.sh && \
  ./scripts/doctor.sh)

# For design-system work: Node 22.23.1.
(cd design-system && \
  ./scripts/ci/install_pnpm_and_dependencies.sh && \
  ./scripts/doctor.sh)

A JavaScript repository installer switches Node to that repository’s pinned version. Run the target repository’s installer or pnpm env use --global <version> after you switch repositories.

Use worktree paths from this pointRun the remaining setup, database, just dev, test, Git, and PR commands inside monolith-<task>/. Keep those commands out of the anchor clone.

Crafting database download

Fetch the approved dump with the download helper

Crafting stores password-protected database dump ZIPs in the permanent dev-dump-upload sandbox. Install the cs CLI from the Crafting web console’s Download menu, then let the monolith helper list, download, and extract a dump.

Open Crafting Crafting CLI setup Team Crafting tips

Run from the generated monolith worktree root

# Confirm the CLI. The helper starts cs login if the session expired.
cs info

# Pick a dump newest-first, enter its ZIP password, and choose a save folder.
uv run scripts/cs_db_download.py

# Or choose the destination now. The script extracts the .sql file.
uv run scripts/cs_db_download.py --output "$HOME/work/diversio/dumps"

Fresh database volume

Copy the extracted SQL file to backend/database-dump/db.sql before PostgreSQL starts for the first time. The initialization flow loads it.

Existing database volume

Stop local app processes. Run cd backend && just up, then just load-db /absolute/path/to/dump.sql from backend/. This drops and recreates only your local database. Run just down before returning to host mode.

Use the correct helpercs_db_download.py fetches a dump. cs_db_snapshot.py publishes one: it uploads and restores a dump in the shared permanent sandbox, prunes old ZIPs, creates a Crafting snapshot, and announces it in Slack. Run the snapshot script only when the database owner asks you to publish an approved dump.

Database seed

Choose empty data or approved realistic data before first DB start

Empty database

Start PostgreSQL, run migrations, then create a local superuser. Good for infrastructure verification.

Approved Crafting dump

Place the extracted dump at backend/database-dump/db.sql before the first database volume starts. PostgreSQL reads init scripts for a fresh volume.

Prepare data

# Optional: copy an approved Crafting dump before first DB start.
cp /path/to/approved-dump.sql backend/database-dump/db.sql

# Pin this first checkout to familiar local dependency ports.
export BACKEND_PRIMARY_CHECKOUT_PATH="$PWD/backend"

# Start PostgreSQL and Redis. just dev owns the app processes.
(cd backend && .bin/docker-compose-local up -d db redis)

# Empty DB or dump: apply current migrations.
(cd backend && .bin/django migrate)

# Create a local login if the dump does not provide one.
(cd backend && .bin/django createsuperuser --configuration=DevApp)

Existing database volume?Changing db.sql will not import it again. Stop app processes, run cd backend && just up, then run just load-db /absolute/path/to/dump.sql from backend/. Confirm the selected path because this drops the local database.

Learn `just dev`: the local control plane

just dev coordinates the local app servers. You choose a worktree, and it loads your commands, checks dependencies and ports, then starts services in tmux, headless, or Docker mode.

1 · choose

Worktree

resolve-worktree.sh opens a status-aware fzf picker.

→

2 · plan

Commands + ports

Your config decides commands; preflight validates ports and dependencies.

→

3 · launch

tmux / headless / Docker

One lifecycle, consistent logs, URLs, stop, and status.

Recommended Arch path

Host app processes + Docker data services

This setup avoids frontend container architecture emulation and keeps PostgreSQL and Redis isolated. Start with the Diversio frontend.

Configure once

just dev config
just dev show

# Confirm the saved commands include:
# Diversio frontend: pnpm start
# Optimo frontend:  pnpm dev
# Backend/RQ auto-uv: enabled

# If a fresh history led the wizard to npm defaults, copy the pnpm example:
# docs/examples/dev-runner.env.pnpm.example

Daily start

export BACKEND_PRIMARY_CHECKOUT_PATH="$PWD/backend"
(cd backend && .bin/docker-compose-local up -d db redis)

# Start Django, RQ, and Diversio frontend in tmux.
just dev mode=tmux with_optimo_frontend=0

# In another shell:
just dev status

Full Docker needs an architecture check

Current frontend images are architecture-split

The current Docker defaults use a Diversio linux/arm64 dev-dependencies image and an Optimo linux/amd64 image. On a typical Arch x86_64 workstation, the Diversio image would need a compatible override or emulation.

Recommendation based on current Compose defaultsUse host-run frontends on Arch until the team confirms a native image for your architecture. Backend Docker remains a good default.

Full Docker after verification

uname -m

# Use after confirming the image/platform path for this machine.
DIVERSIO_FRONTEND_IMAGE="<team-approved-native-image>" \
  just dev docker-up

just dev docker-status
just dev docker-logs service=diversio-frontend

Use these commands

Goal	Command	Result
Open control picker	`just dev`	Shows worktrees, runtime sources, URLs, sync state, and keyboard actions.
Run local panes	`just dev mode=tmux`	One tmux pane per enabled service; best Arch interactive path.
Run in background	`just dev mode=headless`	Headless mode writes PIDs and logs under `.dev-logs/`.
One service	`just dev service=diversio-frontend`	Starts the requested service.
See reality	`just dev status`	Merges tmux, PID, Docker, port, localhost URL, and alias URL state.
Read logs	`just dev logs backend`	Tails headless logs; tmux panes keep their own visible output.
Stop all	`just dev stop`	Stops tmux, headless PIDs, Docker stacks, and reconciles aliases.
Fix config	`just dev reset` then `just dev config`	Rebuilds your user config without editing the shared justfile.

Interactive picker keys

Runtime

F2 tmux up
F3 headless up
F4 Docker up
F5 stop all
F6 / Ctrl-X Docker down

Inspect

Ctrl-R refresh
Ctrl-T attach tmux
Ctrl-L copy logs path
F7 Docker disk use
F8 reclaim selected stack

URLs

1–3 copy one URL
4 copy all URLs
Ctrl-B open backend
Ctrl-D open Diversio
Ctrl-P open Optimo

Success conditionjust dev status shows backend, RQ, and Diversio frontend running. Open the URLs it prints rather than guessing ports. The optional *.localhost aliases appear when the Caddy router can bind port 80; localhost URLs remain the debugging baseline.

Use each repository for one clear job

Repository	Purpose	Toolchain	Daily commands	Normal PR base
backendDjango4Lyfe	Diversio + Optimo APIs, jobs, data, admin	Python 3.14 · uv	`.bin/django` `.bin/pytest` `.bin/ruff` `.bin/ty`	`dev`
frontendDiversio-Frontend	Legacy Diversio dashboard and analytics UI	Node 24.15 · pnpm 10.34.4	`pnpm start` `pnpm test:ci` `pnpm lint`	`dev`
optimo-frontend	Optimo people intelligence UI	Node 24.18 · pnpm 10.34.4	`pnpm dev` `pnpm type-check` `pnpm test:unit`	`dev`
design-systemdiversio-ds	Shared components, tokens, Storybook	Node 22.23 · pnpm 10.34.4	`pnpm start` `pnpm build` `pnpm test`	`dev`
engineering-website	Public hub, broad pages, blog, deploy	Node ≥22.12 · npm	`npm run build` `npm run dev`	`main`
agent-skills-marketplace	Skills, plugin manifests, Pi packages, tool docs	Docs/config + npm for Pi packages	`bash scripts/validate-skills.sh`	`main`
monolith root	Planning hub, submodule pins, worktrees, local orchestration	Git · uv scripts · just	`just dev` `uv run scripts/create_worktree.py`	`main`

Backend wrappers

Use repository commands

Run .bin/django, .bin/pytest, .bin/pytest-parallel, .bin/ruff, and .bin/ty. The wrappers supply the correct uv environment and Django configuration.

Package manager

Follow each repository’s package manager

The product frontends and design system use pnpm. The engineering website uses npm. Keep package-manager changes out of unrelated work.

Daily Git, worktree, validation, and PR loop

Plan

Start from a GitHub issue

New work is GitHub-first. DiversioTeam/monolith is the planning and intake hub; use repo-local execution issues when ownership or hosted tooling requires them.

Isolate

Return to the anchor and create a worktree

Keep feature checkouts out of the anchor. From its clean root, the helper fetches branches, initializes the submodules, copies canonical env files, writes frontend API URLs, and creates detached checkouts to avoid branch locks.

Branch

Create branches in repositories you will change

You can read and run a detached checkout. Create the required monolith and submodule branches before you commit.

Prove

Run focused tests, then local-ci

Use repo wrappers while editing. Run local-ci run --no-github in the changed subrepo before finalizing.

Review

Push, open a ready PR, and publish clean statuses

Open a ready-for-review PR. Use draft status for work in progress. Push the commit before you run and publish clean local-ci statuses.

Feature worktree

# Begin at the clean anchor clone.
cd "$HOME/work/diversio/monolith"
git status --short
uv run scripts/create_worktree.py

# Enter the exact sibling path printed by the script.
# The script leaves branches detached in the new worktree.
cd ../monolith-<worktree-name>
git switch -c gh-<issue>-<short-slug>

# In each submodule you will change:
cd backend
git switch -c feature/<issue>-<short-slug> origin/dev

# Or, for a frontend change:
cd ../frontend
git switch -c feature/<issue>-<short-slug> origin/dev

Validation and PR

# Run inside the changed submodule.
local-ci run --no-github
local-ci resume <run-id>

# After committing and pushing a clean tree:
git push -u origin HEAD
local-ci run
local-ci publish <run-id>

gh pr create --base dev --fill

# If the change moved a submodule commit, commit the pointer at the monolith root.
cd ..
git add backend   # or frontend/design-system/etc.
git commit -m "Update backend submodule for #<issue>"

Dirty loop

`--no-github`

Full local logs while files are still changing. Safe for a dirty worktree.

Clean loop

`run` + `publish`

local-ci posts commit statuses after it confirms that the saved snapshot matches the clean checkout.

Remote CI

Inspect, do not assume

GitHub Actions and CircleCI still own selected security, sandbox, and deploy paths. local-ci is the repo-owned validation path where .local-ci.toml exists.

Use the Engineering website as a reference

The live site documents team habits, stack choices, tools, internship work, and engineering writing. Use it to learn an unfamiliar area, then follow its repository links.

Read first

How We Work

Ownership, system-level curiosity, small feedback loops, proof for bugs, production safety, and turning repeated pain into tooling.

Understand the stack

Systems

Django, React, PostgreSQL, AWS, Terraform, local tooling, docs, analytics, delivery, and agentic workflows.

Install accelerators

Agentic Tools

Registry and deep docs for Claude Code plugins, portable skills, and Pi extensions.

Public site source flow

engineering-websitepages · layouts · styles · blog · deploy

agent-skills-marketplaceSKILL.md · Pi READMEs · marketplace.json

→

engineering.diversio.comone site · build-time content join

Run the engineering website on your machine

cd engineering-website
npm install --package-lock=false
cp ../agent-skills-marketplace/website/src/data/marketplace.json \
  src/data/marketplace.json
npm run build
npm run dev   # http://localhost:4321

Source-of-truth ruleEdit skill docs, Pi docs, and catalog metadata in agent-skills-marketplace. Then copy marketplace.json and rebuild the website. Treat engineering-website/src/data/marketplace.json as a generated copy.

Read the engineering guidelines in this order

The Drive contains migrated ClickUp and Confluence pages. The useful pages teach team habits and dashboard rules; several command snippets describe an older toolchain. Follow the order and warnings below.

Open Diversio Knowledge Base Open Engineering Playbooks

Current source winsUse repository AGENTS.md, README.md, repo-local docs, and the GitHub-first workflow for commands and policy. Ask in #engineering when a Drive page conflicts with them.

Core reading for the first week

Engineer Onboarding

Start with team communication: standups, public Slack questions, huddles, documentation updates, first tasks, and knowledge-sharing sessions. Confirm meeting cadence in Slack because the page came from ClickUp.

Working on a Feature or Bug

Learn the engineering baseline: ask for early feedback, split large features, protect data integrity, design for downtime, check query performance, prove bugs with failing tests, enforce backend authorization, and test with an adversarial mindset.

Code Reviewing

Review early, communicate delays, give partial feedback on large diffs, separate blockers from suggestions, remove debug code, look for missing tests, and keep comments constructive. Use Pi review workflows for another pass; a human reviewer owns the decision.

Frontend Conventions

Keep the sections on semantic HTML, meaningful tests, shared types, design tokens, readable JSX, and Storybook. Ignore its ClickUp branch names and old sandbox or release steps; frontend/AGENTS.md and current package scripts replace them.

Learn the daily tools next

The Drive tool pages explain the ideas. The repository files in the last column define the commands and configuration Aryaman should use.

Order	Tool and reading	What it does	Current Diversio command or source
05	uv	Installs the pinned Python version, creates the backend environment, installs locked dependencies, and runs Python tools inside it.	`cd backend && uv sync` Then prefer `.bin/*` wrappers. Read `backend/UV_MIGRATION_GUIDE.md`; use this guide’s Arch instructions instead of its macOS section.
06	Just	Gives long project commands short, named recipes. The root justfile controls the full stack; the backend justfile owns backend recipes.	`just dev` Read `docs/justfile-primer.md` and `docs/dev-runner.md`.
07	Ruff + Code Quality	Formats Python and catches lint errors such as unused imports before CI.	`cd backend` `.bin/ruff format <changed-files>` `.bin/ruff check --fix <changed-files>` Current rules live in `backend/pyproject.toml`.
08	ty + pre-commit No current Drive page	`ty` checks Python types. Pre-commit runs repository checks before Git accepts a commit or push.	`cd backend` `.bin/ty check .` `just install-hooks` Read `backend/docs/quality/gates.md`.
09	Crafting Dev tips & tricks	Introduces Crafting sandboxes, `cs`, file copies, build logs, and sandbox databases.	Use current Crafting docs and `uv run scripts/cs_db_download.py`. Ignore the page’s Poetry-based RQ restart command and use current sandbox process controls.

Do not mix generationsFor this backend, uv replaces Poetry and project-level pyenv use; ty replaces Mypy. Do not run bare pip install or weaken pyproject.toml to silence Ruff.

Read before dashboard work

Order	Document	Reason
10	Bespoke Question Anonymity Choices	Learn question-level and choice-level protection before you inspect or change survey results.
11	[How-To] Dashboard Creation	Trace SurveyMonkey data, CSV preparation, Django objects, dominant groups, completion, and handoff. Pair with Muhammad’s videos and current code.
12	Dashboard QC Process	Check respondent counts, survey mappings, heatmaps, bespoke questions, prior-year comparisons, and Slack QC output.
13	Dashboard Creation FAQs	Use its topic index for special roles, tenure, office, demographics, free text, and bespoke requests. Some links still point to ClickUp.

Production guardrailPair with an experienced engineer before you run a production Django Admin action, management command, SurveyMonkey download, database change, or credential flow from a migrated page. Use approved backups and current runbooks.

Task-specific references

Score context

Inclusion Scores

Read for old score ranges and participation context. Confirm current values with analytics owners before you use them in product logic.

Benchmark task

Static Benchmark Data Info

Use for benchmark lineage and source notes when a ticket names a benchmark. Confirm the active backend record and generation date.

Product context

User Personas

Use the CHRO, DEI lead, and department-head roles as a starting point. The migrated page leaves several persona fields blank.

Documents outside the main reading path

Drive page	Reason to defer it	Use instead
Software Developer	Lists ClickUp, Postman, individual repository clones, and an old team roster.	GitHub-first docs, Bruno, monolith worktrees, and the access section in this guide.
Branch naming and Creating a Pull Request	Uses `clickup_GH-*` branches and old draft or status rules.	`docs/github-first-branch-and-pr-conventions.md` and repo PR templates.
Backend Local Setup, Postgres, and Muhammad’s setup notes	Uses macOS, Homebrew, local services, raw Django commands, and individual clones.	The Arch, anchor clone, env, and `just dev` sections in this guide.
Poetry and Mypy	These pages describe tools the backend no longer uses.	`uv`, `ty`, current `pyproject.toml`, pre-commit, and backend `.bin/*` wrappers.
Guidelines - Monolith	The page contains a heading and one unfinished sentence.	Root `README.md`, worktree docs, and the system map in this guide.
Universal Survey Processor draft	The named one-off script is absent from the current backend. Passing API keys on a command line also exposes them to shell history and process listings.	`backend/survey/services/survey_processing/`, its tests, and Muhammad’s current recordings.

Use the team’s Pi and cmux workflow

Pi gives the team a terminal coding agent. Diversio’s packages add a shared path for context, review, CI, shipping, and handoff. Learn that path during your first week so your work reaches review in the same shape as the rest of the team.

Read dev-workflow docs Read cmux integration docs Open Pi docs Open cmux

Agent

Pi

Reads the worktree, edits files, runs commands, and saves the session.

→

Shared process

dev-workflow + skills

Provides the team’s context, plan, review, CI, PR, and handoff commands.

→

Terminal lanes

cmux or tmux

cmux adds agent-aware panes on macOS. tmux gives the Arch workstation durable terminal panes.

Arch and cmuxThe official cmux application targets macOS. Use Pi inside tmux on Arch. The dev-workflow commands run in the current Pi session outside cmux; oh-my-pi pane commands require cmux. Do not make an unofficial Linux cmux port part of the team setup without team review.

Install on Arch

Install Pi and the Diversio packages

Shell: install Pi

# PNPM_HOME comes from the monolith bootstrap.
pnpm add -g --ignore-scripts \
  @earendil-works/pi-coding-agent
pi --version
pi

Inside Pi, run /login and choose a subscription or API-key provider. Run /quit after authentication.

Shell: install the team package and start work

pi install git:github.com/DiversioTeam/agent-skills-marketplace
pi list

# Start Pi inside a generated worktree, not the anchor clone.
cd "$HOME/work/diversio/monolith-<task>"
pi --name "gh-<issue>-<slug>"

Inside Pi

/reload
/workflow:help

Pi packages run with your user’s system access. Install reviewed packages from the team repository. Pi may ask you to trust a worktree before it loads project resources; confirm the path before you approve it.

tmux input on Arch

Preserve Pi keyboard shortcuts

tmux can collapse Shift+Enter into plain Enter. Pi recommends CSI-u extended keys with tmux 3.5 or newer.

~/.tmux.conf

tmux -V

grep -qxF 'set -g extended-keys on' "$HOME/.tmux.conf" || \
  echo 'set -g extended-keys on' >> "$HOME/.tmux.conf"
grep -qxF 'set -g extended-keys-format csi-u' "$HOME/.tmux.conf" || \
  echo 'set -g extended-keys-format csi-u' >> "$HOME/.tmux.conf"

# Run this before starting work; it closes existing tmux sessions.
tmux kill-server
tmux

just dev mode=tmux uses tmux for app services. You can keep Pi in another tmux session or terminal tab.

Team workflow from task to pull request

Moment	Pi command	Your goal
Join existing work	`/workflow:context`	Read the PR, branch, diff, and repository instructions before editing.
Map unfamiliar code	`/workflow:scout`	Find files, data flow, callers, tests, and risks.
Challenge the approach	`/workflow:plan` `/workflow:oracle`	Check assumptions and choose a small root-cause change.
Review your change	`/workflow:self` `/workflow:standards`	Reread the diff and run the repository’s quality rules.
Get another view	`/workflow:reviewer` `/workflow:parallel`	Ask for an independent review of correctness, tests, and complexity.
Prove and explain	`/workflow:docs` `/workflow:ci`	Update needed docs and inspect CI failures with logs.
Open or update the PR	`/workflow:ship`	Check CI, commit one change, write the PR description, and push.
Address review	`/workflow:pr-review-comments`	Fix feedback, rerun checks, resolve threads, and request review.
Pause or transfer work	`/workflow:handoff`	Leave the next engineer the branch, state, evidence, and next action.

Inside cmux on a team Mac

Explicit panes with oh-my-pi

/omp-split-right [prompt]: open a Pi lane beside the current lane.
/omp-split-down [prompt]: open a Pi lane below the current lane.
/omp-split-*-command <cmd>: run a shell command in a pane.
/omp-workspace --name "Review" [prompt]: open a named workspace for longer work.

oh-my-pi sends one native cmux notification when a Pi run waits, completes work, or fails.

Automatic workflow lanes

dev-workflow chooses a split when cmux can help

Inside cmux, an idle Pi session opens scout, oracle, reviewer, and parallel commands in a seeded split. The child receives the worktree, branch, Git status, recent context, prompt, and model. Outside cmux, the same command runs in the current Pi session.

One writer per worktreeUse extra lanes for reading, tests, research, and review. Ask one lane to edit files so two agents do not overwrite each other.

Read these during your first week

These posts explain the team’s context-first AI workflow, the monolith that supports it, and the agentic work that shortened CI.

Blog

How we use AI like engineers

Recover context before coding, keep diffs small, and leave risk decisions with engineers.

Blog

The Monolith That Made AI Actually Useful

See why one cross-repository workspace helps people and agents trace a change through the stack.

Blog

No Code by Hand

Read how the team used agentic work to reduce the engineering queue and shorten CI.

Claude Code and Bruno remain available

Claude Code marketplace

Claude Code

claude plugin marketplace add \
  DiversioTeam/agent-skills-marketplace

claude plugin install github-ticket@diversiotech
claude plugin install frontend@diversiotech
claude plugin install backend-atomic-commit@diversiotech
claude plugin install backend-pr-workflow@diversiotech

Bruno for API work

Open backend/bruno/ with Bruno’s Linux AppImage or CLI. Keep credentials out of Git. The team secret manager contains the approved bruno/.env.

Watch Muhammad’s handoff videos

Muhammad recorded the workflows he owned. Sign in with your Diversio Google Workspace account, then watch the recording that matches your task.

Open the video folder Open Muhammad’s onboarding PR notes

Setup and workflow

Start with local context

Local Mac setup with captions.mp4
Use the setup concepts, then follow this guide’s Arch commands.
Crafting how to.mp4
A five-minute process recording; the Drive title does not name a product area.

Debugging

Find evidence across tools

Slack logs Sentry & others.mp4
Use with the troubleshooting and observability sections.

Survey processing

See the legacy survey flows

Create survey surveyprocessingjob.mp4
SurveyProcessingJob creation.
ProcessingJobs w/ captions.mp4
ProcessingJob workflow with captions.
Legacy Survey CSV Method w/ captions.mp4
CSV route through the legacy survey method.
legacy survey method.mp4
Companion recording for the legacy survey flow.

Mac recording on ArchThe local setup video uses macOS. Keep its repository and application concepts; replace its package, Docker Desktop, clipboard, and terminal commands with the Arch steps in this guide.

Video descriptionsThe labels above paraphrase the Drive filenames. Open a recording for its full scope and current context.

Dashboard takeover: where to start reading

These paths cover the main dashboard and survey automation work. Read the nearest tests before you change behavior.

Backend · survey operations

Automatic survey processing

backend/survey/services/survey_processing/
data sources, mappings, response creation, enrichment, parsing
backend/survey/admin/survey_processing.py
admin workflow and validation surfaces
backend/dashboardapp/tasks/benchmark_tasks.py
parallel benchmark jobs, cache, finalization

Product · analytics surfaces

Culture and Inclusion dashboards

backend/dashboardapp/dashboard_functions_v2/analyze/inclusion/
heatmaps, demographic dimensions, score details
frontend/src/components/AnalyzeV2/Inclusion/
main dashboard UI, panels, comments, filters
design-system/src/components/core/BarChart/
shared chart rendering and Brush behavior

First tracePick one visible dashboard number. Follow it from the React component to its hook or API action, then through the Django endpoint, tenant-scoped query, and fixture-backed test. Tracing a value shows how the pieces connect.

Multi-tenant safetyScope each backend query to its tenant. Keep fixture families aligned; mixing dashboardapp and survey tenant fixtures causes false 404s and misleading test failures.

Inspect runtime state before troubleshooting

The Crafting download helper cannot list dumps

Install cs from Crafting’s Download menu, then verify login and access to dev-dump-upload. Ask the database owner for access or the ZIP password; do not copy a password into the guide.

Crafting auth

cs info
cs login
uv run scripts/cs_db_download.py

just dev says a port is busy

Let it remap, stop the other worktree, or choose explicit ports.

Ports

just dev status
just dev backend_port=8010 diversio_frontend_port=3010
DIVERSIO_MONOLITH_STRICT_PORTS=1 just dev

Frontend install returns 401

Confirm your own package token, SSO authorization, and user pnpm auth.

Packages auth

test -n "$NPM_TOKEN" && echo token-present
test "$(pnpm config get //npm.pkg.github.com/:_authToken)" != "undefined" && \
  echo package-auth-configured
(cd frontend && pnpm install --frozen-lockfile)

Frontend starts but points at the wrong backend

Sync API env

just dev sync-api-env
rg "VITE_DASHBOARD_API_ROOT|VITE_API_URL" \
  frontend/.env.local optimo-frontend/.env.local
just dev status

Database dump did not import

PostgreSQL reads init files only for a fresh volume. For an existing local database, stop app processes and use the backend Docker recipes below.

Database

./scripts/docker-worktree-stack.sh context
just dev stop
(cd backend && just up)
(cd backend && just load-db /absolute/path/to/approved-dump.sql)
(cd backend && just down)

Backend host run fails around WeasyPrint

Do not use the macOS DYLD workaround. Verify the Arch libraries and then test the import through uv.

WeasyPrint

sudo pacman -S --needed glib2 cairo pango harfbuzz \
  gobject-introspection fontconfig gdk-pixbuf2 libffi
(cd backend && uv run python -c \
  "from weasyprint import HTML; print('WeasyPrint works')")

Full Docker frontend fails on Arch

Check architecture before chasing app code. The default Diversio image is arm64; the default Optimo image is amd64.

Architecture

uname -m
just dev docker-status
just dev docker-logs service=diversio-frontend

# Recommended recovery: return to host frontend.
just dev docker-down
just dev service=diversio-frontend

local-ci failed or status did not publish

Inspect local-ci

local-ci runs
local-ci show <run-id>
local-ci logs <run-id>
local-ci logs <run-id> --step <step-id>
git status --short
gh auth status

Publishing requires a clean tree that still matches the saved run.

Submodule shows detached or changed at monolith root

create_worktree.py leaves submodules detached. Create a branch in a submodule before you commit there. A changed root entry shows a gitlink pointer; inspect the file diff inside the submodule.

Submodule state

git submodule status
git diff --submodule=log -- backend
(cd backend && git status && git switch -c feature/<issue>-<slug> origin/dev)

Evidence and open questions

Status	Claim	Evidence
Confirmed	Repo layout, branch map, setup scripts, just dev modes, ports, worktrees, and local-ci loops	Monolith `README.md`, root `justfile`, `docs/dev-runner.md`, helper scripts
Confirmed	Backend Python 3.14 + uv; frontend/Optimo/design-system pnpm and Node pins	Each repo’s `AGENTS.md`, `package.json`, `.nvmrc`, `pyproject.toml`
Confirmed	Current Docker frontend defaults are Diversio arm64 and Optimo amd64	`scripts/compose.frontends.dev.yaml` and Docker worktree docs
Confirmed	Live engineering site content and engineering-website/ASM source split	Live pages plus both repositories’ architecture and route ownership docs
Confirmed	Pi installation, provider login, package security, project trust, and tmux extended-key setup	Installed Pi `README.md`, `docs/quickstart.md`, `docs/packages.md`, and `docs/tmux.md`
Confirmed	Diversio Pi workflow commands, automatic cmux splits, explicit oh-my-pi commands, and inline fallback outside cmux	Agent Skills Marketplace `dev-workflow` and `oh-my-pi` package docs
Confirmed	The official cmux terminal targets macOS	`manaflow-ai/cmux` repository description and project docs
Confirmed	The canonical clone stays untouched, owns baseline env files, and creates a worktree for each task with `uv run scripts/create_worktree.py`	Diversio team workflow clarification plus the script’s documented env-copy behavior
Confirmed	Muhammad’s seven handoff videos and onboarding PR notes	Live Google Drive folder, individual Drive file IDs, and the supplied screenshot
Confirmed	Drive guidelines contain migrated ClickUp and Confluence content; several setup, branch, package, and tool instructions conflict with the current repositories	Exported text from Engineer Onboarding, Software Developer, development, backend, frontend, Crafting, dashboard, QC, anonymity, and Muhammad handoff pages
Confirmed	Crafting stores password-protected dump ZIPs in `dev-dump-upload`; the download helper lists, downloads, and extracts them	`scripts/cs_db_download.py`, `scripts/cs_db_snapshot.py`, Crafting’s current CLI guide, and the Drive Crafting tips page
Recommended	The ordered Drive reading path and the list of deferred pages	Full document review cross-checked against current repository docs and code paths
Confirmed	Arch package names in the host install command	Official Arch package search API, checked August 31, 2026 ET
Recommended	Host-run frontends + Docker PostgreSQL/Redis as the first Arch path	Reasoned from current cross-architecture frontend images and first-day simplicity
Team verification	Approved local env values, Crafting access and dump password, secret-manager entries, role-specific cloud/deploy access	The team controls these outside the repositories
Team verification	Native full-Docker Diversio image for the engineer’s exact Arch architecture	Current documentation lists no team-approved Linux override

Primary local sources

README.md: monolith and worktrees
docs/dev-runner.md: complete just dev behavior
docs/dev-guides/local-ci.md: validation loop
backend/AGENTS.md: product boundary and wrappers
frontend/AGENTS.md: dashboard architecture

Primary public sources

engineering.diversio.com/how-we-work
engineering.diversio.com/agentic-tools
engineering.diversio.com/pi/dev-workflow
engineering.diversio.com/pi/oh-my-pi
pi.dev
github.com/manaflow-ai/cmux
archlinux.org/packages

The whole setup in plain English

Keep the main clone as your clean source

Code and Git

Browser to database

Local tool belt

Guide topics at a glance

Your building pass

Your lab equipment

Your local settings

Your service conductor

One campus, several departments

Your draft and submission

Your team handbook

The team’s agent workflow

Follow one visible value

Read the state before changing code

Watch, trace, and ship

Two products share one backend

A dashboard request from click to response

React frontend

Django API

PostgreSQL / RQ

Know the system before installing it

Design system

Monolith + just

Engineering website

Get access before debugging setup

Access checklist

Do not solve access by sharing secrets

Install the Arch-native foundation

Install host packages

Enable Docker Engine

Replace macOS commands on Arch

Authenticate GitHub and package access

SSH key

gh auth login

NPM_TOKEN

Generate your own NPM_TOKEN

Create the token

Authorize the organization

Store your token

Create the permanent anchor clone

Clone the monolith and its submodules once

Install local-ci without Homebrew

Run the monolith toolchain bootstrap

Store safe PATH configuration

Seed env files in the anchor, then create a worktree

Store canonical env files, then generate the real checkout

Install dependencies in the working monolith

Fetch the approved dump with the download helper

Fresh database volume

Existing database volume

Choose empty data or approved realistic data before first DB start

Empty database

Approved Crafting dump

Learn just dev: the local control plane

Worktree

Commands + ports

tmux / headless / Docker

Host app processes + Docker data services

Current frontend images are architecture-split

Use these commands

Runtime

Inspect

URLs

Use each repository for one clear job

Use repository commands

Follow each repository’s package manager

Daily Git, worktree, validation, and PR loop

Start from a GitHub issue

Return to the anchor and create a worktree

Create branches in repositories you will change

Run focused tests, then local-ci

Push, open a ready PR, and publish clean statuses

--no-github

run + publish

Inspect, do not assume

Use the Engineering website as a reference

How We Work

Systems

`gh auth login`

`NPM_TOKEN`

Generate your own `NPM_TOKEN`

Install `local-ci` without Homebrew

Learn `just dev`: the local control plane

`--no-github`

`run` + `publish`