DriftLock: Complete Setup Guide & Demo Walkthrough #11
NalinDalal
announced in
Announcements
Replies: 1 comment
DriftLock: Build Log
What We BuiltA self-maintaining API system that:
Architecture OverviewTech Stack
Development TimelinePhase 1: Foundation (Commits 1-3)Goal: Set up monorepo, database, basic webhook capture. # Initial setup
bun create turborepo DriftLock
cd DriftLock
# Packages created
packages/db/ # Drizzle ORM + PostgreSQL
packages/diff/ # Schema diffing logic
packages/webhook-capture/ # Webhook processing
apps/be/ # Hono backend API
apps/fe/ # React frontend
apps/webhook/ # Bun webhook serverKey decisions:
Phase 2: Schema Detection (Commits 4-6)Goal: Capture webhooks, extract schemas, detect drift. Schema extraction logic ( // Flatten nested objects to dot notation
// Input: { data: { object: { source: "tok_visa" } } }
// Output: { "data.object.source": "string" }
function flattenObject(obj: any, prefix = ''): Shape {
const result: Shape = {};
for (const [key, value] of Object.entries(obj)) {
const path = prefix ? `${prefix}.${key}` : key;
if (typeof value === 'object' && value !== null) {
Object.assign(result, flattenObject(value, path));
} else {
result[path] = { kind: typeof value as ShapeKind };
}
}
return result;
}Drift detection ( // Compare two schemas
function diffShapes(old: Shape, new: Shape): ShapeDiffResult {
const added = Object.keys(new).filter(k => !(k in old));
const removed = Object.keys(old).filter(k => !(k in new));
const typeChanged = Object.keys(old).filter(k =>
k in new && old[k].kind !== new[k].kind
);
// ...
}Challenges:
Phase 3: PR Creation (Commits 7-9)Goal: Scan repos, apply fixes, create GitHub PRs. File scanning ( // Scan repo for affected files
function scanForAffectedFiles(repoPath: string, works: FixWork[]) {
const files = readdirSync(repoPath, { recursive: true })
.filter(file => /\.(ts|tsx|js|jsx)$/.test(file))
.filter(file => !file.includes('node_modules'));
// Match fix patterns against file contents
for (const file of files) {
const content = readFileSync(join(repoPath, file), 'utf8');
for (const work of works) {
if (work.kind === 'field_rename' && work.from) {
const regex = new RegExp(`\\b${work.from}\\b`, 'g');
if (regex.test(content)) {
results.push({ filePath: file, fullPath: join(repoPath, file) });
}
}
}
}
}GitHub PR creation ( // Create branch, commit, push, open PR
async function createWebhookFixPR(input: WebhookPRInput) {
const branch = `driftlock/webhook-fix-${input.endpointId}`;
// 1. Create branch from main
await git.checkoutLocalBranch(branch);
// 2. Apply fixes to files
for (const file of affectedFiles) {
const newContent = applyFixesToSource(content, works);
writeFileSync(file.fullPath, newContent);
}
// 3. Commit and push
await git.add('.');
await git.commit('fix: webhook schema drift');
await git.push('origin', branch);
// 4. Create PR via GitHub API
await octokit.pulls.create({
owner, repo, title, body, head: branch, base: 'main'
});
}Challenges:
Phase 4: GitHub Integration (Commits 10-12)Goal: OAuth login, installation flow, webhook forwarding. OAuth flow: Installation flow: Challenges:
Phase 5: Polish & Testing (Commits 13-15)Goal: Settings UI, error handling, end-to-end testing. Settings page (
Testing with real data: # Created test repo with Stripe-like code
git init ~/repos/stripe-test
cd ~/repos/stripe-test
# Added payment.js with source field usage
cat > payment.js << 'EOF'
const paymentIntent = await stripe.paymentIntents.create({
amount: 2000,
currency: 'usd',
});
console.log(paymentIntent.source); // ← This line gets flagged
EOF
# Pushed to GitHub
git add . && git commit -m "Initial"
git remote add origin https://github.com/NalinDalal/stripe-test.git
git push -u origin mainTest flow: # 1. Send baseline
curl -X POST http://localhost:3001/webhooks/capture/stripe \
-H "Content-Type: application/json" \
-d '{"type":"payment_intent.succeeded","data":{"object":{"source":"tok_visa"}}}'
# 2. Send drift (source removed, payment_method added)
curl -X POST http://localhost:3001/webhooks/capture/stripe \
-H "Content-Type: application/json" \
-d '{"type":"payment_intent.succeeded","data":{"object":{"payment_method":"pm_visa"}}}'
# 3. Check GitHub for new PR
# Branch: driftlock/webhook-fix-stripe
# PR created automatically with TODO commentBugs We FixedBug 1: Reserved Word ConflictProblem: PostgreSQL column named Error: Fix: Renamed to -- Before
CREATE TABLE webhook_drifts (
current_schema JSONB -- ❌ Reserved word
);
-- After
CREATE TABLE webhook_drifts (
new_schema JSONB -- ✅ Safe
);Bug 2: Config Loading TimingProblem: Webhook server loaded config at module load, but DB wasn't ready. Error: Fix: Lazy initialization on first request. // Before
const config = await loadConfig(); // ❌ Runs at import time
// After
let config: WebhookConfig | null = null;
async function getConfig() {
if (!config) {
config = await loadConfig(); // ✅ Runs on first request
}
return config;
}Bug 3: Repo Path ResolutionProblem: Error: Fix: Store full absolute path in settings. // Wrong
repoPath: "stripe-test"
// Correct
repoPath: "/Users/nalindalal/repos/stripe-test"Bug 4: FixKind Not HandledProblem: Error: PR created with Fix: Added } else if (work.kind === 'custom' && work.field) {
const leaf = work.field.split('.').pop();
const regex = new RegExp(`\\b\\w+\\.\\b${leaf}\\b`, 'g');
if (regex.test(content)) {
results.push({ filePath: file, fullPath });
}
}Bug 5: GitHub Installation ReposProblem: Webhook handler used Error: Wrong owner/name stored in database. Fix: // Before
const owner = payload.repository.owner.login;
// After
const owner = payload.installation.account.login;Key Learnings1. Webhooks Are Tricky
2. Git Operations Need Care
3. GitHub Apps Are Complex
4. Schema Evolution Is Real
5. Local Development Is Hard with Webhooks
Code Statistics
What's Next
Try It Yourself# Clone
git clone https://github.com/nerdev-co/DriftLock.git
cd DriftLock
# Install
bun install
# Start DB
docker run -d --name driftlock-postgres \
-e POSTGRES_USER=driftlock \
-e POSTGRES_PASSWORD=driftlock \
-e POSTGRES_DB=driftlock \
-p 5432:5432 postgres:16
# Configure
cp .env.example .env
# Edit .env with your GitHub credentials
# Run
bun run devBuilt with frustration, caffeine, and a lot of |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
DriftLock: Complete Setup Guide & Demo Walkthrough
What is DriftLock?
DriftLock watches your API integrations and opens GitHub PRs when a vendor contract drifts. It captures webhooks, detects schema changes, scans your codebase for affected files, and automatically creates fix PRs.
Setup Guide
Prerequisites
1. Clone & Install
git clone https://github.com/nerdev-co/DriftLock.git cd DriftLock bun install2. Start Database
3. Configure Environment
Create
.envin root:4. Run Migrations
cd packages/db bun run migrate5. Start Development
This starts:
GitHub App Setup
Create GitHub OAuth App
DriftLockhttp://localhost:5173http://localhost:8787/api/auth/github/callbackCreate GitHub App
DriftLock-dev(must be unique)http://localhost:5173https://your-cloudflare-tunnel/webhooks/githubExpose Webhook Server
For local development, use Cloudflare Tunnel:
Copy the generated URL (e.g.,
https://portland-tue-downloaded-welding.trycloudflare.com) and update your GitHub App's webhook URL tohttps://your-tunnel-url/webhooks/github.Demo Walkthrough
Step 1: Login with GitHub
Step 2: Install GitHub App
Step 3: Configure Webhook Capture
your-usernameyour-repo/full/path/to/your/repoStep 4: Send Test Webhooks
Baseline (initial schema)
Response:
{ "status": "ok", "endpointId": "stripe", "eventType": "payment_intent.succeeded", "message": "Schema baseline recorded or unchanged" }Drift (schema changed)
Response:
{ "status": "drift_detected", "endpointId": "stripe", "eventType": "payment_intent.succeeded", "diff": { "added": ["data.object.payment_method"], "removed": ["data.object.source"], "typeChanged": [] }, "pr": "pending" }Step 5: Check the PR
After sending the drift payload, DriftLock:
sourcefield removed,payment_methodfield added/* TODO: field removed */comment where the field is usedCheck your GitHub repo for a new PR:
driftlock/webhook-fix-stripedriftlock: Fix payment_intent.succeeded webhook handlerHow the PR Creation Works
Architecture
Step-by-Step Flow
Webhook Capture (
apps/webhook/capture.ts)Drift Detection (
packages/webhook-capture/driftDetector.ts)Schema Storage (
packages/db/schema.ts)webhook_schemastabledata.object.source)Diff Calculation (
packages/diff/index.ts)SemanticChangeobjects:field_removed: Field no longer in responsefield_added: New field in responsetype_changed: Field type changedbecame_nullable: Field is now optionalFix Works Generation (
packages/diff/index.ts:fixWorksForDiff)field_removed→customkind (adds TODO comment)type_changed→type_coercionkindbecame_nullable→null_checkkindFile Scanning (
packages/webhook-capture/prCreator.ts:scanForAffectedFiles).ts,.tsx,.js,.jsxfiles in reponode_modulesFix Application (
packages/diff/index.ts:applyFixWork)field_rename: Find and replace old field name with newnull_check: Add?? fallbackto field accesscustom: Add TODO comment for removed fieldsGit Operations (
packages/webhook-capture/prCreator.ts:createWebhookFixPR)driftlock/webhook-fix-{endpoint}PR Contents
The generated PR includes:
driftlock: Fix {event_type} webhook handlerWebhook Server Endpoints
/webhooks/capture/:vendor/webhooks/capture/:vendor/webhooks/capture/:vendor/schema/webhooks/capture/:vendor/webhooks/githubDatabase Schema
webhook_schemasStores captured schema snapshots.
webhook_driftsStores detected drifts.
settingsKey-value store for configuration.
Troubleshooting
PR not created
[PR] no_matchesor[PR] no_fixable_files.ts,.tsx,.js,.jsxare scannedWebhook not received
Database errors
docker ps.envcd packages/db && bun run migrateNext Steps
Support
Generated by DriftLock — Self-maintaining APIs.
All reactions