Skip to content

Migrate Wealthboard from Next.js/SQLite to Go, PostgreSQL, and Vite - #16

Merged
wechuli merged 72 commits into
mainfrom
wechuli/migration-to-go
Sep 28, 2026
Merged

wechuli merged 72 commits into
mainfrom
wechuli/migration-to-go

Conversation

@wechuli

@wechuli wechuli commented Sep 28, 2026

Copy link
Copy Markdown
Owner

Replaces Wealthboard’s Next.js/Node.js backend and SQLite/Drizzle persistence with a Go/Chi HTTP service, PostgreSQL, and a React/Vite client.

The Go process now owns authentication, authorization, financial operations, and database access, exposes a versioned /api/v1 API, and serves the compiled frontend. Wealthboard remains self-hosted and local-first, without requiring a separate Node.js application server.

What changed

Backend and API

  • Introduced the Go service with configuration loading, connection pooling, structured request logging, graceful shutdown, and liveness/readiness endpoints.
  • Replaced Server Actions and Next.js API handlers with explicit JSON endpoints.
  • Added an OpenAPI contract and generated TypeScript API types.
  • Implemented owner-scoped reads and mutations across accounts, transactions, investments, goals, reports, settings, estate planning, imports, and exports.

PostgreSQL and financial operations

  • Replaced SQLite/Drizzle with PostgreSQL, append-only Goose migrations, sqlc-generated queries, and transactional Go services.
  • Ported financial calculations to checked integer and exact math/big arithmetic.
  • Carried forward balance and position replay, absolute valuations, atomic paired transfers, idempotent creation workflows, and ownership checks.
  • Ported investment conversion, reconciliation, corporate actions, account/investment imports, and versioned user portability.

Authentication

  • Ported local, OIDC-only, and hybrid authentication to Go.
  • Added session-bound CSRF protection, PostgreSQL-backed login throttling, password management, and operator password reset.
  • Implemented OIDC discovery, PKCE, callback verification, provisioning, linking, and reauthentication.
  • Added scoped personal API keys with one-time token disclosure, expiry, and revocation.

Frontend

  • Replaced the Next.js application with a strict-TypeScript React/Vite SPA.
  • Ported the existing navigation, forms, charts, responsive layouts, themes, and privacy controls.
  • Added authenticated session bootstrap, API-backed workflows, and route-level code splitting.
  • Retained offline-safe PWA behavior without caching private API responses or queuing financial mutations.

Deployment and operations

  • Added a non-root distroless Go application image and updated Compose/Kubernetes examples for PostgreSQL.
  • Kept Node.js outside the main application runtime; PDF/XLSX/DOCX extraction uses a separate bounded worker or direct-install parser.
  • Added PostgreSQL backup and maintenance-mode restore commands with a pre-restore safety dump and post-restore checks.
  • Updated development commands, CI validation gates, deployment guidance, and user/operator documentation.

Compatibility and cutover

  • This is a fresh PostgreSQL deployment boundary, not an in-place database migration. Legacy SQLite data is not imported, converted, or dual-written.
  • OIDC provider registrations must use ${APP_URL}/api/v1/auth/oidc/callback.
  • Removed the superseded Next.js/SQLite runtime after the recorded cutover gates passed, retaining historical migrations and frozen parity fixtures as evidence.

Verification

Added or updated Go unit tests, PostgreSQL integration and migration tests, frontend tests, Playwright workflows, two-user isolation checks, native backup/restore coverage, and retained parity regression checks. Documentation was also reconciled against the current implementation.

Known follow-ups

The runtime migration is complete, but remaining workflow gaps are explicitly documented: goal/estate valuation completeness, exchange-rate edit atomicity and cache refresh, some conversion/import UI capabilities, and AI import consent/cancellation behavior. These remain tracked follow-up work rather than being claimed as resolved by this migration.

- Add api.ts for handling API requests and error management.
- Create app.tsx to manage application state, authentication, and routing.
- Implement login and signup forms with validation using Zod.
- Add session management and logout functionality.
- Create format.ts for formatting currency values and add tests for it.
- Set up main.tsx for rendering the application with React Router.
- Add original styles for the application layout and components.
- Create types.ts for defining TypeScript types based on API schema.
- Configure Vite for development and testing with Vitest.
- Implemented SQL queries and Go functions for goals reports, including counting goals, retrieving goal details, and listing accounts and milestones.
- Added PrivacyBoundary and MoneyValue components to handle privacy mode for displaying monetary values.
- Created tests for MoneyValue and AppShell navigation to ensure proper functionality.
- Introduced reusable UI components such as PageHeader, Card, Badge, and LoadingState for consistent design.
- Developed a useResource hook for managing resource loading states in React components.
…e API functions for consistent formatting; improve PrivacyBoundary component structure; enhance shell component tests and structure; expand type definitions for clarity; update useResource hook for improved error handling.
- Removed unused imports and cleaned up the app.tsx file for better readability.
- Consolidated routing logic within the AuthenticatedApp component.
- Introduced core and feature pages for better organization of components.
- Added core pages: DashboardPage, AccountsPage, AccountDetailPage, TransactionsPage, and GoalsPage.
- Implemented feature pages: CategoriesPage, InstitutionsPage, InstrumentsPage, InstrumentDetailPage, EstatePage, and ReviewPage.
- Created a new SettingsPage component with API key management functionality.
- Developed APIKeysPanel for creating and revoking API keys, including tests for its functionality.
- Improved loading states and error handling across various components.
- Updated sidebar navigation to allow for overflow scrolling.
- Introduced PrivateValue component for masking sensitive information in privacy context.
- Improved APIKeysPanel tests for better readability and structure.
- Refactored SettingsPage for improved layout and readability, including better handling of API key operations.
- Added core page tests for TransactionsPage to ensure proper rendering of transaction data.
…ockerfile and docker-compose for improved service orchestration
…, institutions, and exchange rates

- Added MetadataMutations struct to handle database operations related to metadata.
- Implemented UpdateSettings method to update user settings with validation.
- Created CreateCategory, UpdateCategory, ArchiveCategory, and ReorderCategory methods for category management.
- Added CreateInstitution and UpdateInstitution methods for institution management.
- Implemented CreateExchangeRate and DeleteExchangeRate methods for exchange rate management.
- Added validation functions for settings, categories, institutions, and exchange rates.
- Introduced error handling for metadata operations with specific error types.
…tations

- Implement tests for goal mutations including creation, validation, and milestone handling.
- Add tests for investment mutations focusing on position events and PostgreSQL interactions.
- Create integration tests for ledger transactions, ensuring idempotency and ownership checks.
- Introduce tests for metadata mutations, validating input normalization and error handling.
- Ensure all tests cover various edge cases and enforce business logic invariants.
- Implemented CategoryManager and associated forms for creating, updating, and archiving categories.
- Added InstitutionManager and forms for managing institutions.
- Created GoalManager and GoalForm for managing financial goals and milestones.
- Developed InstrumentManager and InstrumentForm for handling financial instruments and their prices.
- Integrated validation using Zod and form handling with React Hook Form.
- Added unit tests for CategoryManager to ensure proper functionality.
… and maintainability; update InstrumentForm and InstrumentManager components similarly; enhance SettingsPage layout for better structure; modify InstitutionInput type for clearer formatting.
…ons; implement related data structures and services
- Created a web app manifest for Wealthboard to define app metadata and icons.
- Added an offline HTML page to inform users when the app is offline.
- Implemented a service worker to cache essential assets and handle fetch requests, providing offline functionality.
- Introduced extraction worker with package.json configuration.
- Implemented extraction worker daemon for handling document extraction requests.
- Added protocol tests for the extraction worker to ensure proper request handling and environment clearing.
- Created advanced account workflows for managing corporate actions, including form validation and submission.
- Developed estate workflows for managing estate plans, beneficiaries, directives, allocations, and snapshots.
- Implemented PWA manager for handling online/offline states and service worker updates.
- Improved formatting and readability in `feature-pages.tsx` by adjusting line breaks.
- Enhanced `PwaManager` in `pwa.tsx` for better readability and maintainability.
- Refactored `clearUserState` function in `shell.tsx` for improved clarity.
- Introduced new `ai-workflows.tsx` file to manage AI-related workflows, including settings, reviews, and document conversions.
- Added `phase5-settings.tsx` for data portability features, allowing users to download and restore data archives.
- Introduced createCorporateAction function to handle corporate actions with CSRF and idempotency headers.
- Updated api.test.ts to include tests for createCorporateAction.
- Refactored api.ts to include CorporateActionInput type and removed unused types.
- Updated estate-workflows.tsx to modify beneficiary and directive schemas.
- Enhanced feature-pages.tsx description for clarity.
- Improved phase5-settings.tsx for better user experience during data restoration.
- Added advanced-account-workflows.test.tsx for testing import workflows.
- Implemented pwa.test.tsx to ensure PWA safeguards are functioning correctly.
- Add LoginScreen and SignupScreen components for user authentication.
- Create forms with validation using zod for login and signup processes.
- Implement session management and error handling for authentication.
- Introduce button and form control components for consistent UI.
- Add currency selection and privacy toggle features.
- Implement PWA manager for service worker registration and installation prompts.
- Create theme provider and controls for managing light/dark mode preferences.
- Add utility functions for class name merging.
- Write tests for authentication components to ensure functionality.
- Updated tests in auth.test.tsx for better readability.
- Refactored LoginScreen and SignupScreen components for improved formatting and readability.
- Enhanced Button component to maintain consistent formatting.
- Cleaned up currency constants and ensured proper formatting in currencies.ts.
- Improved form controls with consistent formatting in form-controls.tsx.
- Refactored PrivacyToggle and PwaManager components for better readability.
- Streamlined ThemeProvider and ThemeControl components for consistency.
- Cleaned up utility functions in utils.ts for better readability.
- Enhanced AppShell and navigation components for improved structure and readability.
- Improved MobileLink and MobileMore components for better code clarity.
- Created a new migration journal file to track database changes with versioning and breakpoints.
- Added a new expected JSON fixture for phase six parity, detailing account balances, transactions, and goals.
- Introduced ESLint configuration for TypeScript and React, enhancing code quality and consistency.
- Implemented global CSS styles with a dark and light theme, including responsive design and print styles.
…stgreSQL

- Updated project context in GitHub instructions to reflect the new architecture.
- Removed Next.js build artifacts and adjusted Docker and Git ignore files.
- Reorganized service and database structure, moving to internal directories and PostgreSQL.
- Updated authentication and financial data instructions to align with new Go API.
- Revised frontend instructions to accommodate Vite and React changes.
- Enhanced testing instructions for Vitest and Playwright to match new project structure.
- Updated CI workflows to run Go, Vite, and extraction tests.
- Added cutover evidence documentation and legacy migration provenance.
- Implemented teardown script for E2E tests to clean up Docker resources.
- Updated README and architecture documentation to reflect the removal of legacy components.
- Added new images and updated existing screenshots to match the current UI.
…orkflowPage; update tests for new functionality
…dd new screenshots and improve test coverage
…rics; add period changes and account flow metrics
…ty; implement scenarios in goals reports and planning pages
…al ID matching; add tests for idempotency and legacy ID behavior
- Added support for position events in account detail page.
- Introduced PositionsCard component to display account positions with detailed information.
- Implemented InvestmentActivity component to show investment activity history with pagination.
- Created SQL queries for fetching investment position events.
- Updated API client to include new methods for retrieving position events.
- Added tests for PositionsCard and InvestmentActivity components to ensure proper rendering and functionality.
- Simplified type definitions in client.ts and types.ts
- Enhanced formatting and indentation in ledger-forms.tsx and position-account-details.tsx for better clarity
- Updated test files for position-account-details and core to improve structure and readability
- Refactored error handling and message formatting in forms.tsx and core.tsx
- Improved JSX structure in various components for better maintainability
…accuracy. Key changes include:

- Updated definitions and explanations in activity, data portability, estate planning, goals, investments, reports, and security guides.
- Improved descriptions of account transfers, import processes, and AI-assisted features.
- Clarified handling of fees, cash movements, and currency conversions in investment transactions.
- Enhanced warnings and notes regarding valuation limits, linked account values, and estate calculations.
- Revised instructions for managing exchange rates and API authentication.
- Added historical context to the migration brief, indicating the completion of the Go/Vite/PostgreSQL cutover.
- Updated PostgreSQL version in various configurations and documentation.
- Introduced a new service test database setup for improved isolation and cleanup.
- Refactored tests to utilize the new database setup, ensuring better reliability and clarity.
- Enhanced documentation to reflect changes in database requirements and upgrade procedures.
@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

Comment thread internal/service/core_reads.go Fixed
Comment thread internal/service/investment_mutations.go Fixed
Comment thread internal/api/oidc.go Fixed
@wechuli
wechuli merged commit 70d1ab6 into main Sep 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants