Skip to content

feat: add reusable source lists with nesting and RFC1918 preset - #5

Merged
mstrhakr merged 4 commits into
mainfrom
mstrhakr/issue3
Oct 6, 2026
Merged

mstrhakr merged 4 commits into
mainfrom
mstrhakr/issue3

Conversation

@mstrhakr

@mstrhakr mstrhakr commented Oct 6, 2026

Copy link
Copy Markdown
Owner

This pull request introduces a major new feature: reusable source lists for jobs, allowing sets of hostnames, IPs, CIDRs, and nested source references to be defined once and included in multiple jobs. The implementation includes database schema changes, new store logic for managing source lists, API and workflow documentation updates, and CI workflow extensions for Node.js-based UI tests. Below are the most important changes grouped by theme.

Reusable Source Lists Feature

  • Added a new SourceList type and comprehensive management logic in internal/store/source_lists.go, supporting creation, update, deletion (with cycle and reference checks), and expansion of nested source lists. This includes logic to prevent cycles, block deletion of referenced sources, and ensure atomic updates with locking and transactions.
  • Updated the SyncJob struct and related database logic to support an included_list_ids field, enabling jobs to reference reusable source lists in addition to inline entries. This includes schema migration, read/write logic, and validation. [1] [2] [3] [4] [5] [6]
  • Introduced a mutex (sourceMu) to serialize source list and job reference mutations for data integrity. [1] [2]

Documentation Updates

  • Expanded the README.md to document reusable source lists, including their usage, API endpoints, cycle prevention, RFC1918 preset, and operational details. The API section now describes CRUD endpoints for source lists and updates to job/resolve endpoints for including source lists. [1] [2] [3] [4] [5] [6]

Continuous Integration

  • Extended the CI workflow (.github/workflows/ci.yml) to set up Node.js 24 and run syntax and test checks for the UI's reusable source list functionality.

These changes collectively enable robust, reusable, and composable source management for jobs, improving maintainability, safety, and flexibility of input management.

Closes #3

Copilot AI balanced review requested due to automatic review settings October 6, 2026 15:11

This comment was marked as resolved.

@mstrhakr
mstrhakr merged commit 71d07ef into main Oct 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Reuse list in other list

2 participants