Skip to content

chore: remove blob download branch restriction #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Wenzel merged 1 commit into from
Sep 7, 2026
Merged

chore: remove blob download branch restriction #70

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 12 additions & 22 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

This is a GraphQL API server for OSWatcher that uses Neo4j as the database and provides filesystem analysis, diff computation, and search capabilities. The server includes:
- GraphQL API with WebSocket subscriptions
- REST API for blob downloads (authorization check disabled by default -- see below)
- REST API for blob downloads
- S3/MinIO integration for blob storage
- Windows registry value filtering for sensitive data (disabled by default -- see below)

**Open-source release note (2026-08-11):** blob download authorization and
sensitive registry value redaction are commented out by default (not
deleted) in `src/rest-routes.ts` and `src/index.ts`, so a self-hosted
instance has full functionality with no configuration required. See
`docs/reference/access-restrictions.md` for what each restriction does and
how to re-enable it.
**Open-source release note:** sensitive registry value redaction is commented
out by default (not deleted) in `src/index.ts`, so a self-hosted instance has
full functionality with no configuration required. See
`docs/reference/access-restrictions.md` for what it does and how to re-enable
it. Blob download authorization (branch-based blocking of Windows binaries) was
removed entirely on 2026-09-08; it is in git history.

## Development Commands

Expand Down Expand Up @@ -81,17 +81,12 @@ Custom resolvers for operations not auto-generated by Neo4j GraphQL:
- `search` - Searches filesystem paths across commits
- `traversePath` - Navigates filesystem tree structure

#### Blob Storage & Authorization
#### Blob Storage

**REST Routes (`src/rest-routes.ts`)**
- Express router for blob downloads at `/blob/:hash`
- Uses S3 SDK to fetch blobs from MinIO with authenticated requests
- Integrates authorization check before serving blobs

**Authorization (`src/blob-authorization.ts`)**
- `isBlobRestricted()` checks if blob belongs only to restricted branches
- Uses `CHECK_BLOB_RESTRICTED_QUERY` Cypher query from `src/queries.ts`
- Fails safe: treats unknown/error cases as restricted
- No per-blob access control: every stored blob is downloadable by hash

#### Data Processing

Expand Down Expand Up @@ -123,7 +118,6 @@ Custom resolvers for operations not auto-generated by Neo4j GraphQL:
See README.md for complete `.env` setup. Key variables:
- **Neo4j**: `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD`
- **Storage**: `OBJECT_STORAGE_URI`, `MINIO_ACCESS_KEY`, `MINIO_SECRET_KEY`, `MINIO_OBJECTS_BUCKET_NAME`
- **Authorization**: `RESTRICTED_BRANCH_NAME` - branch containing restricted blobs
- **Optional**: `NODE_ENV`, `SENSITIVE_REGISTRY_VALUES`

## Testing Strategy
Expand All @@ -143,16 +137,12 @@ The project includes comprehensive documentation in the `docs/` directory follow
- [Blob Download API](docs/reference/blob-api.md) - Complete REST API spec with error codes, rate limits, examples
- [Access Restrictions](docs/reference/access-restrictions.md) - Security controls and filtering mechanisms
- **How-To Guides** (`docs/how-to/`) - Step-by-step task instructions
- [Configure Blob Restrictions](docs/how-to/configure-blob-restrictions.md) - Set up branch-based access control
- [Integrate Blob API](docs/how-to/integrate-blob-api.md) - Frontend integration guide
- **Explanation** (`docs/explanation/`) - Conceptual deep dives
- [Blob Authorization Deep Dive](docs/explanation/blob-authorization.md) - Comprehensive explanation of restriction logic, Cypher queries, design decisions, and security considerations
- [Query Optimization](docs/explanation/query-optimization.md) - CALL {} subquery pattern for variable-length path queries

**When working on Cypher queries or the graph data model, ALWAYS consult [docs/reference/data-model.md](docs/reference/data-model.md) first.**

**When working on blob authorization or downloads, always consult the docs/ directory first.**

## Important Patterns

### Module System
Expand All @@ -168,9 +158,9 @@ The project includes comprehensive documentation in the `docs/` directory follow

### Security Patterns
- No per-user authentication. Auth0 was removed when the project went fully
open source; the access restrictions it guarded are disabled by default.
- Sensitive data filtering happens at response time, not query time
- Fail-safe approach: default to restrictive on errors
open source; the access restrictions it guarded are disabled or removed.
- Sensitive data filtering (registry redaction) happens at response time, not
query time, and fails open on error

### GraphQL Patterns
- Schema-first development with `type-defs.graphql`
Expand Down
5 changes: 0 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,6 @@ MINIO_ACCESS_KEY=
MINIO_SECRET_KEY=
MINIO_OBJECTS_BUCKET_NAME=objects

# Optional - Blob Access Control
# Unused unless the restriction commented out in src/rest-routes.ts is
# re-enabled (see docs/reference/access-restrictions.md)
RESTRICTED_BRANCH_NAME=

# Optional - Environment
NODE_ENV=development

Expand Down
38 changes: 1 addition & 37 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,14 @@ Reference documentation provides technical specifications and API details.

How-to guides show you how to solve specific problems and complete tasks.

- [Configure Blob Restrictions](./how-to/configure-blob-restrictions.md) - Set up branch-based blob access control
- [Integrate Blob API (Frontend)](./how-to/integrate-blob-api.md) - Use the blob API from your frontend application

### 💡 [Explanation](./explanation/)
*Understanding-oriented - Conceptual information*

Explanation documentation helps you understand the concepts and design decisions.

- [Blob Authorization Deep Dive](./explanation/blob-authorization.md) - How blob restrictions work internally
- [Query Optimization](./explanation/query-optimization.md) - CALL {} subquery pattern for variable-length path queries

### 🎓 Tutorials
*Learning-oriented - Step-by-step learning paths*
Expand All @@ -37,49 +36,14 @@ Explanation documentation helps you understand the concepts and design decisions

### For Developers

**Just getting started?**
- Read [Blob Authorization Explanation](./explanation/blob-authorization.md) to understand why restrictions exist
- Follow [Configure Blob Restrictions](./how-to/configure-blob-restrictions.md) to set up your environment

**Integrating the API?**
- Check [Blob API Reference](./reference/blob-api.md) for complete endpoint specifications
- Review [Access Restrictions](./reference/access-restrictions.md) to understand security controls
- Follow [Frontend Integration Guide](./how-to/integrate-blob-api.md) for client implementation

**Troubleshooting?**
- See troubleshooting section in [Configuration Guide](./how-to/configure-blob-restrictions.md#troubleshooting)
- Check error codes in [API Reference](./reference/blob-api.md#error-responses)

### For System Administrators

**Configuring restrictions:**
1. Set `OBJECT_STORAGE_URI` and `RESTRICTED_BRANCH_NAME` in `.env`
2. Verify with test requests (see [Configuration Guide](./how-to/configure-blob-restrictions.md#step-3-test-blob-restrictions))
3. Monitor logs for authorization decisions

**Security considerations:**
- Review [Security Considerations](./explanation/blob-authorization.md#security-considerations)
- Configure S3 bucket policies to prevent direct access
- Monitor 403 responses for unauthorized access attempts

## Feature Overview: Blob Download Restrictions

### Problem
Windows binaries cannot be redistributed due to Microsoft licensing restrictions, but the system needs to allow exploration and downloading of Linux files.

### Solution
A two-tier approach:
1. **GraphQL API** - Returns blob hashes freely for exploration (browsing, searching, diffing)
2. **REST API** - Controls actual downloads with branch-based authorization

### Key Concepts

**Branch-Based Restrictions**: Blobs are restricted if they belong exclusively to a configured restricted branch (e.g., `windows-10`).

**Fail-Safe Default**: If authorization check fails, access is denied (better to block accidentally than allow).

**Streaming Architecture**: Large files are streamed directly from object storage without buffering in memory.

## Contributing to Documentation

When adding new documentation:
Expand Down
Loading
Loading