Merged
Conversation
…ility - Clear decompressed JSON string immediately after parsing to free memory - Clear serialized state JSON after checksum verification to reduce memory peak - Clear all existing multi-instance objects before importing snapshot to avoid conflicts - Add clearing of numerous additional multi-instance object indices for hot-reload robustness - Modify existing singleton objects in place instead of removing them - Reset fork_db before seeding head block to prevent stale entries on reload - Improve logging messages for snapshot import phases
- Changed SHARED_LIBS setting from ON to OFF in build-linux.sh - Adjusted build configuration to not use shared libraries on Linux - Ensured consistent build behavior by modifying build script flag
…lity - Improve IP address extraction reliability in peer_connection component - Add robust try-catch fallback mechanisms in peer statistics logging - Strengthen connection lifecycle management with enhanced exception safety - Enhance message queuing, priority handling, and multiplexing with better error recovery - Improve metrics, reputation tracking, and peer database handling with enhanced logging - Harden message_oriented_connection with improved logging and error monitoring - Refine stcp_socket with enhanced error handling during ECDH key exchange and AES setup - Update handshake protocol to support reliable IP extraction and enhanced logging - Increase overall stability and monitoring in node-level peer orchestration and connection management fix(database): improve memory allocation error handling in blockchain layer - Enhance handling of boost::interprocess::bad_alloc exceptions in push_block() - Implement deferred shared memory resize mechanism with proper lock acquisition - Avoid throwing on memory exhaustion to maintain peer connectivity - Add detailed memory pressure logging during peer connectivity operations - Safely schedule deferred resize and preserve memory state for automatic recovery
- Add block number to the log message for better identification - Enhance debug information when block data is unavailable in DLT mode
- Replace wlog with ilog for shared memory resize messages with yellow color codes - Log actual used memory before and after resize instead of free memory - Clarify deferred and immediate resize log messages with usage and total memory info - Add used memory calculation before and after resizing in multiple places - Update deferred resize logs to mirror immediate resize format and details
…ration search - Introduce block-log-viewer.js for interactive terminal UI browsing of block logs - Implement navigation commands including fast next/prev with non-free ops via bitmask - Add scanning command to build and save bitmask marking blocks with non-free operations - Support string search in full operation JSON with export to JSON file functionality - Enhance documentation with usage instructions and bitmask file format details - Update index.md to reference block-log tools: reader, viewer, bitmask, search and export
- Add CLI option --reader to specify the path of block-log-reader.js module - Implement fallback search for block-log-reader.js in script directory and BLOCK_LOG_READER env var - Improve error messages when block-log-reader module is not found - Enhance usage information to document new --reader option and path argument formats - Auto-detect and switch to DLT mode if input path is a directory containing only dlt_block_log - Validate existence of block_log or dlt_block_log when input path is a directory - Refactor main argument parsing to handle directory and file inputs more flexibly
- Introduce batchSearch function to process searches in large blocks iteratively - Add lastSearch state to track last search type and term - Implement 'c' command to continue the last search operation - Optimize searchOpForward and searchStringForward using batchSearch for better memory usage - Add export feature to stream matching operations to JSON file with buffer and bigint serialization handling - Provide progress updates and handle large exports incrementally to prevent OOM - Improve block skipping logic using bitmask and lightweight transaction checks - Enhance CLI help to include the new 'continue last search' command option
…ety checks - Introduce base58 encoding for public key conversion with VIZ prefix - Add function to convert compressed public keys to VIZ address string - Implement safety caps on readVector and readOperation to prevent OOM and invalid types - Add detailed error handling and logging during operation deserialization - Extend operation readers for many missing and deprecated operations (IDs 0-63) - Enhance readSignedBlock to safely read transactions with error handling and limits - Export publicKeyToString and base58Encode utilities for external use feat(block-log-viewer): improve operation searching and raw block scanning - Add case-insensitive ASCII byte search utility for raw searches - Implement fast raw ASCII byte search forward without full deserialization - Improve operation forward search to use name includes with correct casing - Replace JSON.stringify with zero-allocation recursive search on operation data - Add deserialization helper from raw buffer with error handling - Provide verbose debug output for operation byte consumption during parsing - Enhance user commands with raw ASCII search capability (R command)
…commands - Add parseSearch function to handle '=' prefix for exact match detection - Update deepIncludes and findOpsByString to support exact matching - Modify searchStringForward and searchExport to accept exact match parameter - Enhance help output with instructions on using '=' prefix for exact matches - Update command parsing to handle exact match flags for S and e commands - Refactor continuation command to pass exact flag for repeated searches - Improve console messages to indicate if search/export is exact or substring match - Fix fs.writeSync calls to use consistent argument style without zero offsets
…nd operations - Add definitions for public_key_type, flat_set, flat_map, optional, and extensions_type - Refine common_types: asset wire_size fixed to 16 bytes, change account_auths and key_auths to flat_map - Add chain_properties_init and versioned_chain_properties with detailed variant fields - Introduce multiple new operation schemas including invite operations, subscription, account sale, and virtual operations - Enhance file_formats with block_log_bitmask and search_export formats for improved indexing and export - Expand viewer_commands with advanced search (exact match), raw search, export, and navigation by non-free operations - Update block-log-spec.md with detailed descriptions and examples of new types, wire formats, and operations - Improve user guidance on search modes and bitmask usage in block-log-viewer commands
- Restore fork_db head to original chain tip after pop_block and apply_block - Update fork_db head to new chain tip after successful fork switch - Prevent _head from being stuck at fork point, ensuring correct synopsis fetching fix(network): handle silently rejected blocks in P2P sync and processing - Check acceptance status of blocks handled by delegate in sync and regular processing - Log warning and throw unlinkable_block_exception on silent rejection - Soft-ban peers sending blocks from unknown forks to prevent dead forks requests
…o enable peer soft-ban - Throw unlinkable_block_exception when a block's parent is not in fork_db instead of silently rejecting - Update fork_db handling to distinguish dead forks from micro-forks - Modify node.cpp to log block acceptance and handle false returns without error for non-dead-fork cases - Convert unlinkable_block_exception in p2p_plugin to network exception for proper peer soft-ban or resync - Improve logging to clearly indicate block rejection reasons related to dead forks and unknown parents
…plication - Replace returning false on memory exhaustion with deferred_resize_exception throw - Declare deferred_resize_exception in chain and network exception hierarchies - Catch deferred_resize_exception in network node and p2p plugin to defer block acceptance - Add logs indicating shared memory resize and block retry on next push - Prevent peer penalization or block marking during deferred resize condition
…linkable_block_exception - Introduce unlinkable_block_exception for proper dead fork detection in fork database - Modify fork_database::push_block() to throw unlinkable_block_exception on linking failure - Cache unlinked blocks and improve logging for fork linking attempts - Improve fork switching logic to handle unlinkable_block_exception and avoid processing dead forks - Enhance fork switching with emergency consensus tie-breaking and safe branch comparison - Update database exceptions to integrate enhanced fork handling and detection feat(network): implement P2P soft-ban and exception propagation for better peer handling - Add soft-ban state fields (fork_rejected_until, inhibit_fetching_sync_blocks) in peer_connection - Convert unlinkable_block_exception to network exceptions for soft-ban enforcement - Enhance peer_connection and node with soft-ban logic on fork rejection scenarios - Improve P2P block processing with exception handling and peer reputation management - Strengthen connection lifecycle management with robust exception safety and soft-ban support - Integrate chain exception propagation into P2P layer for better network reliability
- Introduce ANSI color codes for red text to highlight ban messages - Log ban notifications with red color and detailed info including timestamp and reason - Apply colorized logs to soft-bans for old fork sync blocks, rejected sync blocks, unlinkable blocks - Enhance visibility of ban messages for blocks too old or causing fork rejection - Maintain existing ban logic while improving console output readability
- Bump fc subproject commit from 7ada0a5 to fa5b500 - Sync thirdparty dependencies with latest fc changes
…ergency consensus tie-breaking - Implement unlinkable_block_exception throwing for accurate dead fork detection - Improve fork switching logic with deterministic hash-based tie-breaking for emergency mode - Add emergency consensus mode with deterministic block selection criteria - Enhance fork database error handling and logging for better diagnostics - Ensure consistent block selection across nodes to prevent split-brain during emergencies
…stalls - Add early rejection of blocks whose parent is unknown and block number is ahead of head - Prevent fork_db from throwing unlinkable_block_exception for these blocks - Allow blocks linking directly to head for correct sync progression - In network node, handle unlinkable_block_exception by soft-banning stale forks below or at head - Restart sync instead of banning when unlinkable block is ahead of current head to catch up - Introduce deferred resize flag for handling shared memory resize during block sync - Restart synchronization with all peers after deferred resize or ahead unlinkable blocks to refetch missing blocks and avoid sync stalls
…eptions - Added catch for `unlinkable_block_exception` in sync path with distinction for block position relative to head to decide soft-ban or sync restart - Fixed `deferred_resize_exception` handling to restart sync without soft-banning peers, preventing loss of blocks due to local shared memory resize conditions - Restored early rejection of far-ahead blocks with unknown parent in `_push_block` to silently reject and avoid sync restart storms - Documented comprehensive soft-ban triggers with explicit actions for sync and broadcast code paths to improve peer classification and prevent infinite sync loops - Enhanced flags auto-reset mechanism after soft-ban expiration and added monitoring info - Clarified handling rules to avoid cascading disconnects during emergency consensus mode - Updated snapshot-plugin docs with detailed early-rejection and deferred-resize logic to prevent sync stalls and maintain sequential sync progress
- Add support for setting trusted peer endpoints with reduced soft-ban times - Trusted peers (from trusted-snapshot-peer config) get 5-minute bans instead of 1 hour - Modify soft-ban logic to apply different durations based on peer trust status - Update documentation to reflect new soft-ban behavior and peer classification - Integrate snapshot plugin with p2p plugin to register trusted snapshot peer IPs - Improve logging to show ban duration per peer dynamically - Enhance peer handling to prevent cascading failures and support faster recovery from transient errors
…s and sync loops - Implement intelligent early block rejection for far-ahead blocks with unknown parents - Prevent fork database exceptions by rejecting blocks at or below head on dead forks early - Avoid sync restart loops during snapshot imports by silently rejecting invalid blocks - Ensure safe acceptance of blocks whose previous equals the head block ID - Integrate early rejection logic with fork database push_block flow - Enhance P2P synchronization to handle unlinkable blocks with improved classification - Support deterministic tie-breaking for emergency consensus in fork switching - Maintain stability by avoiding unnecessary fork database operations and excessive processing
- Added graphene::snapshot to target_link_libraries in p2p plugin CMakeLists.txt - Ensured snapshot functionality is included in p2p build process
- Replace debug logs with info logs for sync restart reasons - Add detailed info logs for sync start with peer and block acceptance - Include current head block number and gap in block pushing logs - Change error logs to warnings or errors with clearer messages - Log start and end of sync mode with relevant block numbers - Refine exception handling logs for better clarity in p2p_plugin - Ensure consistent and informative logging for sync and block events
…and frequency - Use info-level logging for sync restart reasons to improve visibility - Add block processing logs with current head block numbers and gap calculations - Increase logging frequency from every 10,000 to every 500 blocks for finer progress monitoring - Enhance logging system documentation with ANSI color codes for operational clarity - Prevent duplicate sync start messages with guard variable for consistent reporting fix(database): improve fork database exception prevention via early rejection - Implement intelligent early rejection for blocks far ahead with unknown parents - Prevent fork database exceptions and sync restart loops during snapshot imports - Reject dead fork blocks at or below head with unlinkable_block_exception - Properly classify unlinkable blocks to support P2P soft-ban handling - Enhance error propagation to improve fork switching and synchronization stability feat(network): add dual-tier trusted peer support with soft-ban improvements - Introduce trusted-snapshot-peer configuration for trusted peer identification - Apply 5-minute soft-ban duration for trusted peers versus 1-hour for regular peers - Integrate trusted peer management across P2P and snapshot plugins for consistency - Enhance peer trust detection and soft-ban duration calculation for operational efficiency - Add ANSI color-coded ban notifications for improved visibility during soft-ban events
- Use <data-dir>/snapshots as default if --snapshot-dir is not specified - Ensure snapshot directory exists by creating it if missing - Update auto-latest snapshot discovery to handle the default directory properly - Remove fallback to current directory for snapshot operations - Modify logging to reflect actual snapshot directory used - Clarify --snapshot-dir option description with default path information
…ility - Updated block pushing log message to appear in gray color - Colored new peer connection log message in yellow - Added white color define for p2p stats console log messages - Updated transaction receipt log message to use white color for clarity - Preserved existing log message contents while enhancing readability with colors
- Introduce resync() method in network node to restart synchronization with all connected peers - Implement stale sync detection in p2p_plugin that triggers when no blocks received within configurable timeout (default 120 seconds) - Reset sync from last irreversible block and force resync on stale sync detection to recover node state - Reconnect all seed nodes to ensure network connectivity during recovery - Add command line options to enable stale sync detection and configure timeout - Schedule periodic stale sync check task while enabled - Safely cancel stale sync task on plugin shutdown to prevent dangling tasks
…eers - Default snapshot directory changed to <data_dir>/snapshots from current working directory - Added automatic creation of default snapshot directory if missing - Integrated trusted snapshot peer configuration for improved bootstrap - Documented snapshot serving settings and anti-spam protection features - Included configuration examples for snapshot, trusted peers, and serving options - Updated troubleshooting guide with snapshot directory, file integrity, and peer connectivity checks - Enhanced network synchronization logging with current head block numbers and gap calculations - Added trusted peer support with reduced soft-ban duration in P2P network configuration - Improved P2P plugin integration with snapshot plugin for trusted peer endpoint registration - Updated witness configuration defaults for more accurate participation calculations and reliability
- Changed resync() method to virtual in node.hpp header - Added resync() declaration in node.cpp source file - Improved extensibility for resync behavior in derived classes
…ogging - Introduce programmatic synchronization control via the new resync() method - Update network library documentation to include resync functionality and usage scenarios - Enhance peer statistics system documentation with improved IP address extraction and conversion optimization - Add detailed troubleshooting guidance for synchronization stalls and resync usage - Integrate resync method description in node management and P2P plugin sections - Document automatic network recovery integration using resync in the P2P plugin - Add configuration options related to stale sync detection and resync control - Fix DENY_UNTRUSTED message formatting in Snapshot Plugin System documentation
- Change debug log to info log in json_rpc plugin for better visibility - Adjust info logs to debug logs in p2p plugin to reduce verbosity during sync - Ensure consistent log level usage across both plugins
… control - Add detailed documentation on the new virtual resync() method enabling manual sync restart - Explain virtual design for extensibility by derived classes including simulated_network override - Update node management and synchronization sections to reflect virtual resync implementation - Enhance troubleshooting with guidance for virtual method override conflicts - Include resync integration in dependency analysis and diagrams - Clarify usage scenarios and benefits of programmatic synchronization control - Improve clarity on network recovery mechanisms using the resync method
…ook-ahead - Change production loop timer from 1000ms to 250ms interval for finer granularity - Adjust look-ahead offset from 500ms to 250ms to align 'now' with slot boundary - Update scheduling logic to wait for 250ms ticks instead of 1 second - Provide a 500ms lag threshold to allow safe block production despite OS jitter - Ensure block timestamps use deterministic slot time, not current clock time - Document the new production loop mechanism, timing rationale, and missed block behavior - Improve safety margin from 0ms to 500ms compared to previous implementation - Clarify production conditions and error cases in documentation files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.