diff --git a/.agent/README.md b/.agent/README.md index 758b3dc00..11b8947e6 100644 --- a/.agent/README.md +++ b/.agent/README.md @@ -36,6 +36,7 @@ This directory contains the project's technical constitution, specialized skills | [`hey-agent.md`](./workflows/hey-agent.md) | Unified management for Rules, Skills, and Workflows. | | [`lab-down.md`](./workflows/lab-down.md) | Stops and cleans up the database laboratory. | | [`lab-up.md`](./workflows/lab-up.md) | Starts a persistent database laboratory and injects data. | +| [`local-dev-sync.md`](./workflows/local-dev-sync.md) | Synchronize developer changes, run unit tests, and update changelog and release notes. | | [`markdown-lint.md`](./workflows/markdown-lint.md) | Check markdown content for cleanliness and project standard compliance (AFF, keywords, links) | | [`plan.md`](./workflows/plan.md) | Create or update an implementation plan (implementation_plan.md) | | [`release-manager.md`](./workflows/release-manager.md) | High-level release orchestrator for the Release Manager role | @@ -48,4 +49,4 @@ This directory contains the project's technical constitution, specialized skills --- -*Generated automatically by `/doc-sync` on 2026-06-12 20:03:20* \ No newline at end of file +*Generated automatically by `/doc-sync`* \ No newline at end of file diff --git a/.agent/rules/03_execution_rules.md b/.agent/rules/03_execution_rules.md index c23357e43..ded92a982 100644 --- a/.agent/rules/03_execution_rules.md +++ b/.agent/rules/03_execution_rules.md @@ -76,11 +76,10 @@ To ensure quality and clarity in every development cycle, all non-trivial featur - **Commit Validation:** Commits are automatically linted via `commitlint`. Non-compliant messages will be rejected by the pre-commit hook. - **History Documentation:** Use `npm run commit` to generate structured history. -1. **Changelog:** All changes MUST be traced and documented inside `@Changelog`. - - _Exception_: Documentation-only updates (`docs:`) following Conventional Commits may skip the manual `@Changelog` entry if they are primarily intended for README synchronization. - - _Requirement_: Adding a new test MUST have a `test:` entry in the `@Changelog`. - - _Requirement_: Changing test scripts or updating infrastructure MUST have a `ci:` entry in the `@Changelog`. - - _Requirement_: Changing `Makefile` or files under `build/` MUST be traced in the `@Changelog` (usually via `ci:` or `chore:`). +1. **Changelog:** All changes, without exception (including documentation `docs:` and unit tests `test:`), MUST be traced and documented inside the `Changelog` file to ensure granular change tracking. + - _Requirement_: Adding a new test MUST have a `test:` entry in the `Changelog`. + - _Requirement_: Changing test scripts or updating infrastructure MUST have a `ci:` entry in the `Changelog`. + - _Requirement_: Changing `Makefile` or files under `build/` MUST be traced in the `Changelog` (usually via `ci:` or `chore:`). - _Requirement_: All feature completions MUST be synchronized with `ROADMAP.md` before final PR/Commit. - _Ordering_: Changelog entries MUST be ordered by category: `chore`, `feat`, `fix`, `test`, `ci`, then others. - _Release Notes_: All release notes generated in `releases/` MUST follow the same category ordering in their "Executive Summary" section. diff --git a/.agent/workflows/doc-sync.md b/.agent/workflows/doc-sync.md index 02fe62df1..4c03b7cbb 100644 --- a/.agent/workflows/doc-sync.md +++ b/.agent/workflows/doc-sync.md @@ -10,7 +10,7 @@ category: Documentation // turbo ```bash -python3 build/doc_sync.py +perl build/doc_sync.pl ``` 1. **Full Documentation Review Checklist**: diff --git a/.agent/workflows/local-dev-sync.md b/.agent/workflows/local-dev-sync.md new file mode 100644 index 000000000..340821403 --- /dev/null +++ b/.agent/workflows/local-dev-sync.md @@ -0,0 +1,32 @@ +--- +trigger: explicit_call +description: Synchronize developer changes, run unit tests, and update changelog and release notes. +category: tool +--- + +# Local Developer Sync Workflow + +This workflow provides local synchronization automation for the developer, ensuring version consistency, automatic changelog sorting, release notes updating, unit testing, and git delivery. + +## 🧠 Rationale + +Ensuring a clean and synchronized branch history, accurate release documentation, and fully passing unit tests before any push to the remote repository. + +## đŸ› ïž Implementation + +Run the local dev-sync orchestrator script: + +// turbo + +```bash +perl build/dev_sync.pl +``` + +## ✅ Verification + +- The script returns exit code 0. +- All version files are confirmed to be consistent. +- `Changelog` is updated and sorted according to the Conventional Commit categories. +- `releases/v[VERSION].md` matches the latest entries. +- All local unit tests passed successfully. +- Modified release files are committed and pushed to `origin`. diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 000000000..859b3921d --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,43 @@ +--- +name: Bug report +about: Create a report to help us improve MySQLTuner +title: '[BUG] ' +labels: bug +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**Execution Command** +Please provide the exact command used to run MySQLTuner: +```bash +perl mysqltuner.pl [your arguments here] +``` + +**Environment info:** +- MySQLTuner version: (e.g. 2.9.0) +- Database Server: (e.g. MySQL, MariaDB, Percona Server) +- Database Version: (e.g. 8.0.35, 10.6.15) +- Operating System & Version: (e.g. Rocky Linux 9, Ubuntu 22.04) +- Perl version: (e.g. v5.32.1) + +**MySQLTuner Debug Output** +Please run MySQLTuner with the `--debug` option and paste the output here. +> ⚠ **IMPORTANT:** Obfuscate/remove any sensitive database credentials, IP addresses, hostnames, or passwords before pasting. + +```markdown + +``` + +**To Reproduce** +Steps to reproduce the behavior if not fully captured by the debug output: +1. Run command '...' +2. See error '...' + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Additional context** +Add any other context about the problem here (e.g. custom configuration files, /etc/my.cnf). diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 000000000..3a7c982cc --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: true +contact_links: + - name: Stack Overflow + url: http://stackoverflow.com/questions/tagged/mysqltuner + about: Please post usage questions or support issues on Stack Overflow using the 'mysqltuner' tag. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 000000000..17e3c4955 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,20 @@ +--- +name: Feature request +about: Suggest an idea for MySQLTuner +title: '[ENHANCEMENT] ' +labels: enhancement +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. (e.g., I'm always frustrated when [...]) + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.husky/commit-msg b/.husky/commit-msg index 70bd3dd23..f92fa5d0c 100644 --- a/.husky/commit-msg +++ b/.husky/commit-msg @@ -1 +1,16 @@ -npx --no-install commitlint --edit "$1" +#!/bin/sh + +# Run commitlint first to validate the commit message format +npx --no-install commitlint --edit "$1" || exit 1 + +# Check if Changelog is staged when commit type is feat or fix +COMMIT_MSG_FILE="$1" +FIRST_LINE=$(head -n 1 "$COMMIT_MSG_FILE") + +if echo "$FIRST_LINE" | grep -qE "^(feat|fix)(\([^)]+\))?!?:\s" ; then + if ! git diff --cached --name-only | grep -q "^Changelog$"; then + echo "ERROR: Commit type 'feat' or 'fix' detected, but 'Changelog' is not staged." + echo "Please update and stage 'Changelog' before committing." + exit 1 + fi +fi diff --git a/.husky/pre-commit b/.husky/pre-commit old mode 100644 new mode 100755 index 72c4429bc..de1f3da3b --- a/.husky/pre-commit +++ b/.husky/pre-commit @@ -1 +1,8 @@ -npm test +#!/bin/sh + +# Only run tests if code, tests, build scripts, Makefile, or package configuration has changed +if git diff --cached --name-only | grep -qE '\.(pl|pm)$|^tests/|^build/|^Makefile$|^package(-lock)?\.json$'; then + npm test +else + echo "Husky [pre-commit]: No code or test files modified. Skipping unit tests." +fi diff --git a/CURRENT_VERSION.txt b/CURRENT_VERSION.txt index b33624976..c8e38b614 100644 --- a/CURRENT_VERSION.txt +++ b/CURRENT_VERSION.txt @@ -1 +1 @@ -2.8.45 +2.9.0 diff --git a/Changelog b/Changelog index b226931eb..1c2881589 100644 --- a/Changelog +++ b/Changelog @@ -1,7 +1,89 @@ # MySQLTuner Changelog +2.9.0 2026-07-03 + +- chore(build): allow build scope in compliance auditor +- chore(build): rewrite dev_sync and doc_sync in Perl for consistency +- chore(deps): update devops-infra/action-commit-push digest to fa0c793 (#929) +- chore(deps): update shogo82148/actions-setup-mysql digest to 076e636 (#930) +- chore(main): add doc links in localhost warnings and support custom local subdomains +- chore(main): add roadmap to the whitelist of allowed scopes in compliance checks. +- chore(main): hide hostname, ssl, and replication warnings on localhost (#933) +- chore(main): whitelist deps and system commit scopes in check_compliance.pl to support Dependabot and host metrics commits. +- chore(metadata): add GitHub issue templates for bugs and feature requests +- chore: automated project metadata update +- chore: remove execution.log from git repository and sync docs +- feat(cli): create an agent-ready output format (JSON/YAML) so that MySQLTuner can be easily integrated by AI agents. +- feat(cli): resolve remote memory, socket override, and temptable sizing issues +- feat(main): recommend enabling slow query log if disabled (#517) +- feat(metadata): fix test badge and update version references in READMEs +- feat(report): add pgBadger-inspired query distribution, locking latency, and temp table memory/disk spill visual analytics to HTML report +- feat(report): add verbose execution timing measurements for each section, showing both elapsed time and its percentage relative to the total script execution time. +- feat(report): add verbose timings, step percentages, and snapshot summary +- feat(report): categorize recommendations in HTML report and add Connections/Performance panels, Storage engines table, and SQL modeling stats grid. +- feat(report): finalize HTML report, YAML output, and historical comparison +- feat(report): finalize a complete HTML report file beginning in v2.8.45. +- feat(report): implement Phase 13 sectional global indicators and KPIs +- feat(report): move dump_csv_files execution step to immediately before make_recommendations. +- feat(report): print an environment audit snapshot summary (server, user, RAM, swap, versions, uptime) right after get_all_vars. +- feat(report): support historical comparison of database diagnostics and performance metrics over time. +- feat: recommend slow query log when disabled (#517) +- feat: update Total buffers output message format to display temptable sizing when active. +- fix(cli): add mutually exclusive guard for json and yaml options. +- fix(cli): preserve --socket in connection arguments when --host is specified. +- fix(main): add undefined and 'NULL' guards to hr_num to eliminate uninitialized value warnings. +- fix(main): add undefined/NULL guards to hr_num to resolve uninitialized warnings +- fix(main): address PR #931 code review feedback and enhance test validations +- fix(main): calculate health score early in historical comparison to ensure scores exist for trend analysis. +- fix(main): calculate query cache efficiency using Com_select on MariaDB (MDEV-4981) +- fix(main): calculate query cache efficiency using Com_select on MariaDB, where Com_select includes query cache hits (MDEV-4981). +- fix(main): exclude MariaDB user roles and support zero-config TLS +- fix(main): exclude MariaDB user roles from SSL remote user audit +- fix(main): format YAML null as tilde (~) and multiline values using literal block style. +- fix(main): guard InnoDB log file size and log size percentage checks against uninitialized variables. +- fix(main): guard version and version comment checks in MariaDB parallel replication and query cache blocks. +- fix(main): implement cached version comparison parser to eliminate uninitialized value warnings and improve performance. +- fix(main): only recommend increasing innodb_log_buffer_size when log waits occur +- fix(main): sanitize and redact sensitive credentials from json/yaml output exports. +- fix(main): skip local /proc/loadavg read during remote database audits. +- fix(main): support MariaDB 11.4+ zero-configuration TLS and avoid false missing certificate warnings +- fix(report): flush console output trace when opening raw log files late in dumpdir mode. +- fix(test): add undef fallbacks to human_size, hr_bytes and hr_bytes_rnd mocks in tests. +- fix: update commit process +- fix: update documentation and code +- test(hook): verify pre-commit hook runs tests when test files change +- test(lab): add unit test test_issue_480.t for table_open_cache_instances recommendation (#480) +- test(lab): add unit test test_issue_517.t for slow query log recommendations (#517) +- test(lab): add unit test test_issue_810.t for issue 810 +- test(lab): add unit test test_issue_810.t for remote host forcemem MB interpretation correctness (issue #810) +- test(lab): add unit test test_issue_938.t for InnoDB write log efficiency +- test(lab): add unit tests for query cache efficiency logic on MySQL and MariaDB in tests/test_issue_927.t (renamed from tests/issue_927.t). +- test(lab): normalize all repro_issue_*.t and issue_*.t test file names to test_issue_*.t +- test(lab): split unit_coverage_boost4.t into smaller topic-oriented unit tests: unit_cli_helpers.t, unit_client_privileges.t, unit_cloud_commands.t, unit_fs_info.t, unit_os_vm.t +- test(lab): utilize isolated tempfile for mock json to avoid race conditions in tests/unit_phase13_kpis.t. +- test(report): add verbose timing and audit snapshot summary formatting checks to tests/verbose_timing.t. +- test(report): update HTML report unit tests to verify connections, performance, storage, and modeling HTML sections. +- test(security): add test cases for MariaDB role exclusion and zero-configuration TLS +- test(versions): add unit tests for version caching and comparisons, resolve redundant warnings, and use tempfile in tests/unit_versions.t. +- test: add socket preservation tests to test_issue_781.t. +- test: add unit test for remote host physical memory detection and validation (issue #796). +- test: add unit test for temptable output formatting in Total buffers. +- ci: optimize pre-commit hook to only run unit tests when code, tests, or dependencies are modified. +- docs(metadata): promote HTML reports and add E2E examples +- docs(metadata): remove timestamp from doc-sync generated files +- docs(roadmap): link strategic technical evolutions specification and enforce changelog staging +- docs(roadmap): specify and implement Phase XIV pgBadger-inspired HTML report modules +- docs(roadmap): specify Phase XVI (AI Agent Integration) and Phase XVII (Dockerized MCP Daemon). +- docs: add LightPath as sponsor, relocate coffee button, and use star-history chart +- docs: regenerate release notes +- docs: update repository links to major/MySQLTuner-perl and add GitHub stars badge + 2.8.45 2026-06-04 + + + + - chore: restore doc_sync.py utility script and run doc synchronization. - feat: export both complete (unfiltered) and filtered CSV files for sys views in dumpdir. - feat: filter system databases from PFS/sys schema analysis queries and dumpdir exports. @@ -26,6 +108,10 @@ 2.8.44 2026-05-27 + + + + - feat: develop automated end-of-life (EOL) date synchronization check script to audit LTS versions in CI. - feat: develop automated specification consistency auditor and Spec-to-Test Mapping Matrix. - feat: develop LTS API auto-bumping utility with GitHub Actions integration. @@ -56,6 +142,10 @@ 2.8.43 2026-05-25 + + + + - chore: bump version to 2.8.43. - feat: add --compress-dump option to compress SQL schema dumps using gzip. - feat: add --dump-limit option to limit row extraction count for CSV dumps. @@ -82,11 +172,19 @@ 2.8.42 2026-05-17 + + + + - chore: automated project metadata update (empty release). 2.8.41 2026-05-17 + + + + - feat: enhance --forcemem and --forceswap to support human-readable memory units (B, K, M, G, T, P). - feat: implement idiomatic Perl Boolean practices across the project (#34). - feat: add recommendation for `table_open_cache_instances` based on CPU cores (#480). @@ -129,6 +227,10 @@ 2.8.40 2026-02-15 + + + + - feat: overhaul InnoDB Redo Log Capacity logic to consider RAM size and workload writes. - feat: add support for `innodb_dedicated_server` detection in Redo Log diagnostics. - feat: implement multi-cloud autodiscovery for AWS RDS/Aurora, GCP Cloud SQL, Azure (Flexible/Managed), and DigitalOcean. @@ -144,6 +246,10 @@ 2.8.39 2026-02-15 + + + + - feat: implement authentication plugin security checks to detect insecure or deprecated plugins like mysql_native_password and sha256_password. - feat: add MySQL 9.x readiness diagnostics for eliminated authentication methods. - feat: update MariaDB recommendations to suggest ed25519 and unix_socket for enhanced security. @@ -169,6 +275,10 @@ 2.8.36 2026-02-13 + + + + - fix: migrate CI workflows to native GitHub services to resolve Docker API version mismatch. - fix: modernize release workflow using softprops/action-gh-release@v2. - fix: enhance Docker publishing with Buildx setup for reliable multi-platform builds. @@ -186,6 +296,10 @@ 2.8.35 2026-02-02 + + + + - feat: modernize version check using `HTTP::Tiny` with robust fallback to `curl`/`wget` (PR #18 and #17). - feat: integrate `perltidy` in `release-preflight` workflow and enforce script formatting (issue #19). - fix: resolve inaccurate `innodb_log_file_size` recommendations caused by integer rounding (issue #770). @@ -198,6 +312,10 @@ 2.8.33 2026-01-31 + + + + - fix: improved cPanel/Flex detection and refined `skip-name-resolve` recommendation (issue #863). - test: add enhanced unit test `tests/issue_863_enhanced.t` for cPanel detection verification. - docs: consolidate project governance rules and resolve backwards compatibility contradiction (00_constitution.md, 03_execution_rules.md). @@ -219,6 +337,10 @@ 2.8.32 2026-01-30 + + + + - feat: remove `--skippassword` from test laboratory to enable security checks. - fix: resolve false positive weak password warnings on MariaDB socket authentication (issue #875) and prevent dictionary corruption by silencing `curl`/`wget` output. - test: add unit test [tests/test_issue_875.t](file:///tests/test_issue_875.t) to verify socket authentication detection. @@ -228,6 +350,10 @@ 2.8.31 2026-01-27 + + + + - feat: add `--schemadir ` option to generate per-schema markdown documentation. - feat: support independent schema documentation generation without requiring `--dumpdir`. - feat: restructure specifications into `documentation/specifications/` (/hey-agent). @@ -263,6 +389,10 @@ 2.8.30 2026-01-24 + + + + - feat: auto-generate `raw_mysqltuner.txt` report in `dumps/` directory when using `--dumpdir` - feat: add InnoDB transaction isolation levels and metrics (active count, longest duration) - feat: add MariaDB innodb_snapshot_isolation detection and recommendation @@ -289,21 +419,37 @@ 2.8.29 2026-01-24 + + + + - fix: synchronize all version occurrences in mysqltuner.pl and update release workflows (issue #15) - feat: add version consistency check to release-preflight and git-flow workflows - docs: update copyright years to 2026 2.8.28 2026-01-22 + + + + - feat: ajoute l'option --no-pfstat pour la partie performance schema - feat: ajoute l'option --no-colstat pour la partie colonne stat - fix: skip innodb_buffer_stats during sys schema dump to avoid performance issues 2.8.27 2026-01-18 + + + + - refactor: replace massive raw backtick usage with execute_system_command wrapper for better security and compliance (Compliance Sentinel) 2.8.26 2026-01-18 + + + + - fix: inverted replication command logic causing wrong SQL on MySQL 8.0+/MariaDB 10.5+ (issue #553) - feat: add MySQL/MariaDB version detection to prevent version number conflicts in replication logic - test: add comprehensive test suite (test_issue_553.t) for replication command compatibility @@ -311,17 +457,29 @@ 2.8.24 2026-01-18 + + + + - fix: improve MariaDB 11+ detection by checking version_comment (issue #869) - fix: handle innodb_buffer_pool_chunk_size=0 (autosize) in MariaDB 10.8+ (#869) - chore: bump version to 2.8.24 2.8.23 2026-01-18 + + + + - feat: add --ignore-tables CLI option to filter specific tables from analysis (#749) - chore: bump version to 2.8.23 2.8.22 2026-01-18 + + + + - feat: update all repository links from 'major' to 'jmrenouard' (issue #410) - docs: add Changelog information and Useful Links to all README files (issue #411) - feat: improve thread_pool_size recommendations based on logical CPU count (issue #404) @@ -332,65 +490,113 @@ 2.8.21 2026-01-18 + + + + - fix: remove contradictory query_cache_limit recommendation when disabling query cache (issue #671) - fix: cap join_buffer_size recommendation at 4MB and prefer index optimization (issue #671) - chore: bump version to 2.8.21 2.8.20 2026-01-18 + + + + - feat: add automated regression test for forcemem MB interpretation (issues #780, #810) - chore: bump version to 2.8.20 2.8.18 2026-01-18 + + + + - feat: add --max-password-checks option to limit dictionary checks (default: 100) - fix: ensure Machine type is reported as 'Container' when --container option is used - chore: bump version to 2.8.18 2.8.17 2026-01-18 + + + + - feat: implementation of issue #403 to check weak passwords on MySQL 8.0+ and flush hosts every 100 attempts - chore: bump version to 2.8.17 2.8.16 2026-01-18 + + + + - chore: bump version to 2.8.16 2.8.15 2026-01-18 + + + + - feat: update all GitHub links from 'major' to 'jmrenouard' organization - feat: refactor plugin information to filter ACTIVE status and display specific columns grouped by type - chore: bump version to 2.8.15 2.8.13 2026-01-18 + + + + - docs: add Useful Links section to all README files (English, French, Russian, Italian) - chore: bump version to 2.8.13 2.8.12 2026-01-17 + + + + - feat: update is_docker() to detect containerd and podman runtimes - chore: bump version to 2.8.12 2.8.11 2026-01-17 + + + + - docs: update INTERNALS.md with information about Cloud, SSH, Containers, and Plugins - chore: bump version to 2.8.11 2.8.10 2026-01-17 + + + + - feat: add dates and commands to log files in test_envs.sh - feat: add separators (=) at the end of log files in test_envs.sh - chore: synchronize version strings across script, POD, and version file 2.8.9 2026-01-17 + + + + - feat: improve container log detection by excluding proxy containers (traefik, haproxy, maxscale, proxy) - feat: prioritize database-related container names (mysql, mariadb, percona, db, database) - chore: bump version to 2.8.9 2.8.8 2026-01-17 + + + + - feat: add -d/--database parameter to test_envs.sh to tune specific databases - feat: add -c/--configs parameter to test_envs.sh for easier configuration selection - feat: add timestamps to major steps in test_envs.sh logs @@ -399,12 +605,20 @@ 2.8.7 2026-01-17 + + + + - docs: add standardized comment headers to all build shell scripts - chore: synchronize version strings across script, POD, and version file - fix: ensure version consistency between Changelog and CURRENT_VERSION.txt 2.8.6 2026-01-17 + + + + - feat: add Plugin Information section and --plugininfo flag (#794) - fix: memory calculation bug in system_recommendations (1.5GB check) - fix: ensure forcemem is correctly interpreted and displayed as MB in os_setup @@ -412,30 +626,54 @@ 2.8.5 2026-01-17 + + + + - fix: noisy sysctl errors for sunrpc parameters when kernel module is not loaded - fix: refactor get_kernel_info to handle missing sysctl parameters gracefully 2.8.4 2026-01-17 + + + + - fix: database injection failing to find dump files due to incorrect working directory - fix: ensure correct path handling for 'source' commands in employees.sql 2.8.3 2026-01-17 + + + + - feat: detect docker/podman environment and automatically grab logs from container if local log file is not found - feat: add --container option to manually specify a container for log retrieval 2.8.2 2026-01-17 + + + + - fix: system command failures (ping/ifconfig/redirection) on modern Linux (Ubuntu 22.04/WSL2) - feat: integrate external test dependencies (multi-db-docker-env, test_db) and automated employees database injection 2.8.1 2026-01-17 + + + + - fix: resilient memory checks with /proc fallback on Linux and silencing expected ps failures 2.8.0 2026-01-17 + + + + - Bump version to 2.8.0 - enhance user hostname restriction checks - feat: Translate comments and messages in updateCVElist.py to English diff --git a/POTENTIAL_ISSUES.md b/POTENTIAL_ISSUES.md index cfc248b21..c85c11739 100644 --- a/POTENTIAL_ISSUES.md +++ b/POTENTIAL_ISSUES.md @@ -2,13 +2,13 @@ This file records anomalies discovered during laboratory testing (Perl warnings, SQL errors, etc.). -## [2026-05-29 Audit] Status Refresh v2.8.44 +## [2026-06-16 Audit] Status Refresh v2.9.0 ### Unit Test Results - **Status**: ✅ ALL PASS -- **Files**: 72 test files -- **Assertions**: 362 tests +- **Files**: 81 test files +- **Assertions**: 462 tests - **Perl Syntax**: Clean (`perl -cw mysqltuner.pl` — no warnings) ### Test Coverage Analysis @@ -16,15 +16,12 @@ This file records anomalies discovered during laboratory testing (Perl warnings, | Metric | Value | |:---|:---| | Total Subroutines | 167 | -| Tested Subroutines | ~154 (~92%) | -| Untested Subroutines | ~13 (~8%) | +| Tested Subroutines | 167 (100%) | +| Untested Subroutines | 0 (0%) | #### Remaining Untested Subroutines (System/IO-Heavy) -- `check_privileges`, `cloud_setup`, `get_fs_info`, `get_fs_info_win` -- `get_http_cli`, `get_os_release`, `get_tuning_info` -- `infoprintcmd`, `infoprinthcmd`, `is_virtual_machine` -- `parse_cli_args`, `show_help` (x2) +- None (100% subroutine coverage reached) ### 🔮 Critical Issues @@ -58,7 +55,8 @@ This file records anomalies discovered during laboratory testing (Perl warnings, #### PI-006: 13 out of 167 subroutines have zero test coverage - **Impact**: Remaining untested functions are mostly system-level (filesystem, OS detection, cloud setup) or CLI helpers (`show_help`, `parse_cli_args`) - **Severity**: 🟱 LOW — core diagnostic functions now fully covered -- **Coverage rate**: ~92% of subroutines referenced in at least one test (improved from ~55% → 62% → 78% → 92%) +- **Coverage rate**: 100% of subroutines referenced in at least one test (improved from ~55% → 62% → 78% → 92% → 100%) +- **Status**: [x] **FIXED** — All remaining subroutines covered in `tests/unit_coverage_boost4.t`. #### PI-007: Extremely large subroutines - **Impact**: Several functions exceed 500+ lines, making maintenance difficult @@ -70,6 +68,7 @@ This file records anomalies discovered during laboratory testing (Perl warnings, - **Source**: Each call to `mysql_version_ge()`, `mysql_version_le()`, `mysql_version_eq()` re-parses `$myvar{'version'}` via regex - **Impact**: Redundant computation — called 100+ times across the script - **Severity**: 🟱 LOW — performance impact minimal but code duplication +- **Status**: [x] **FIXED** — Implemented version parsing caching via `_parse_version()`. #### PI-009: MariaDB 10.6 Approaching EOL - **Source**: [mariadb_support.md](file:///mariadb_support.md) @@ -114,12 +113,13 @@ This file records anomalies discovered during laboratory testing (Perl warnings, - Binlog checksum, doublewrite consistency: NOT implemented - **Status**: Phase 9 partially implemented -#### PI-016: ROADMAP Phases 10-12 — Not started +#### PI-016: ROADMAP Phases 11-12 — Not started - Workload Analysis & Traffic Profiling: Not implemented - Advanced Log Parser & Lock Monitoring: Not implemented -- Sectional Global Indicators: Not implemented -#### PI-017: ROADMAP Phase 13 (Export Optimization) — COMPLETED ✅ +#### PI-017: ROADMAP Phase 13 (Sectional Global Indicators) — COMPLETED ✅ + +#### PI-018: ROADMAP Phase 14 (Export Optimization) — COMPLETED ✅ --- @@ -212,3 +212,13 @@ This file records anomalies discovered during laboratory testing (Perl warnings, - [x] **Doc-Sync**: `.agent/README.md` synchronized with 18 workflows. - [x] **SECURITY.md**: Version reference updated to v2.8.44. - [x] **ROADMAP PI-010**: I/O Pressure status already corrected. + +### [2026-06-04] Release v2.8.45 + +- [x] **Temptable Sizing Limits**: Integrated `temptable_max_ram` calculations and mmap checks. +- [x] **InnoDB Index/Data Ratio Check**: Added advisory and CSV dump for tables > 50,000 rows. +- [x] **Uptime Observability**: Exposed Database Server Uptime in all modes. +- [x] **Performance Optimization**: Bulk-fetched table engine and column details to reduce queries (from 2N+3 to 2 per table). +- [x] **System DB Filtering**: Excluded system tables from schema analysis and dumpdir exports. +- [x] **SQL Escaping Fixes**: Safe dollar sign escaping in system call wrappers. + diff --git a/README.fr.md b/README.fr.md index 56fd51363..c5d74e307 100644 --- a/README.fr.md +++ b/README.fr.md @@ -1,11 +1,11 @@ ![MySQLTuner-perl](mtlogo2.png) -[!["Offrez-nous un cafĂ©"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/jmrenouard) +[![GitHub stars](https://img.shields.io/github/stars/major/MySQLTuner-perl?style=for-the-badge&logo=github)](https://github.com/major/MySQLTuner-perl) [![État du projet](https://opensource.box.com/badges/active.svg)](https://opensource.box.com/badges) -[![État des tests](https://github.com/jmrenouard/MySQLTuner-perl/workflows/Test/badge.svg)](https://github.com/jmrenouard/MySQLTuner-perl/actions) -[![Temps moyen de rĂ©solution d'un problĂšme](https://isitmaintained.com/badge/resolution/jmrenouard/MySQLTuner-perl.svg)](https://isitmaintained.com/project/jmrenouard/MySQLTuner-perl "Temps moyen de rĂ©solution d'un problĂšme") -[![Pourcentage de problĂšmes ouverts](https://isitmaintained.com/badge/open/jmrenouard/MySQLTuner-perl.svg)](https://isitmaintained.com/project/jmrenouard/MySQLTuner-perl "Pourcentage de problĂšmes encore ouverts") +[![État des tests](https://github.com/major/MySQLTuner-perl/actions/workflows/pull_request.yml/badge.svg)](https://github.com/major/MySQLTuner-perl/actions) +[![Temps moyen de rĂ©solution d'un problĂšme](https://isitmaintained.com/badge/resolution/major/MySQLTuner-perl.svg)](https://isitmaintained.com/project/major/MySQLTuner-perl "Temps moyen de rĂ©solution d'un problĂšme") +[![Pourcentage de problĂšmes ouverts](https://isitmaintained.com/badge/open/major/MySQLTuner-perl.svg)](https://isitmaintained.com/project/major/MySQLTuner-perl "Pourcentage de problĂšmes encore ouverts") [![Licence GPL](https://badges.frapsoft.com/os/gpl/gpl.png?v=103)](https://opensource.org/licenses/GPL-3.0/) **MySQLTuner** est un script Ă©crit en Perl qui vous permet d'examiner rapidement une installation MySQL et de faire des ajustements pour augmenter les performances et la stabilitĂ©. Les variables de configuration actuelles et les donnĂ©es d'Ă©tat sont rĂ©cupĂ©rĂ©es et prĂ©sentĂ©es dans un bref format avec quelques suggestions de performances de base. @@ -15,34 +15,52 @@ **MySQLTuner** est activement maintenu et prend en charge de nombreuses configurations telles que [Galera Cluster](https://galeracluster.com/), [TokuDB](https://www.percona.com/software/mysql-database/percona-tokudb), [SchĂ©ma de performance](https://github.com/mysql/mysql-sys), les mĂ©triques du systĂšme d'exploitation Linux, [InnoDB](https://dev.mysql.com/doc/refman/5.7/en/innodb-storage-engine.html), [MyISAM](https://dev.mysql.com/doc/refman/5.7/en/myisam-storage-engine.html), [Aria](https://mariadb.com/docs/server/server-usage/storage-engines/aria/aria-storage-engine), ... Vous pouvez trouver plus de dĂ©tails sur ces indicateurs ici : -[Description des indicateurs](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/INTERNALS.md). +[Description des indicateurs](https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md). -![MysqlTuner](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mysqltuner.png) +![MysqlTuner](https://github.com/major/MySQLTuner-perl/blob/master/mysqltuner.png) Liens utiles == -* **DĂ©veloppement actif :** [https://github.com/jmrenouard/MySQLTuner-perl](https://github.com/jmrenouard/MySQLTuner-perl) -* **Versions/Tags :** [https://github.com/jmrenouard/MySQLTuner-perl/tags](https://github.com/jmrenouard/MySQLTuner-perl/tags) -* **Changelog :** [https://github.com/jmrenouard/MySQLTuner-perl/blob/master/Changelog](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/Changelog) +* **DĂ©veloppement actif :** [https://github.com/major/MySQLTuner-perl](https://github.com/major/MySQLTuner-perl) +* **Versions/Tags :** [https://github.com/major/MySQLTuner-perl/tags](https://github.com/major/MySQLTuner-perl/tags) +* **Changelog :** [https://github.com/major/MySQLTuner-perl/blob/master/Changelog](https://github.com/major/MySQLTuner-perl/blob/master/Changelog) * **Images Docker :** [https://hub.docker.com/repository/docker/jmrenouard/mysqltuner/tags](https://hub.docker.com/repository/docker/jmrenouard/mysqltuner/tags) +* **Rapports HTML Interactifs (v2.9.0+) :** + * [Exemple de rapport MariaDB 11.4](https://lightpath.fr/MySQLtuner_reports/MySQLTuner-v290_mariadb114/Schemadir/mysqltuner_report.html) + * [Exemple de rapport MySQL 8.4](https://lightpath.fr/MySQLtuner_reports/MySQLTuner-v290_mysql84/Schemadir/mysqltuner_report.html) + * [Exemple de rapport Percona 8.0](https://lightpath.fr/MySQLtuner_reports/MySQLTuner-v290_percona80/Schemadir/mysqltuner_report.html) MySQLTuner a besoin de vous === **MySQLTuner** a besoin de contributeurs pour la documentation, le code et les commentaires : -* Veuillez nous rejoindre sur notre outil de suivi des problĂšmes sur [le suivi GitHub](https://github.com/jmrenouard/MySQLTuner-perl/issues). -* Le guide de contribution est disponible en suivant [le guide de contribution de MySQLTuner](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/CONTRIBUTING.md) -* Mettez une Ă©toile au **projet MySQLTuner** sur [le projet Git Hub de MySQLTuner](https://github.com/jmrenouard/MySQLTuner-perl/) +* Veuillez nous rejoindre sur notre outil de suivi des problĂšmes sur [le suivi GitHub](https://github.com/major/MySQLTuner-perl/issues). +* Le guide de contribution est disponible en suivant [le guide de contribution de MySQLTuner](https://github.com/major/MySQLTuner-perl/blob/master/CONTRIBUTING.md) +* Mettez une Ă©toile au **projet MySQLTuner** sur [le projet Git Hub de MySQLTuner](https://github.com/major/MySQLTuner-perl/) * Support payant pour LightPath ici : [jmrenouard@lightpath.fr](jmrenouard@lightpath.fr) * Support payant pour Releem disponible ici : [Application Releem](https://releem.com/) +### Sponsors + +Le dĂ©veloppement actif est sponsorisĂ© par : + +

+ + LightPath + +

+ +Merci à LightPath pour la mise à disposition des ressources (serveurs de développement, abonnement IA, environnements de recette & fonctionnalités). + ![Statistiques GitHub de jmrenouard](https://github-readme-stats.vercel.app/api?username=jmrenouard&show_icons=true&theme=radical) -## Stargazers au fil du temps +[!["Offrez-nous un café"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/jmrenouard) + +## Historique des étoiles -[![Stargazers au fil du temps](https://starchart.cc/jmrenouard/MySQLTuner-perl.svg)](https://starchart.cc/jmrenouard/MySQLTuner-perl) +[![Star History Chart](https://api.star-history.com/svg?repos=major/MySQLTuner-perl&type=Date)](https://star-history.com/#major/MySQLTuner-perl&Date) Compatibilité ==== @@ -58,8 +76,8 @@ Les résultats des tests sont disponibles ici uniquement pour les versions LTS  Merci à [endoflife.date](https://endoflife.date/) -* Reportez-vous aux [versions prises en charge de MariaDB](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mariadb_support.md). -* Reportez-vous aux [versions prises en charge de MySQL](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mysql_support.md). +* Reportez-vous aux [versions prises en charge de MariaDB](https://github.com/major/MySQLTuner-perl/blob/master/mariadb_support.md). +* Reportez-vous aux [versions prises en charge de MySQL](https://github.com/major/MySQLTuner-perl/blob/master/mysql_support.md). ***La prise en charge de Windows est partielle*** @@ -156,12 +174,12 @@ Recommandations de sécurité Salut l'utilisateur de directadmin ! Nous avons détecté que vous exécutez mysqltuner avec les informations d'identification de da_admin extraites de `/usr/local/directadmin/conf/my.cnf`, ce qui pourrait entraßner une découverte de mot de passe ! -Lisez le lien pour plus de détails [ProblÚme n°289](https://github.com/jmrenouard/MySQLTuner-perl/issues/289). +Lisez le lien pour plus de détails [ProblÚme n°289](https://github.com/major/MySQLTuner-perl/issues/289). Que vérifie exactement MySQLTuner ? -- -Toutes les vérifications effectuées par **MySQLTuner** sont documentées dans la documentation [MySQLTuner Internals](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/INTERNALS.md). +Toutes les vérifications effectuées par **MySQLTuner** sont documentées dans la documentation [MySQLTuner Internals](https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md). **MySQLTuner** analyse les domaines suivants : @@ -190,14 +208,14 @@ Choisissez l'une de ces méthodes : ```bash wget https://mysqltuner.pl/ -O mysqltuner.pl -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv ``` 2) Vous pouvez télécharger l'intégralité du référentiel en utilisant `git clone` ou `git clone --depth 1 -b master` suivi de l'URL de clonage ci-dessus. ```bash -git clone --depth 1 -b master https://github.com/jmrenouard/MySQLTuner-perl.git +git clone --depth 1 -b master https://github.com/major/MySQLTuner-perl.git ``` 3) Sur Apple macOS, installez via [Homebrew](https://brew.sh/) : @@ -217,8 +235,8 @@ docker run --rm -it jmrenouard/mysqltuner --host --user + + LightPath + +

+ +Grazie a LightPath per aver fornito risorse (server di sviluppo, abbonamento IA, staging e funzionalità). + ![Statistiche GitHub di jmrenouard](https://github-readme-stats.vercel.app/api?username=jmrenouard&show_icons=true&theme=radical) -## Stargazer nel tempo +[!["Offrici un caffÚ"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/jmrenouard) + +## Cronologia delle stelle -[![Stargazer nel tempo](https://starchart.cc/jmrenouard/MySQLTuner-perl.svg)](https://starchart.cc/jmrenouard/MySQLTuner-perl) +[![Star History Chart](https://api.star-history.com/svg?repos=major/MySQLTuner-perl&type=Date)](https://star-history.com/#major/MySQLTuner-perl&Date) Compatibilità ==== @@ -58,8 +76,8 @@ I risultati dei test sono disponibili qui solo per LTS: Grazie a [endoflife.date](https://endoflife.date/) -* Fare riferimento a [Versioni supportate di MariaDB](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mariadb_support.md). -* Fare riferimento a [Versioni supportate di MySQL](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mysql_support.md). +* Fare riferimento a [Versioni supportate di MariaDB](https://github.com/major/MySQLTuner-perl/blob/master/mariadb_support.md). +* Fare riferimento a [Versioni supportate di MySQL](https://github.com/major/MySQLTuner-perl/blob/master/mysql_support.md). ***Il supporto per Windows Ú parziale*** @@ -156,12 +174,12 @@ Raccomandazioni di sicurezza Ciao utente di directadmin! Abbiamo rilevato che esegui mysqltuner con le credenziali di da_admin prese da `/usr/local/directadmin/conf/my.cnf`, il che potrebbe portare alla scoperta di una password! -Leggi il link per maggiori dettagli [Problema #289](https://github.com/jmrenouard/MySQLTuner-perl/issues/289). +Leggi il link per maggiori dettagli [Problema #289](https://github.com/major/MySQLTuner-perl/issues/289). Cosa sta controllando esattamente MySQLTuner? -- -Tutti i controlli eseguiti da **MySQLTuner** sono documentati nella documentazione [MySQLTuner Internals](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/INTERNALS.md). +Tutti i controlli eseguiti da **MySQLTuner** sono documentati nella documentazione [MySQLTuner Internals](https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md). **MySQLTuner** analizza le seguenti aree: @@ -190,14 +208,14 @@ Scegli uno di questi metodi: ```bash wget https://mysqltuner.pl/ -O mysqltuner.pl -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv ``` 2) È possibile scaricare l'intero repository utilizzando `git clone` o `git clone --depth 1 -b master` seguito dall'URL di clonazione sopra. ```bash -git clone --depth 1 -b master https://github.com/jmrenouard/MySQLTuner-perl.git +git clone --depth 1 -b master https://github.com/major/MySQLTuner-perl.git ``` 3) Su Apple macOS, installa tramite [Homebrew](https://brew.sh/): @@ -217,8 +235,8 @@ docker run --rm -it jmrenouard/mysqltuner --host --user + + LightPath + +

+ +Thanks to LightPath for providing resources (dev servers, AI subscriptions, staging & features). + ![jmrenouard's GitHub stats](https://github-readme-stats.vercel.app/api?username=jmrenouard&show_icons=true&theme=radical) -## Stargazers over time +[!["Buy Us A Coffee"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/jmrenouard) + +## Star History -[![Stargazers over time](https://starchart.cc/jmrenouard/MySQLTuner-perl.svg)](https://starchart.cc/jmrenouard/MySQLTuner-perl) +[![Star History Chart](https://api.star-history.com/svg?repos=major/MySQLTuner-perl&type=Date)](https://star-history.com/#major/MySQLTuner-perl&Date) Compatibility ==== @@ -60,8 +78,8 @@ Test result are available here for LTS only: Thanks to [endoflife.date](https://endoflife.date/) -* Refer to [MariaDB Supported versions](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mariadb_support.md). -* Refer to [MySQL Supported versions](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mysql_support.md). +* Refer to [MariaDB Supported versions](https://github.com/major/MySQLTuner-perl/blob/master/mariadb_support.md). +* Refer to [MySQL Supported versions](https://github.com/major/MySQLTuner-perl/blob/master/mysql_support.md). ***Windows Support is partial*** @@ -158,12 +176,12 @@ Security recommendations Hi directadmin user! We detected that you run mysqltuner with da_admin's credentials taken from `/usr/local/directadmin/conf/my.cnf`, which might bring to a password discovery! -Read link for more details [Issue #289](https://github.com/jmrenouard/MySQLTuner-perl/issues/289). +Read link for more details [Issue #289](https://github.com/major/MySQLTuner-perl/issues/289). What is MySQLTuner checking exactly ? -- -All checks done by **MySQLTuner** are documented in [MySQLTuner Internals](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/INTERNALS.md) documentation. +All checks done by **MySQLTuner** are documented in [MySQLTuner Internals](https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md) documentation. **MySQLTuner** analyzes the following areas: @@ -192,14 +210,14 @@ Choose one of these methods: ```bash wget https://mysqltuner.pl/ -O mysqltuner.pl -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv ``` 2) You can download the entire repository by using `git clone` or `git clone --depth 1 -b master` followed by the cloning URL above. ```bash -git clone --depth 1 -b master https://github.com/jmrenouard/MySQLTuner-perl.git +git clone --depth 1 -b master https://github.com/major/MySQLTuner-perl.git ``` 3) On Apple macOS, install via [Homebrew](https://brew.sh/): @@ -219,8 +237,8 @@ docker run --rm -it jmrenouard/mysqltuner --host --user + + LightPath + +

+ +ĐĄĐżĐ°ŃĐžĐ±ĐŸ LightPath Đ·Đ° ĐżŃ€Đ”ĐŽĐŸŃŃ‚Đ°ĐČĐ»Đ”ĐœĐžĐ” Ń€Đ”ŃŃƒŃ€ŃĐŸĐČ (сДрĐČДры Ń€Đ°Đ·Ń€Đ°Đ±ĐŸŃ‚ĐșĐž, ĐżĐŸĐŽĐżĐžŃĐșĐ° ĐœĐ° ИИ, стДĐčĐŽĐ¶ĐžĐœĐł Đž ĐœĐŸĐČŃ‹Đ” Ń„ŃƒĐœĐșцоо). + ![СтатостоĐșĐ° GitHub jmrenouard](https://github-readme-stats.vercel.app/api?username=jmrenouard&show_icons=true&theme=radical) -## ЗĐČĐ”Đ·ĐŽĐŸŃ‡Đ”Ń‚Ń‹ с Ń‚Đ”Ń‡Đ”ĐœĐžĐ”ĐŒ ĐČŃ€Đ”ĐŒĐ”ĐœĐž +[!["ĐšŃƒĐżĐžŃ‚Đ” ĐœĐ°ĐŒ ĐșĐŸŃ„Đ”"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/jmrenouard) + +## Đ˜ŃŃ‚ĐŸŃ€ĐžŃ Đ·ĐČДзЎ -[![ЗĐČĐ”Đ·ĐŽĐŸŃ‡Đ”Ń‚Ń‹ с Ń‚Đ”Ń‡Đ”ĐœĐžĐ”ĐŒ ĐČŃ€Đ”ĐŒĐ”ĐœĐž](https://starchart.cc/jmrenouard/MySQLTuner-perl.svg)](https://starchart.cc/jmrenouard/MySQLTuner-perl) +[![Star History Chart](https://api.star-history.com/svg?repos=major/MySQLTuner-perl&type=Date)](https://star-history.com/#major/MySQLTuner-perl&Date) ĐĄĐŸĐČĐŒĐ”ŃŃ‚ĐžĐŒĐŸŃŃ‚ŃŒ ==== @@ -58,8 +76,8 @@ MySQLTuner ĐœŃƒĐ¶ĐŽĐ°Đ”Ń‚ŃŃ ĐČ ĐČас ĐĄĐżĐ°ŃĐžĐ±ĐŸ [endoflife.date](https://endoflife.date/) -* ĐĄĐŒ. [ĐŸĐŸĐŽĐŽĐ”Ń€Đ¶ĐžĐČĐ°Đ”ĐŒŃ‹Đ” ĐČДрсОО MariaDB](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mariadb_support.md). -* ĐĄĐŒ. [ĐŸĐŸĐŽĐŽĐ”Ń€Đ¶ĐžĐČĐ°Đ”ĐŒŃ‹Đ” ĐČДрсОО MySQL](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/mysql_support.md). +* ĐĄĐŒ. [ĐŸĐŸĐŽĐŽĐ”Ń€Đ¶ĐžĐČĐ°Đ”ĐŒŃ‹Đ” ĐČДрсОО MariaDB](https://github.com/major/MySQLTuner-perl/blob/master/mariadb_support.md). +* ĐĄĐŒ. [ĐŸĐŸĐŽĐŽĐ”Ń€Đ¶ĐžĐČĐ°Đ”ĐŒŃ‹Đ” ĐČДрсОО MySQL](https://github.com/major/MySQLTuner-perl/blob/master/mysql_support.md). ***ĐŸĐŸĐŽĐŽĐ”Ń€Đ¶ĐșĐ° Windows Ń‡Đ°ŃŃ‚ĐžŃ‡ĐœĐ°*** @@ -156,12 +174,12 @@ GRANT SELECT, PROCESS, EXECUTE, REPLICATION CLIENT, SHOW DATABASES, SHOW VIEW ON ПроĐČДт, ĐżĐŸĐ»ŃŒĐ·ĐŸĐČĐ°Ń‚Đ”Đ»ŃŒ directadmin! Мы ĐŸĐ±ĐœĐ°Ń€ŃƒĐ¶ĐžĐ»Đž, Ń‡Ń‚ĐŸ ĐČы запусĐșаДтД mysqltuner с ŃƒŃ‡Đ”Ń‚ĐœŃ‹ĐŒĐž ĐŽĐ°ĐœĐœŃ‹ĐŒĐž da_admin, ĐČĐ·ŃŃ‚Ń‹ĐŒĐž Оз `/usr/local/directadmin/conf/my.cnf`, Ń‡Ń‚ĐŸ ĐŒĐŸĐ¶Đ”Ń‚ проĐČДстО Đș расĐșрытою ĐżĐ°Ń€ĐŸĐ»Ń! -ĐŸĐŸĐŽŃ€ĐŸĐ±ĐœĐ”Đ” чотаĐčŃ‚Đ” ĐżĐŸ ссылĐșĐ” [ĐŸŃ€ĐŸĐ±Đ»Đ”ĐŒĐ° №289](https://github.com/jmrenouard/MySQLTuner-perl/issues/289). +ĐŸĐŸĐŽŃ€ĐŸĐ±ĐœĐ”Đ” чотаĐčŃ‚Đ” ĐżĐŸ ссылĐșĐ” [ĐŸŃ€ĐŸĐ±Đ»Đ”ĐŒĐ° №289](https://github.com/major/MySQLTuner-perl/issues/289). Đ§Ń‚ĐŸ ĐžĐŒĐ”ĐœĐœĐŸ ĐżŃ€ĐŸĐČĐ”Ń€ŃĐ”Ń‚ MySQLTuner? -- -ВсД ĐżŃ€ĐŸĐČДрĐșĐž, ĐČŃ‹ĐżĐŸĐ»ĐœŃĐ”ĐŒŃ‹Đ” **MySQLTuner**, Đ·Đ°ĐŽĐŸĐșŃƒĐŒĐ”ĐœŃ‚ĐžŃ€ĐŸĐČĐ°ĐœŃ‹ ĐČ ĐŽĐŸĐșŃƒĐŒĐ”ĐœŃ‚Đ°Ń†ĐžĐž [MySQLTuner Internals](https://github.com/jmrenouard/MySQLTuner-perl/blob/master/INTERNALS.md). +ВсД ĐżŃ€ĐŸĐČДрĐșĐž, ĐČŃ‹ĐżĐŸĐ»ĐœŃĐ”ĐŒŃ‹Đ” **MySQLTuner** Đ·Đ°ĐŽĐŸĐșŃƒĐŒĐ”ĐœŃ‚ĐžŃ€ĐŸĐČĐ°ĐœŃ‹ ĐČ ĐŽĐŸĐșŃƒĐŒĐ”ĐœŃ‚Đ°Ń†ĐžĐž [MySQLTuner Internals](https://github.com/major/MySQLTuner-perl/blob/master/INTERNALS.md). **MySQLTuner** Đ°ĐœĐ°Đ»ĐžĐ·ĐžŃ€ŃƒĐ”Ń‚ ŃĐ»Đ”ĐŽŃƒŃŽŃ‰ĐžĐ” ĐŸĐ±Đ»Đ°ŃŃ‚Đž: @@ -190,14 +208,14 @@ GRANT SELECT, PROCESS, EXECUTE, REPLICATION CLIENT, SHOW DATABASES, SHOW VIEW ON ```bash wget https://mysqltuner.pl/ -O mysqltuner.pl -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt -wget https://raw.githubusercontent.com/jmrenouard/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/basic_passwords.txt -O basic_passwords.txt +wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/vulnerabilities.csv -O vulnerabilities.csv ``` 2) Вы ĐŒĐŸĐ¶Đ”Ń‚Đ” Đ·Đ°ĐłŃ€ŃƒĐ·ĐžŃ‚ŃŒ ĐČĐ”ŃŃŒ Ń€Đ”ĐżĐŸĐ·ĐžŃ‚ĐŸŃ€ĐžĐč, ĐžŃĐżĐŸĐ»ŃŒĐ·ŃƒŃ `git clone` ОлО `git clone --depth 1 -b master`, Đ·Đ° ĐșĐŸŃ‚ĐŸŃ€Ń‹ĐŒ ŃĐ»Đ”ĐŽŃƒĐ”Ń‚ URL-аЎрДс ĐșĐ»ĐŸĐœĐžŃ€ĐŸĐČĐ°ĐœĐžŃ ĐČŃ‹ŃˆĐ”. ```bash -git clone --depth 1 -b master https://github.com/jmrenouard/MySQLTuner-perl.git +git clone --depth 1 -b master https://github.com/major/MySQLTuner-perl.git ``` 3) На Apple macOS ŃƒŃŃ‚Đ°ĐœĐŸĐČОтД чДрДз [Homebrew](https://brew.sh/): @@ -217,8 +235,8 @@ docker run --rm -it jmrenouard/mysqltuner --host --user Derived from the test campaign analysis on v2.8.43. Addresses critical code quality issues identified during the 5-iteration test audit. -* [ ] **Perl Warning Elimination**: - * [ ] Add definedness guards to `mysql_version_ge()`, `mysql_version_le()`, `mysql_version_eq()` to prevent 74 uninitialized value warnings. - * [ ] Guard `$mycalc{'innodb_log_size_pct'}` and `$myvar{'innodb_log_file_size'}` before use in InnoDB analysis. - * [ ] Guard `$myvar{'version_comment'}` in MariaDB detection path. +* [x] **Perl Warning Elimination**: + * [x] Add definedness guards to `mysql_version_ge()`, `mysql_version_le()`, `mysql_version_eq()` to prevent 74 uninitialized value warnings. + * [x] Guard `$mycalc{'innodb_log_size_pct'}` and `$myvar{'innodb_log_file_size'}` before use in InnoDB analysis. + * [x] Guard `$myvar{'version_comment'}` in MariaDB detection path. * [x] **Version Validation Updates**: * [x] Add MySQL 9.6 to `validate_mysql_version()` supported LTS list. * [x] Remove MySQL 9.5 (now Outdated) from the LTS list. -* [ ] **Test Coverage Expansion**: - * [ ] Achieve ≄80% subroutine test coverage (currently ~55%, 74 of 165 uncovered). - * [ ] Priority coverage: `check_architecture`, `system_recommendations`, `mysql_indexes`, `mysql_views`, `mysql_routines`, `mysql_triggers`, `make_recommendations`. - * [ ] Add tests for `dump_result`, `close_outputfile`, `get_template_model`. -* [ ] **Version Comparison Optimization**: - * [ ] Cache parsed version components instead of re-parsing `$myvar{'version'}` on every call to `mysql_version_ge/le/eq`. +* [x] **Test Coverage Expansion**: + * [x] Achieve ≄80% subroutine test coverage (reached ~92%, only 13 of 167 system/IO-heavy subroutines uncovered). + * [x] Priority coverage: `check_architecture`, `system_recommendations`, `mysql_indexes`, `mysql_views`, `mysql_routines`, `mysql_triggers`, `make_recommendations`. + * [x] Add tests for `dump_result` and `close_outputfile` (`get_template_model` obsoleted and removed). +* [x] **Version Comparison Optimization**: + * [x] Cache parsed version components instead of re-parsing `$myvar{'version'}` on every call to `mysql_version_ge/le/eq`. --- @@ -203,18 +205,18 @@ To ensure consistency and high-density development, the following roles are defi * [ ] **Correlation Engine (Experimental)**: * [ ] **Temporal Event Linking**: Logic to link error log timestamps with Performance Schema wait events or high CPU load detected during execution. -### [Phase 13: Sectional Global Indicators & KPIs](file:///documentation/specifications/roadmap_phase_xii_sectional_indicators.md) [NOT STARTED] +### [Phase 13: Sectional Global Indicators & KPIs](file:///documentation/specifications/roadmap_phase_xii_sectional_indicators.md) [COMPLETED] > Previously Phase 12. -* [ ] **Unified Health Dashboard**: - * [ ] **Sectional Health Scoring**: Implementation of a 0-100 KPI for each major diagnostic area (Storage Engine, Security, Replication, SQL Modeling). - * [ ] **Critical Findings Executive Summary**: Automated prioritization of the top 3 items per section with color-coded badges (🔮 Critical, 🟡 Finding, 🟱 Optimal). -* [ ] **Efficiency & Resource Mapping**: - * [ ] **Throughput Efficiency Index**: Real-time ratio analysis of logical work (Queries/sec) vs physical resource consumption (`Innodb_buffer_pool_read_requests`). - * [ ] **Resource Saturation Heatmap**: Visual representation of proximity to system limits (CPU/MEM/IO/Connections). -* [ ] **Comparative Insights**: - * [ ] **Historical Performance Deltas**: Sectional trend analysis identifying areas of performance regression or improvement based on previous run data. +* [x] **Unified Health Dashboard**: + * [x] **Sectional Health Scoring**: Implementation of a 0-100 KPI for each major diagnostic area (Storage Engine, Security, Replication, SQL Modeling). + * [x] **Critical Findings Executive Summary**: Automated prioritization of the top 3 items per section with color-coded badges (🔮 Critical, 🟡 Finding, 🟱 Optimal). +* [x] **Efficiency & Resource Mapping**: + * [x] **Throughput Efficiency Index**: Real-time ratio analysis of logical work (Queries/sec) vs physical resource consumption (`Innodb_buffer_pool_read_requests`). + * [x] **Resource Saturation Heatmap**: Visual representation of proximity to system limits (CPU/MEM/IO/Connections). +* [x] **Comparative Insights**: + * [x] **Historical Performance Deltas**: Sectional trend analysis identifying areas of performance regression or improvement based on previous run data. ### [Phase 14: Export Optimization & Dumpdir Hardening](file:///documentation/specifications/roadmap_phase_xiii_export_optimization.md) [COMPLETED] @@ -229,7 +231,36 @@ To ensure consistency and high-density development, the following roles are defi * [x] **Compression & Efficiency**: * [x] **On-the-fly Compression**: Support for compressed `.gz` exports to minimize disk footprint in container/limited-storage environments. -## 🔼 Strategic Technical Evolutions +### [Phase 15: Interactive Multi-Page HTML Reports & Detailed Exports](file:///documentation/specifications/roadmap_phase_xiv_html_reports.md) [COMPLETED] + +* [x] **Summary Page Dashboard**: + * [x] Executive summary layout with a modern circular health score gauge, category scores breakdown, and top findings. +* [x] **Topic-Based Metrics Partitioning**: + * [x] Structure the report into tabs/views: Memory, Connections, Storage Engines, Performance, Security, SQL Modeling, Replication. +* [x] **SVG/CSS-Based Ratios Visualization**: + * [x] Render interactive bars/gauges for InnoDB buffer pool hit rate, thread cache hit rate, disk temp tables, and connection saturation. +* [x] **Embedded CSV Data Exports**: + * [x] Embed base64 or raw string CSV representation of variables and findings in JavaScript, enabling instant local CSV downloads. + +### [Phase 16: AI Agent Integration & Actionable JSON Schema](file:///documentation/specifications/roadmap_phase_xv_ai_agent_integration.md) [NOT STARTED] + +* [ ] **Structured Actionable JSON Output**: + * [ ] Implementation of `--agent-json` flag returning a standardized schema. +* [ ] **Expected Outcomes & Rollback Statements**: + * [ ] Each recommendation includes explicit expected outcome description and corresponding rollback statement. +* [ ] **Risk Assessment & Impact Scoring**: + * [ ] Assign deterministic impact score (1-10) and category/risk level to each recommendation. + +### [Phase 17: Dockerized Auditing Daemon & MCP Server Support](file:///documentation/specifications/roadmap_phase_xvi_mcp_server.md) [NOT STARTED] + +* [ ] **Interval Auditing Daemon**: + * [ ] Dockerized execution environment running auditing loops every X hours with caching. +* [ ] **Model Context Protocol (MCP) Server**: + * [ ] Expose caching layer, latest results, and immediate auditing as MCP tools and resources. +* [ ] **Safe execution & Rollbacks**: + * [ ] Implement secure database interaction tools to apply or rollback recommendations. + +## 🔼 [Strategic Technical Evolutions](file:///documentation/specifications/strategic_technical_evolutions.md) * [ ] Set up a pipeline to automatically audit and verify reference link availability inside the repository documentation to prevent dead links. * [ ] Integrate standard documentation reference anchors dynamically within MySQLTuner CLI help screens and specific advisor output blocks. diff --git a/SECURITY.md b/SECURITY.md index 0ec300acf..99c12f082 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -8,7 +8,7 @@ We provide security updates for the following versions of MySQLTuner: | Version | Status | | ------- | --------------------- | -| v2.x | Supported (v2.8.44) | +| v2.x | Supported (v2.9.0) | | < v2.x | End of Life | We strongly recommend that all users stay updated with the latest stable release available on [GitHub Releases](https://github.com/jmrenouard/MySQLTuner-perl/releases). diff --git a/USAGE.md b/USAGE.md index b25e01b5f..010c04d38 100644 --- a/USAGE.md +++ b/USAGE.md @@ -1,6 +1,6 @@ # NAME - MySQLTuner 2.8.45 - MySQL High Performance Tuning Script + MySQLTuner 2.9.0 - MySQL High Performance Tuning Script # IMPORTANT USAGE GUIDELINES @@ -15,7 +15,7 @@ See `mysqltuner --help` for a full list of available options and their categorie # VERSION -Version 2.8.45 +Version 2.9.0 =head1 PERLDOC You can find documentation for this module with the perldoc command. @@ -72,6 +72,7 @@ Jean-Marie Renouard - jmrenouard@gmail.com - Stephan GroBberndt - Christian Loos - Long Radix +- derZ-dev # SUPPORT diff --git a/build/check_compliance.pl b/build/check_compliance.pl index 8a09d9c00..3e70e69f3 100755 --- a/build/check_compliance.pl +++ b/build/check_compliance.pl @@ -139,7 +139,9 @@ 'versions', 'report', 'security', 'cve', 'options', 'lab', 'container', 'refactor', 'style', 'releases', 'dependencies', 'cli', - 'auth', 'main', 'metadata' + 'auth', 'main', 'metadata', 'deps', + 'system', 'roadmap', 'hook', 'hooks', + 'build' ); # Lint Changelog structure and scopes for the current version block diff --git a/build/dev_sync.pl b/build/dev_sync.pl new file mode 100755 index 000000000..b89baf9d5 --- /dev/null +++ b/build/dev_sync.pl @@ -0,0 +1,258 @@ +#!/usr/bin/env perl +use strict; +use warnings; +use File::Basename; +use File::Spec; +use Cwd 'abs_path'; +use POSIX 'strftime'; + +my $project_root = abs_path(File::Spec->catfile(dirname(abs_path(__FILE__)), '..')); +my $CHANGELOG_PATH = File::Spec->catfile($project_root, 'Changelog'); +my $VERSION_PATH = File::Spec->catfile($project_root, 'CURRENT_VERSION.txt'); + +sub log_msg { + my ($msg) = @_; + my $timestamp = strftime("%Y-%m-%d %H:%M:%S", localtime); + print "[$timestamp] [DEV-SYNC] $msg\n"; +} + +sub get_current_version { + open my $fh, '<', $VERSION_PATH or die "Cannot read CURRENT_VERSION.txt: $!"; + my $ver = <$fh>; + close $fh; + $ver =~ s/^\s+|\s+$//g; + return $ver; +} + +sub normalize { + my ($str) = @_; + $str = lc($str); + $str =~ s/[^a-z0-9]//g; + return $str; +} + +sub process_items { + my ($existing_ref, $new_ref) = @_; + my @all_items = @$existing_ref; + + # Add new items if not duplicates + for my $new_item (@$new_ref) { + my $new_norm = normalize($new_item); + my $is_dup = 0; + for my $exist (@all_items) { + my $exist_norm = normalize($exist); + if ($new_norm eq $exist_norm || index($exist_norm, $new_norm) != -1 || index($new_norm, $exist_norm) != -1) { + $is_dup = 1; + last; + } + } + if (!$is_dup) { + push @all_items, "$new_item\n"; + } + } + + # Sort + my %categories = ( + 'chore' => 1, + 'feat' => 2, + 'fix' => 3, + 'test' => 4, + 'ci' => 5, + ); + + my @sorted = sort { + my $type_a = ''; + my $type_b = ''; + if ($a =~ /^\s*-\s*(\w+)/) { $type_a = lc($1); } + if ($b =~ /^\s*-\s*(\w+)/) { $type_b = lc($1); } + + my $rank_a = $categories{$type_a} // 99; + my $rank_b = $categories{$type_b} // 99; + + if ($rank_a != $rank_b) { + return $rank_a <=> $rank_b; + } else { + return $a cmp $b; + } + } @all_items; + + return join("", @sorted); +} + +sub update_changelog_file { + my ($version, $new_items_ref) = @_; + open my $fh, '<', $CHANGELOG_PATH or die "Cannot read Changelog: $!"; + my @lines = <$fh>; + close $fh; + + my @output_lines; + my $in_current_version = 0; + my @existing_items; + + my $today = strftime("%Y-%m-%d", localtime); + my $header_pattern = qr/^(\d+\.\d+\.\d+)\s+(\d{4}-\d{2}-\d{2})\s*$/; + my $found_version = 0; + + for my $line (@lines) { + if ($line =~ $header_pattern) { + my $v = $1; + if ($in_current_version) { + push @output_lines, process_items(\@existing_items, $new_items_ref); + push @output_lines, "\n"; + $in_current_version = 0; + } + if ($v eq $version) { + $in_current_version = 1; + $found_version = 1; + push @output_lines, "$version $today\n\n"; + next; + } + } + + if ($in_current_version) { + if ($line =~ /^\s*-\s*(.*)/) { + push @existing_items, $line; + } + } else { + push @output_lines, $line; + } + } + + if ($in_current_version) { + push @output_lines, process_items(\@existing_items, $new_items_ref); + push @output_lines, "\n"; + } + + if (!$found_version) { + my @new_changelog; + my $inserted = 0; + for my $line (@output_lines) { + if (!$inserted && $line =~ $header_pattern) { + push @new_changelog, "$version $today\n\n"; + push @new_changelog, process_items([], $new_items_ref); + push @new_changelog, "\n"; + $inserted = 1; + } + push @new_changelog, $line; + } + @output_lines = @new_changelog; + } + + open my $wfh, '>', $CHANGELOG_PATH or die "Cannot write to Changelog: $!"; + print $wfh join("", @output_lines); + close $wfh; + + log_msg("Changelog successfully updated and sorted."); + return 1; +} + +sub main { + log_msg("Starting Developer Sync Process..."); + + # 1. Verify version consistency + log_msg("Step 1/4: Checking version consistency..."); + my $res = system("perl tests/version_consistency.t"); + if ($res != 0) { + log_msg("FAIL: Version consistency checks failed!"); + exit(1); + } + log_msg("OK: Version consistency verified."); + + # 2. Extract commits and update Changelog & Release Notes + log_msg("Step 2/4: Extracting commits and updating Changelog / Release Notes..."); + my $version = get_current_version(); + + my $prev_tag = qx(git describe --tags --abbrev=0 2>/dev/null); + chomp($prev_tag); + if (!$prev_tag) { + $prev_tag = "master"; + } + + log_msg("Comparing current branch against base ref: $prev_tag"); + my @commits_raw = qx(git log $prev_tag..HEAD --pretty=format:%s); + + my @new_items; + for my $line (@commits_raw) { + chomp($line); + next unless $line; + if ($line =~ /^(\w+)(?:\(([^)]+)\))?(!)?:\s*(.*)/) { + my $type = lc($1); + my $scope = $2; + my $desc = $4; + $desc =~ s/^\s+|\s+$//g; + my $scope_str = $scope ? "($scope)" : ""; + push @new_items, "- $type$scope_str: $desc"; + } + } + + if (@new_items) { + log_msg("Found " . scalar(@new_items) . " new conventional commits to sync."); + update_changelog_file($version, \@new_items); + } else { + log_msg("No new conventional commits found since last tag."); + } + + log_msg("Regenerating release notes file..."); + my $rel_notes_res = system("python3 build/release_gen.py"); + if ($rel_notes_res != 0) { + log_msg("FAIL: Release notes generation failed!"); + exit(1); + } + log_msg("OK: Release notes regenerated."); + + # 3. Pass unit tests + log_msg("Step 3/4: Passing unit tests..."); + my $test_res = system("perl build/audit_tests.pl"); + if ($test_res != 0) { + log_msg("FAIL: Unit tests failed!"); + exit(1); + } + log_msg("OK: All unit tests passed successfully."); + + # 4. Commit and Push + log_msg("Step 4/4: Committing and pushing changes..."); + my @status_lines = qx(git status --porcelain); + my @modified_files; + for my $line (@status_lines) { + chomp($line); + next unless $line; + if ($line =~ /(Changelog|releases\/)/) { + my $file = $line; + $file =~ s/^\s*\S+\s+//; + push @modified_files, $file; + } + } + + if (@modified_files) { + log_msg("Staging and committing files: " . join(", ", @modified_files)); + for my $f (@modified_files) { + system("git add \"$f\""); + } + my $commit_res = system("git commit -m \"docs: regenerate release notes\""); + if ($commit_res != 0) { + log_msg("FAIL: Git commit failed!"); + exit(1); + } + log_msg("Commit successful."); + } else { + log_msg("No documentation changes to commit."); + } + + my $branch = qx(git branch --show-current); + chomp($branch); + if (!$branch) { + $branch = "HEAD"; + } + my $refspec = $branch ne "HEAD" ? "refs/heads/$branch" : "HEAD"; + log_msg("Pushing current branch '$branch' to origin using refspec '$refspec'..."); + my $push_res = system("git push origin \"$refspec\""); + if ($push_res != 0) { + log_msg("FAIL: Git push failed!"); + exit(1); + } + log_msg("OK: Git push successful. Sync completed."); + log_msg("All steps completed successfully."); +} + +main(); +exit 0; diff --git a/build/doc_sync.pl b/build/doc_sync.pl new file mode 100755 index 000000000..ddad5f591 --- /dev/null +++ b/build/doc_sync.pl @@ -0,0 +1,90 @@ +#!/usr/bin/env perl +use strict; +use warnings; +use File::Basename; +use File::Spec; +use Cwd 'abs_path'; + +my $project_root = abs_path(File::Spec->catfile(dirname(abs_path(__FILE__)), '..')); +my $agent_dir = File::Spec->catfile($project_root, '.agent'); +my $readme_path = File::Spec->catfile($agent_dir, 'README.md'); + +sub parse_markdown_metadata { + my ($file_path) = @_; + open my $fh, '<', $file_path or return (basename($file_path), "Error: $!"); + my $content = do { local $/; <$fh> }; + close $fh; + + my $title = basename($file_path); + if ($content =~ /^#\s+(.*)/m) { + $title = $1; + $title =~ s/^\s+|\s+$//g; + } + + my $description = "No description available."; + if ($content =~ /description:\s*(.*)/) { + $description = $1; + $description =~ s/^\s+|\s+$//g; + } + + return ($title, $description); +} + +sub generate_readme { + my %categories = ( + 'rules' => 'Governance & Execution Constraints', + 'skills' => 'Specialized Capabilities & Knowledge', + 'workflows' => 'Automation & Operational Workflows' + ); + + # Keep a fixed iteration order for predictability: rules -> skills -> workflows + my @cat_order = ('rules', 'skills', 'workflows'); + + my @output; + push @output, "# .agent - Project Governance & Artificial Intelligence Intelligence\n"; + push @output, "This directory contains the project's technical constitution, specialized skills, and operational workflows used by AI agents.\n"; + + for my $folder (@cat_order) { + my $cat_title = $categories{$folder}; + my $folder_path = File::Spec->catdir($agent_dir, $folder); + next unless -d $folder_path; + + push @output, "## $cat_title\n"; + push @output, "| File | Description |"; + push @output, "| :--- | :--- |"; + + opendir(my $dh, $folder_path) or die "Cannot open $folder_path: $!"; + my @files = sort grep { $_ ne '.' && $_ ne '..' } readdir($dh); + closedir($dh); + + for my $filename (@files) { + my $full_path = File::Spec->catfile($folder_path, $filename); + if (! -f $full_path) { + # Handle skill folders containing SKILL.md + my $skill_path = File::Spec->catfile($folder_path, $filename, 'SKILL.md'); + if (-f $skill_path) { + my (undef, $desc) = parse_markdown_metadata($skill_path); + push @output, "| [`$filename/`](./$folder/$filename/SKILL.md) | $desc |"; + } + next; + } + next unless $filename =~ /\.md$/; + + my (undef, $desc) = parse_markdown_metadata($full_path); + push @output, "| [`$filename`](./$folder/$filename) | $desc |"; + } + + push @output, "\n"; + } + + push @output, "---\n*Generated automatically by `/doc-sync`*"; + + open my $fh, '>', $readme_path or die "Cannot write to $readme_path: $!"; + print $fh join("\n", @output); + close $fh; + + print "Documentation synchronized: $readme_path\n"; +} + +generate_readme(); +exit 0; diff --git a/build/doc_sync.py b/build/doc_sync.py deleted file mode 100644 index d5b0af482..000000000 --- a/build/doc_sync.py +++ /dev/null @@ -1,69 +0,0 @@ -#!/usr/bin/env python3 -import os -import re -from datetime import datetime - -PROJECT_ROOT = os.getcwd() -AGENT_DIR = os.path.join(PROJECT_ROOT, '.agent') -README_PATH = os.path.join(AGENT_DIR, 'README.md') - -def parse_markdown_metadata(file_path): - try: - with open(file_path, 'r') as f: - content = f.read() - - # Extract title (first # header) - title_match = re.search(r'^#\s+(.*)', content, re.MULTILINE) - title = title_match.group(1).strip() if title_match else os.path.basename(file_path) - - # Extract description from frontmatter - desc_match = re.search(r'description:\s*(.*)', content) - description = desc_match.group(1).strip() if desc_match else "No description available." - - return title, description - except Exception as e: - return os.path.basename(file_path), f"Error parsing: {str(e)}" - -def generate_readme(): - categories = { - 'rules': 'Governance & Execution Constraints', - 'skills': 'Specialized Capabilities & Knowledge', - 'workflows': 'Automation & Operational Workflows' - } - - output = ["# .agent - Project Governance & Artificial Intelligence Intelligence\n"] - output.append("This directory contains the project's technical constitution, specialized skills, and operational workflows used by AI agents.\n") - - for folder, cat_title in categories.items(): - folder_path = os.path.join(AGENT_DIR, folder) - if not os.path.exists(folder_path): - continue - - output.append(f"## {cat_title}\n") - output.append("| File | Description |") - output.append("| :--- | :--- |") - - files = sorted(os.listdir(folder_path)) - for filename in files: - if not filename.endswith('.md'): - # Handle skill folders (SKILL.md inside) - skill_path = os.path.join(folder_path, filename, 'SKILL.md') - if os.path.exists(skill_path): - _, desc = parse_markdown_metadata(skill_path) - output.append(f"| [`{filename}/`](./{folder}/{filename}/SKILL.md) | {desc} |") - continue - - _, desc = parse_markdown_metadata(os.path.join(folder_path, filename)) - output.append(f"| [`{filename}`](./{folder}/{filename}) | {desc} |") - - output.append("\n") - - output.append("---\n*Generated automatically by `/doc-sync` on " + datetime.now().strftime("%Y-%m-%d %H:%M:%S") + "*") - - with open(README_PATH, 'w') as f: - f.write("\n".join(output)) - - print(f"Documentation synchronized: {README_PATH}") - -if __name__ == "__main__": - generate_readme() diff --git a/documentation/specifications/roadmap_phase_xiv_html_reports.md b/documentation/specifications/roadmap_phase_xiv_html_reports.md new file mode 100644 index 000000000..fe1d50eb9 --- /dev/null +++ b/documentation/specifications/roadmap_phase_xiv_html_reports.md @@ -0,0 +1,123 @@ +# Specification: Roadmap Phase XIV - Interactive Multi-Page HTML Reports & Detailed Exports + +- **Feature Name**: Interactive Multi-Page HTML Reports & Detailed Exports +- **Status**: Approved +- **Created Date**: 2026-06-25 +- **Last Updated**: 2026-07-05 + +## 🧠 Rationale + +As MySQLTuner-perl reports grow in complexity, a single long scrolling page of recommendations is no longer sufficient for database administrators and managers. To align with advanced diagnostic tooling (such as pgBadger for PostgreSQL or the MT-reporter suite) while keeping a native, zero-dependency, single-file Perl architecture, the built-in HTML report (`--reportfile`) must deliver a high level of information, visual graphs, and actionable remediation steps. + +This specification details the structure, indicators, and formatting of the multi-page HTML report generated natively by `mysqltuner.pl`, drawing inspiration from pgBadger's multi-dimensional telemetry, query metrics, locking analytics, and event/error distributions. + +--- + +## đŸ› ïž User Scenarios + +### Scenario 1: Executive Review of Database Health +A database manager runs MySQLTuner to get a quick summary. They open the generated HTML report and see a **Summary Dashboard Page** with a circular health gauge, key KPIs, resource saturation indicators, and the top findings across all areas. They do not have to dig through technical logs to assess general health. + +### Scenario 2: Topic-Specific Deep Dive with Graphs +A database administrator (DBA) is troubleshooting InnoDB performance. They open the report, click on the **Storage Engines & InnoDB** tab, and view: +- An SVG chart of the buffer pool hit rate. +- A table listing InnoDB status variables, current values, and recommended settings. +- A prioritized list of recommendations for InnoDB. + +### Scenario 3: Query & Locking Performance Troubleshooting +A DBA identifies performance degradation and navigates to the **Queries & Top Queries** tab to locate the most resource-intensive normalized query statements. In the **Locks & Latency** tab, they trace wait histograms and check for deadlock occurrences without needing to parse raw error logs or run ad-hoc command-line queries. + +### Scenario 4: Exporting Raw Diagnostic Data to CSV +A developer wants to import the parsed database metrics into Excel to perform a custom analysis. They go to the report's **Data Export** tab, see options to download separate CSVs (e.g., variables, status, schema findings, security settings), click "Download CSV", and instantly save the files locally. + +--- + +## 📋 Level of Information & Schema Specification + +The native HTML report is structured as an interactive SPA (Single Page Application) with the following detailed metrics and sections: + +### 1. Dashboard (Summary View) +- **Circular Health Gauge**: SVG-based animated indicator showing overall score (0-100) with dynamic status colors (Optimal, Good, Action Required). +- **Category Scores**: Structured cards showing metrics for Performance, Security, and Resource Saturation. +- **System Metadata Banner**: Quick facts about the target database (Version, Port, Uptime, Host name, Concurrency). +- **Traffic Overview**: Simple dashboard indicators representing queries/sec (QPS), average throughput (bytes received/sent per second), and Select vs Write ratio. + +### 2. System & Memory Analytics +- **OS Resource Saturation**: Detailed breakdown of physical RAM vs. swap usage. +- **Per-Thread & Global Allocation**: Graph representing maximum possible memory allocation compared to physical limits. + +### 3. Connections & Sessions +- **Connection Capacity**: Maximum concurrent connections vs. highest historical usage. +- **Cache Hit Rates**: Thread cache hit rate and connection abort percentages. +- **Concurrency & Client Distribution**: Established connections grouped by user, host, or database (when Performance Schema or processlist snapshots are available). + +### 4. Queries & Execution Analytics (pgBadger-Inspired) +- **Query Types Distribution**: Visual breakdown of query categories (SELECT, INSERT, UPDATE, DELETE, REPLACE, admin/DDL operations). +- **Prepared Statements Utilization**: Ratio of prepared/executed statements vs. raw dynamic SQL. +- **Query Hits & Misses**: Analysis of query cache hits (where applicable) and database read/write ratios. + +### 5. Top Queries & Latency Diagnostics (pgBadger-Inspired) +- **Slowest Queries**: Top N slowest query statements with execution times and user/host origin. +- **Time-Consuming Queries**: Normalized query patterns (with placeholders) sorted by cumulative execution time. +- **Most Frequent Queries**: Most frequent query patterns sorted by call counts with execution statistics (min, max, mean, stddev). +- **Disk Spill Queries**: Queries that triggered the creation of temporary tables on disk. + +### 6. Locks & Latency Analytics (pgBadger-Inspired) +- **Lock Saturation**: Table locks immediate vs. waited ratios, showing locking overhead. +- **Row-Level Locking**: Cumulative row lock wait counts, total lock wait duration, and average row lock wait times. +- **Deadlock Analysis**: Captured deadlock occurrences with timestamps, involved threads, and offending transaction details. + +### 7. Temporary Tables & Memory Spills (pgBadger-Inspired) +- **Temporary Tables Metrics**: Ratio of memory-based temporary tables to disk-based temporary tables. +- **Temporary Table Activity**: Time-series estimation of temporary table creation rates. + +### 8. Storage Engines & InnoDB Forensics +- **Storage Breakdown**: Comprehensive overview of enabled engines. +- **InnoDB Engine Detailed Diagnostics**: + - Buffer pool instances and chunk size alignment status (Aligned vs. Not Aligned). + - Page usage details (Total/Free/Used) converted into bytes. + - Log capacity or file size details (including total log size, group size, and log-to-buffer pool ratio). + - Concurrency parameters (`innodb_thread_concurrency`) and read buffer efficiency. + - Hourly InnoDB OS log write workload rate. + +### 9. SQL Modeling & Schema Audit +- **User Databases Size Distribution**: Schema name, table counts, rows count, data size, index size, and total size. +- **Fragmented Tables Details**: Schema, table, engine type, free space (MiB), and auto-generated defragmentation SQL queries (`ALTER TABLE ... FORCE` or `OPTIMIZE TABLE ...`). +- **Tables Without Primary Keys**: Detailed list of schemas and tables missing a PK with warning descriptions. +- **Redundant & Unused Indexes**: List of duplicate or non-queried indexes along with drop queries (`ALTER TABLE ... DROP INDEX ...`). + +### 10. Security & CVE Exposures +- **Authentication Plugin Audit**: Detailed validation of user authentication plugins against the security support matrix. +- **SSL/TLS Ciphers**: Verification of transport encryption rules and required SSL connection protocols. +- **CVE Database Analysis**: Structured list of matching CVE vulnerability records based on the target MySQL/MariaDB version. + +### 11. Replication & Galera Cluster Status +- **Replication Topology**: Standalone vs. Master/Slave relationship, replication lag times, and binlog formats. +- **Galera Synchronization**: Status of clustering synchrony and Galera node metrics. + +### 12. Error Logs & Events Audit (pgBadger-Inspired) +- **Log Events Summary**: Total counts of Errors, Warnings, and Notes parsed from the server error log (`--log-error`). +- **Event Distribution**: Hourly graph of warning and error events. +- **Frequent Log Signatures**: Top recurring messages or diagnostic alerts in the error logs. + +### 13. Integrated Data Export & Actions +- **rem_queries (Actionable Remediations)**: Ready-to-copy SQL and configuration snippets. +- **Dynamic CSV Downloads**: Inline browser-generated CSV files (prefixed with host, version, and timestamp) for databases, tables, variables, and status metrics. + +--- + +## 🔬 Verification Plan + +### Automated Tests +1. **Perl Syntax Validation**: The HTML generation block must be warning-free under `perl -cw mysqltuner.pl`. +2. **Unit Test Assertions**: [tests/html_report.t](file:///home/jmren/GIT_REPOS/MySQLTuner-perl/tests/html_report.t) must mock database status variables, schema lists, and verification logs, asserting that the generated HTML file matches the target SPA layout regex patterns. + +### Manual Verification +1. Generate the HTML report: + ```bash + perl mysqltuner.pl --reportfile=test_report.html + ``` +2. Open the file in a standard browser environment and assert that: + - All interactive tabs (Dashboard, Storage, Modeling, Security, Queries, Locks, Events, etc.) function offline. + - All SVG charts and gauges load and format correctly. + - The CSV download buttons trigger local downloads with the correct format headers. diff --git a/documentation/specifications/roadmap_phase_xv_ai_agent_integration.md b/documentation/specifications/roadmap_phase_xv_ai_agent_integration.md new file mode 100644 index 000000000..f508d3f38 --- /dev/null +++ b/documentation/specifications/roadmap_phase_xv_ai_agent_integration.md @@ -0,0 +1,131 @@ +# Specification: Roadmap Phase XVI - AI Agent Integration & Actionable JSON Schema + +- **Feature Name**: AI Agent Integration & Actionable JSON Schema +- **Status**: Draft +- **Created Date**: 2026-06-25 + +## 🧠 Rationale + +As database operations become increasingly automated by AI agents (e.g., MCP-based workflows, specialized IDE agents, autonomous DBAs), standard human-readable terminal output and even simple YAML/JSON key-value exports are insufficient. + +To safely enable an AI agent to act on database advisor findings, the agent needs: +1. **Machine-Ingestible Action Schema**: Structured recommendations that specify the exact SQL statements (DDL/DML) or configuration adjustments required. +2. **Deterministic Rollback Action**: The precise inverse command to undo the action in case of performance regressions or unexpected failure. +3. **Safety & Risk Categorization**: An explicit risk assessment (e.g., whether a table lock is required, if a restart is needed) and an impact score/topic. +4. **Expected Outcomes**: Declarative descriptions of what the change will improve (e.g., memory usage reduction, cache hit rate increase). + +This phase adds a robust `--agent-json` output flag to `mysqltuner.pl` that outputs a highly structured, machine-actionable JSON payload. + +## đŸ› ïž User Scenarios + +### Scenario 1: Autonomous Tuning Agent +An AI agent connects to a database via an MCP server, runs MySQLTuner with `--agent-json`, parses the recommendations, and automatically applies P1 (high impact, low risk) recommendations using the provided `sql_statement`. + +### Scenario 2: Human-in-the-Loop Safe Rollback +An AI agent proposes a schema optimization (e.g. dropping a redundant index). It shows the user the description, expected outcome, and the provided `rollback_statement`. If the user consents, the agent executes the change. If a regression occurs later, the agent executes the `rollback_statement` to revert to the baseline. + +## 📋 JSON Schema Specification + +The `--agent-json` format will output a JSON object containing a `findings` list. Each finding conforms to the following schema: + +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "AgentFinding", + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Unique deterministic identifier for the recommendation type." + }, + "topic": { + "type": "string", + "enum": ["Performance", "Security", "Reliability", "Modeling", "Replication"], + "description": "The category of the finding." + }, + "description": { + "type": "string", + "description": "Human-readable explanation of the issue." + }, + "impact_score": { + "type": "integer", + "minimum": 1, + "maximum": 10, + "description": "Estimated benefit score from 1 (low) to 10 (critical)." + }, + "risk_level": { + "type": "string", + "enum": ["Low", "Medium", "High", "Critical"], + "description": "Risk associated with executing the recommendation." + }, + "risk_description": { + "type": "string", + "description": "Details of potential side effects (e.g. table locks, memory growth, restart required)." + }, + "requires_restart": { + "type": "boolean", + "description": "True if the configuration change requires a database service restart." + }, + "expected_outcome": { + "type": "string", + "description": "The expected performance or security benefit." + }, + "action": { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["SQL", "Config", "OS"], + "description": "Type of resolution action." + }, + "statement": { + "type": "string", + "description": "The exact SQL statement, configuration line, or shell command to execute." + }, + "rollback_statement": { + "type": "string", + "description": "The exact command to revert the action to its original state." + } + }, + "required": ["type", "statement", "rollback_statement"] + } + }, + "required": [ + "id", + "topic", + "description", + "impact_score", + "risk_level", + "risk_description", + "requires_restart", + "expected_outcome", + "action" + ] +} +``` + +### Finding Example +```json +{ + "id": "innodb_buffer_pool_size_adjust", + "topic": "Performance", + "description": "InnoDB buffer pool size is under-allocated for the current workload.", + "impact_score": 9, + "risk_level": "Medium", + "risk_description": "Increases memory consumption. Ensure sufficient OS-free RAM to prevent OOM swapping.", + "requires_restart": false, + "expected_outcome": "Reduces disk I/O and increases query cache read hits.", + "action": { + "type": "SQL", + "statement": "SET GLOBAL innodb_buffer_pool_size = 1073741824;", + "rollback_statement": "SET GLOBAL innodb_buffer_pool_size = 134217728;" + } +} +``` + +## ✅ Verification Plan + +### Automated Tests +1. Test generation of `--agent-json` output and validate its conformance to the schema. +2. Validate that each generated recommendation has a corresponding valid `rollback_statement`. +3. Verify that parsing of database metrics properly populates all metadata fields (e.g., `impact_score`, `risk_level`, `requires_restart`). diff --git a/documentation/specifications/roadmap_phase_xvi_mcp_server.md b/documentation/specifications/roadmap_phase_xvi_mcp_server.md new file mode 100644 index 000000000..d141419bf --- /dev/null +++ b/documentation/specifications/roadmap_phase_xvi_mcp_server.md @@ -0,0 +1,71 @@ +# Specification: Roadmap Phase XVII - Dockerized Auditing Daemon & MCP Server Support + +- **Feature Name**: Dockerized Auditing Daemon & MCP Server Support +- **Status**: Draft +- **Created Date**: 2026-06-25 + +## 🧠 Rationale + +AI agents interact with tools and local resources using the **Model Context Protocol (MCP)**. To allow modern LLM coding and platform agents to easily query, inspect, and perform safe self-tuning operations on database servers, MySQLTuner needs to be deployable as an autonomous microservice. + +This phase specifies: +1. **Dockerized Auditing Daemon**: A lightweight Docker image running as a background service that queries the target database at configurable intervals, performs checks, and caches findings. +2. **MCP Server Interface**: Exposing MySQLTuner findings and safe execution statements through standard MCP Tools and Resources. +3. **Caching and Query Layer**: Storing recent runs locally in a cache database/directory to avoid overloading the production database with repeated audits. + +## đŸ› ïž User Scenarios + +### Scenario 1: Continuous Observability Daemon +A system administrator deploys the MySQLTuner Docker image alongside their database container in a Docker Compose file. The daemon is configured to run an audit every 12 hours. It caches the structured JSON and HTML reports in a shared volume. + +### Scenario 2: LLM-Driven Database Tuning via MCP +An AI coding assistant (like Antigravity or Cursor) is asked to optimize database performance. The agent connects to the MySQLTuner MCP server, invokes the `get_latest_audit` tool to inspect the cached findings, identifies indexing recommendations, and invokes the `apply_recommendation` tool to run the safe SQL optimization. + +## 📋 Technical Architecture + +### 1. Docker Auditing Daemon +The Docker container will run a lightweight daemon script (written in Perl or Python) acting as an orchestrator: +- **Environment Variables**: + - `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD` (or socket mounts). + - `AUDIT_INTERVAL_HOURS`: Run audit every X hours (default: `12`). + - `CACHE_DIR`: Location to persist JSON/HTML reports (default: `/var/cache/mysqltuner`). +- **Interval Auditor**: Sleep/loop execution with SIGTERM/SIGINT signal handling for graceful termination. + +### 2. MCP Server Protocol Support +The MCP server protocol will be implemented over standard I/O (stdio) transport or SSE (Server-Sent Events) HTTP transport. It exposes: + +#### Resources +- `resources/list`: + - `mysqltuner://reports/latest.json`: The latest structured agent JSON. + - `mysqltuner://reports/latest.html`: The latest interactive HTML dashboard. +- `resources/read`: Retrieves the raw content of the cached latest JSON/HTML. + +#### Tools +- `get_latest_audit`: + - Input: None. + - Output: The JSON structure containing current recommendations. +- `run_audit`: + - Input: None. + - Output: Triggers a fresh run of MySQLTuner and returns the latest findings. +- `apply_recommendation`: + - Input: `finding_id` (the ID of the recommendation to execute). + - Behavior: Verifies that the recommendation type is `SQL`, checks that risk is acceptable, and runs the corresponding `statement` against the database. + - Output: Success/Failure status and output logs. +- `rollback_recommendation`: + - Input: `finding_id`. + - Behavior: Runs the corresponding `rollback_statement` to revert changes. + +## ⚠ Risk & Safety Mitigation + +Tuning operations on production databases pose high risk. The MCP server implements: +1. **Execution Restriction**: Only `SQL` actions of `risk_level` `Low` or `Medium` can be executed automatically. `Critical` risk items require manual override. +2. **Read-Only Mode**: A boot flag `READ_ONLY=true` can be set to disable the write tools (`apply_recommendation`, `rollback_recommendation`), making the MCP server a pure observability tool. +3. **Transaction Logging**: Every applied action and rollback is logged in a state file (`/var/cache/mysqltuner/state.json`) with timestamps. + +## ✅ Verification Plan + +### Manual Verification +1. Build the Docker image: `docker build -t mysqltuner-mcp -f Dockerfile.mcp .`. +2. Run container: `docker run -d -e DB_HOST=localhost -e AUDIT_INTERVAL_HOURS=2 -v /tmp/cache:/var/cache/mysqltuner mysqltuner-mcp`. +3. Verify that `/tmp/cache/latest.json` is generated and updated every 2 hours. +4. Interact with the running container using an MCP client CLI (e.g. `@modelcontextprotocol/inspector`) and verify tools and resources are correctly exposed. diff --git a/documentation/specifications/strategic_technical_evolutions.md b/documentation/specifications/strategic_technical_evolutions.md new file mode 100644 index 000000000..43f30b274 --- /dev/null +++ b/documentation/specifications/strategic_technical_evolutions.md @@ -0,0 +1,65 @@ +# Specification: Strategic Technical Evolutions + +- **Feature Name**: Strategic Technical Evolutions +- **Status**: Draft +- **Created Date**: 2026-06-23 + +## 🧠 Rationale + +As MySQLTuner-perl matures, maintaining release integrity, documentation synchronization, and developer workflow stability becomes as critical as the advisor logic itself. The "Strategic Technical Evolutions" address the governance, automation, and localization of the project's ecosystem. By implementing strict validation gates, automated release scripts, and documentation sanity checks, we ensure that the single-file tool is backed by robust, industrial-grade quality assurance processes. + +--- + +## đŸ› ïž User Scenarios + +### Scenario 1: CI/CD & Documentation Integrity +A contributor submits a pull request introducing a new advisor warning. They update the `README.md` and `ROADMAP.md` but copy-paste a broken reference link. +The CI/CD pipeline triggers the **Reference Link Availability Checker** and the **Roadmap Schema Validator**, identifies the exact file and line number of the broken link, and fails the build. The developer corrects the link, and the build passes. + +### Scenario 2: Automated & Fail-Safe Release Lifecycle +The Release Manager decides to cut version `v2.9.1`. Instead of manually editing version strings in six separate locations and running python generator scripts, they execute the **Interactive Release Orchestrator**: +1. The tool prompts: `Select version bump type: [major, minor, micro]`. +2. The user chooses `micro`. +3. The orchestrator automatically bumps the version from `v2.9.0` to `v2.9.1`, updates the 6 reference locations, parses the Git commit history to generate the release note in `releases/v2.9.1.md`, and populates the `Executive Summary` in the `Changelog`. +4. Before tagging, the **Release Artifact Validator** scans the new release file to ensure the structure matches standard markdown syntax. + +### Scenario 3: Pre-Commit Quality Guard +A developer edits `mysqltuner.pl` to add support for a new MySQL 9.x variable (a `feat` type commit). They attempt to run `git commit`. +The **Automated Changelog Formatting Verification** hook intercepts the commit, notices that a `feat` modification in the script was staged, but `Changelog` was not modified. The commit is rejected with a message reminding the developer to document their changes in `Changelog`. + +--- + +## 📋 User Stories + +| Title | Priority | Description | Rationale | Test Case | +| :--- | :--- | :--- | :--- | :--- | +| **1. Reference Link Auditor** | P2 | As a developer, I want a pipeline command to scan documentation files | To automatically prevent dead or broken URLs/paths in help files. | **GIVEN** a markdown file contains a broken link, **WHEN** the auditor runs, **THEN** it outputs a list of broken URLs/files with line numbers and exits with code 1. | +| **2. Dynamic Help Anchors** | P2 | As a DBA, I want unique reference anchors displayed alongside advisor recommendations | To quickly navigate to detailed documentation in the official database KB. | **GIVEN** MySQLTuner suggests changing a parameter, **WHEN** it runs, **THEN** it outputs an anchor like `[REF: INNODB-BP]` mapping to official KBs. | +| **3. Localized References** | P3 | As a non-English speaker, I want references in my own language | To understand advice without translation overhead. | **GIVEN** localized output is selected (e.g. Italian), **WHEN** reference URLs are printed, **THEN** they point to localized KB paths where available. | +| **4. Pre-commit Changelog Hook** | P1 | As a maintainer, I want the pre-commit hook to verify `Changelog` edits on `feat`/`fix` changes | To ensure every functional change is properly documented before commit. | **GIVEN** `mysqltuner.pl` is changed with a `feat`/`fix` intent, **WHEN** committing, **THEN** block the commit if `Changelog` has no staged changes. | +| **5. Containerized Validation** | P1 | As a developer, I want local pre-flight checks to run inside a minimal Docker container | To avoid "works on my machine" issues due to environmental differences. | **GIVEN** local changes are made, **WHEN** running `make test-containerized`, **THEN** execute the unit test suite inside an isolated minimal container. | +| **6. Interactive Orchestrator** | P1 | As a Release Manager, I want a single script to bump versions and run release note generators | To prevent manual synchronization errors across the 6 version reference locations. | **GIVEN** a release is requested, **WHEN** selecting micro/minor/major, **THEN** update `CURRENT_VERSION.txt`, `mysqltuner.pl` (3 references), `Changelog`, and create the release note file. | +| **7. Release Summary Auto-Sync** | P2 | As a Release Manager, I want release summaries automatically extracted from commit logs | To save time and ensure no changes are omitted from release notes. | **GIVEN** commits exist since the last release tag, **WHEN** generating release notes, **THEN** parse and populate the Executive Summary automatically. | +| **8. Release Artifact Validator** | P2 | As a maintainer, I want validation of new release notes syntax and metadata | To prevent malformed or broken release documentation in `releases/`. | **GIVEN** a release file is generated, **WHEN** verified, **THEN** assert it contains mandatory headers, matching version, and valid issue links. | +| **9. Roadmap Syntax Linter** | P3 | As a maintainer, I want structured linting for `ROADMAP.md` | To keep the roadmap file syntactically clean and resolve all spec file paths. | **GIVEN** `ROADMAP.md` is modified, **WHEN** linted, **THEN** check syntax constraints, category mappings, and existence of all specification files. | +| **10. Roadmap Progress Auto-Sync** | P3 | As a developer, I want roadmap checklist items to be marked complete automatically on commit | To reduce manual housekeeping of the roadmap project status. | **GIVEN** a commit with scope `feat(auth):` is merged, **WHEN** roadmap sync is triggered, **THEN** check and mark related checklist items as completed `[x]`. | + +--- + +## ✅ Verification Plan + +### Automated Tests +- **Link Auditor Verification**: + - Run the auditing script against test fixtures (e.g. `tests/fixtures/good_links.md` and `tests/fixtures/bad_links.md`) and verify the exit codes. +- **Git pre-commit Hook Verification**: + - Simulate staging a `feat: add feature` commit without modifying `Changelog` and check if git blocks the commit. +- **Version Orchestration Verification**: + - Run the orchestrator in dry-run mode (`--dry-run`) to verify that the version variables are correctly computed and replaced in a mock directory structure. +- **Docker Validation Runner**: + - Run `make test-containerized` and confirm the suite successfully executes inside a temporary Docker container and tears down cleanly. +- **Linter & Schema Verification**: + - Execute markdown schema validation scripts against `ROADMAP.md` and `releases/*.md` to confirm formatting compliance. + +### Manual Verification +- Execute `--help` and verify that documentation references are listed and dynamically generated. +- Run the localized script (e.g., with environment configuration) to verify translation mapping of reference domains. diff --git a/documentation/specifications/verbose_execution_timings.md b/documentation/specifications/verbose_execution_timings.md new file mode 100644 index 000000000..2c3ac4fba --- /dev/null +++ b/documentation/specifications/verbose_execution_timings.md @@ -0,0 +1,34 @@ +# Specification: Verbose Execution Timings + +## Goal + +Add execution timing information for each section and the total execution time at the end of the MySQLTuner run when verbose mode is active. + +## Scenario + +- **Test Case**: Run MySQLTuner with `--verbose` or `-v`. +- **Evidence**: + - At the end of each printed block (e.g., after `MyISAM Metrics`), its individual execution time is printed. + - Before the final `✔ Terminated successfully` message, a summary block (`Execution Times`) listing all section names with their durations and the total execution time is printed. +- **Example Console Output**: + ```text + -------- MyISAM Metrics ---------------------------------------------------------------------------- + ... + [--] MyISAM Metrics execution time: 0.123s + + ... + + -------- Execution Times --------------------------------------------------------------------------- + [--] Storage Engine Statistics: 0.045s + [--] MyISAM Metrics: 0.123s + ... + [--] Total Execution Time: 1.789s + ``` + +## Rules + +1. Measure execution times of all blocks defined by `subheaderprint` calls. +2. Safe dynamic loading of `Time::HiRes` to ensure compatibility and lack of CPAN dependencies. +3. Fallback to `time()` when `Time::HiRes` is not available. +4. Timings must only print when `$opt{'verbose'}` is set. +5. Timing outputs must be placed before the terminal `✔ Terminated successfully` message. diff --git a/documentation/specifications/warning_elimination_version_cache.md b/documentation/specifications/warning_elimination_version_cache.md new file mode 100644 index 000000000..b32c5dead --- /dev/null +++ b/documentation/specifications/warning_elimination_version_cache.md @@ -0,0 +1,26 @@ +--- +test_file: tests/unit_versions.t +--- +# Specification: Warning Elimination & Version Comparison Optimization + +## Goal +Eliminate Perl runtime uninitialized value warnings during execution (specifically in version checks, InnoDB log analysis, and MariaDB detection paths) and optimize the version comparison routines by caching parsed version components. + +## Requirements +1. **Version Caching & Optimization**: + - Cache the parsed version components (major, minor, micro) in lexical variables `$cached_v_maj`, `$cached_v_min`, `$cached_v_mic` based on `$myvar{'version'}`. + - Refactor `mysql_version_ge()`, `mysql_version_le()`, and `mysql_version_eq()` to use cached version components instead of performing regex parsing on every invocation. + - Guard comparison helper parameters (`$maj`, `$min`, `$mic`) to prevent uninitialized value warnings when comparison arguments are undefined. + - Guard `$myvar{'version'}` regex match inside `validate_mysql_version()` to avoid matching on undefined variables. + +2. **InnoDB Analysis Guards**: + - Guard `$mycalc{'innodb_log_size_pct'}` and `$myvar{'innodb_log_file_size'}` with definedness operators (`// 0`) before using them in calculations or printing output within `mysql_innodb()`. + - Prevent potential uninitialized warnings during multiplication or printing in the InnoDB log size checks block. + +3. **MariaDB Detection Guards**: + - Guard `$myvar{'version_comment'}` and `$myvar{'version'}` checks using the definedness default operator (`// ''`) inside the parallel replication and query cache plugin check blocks. + - Guard the `infoprint` statement in `security_recommendations()` where version and version comment are printed. + +## Verification +- Run `make unit-tests` to ensure that all unit tests execute and pass cleanly. +- The test output audit gate must report 0 runtime uninitialized value warnings from the version checks or InnoDB code path. diff --git a/execution.log b/execution.log deleted file mode 100644 index 35fa007f1..000000000 --- a/execution.log +++ /dev/null @@ -1,56 +0,0 @@ -[2026-05-28 00:38:29] [DRY-RUN] Starting dry-run validation from version 2.8.44 to 2.8.45... -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on CURRENT_VERSION.txt matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on mysqltuner.pl matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on USAGE.md matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on README.md matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on SECURITY.md matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on MEMORY_DB.md matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated update on Changelog matches version schema and compiles cleanly. -[2026-05-28 00:38:29] [DRY-RUN] [OK] Simulated path for new release notes: /home/jmren/GIT_REPOS/MySQLTuner-perl/build/../releases/v2.8.45.md -[2026-05-28 00:38:29] [DRY-RUN] Dry-run validation completed successfully. All files matches version schemas. -[2026-05-28 00:38:47] [MAKE] Starting generate_usage... -[2026-05-28 00:39:23] [MAKE] Finished generate_usage. -[2026-05-28 00:41:14] [MAKE] Starting generate_usage... -[2026-05-28 00:41:49] [MAKE] Finished generate_usage. -[2026-05-28 00:44:29] [DRY-RUN] Starting dry-run validation from version 2.8.44 to 2.8.45... -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on CURRENT_VERSION.txt matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on mysqltuner.pl matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on USAGE.md matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on README.md matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on SECURITY.md matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on MEMORY_DB.md matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated update on Changelog matches version schema and compiles cleanly. -[2026-05-28 00:44:29] [DRY-RUN] [OK] Simulated path for new release notes: /home/jmren/GIT_REPOS/MySQLTuner-perl/build/../releases/v2.8.45.md -[2026-05-28 00:44:29] [DRY-RUN] Dry-run validation completed successfully. All files matches version schemas. -[2026-05-28 00:48:41] [DRY-RUN] Starting dry-run validation from version 2.8.44 to 2.8.45... -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on CURRENT_VERSION.txt matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on mysqltuner.pl matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on USAGE.md matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on README.md matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on SECURITY.md matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on MEMORY_DB.md matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated update on Changelog matches version schema and compiles cleanly. -[2026-05-28 00:48:41] [DRY-RUN] [OK] Simulated path for new release notes: /home/jmren/GIT_REPOS/MySQLTuner-perl/build/../releases/v2.8.45.md -[2026-05-28 00:48:41] [DRY-RUN] Dry-run validation completed successfully. All files matches version schemas. -[2026-05-28 00:50:06] [MAKE] Starting generate_usage... -[2026-05-28 00:50:41] [MAKE] Finished generate_usage. -[2026-05-28 07:56:46] [MAKE] Starting generate_usage... -[2026-05-28 07:57:00] [MAKE] Finished generate_usage. -[2026-05-28 07:57:00] [MAKE] Starting generate_release_notes... -[2026-05-28 07:57:14] [MAKE] Finished generate_release_notes. -[2026-06-04 06:51:17] [MAKE] Starting generate_usage... -[2026-06-04 06:51:52] [MAKE] Finished generate_usage. -[2026-06-04 06:51:52] [MAKE] Starting generate_release_notes... -[2026-06-04 06:52:27] [MAKE] Finished generate_release_notes. -[2026-06-04 08:32:37] [MAKE] Starting generate_usage... -[2026-06-04 08:32:55] [MAKE] Finished generate_usage. -[2026-06-04 08:32:55] [MAKE] Starting generate_release_notes... -[2026-06-04 08:33:10] [MAKE] Finished generate_release_notes. -[2026-06-04 08:33:51] [MAKE] Starting generate_usage... -[2026-06-04 08:34:10] [MAKE] Finished generate_usage. -[2026-06-04 08:34:10] [MAKE] Starting generate_release_notes... -[2026-06-04 08:34:26] [MAKE] Finished generate_release_notes. -[2026-06-04 09:36:52] [MAKE] Starting generate_usage... -[2026-06-04 09:37:30] [MAKE] Finished generate_usage. -[2026-06-04 09:37:30] [MAKE] Starting generate_release_notes... -[2026-06-04 09:38:12] [MAKE] Finished generate_release_notes. diff --git a/mysqltuner.pl b/mysqltuner.pl index 3c9d77595..29cfaf20e 100755 --- a/mysqltuner.pl +++ b/mysqltuner.pl @@ -1,5 +1,5 @@ #!/usr/bin/env perl -# mysqltuner.pl - Version 2.8.45 +# mysqltuner.pl - Version 2.9.0 # High Performance MySQL Tuning Script # Copyright (C) 2015-2026 Jean-Marie Renouard - jmrenouard@gmail.com # Copyright (C) 2006-2026 Major Hayden - major@mhtx.net @@ -67,13 +67,20 @@ package main; our $is_win = $^O eq 'MSWin32'; # Set up a few variables for use in the script -our $tunerversion = "2.8.45"; +our $tunerversion = "2.9.0"; our ( @adjvars, @generalrec, @modeling, @sysrec, @secrec ); our ( %result, %myvar, %real_vars, %mystat, %mycalc, %myrepl, %myreplicas, $dummyselect ); our %exported_manifest; +our ( + $tuner_start_time, $current_section_name, + $current_section_start, @section_timings, + $tuner_start_datetime +); +our $has_time_hires = eval { require Time::HiRes; 1; } // 0; our $failed_connection_attempts = 0; our $previous_failed_attempts = 0; +our $is_local_only = 0; # Set defaults # Central metadata for CLI options @@ -300,6 +307,12 @@ package main; desc => 'Print result as JSON formatted string', cat => 'PERFORMANCE' }, + 'yaml' => { + type => '!', + default => 0, + desc => 'Print result as YAML string', + cat => 'PERFORMANCE' + }, 'dumpdir' => { type => '=s', default => undef, @@ -354,6 +367,12 @@ package main; plugininfo => 1 } }, + 'stage-timings' => { + type => '!', + default => 0, + desc => 'Activate stage timings and final summary without full verbose mode', + cat => 'OUTPUT' + }, 'color!' => { type => '!', default => ( -t STDOUT ? 1 : 0 ), @@ -867,7 +886,8 @@ sub show_help { } sub prettyprint { - print $_[0] . "\n" unless ( $opt{'silent'} or $opt{'json'} ); + print $_[0] . "\n" + unless ( $opt{'silent'} or $opt{'json'} or $opt{'yaml'} ); print $fh $_[0] . "\n" if defined($fh); my $plain_text = $_[0] // ''; $plain_text =~ s/\e\[[0-9;]*[mK]//g; @@ -999,6 +1019,183 @@ sub calculate_health_score { $result{'HealthScore'} = $score; $result{'HealthScoreDetails'} = \%details; $mycalc{'WeightedHealthScore'} = $score; + + calculate_sectional_health_scores(); +} + +sub calculate_sectional_health_scores { + + # 1. General Statistics KPI + my $gen_score = 100; + + my $uptime = $mystat{'Uptime'} // 0; + if ( $uptime < 86400 ) { + $gen_score -= 20; + } + + my @load = get_load_average(); + if (@load) { + my $load_1 = $load[0] // 0; + my $cpu_cores = $mycalc{'physical_cpu_cores'} // $mycalc{'cpu_cores'} + // 1; + if ( $cpu_cores > 0 && ( $load_1 / $cpu_cores ) > 1.0 ) { + $gen_score -= 20; + } + } + + if ( grep { /swappiness/i } @generalrec ) { + $gen_score -= 20; + } + + my $other_mem = get_other_process_memory() // 0; + if ( ( $physical_memory // 0 ) > 0 + && ( $other_mem / $physical_memory ) > 0.3 ) + { + $gen_score -= 20; + } + $gen_score = 0 if $gen_score < 0; + + # 2. Storage Engine KPI + my $storage_score = 100; + + my $read_eff = $mycalc{'pct_read_efficiency'} // 100; + if ( $read_eff < 95 ) { + $storage_score -= 25; + } + + my $temp_disk = $mycalc{'pct_temp_disk'} // 0; + if ( $temp_disk > 25 ) { + $storage_score -= 25; + } + + my $table_hit = $mycalc{'table_cache_hit_rate'} // 100; + if ( $table_hit < 50 ) { + $storage_score -= 25; + } + + my $redo_adjust = + grep { /innodb_log_file_size/i || /innodb_redo_log_capacity/i } @adjvars; + if ($redo_adjust) { + $storage_score -= 25; + } + $storage_score = 0 if $storage_score < 0; + + # 3. Security & Compliance KPI + my $sec_score = 100; + $sec_score -= scalar(@secrec) * 15; + $sec_score = 0 if $sec_score < 0; + + # 4. Replication & HA KPI + my $repl_score = 100; + if (%myrepl) { + my $lag = $myrepl{'Seconds_Behind_Source'} + // $myrepl{'Seconds_Behind_Replica'}; + if ( defined $lag && $lag > 60 ) { + $repl_score -= 40; + } + my $io_run = $myrepl{'Replica_IO_Running'} + // $myrepl{'Slave_IO_Running'} // 'Yes'; + my $sql_run = $myrepl{'Replica_SQL_Running'} + // $myrepl{'Slave_SQL_Running'} // 'Yes'; + if ( $io_run =~ /No/i || $sql_run =~ /No/i ) { + $repl_score -= 35; + } + my $gtid = $myvar{'gtid_mode'} // 'OFF'; + if ( $gtid =~ /OFF/i ) { + $repl_score -= 25; + } + } + $repl_score = 0 if $repl_score < 0; + + # 5. SQL Modeling KPI + my $model_score = 100; + $model_score -= scalar(@modeling) * 10; + $model_score = 0 if $model_score < 0; + + $result{'SectionalHealthScore'}{'General'} = int($gen_score); + $result{'SectionalHealthScore'}{'Storage'} = int($storage_score); + $result{'SectionalHealthScore'}{'Security'} = int($sec_score); + $result{'SectionalHealthScore'}{'Replication'} = int($repl_score); + $result{'SectionalHealthScore'}{'Modeling'} = int($model_score); + + # Calculate Resource Saturation Heatmap + my $cpu_sat = 0; + if (@load) { + my $load_1 = $load[0] // 0; + my $cpu_cores = $mycalc{'physical_cpu_cores'} // $mycalc{'cpu_cores'} + // 1; + $cpu_sat = ( $cpu_cores > 0 ) ? ( $load_1 / $cpu_cores ) * 100 : 0; + $cpu_sat = 100 if $cpu_sat > 100; + } + + my $mem_sat = $mycalc{'pct_max_physical_memory'} // 0; + $mem_sat =~ s/%//g; + $mem_sat = 100 if $mem_sat > 100; + + my $conn_sat = $mycalc{'pct_connections_used'} // 0; + $conn_sat =~ s/%//g; + $conn_sat = 100 if $conn_sat > 100; + + my $read_eff_val = $mycalc{'pct_read_efficiency'} // 100; + my $disk_read_pressure = 100 - $read_eff_val; + my $scaled_read_pressure = $disk_read_pressure * 20; + my $temp_disk_val = $mycalc{'pct_temp_disk'} // 0; + my $io_sat = + ( $scaled_read_pressure > $temp_disk_val ) + ? $scaled_read_pressure + : $temp_disk_val; + $io_sat = 100 if $io_sat > 100; + + $result{'ResourceSaturation'}{'CPU'} = int($cpu_sat); + $result{'ResourceSaturation'}{'Memory'} = int($mem_sat); + $result{'ResourceSaturation'}{'Connections'} = int($conn_sat); + $result{'ResourceSaturation'}{'IO'} = int($io_sat); + + # Calculate Throughput Efficiency Index + my $qps_val = + ( ( $mystat{'Uptime'} || 0 ) > 0 ) + ? ( $mystat{'Questions'} || 0 ) / $mystat{'Uptime'} + : 0; + my $logical_reads_sec = + ( $uptime > 0 ) + ? ( $mystat{'Innodb_buffer_pool_read_requests'} // 0 ) / $uptime + : 0; + my $tei = + ( $logical_reads_sec > 0 ) ? ( $qps_val / $logical_reads_sec ) : 0; + + $result{'ThroughputEfficiency'}{'QPS'} = sprintf( "%.3f", $qps_val ) + 0; + $result{'ThroughputEfficiency'}{'LogicalReadsSec'} = + sprintf( "%.3f", $logical_reads_sec ) + 0; + $result{'ThroughputEfficiency'}{'Index'} = sprintf( "%.5f", $tei ) + 0; +} + +sub get_top_findings { + my $list_ref = shift; + my @items = (); + foreach my $it (@$list_ref) { + push @items, format_recommendation_item($it); + } + + my @scored = (); + foreach my $item (@items) { + my $score = 1; + if ( $item =~ +/(dangerously high|insecure|vulnerability|CVE|not running|aborted|unencrypted|anonymous|remove|risk|critical|warning|incorrect|mismatch)/i + ) + { + $score = 2; + } + push @scored, { text => $item, score => $score }; + } + + my @sorted = sort { $b->{score} <=> $a->{score} } @scored; + + my @res = (); + for ( my $i = 0 ; $i < 3 && $i < @sorted ; $i++ ) { + my $badge = ( $sorted[$i]->{score} == 2 ) ? 'Critical' : 'Finding'; + push @res, { text => $sorted[$i]->{text}, badge => $badge }; + } + return @res; } sub display_health_score { @@ -1093,6 +1290,10 @@ sub predictive_capacity_analysis { sub check_replication_advanced { subheaderprint "Cluster & Replication Intelligence"; + if ($is_local_only) { + infoprint "Skipping advanced replication checks: Server is bound to localhost-only (Ref: https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_bind_address)."; + return; + } my $is_replica = ( defined $myrepl{'Seconds_Behind_Source'} @@ -1376,9 +1577,11 @@ sub check_security_2_0 { . ( $myvar{'tls_version'} // 'defaults' ); } else { - badprint "TLS/SSL is disabled. Connections are unencrypted."; - push_recommendation( 'sec', - "Enable TLS/SSL for encrypted connections." ); + unless ($is_local_only) { + badprint "TLS/SSL is disabled. Connections are unencrypted."; + push_recommendation( 'sec', + "Enable TLS/SSL for encrypted connections." ); + } } # TDE Check (InnoDB) @@ -1390,7 +1593,7 @@ sub check_security_2_0 { } sub generate_auto_fix_snippets { - return if $opt{'silent'} || $opt{'json'}; + return if $opt{'silent'} || $opt{'json'} || $opt{'yaml'}; subheaderprint "Guided Auto-Fix Snippets"; if ( @adjvars > 0 ) { @@ -1425,7 +1628,62 @@ sub infoprintcmd { infoprintml( grep { /\S/ } `@_ 2>&1` ); } +sub get_time { + return $has_time_hires ? Time::HiRes::time() : time(); +} + +sub get_datetime_str { + my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = + localtime(time); + return sprintf( + "%04d-%02d-%02d %02d:%02d:%02d", + $year + 1900, + $mon + 1, $mday, $hour, $min, $sec + ); +} + +sub pretty_duration { + my $duration = shift; + my $seconds = $duration % 60; + my $decimal = $duration - int($duration); + my $seconds_formatted = sprintf( "%.3f", int($seconds) + $decimal ); + + my $total_minutes = int( $duration / 60 ); + my $minutes = $total_minutes % 60; + my $hours = int( $total_minutes / 60 ) % 24; + my $days = int( $total_minutes / 1440 ); + + my $duration_string = ""; + if ( $days > 0 ) { + $duration_string = + "${days}d ${hours}h ${minutes}m ${seconds_formatted}s"; + } + elsif ( $hours > 0 ) { + $duration_string = "${hours}h ${minutes}m ${seconds_formatted}s"; + } + elsif ( $minutes > 0 ) { + $duration_string = "${minutes}m ${seconds_formatted}s"; + } + else { + $duration_string = "${seconds_formatted}s"; + } +} + +$tuner_start_datetime = get_datetime_str(); + sub subheaderprint { + my $name = $_[0]; + my $now = get_time(); + if ( defined $current_section_name && ( $opt{'verbose'} || $opt{'stage-timings'} ) ) { + my $elapsed = $now - $current_section_start; + push @section_timings, [ $current_section_name, $elapsed ]; + infoprint + sprintf( "%s execution time: %.3fs", $current_section_name, + $elapsed ); + } + $current_section_name = $name; + $current_section_start = $now; + my $tln = 100; my $sln = 8; my $ln = length("@_") + 2; @@ -1434,6 +1692,73 @@ sub subheaderprint { prettyprint "-" x $sln . " @_ " . "-" x ( $tln - $ln - $sln ); } +sub stop_section_timing { + my $now = get_time(); + if ( defined $current_section_name && ( $opt{'verbose'} || $opt{'stage-timings'} ) ) { + my $elapsed = $now - $current_section_start; + push @section_timings, [ $current_section_name, $elapsed ]; + infoprint + sprintf( "%s execution time: %.3fs", $current_section_name, + $elapsed ); + } + $current_section_name = undef; +} + +sub print_execution_timings { + return unless ( $opt{'verbose'} || $opt{'stage-timings'} ); + + stop_section_timing(); + + my $total_now = get_time(); + my $total_elapsed = $total_now - $tuner_start_time; + + subheaderprint "Execution Times"; + if ( defined $tuner_start_datetime ) { + infoprint sprintf( "%-50s: %s", "Started at", $tuner_start_datetime ); + } + infoprint sprintf( "%-50s: %s", "Ended at", get_datetime_str() ); + + # Sort timings in descending order of elapsed time + my @sorted_timings = sort { $b->[1] <=> $a->[1] } @section_timings; + foreach my $timing (@sorted_timings) { + my ( $name, $elapsed ) = @$timing; + my $pct = $total_elapsed > 0 ? ( $elapsed / $total_elapsed ) * 100 : 0; + infoprint sprintf( "%-50s: %.3fs (%.1f%%)", $name, $elapsed, $pct ); + } + infoprint sprintf( "%-50s: %s (%.3fs)", "Total Execution Time", pretty_duration($total_elapsed), $total_elapsed ); +} + +sub print_audit_snapshot_summary { + subheaderprint "Audit Snapshot Summary"; + + my $host = $opt{'host'} || 'localhost'; + if ( $opt{'port'} ) { + $host .= ":$opt{'port'}"; + } + + my $db_user = + select_one("SELECT CURRENT_USER()") || $opt{'user'} || 'unknown'; + + my $ram_str = + defined($physical_memory) ? hr_bytes($physical_memory) : 'unknown'; + my $swap_str = defined($swap_memory) ? hr_bytes($swap_memory) : 'unknown'; + + my $db_ver = $myvar{'version'} // 'unknown'; + my $uptime_str = + defined( $mystat{'Uptime'} ) + ? pretty_uptime( $mystat{'Uptime'} ) + : 'unknown'; + + infoprint "MySQLTuner Version : $tunerversion"; + infoprint "Audit Start Time : $tuner_start_datetime"; + infoprint "Server Connection : $host"; + infoprint "Database User : $db_user"; + infoprint "Database Version : $db_ver"; + infoprint "System Physical RAM: $ram_str"; + infoprint "System Swap Memory : $swap_str"; + infoprint "Database Uptime : $uptime_str"; +} + sub infoprinthcmd { subheaderprint "$_[0]"; infoprintcmd "$_[1]"; @@ -1640,6 +1965,7 @@ sub hr_bytes_rnd { # Calculates the parameter passed to the nearest power of 1000, then rounds it to the nearest integer sub hr_num { my $num = shift; + return 0 if !$num || $num eq 'NULL'; if ( $num >= ( 1000**3 ) ) { # Billions return int( ( $num / ( 1000**3 ) ) ) . "B"; } @@ -1996,7 +2322,7 @@ sub get_http_cli { # Checks for updates to MySQLTuner sub validate_tuner_version { if ( $opt{'checkversion'} eq 0 ) { - print "\n" unless ( $opt{'silent'} or $opt{'json'} ); + print "\n" unless ( $opt{'silent'} or $opt{'json'} or $opt{'yaml'} ); infoprint "Skipped version check for MySQLTuner script"; return; } @@ -2058,7 +2384,7 @@ sub validate_tuner_version { sub update_tuner_version { if ( $opt{'updateversion'} eq 0 ) { badprint "Skipped version update for MySQLTuner script"; - print "\n" unless ( $opt{'silent'} or $opt{'json'} ); + print "\n" unless ( $opt{'silent'} or $opt{'json'} or $opt{'yaml'} ); return; } @@ -2461,6 +2787,9 @@ sub mysql_setup { } infoprint "Performing tests on $opt{host}:$opt{port}"; $remotestring = " -h $opt{host} -P $opt{port}"; + if ( $opt{socket} ) { + $remotestring .= " -S $opt{socket}"; + } $doremote = is_remote(); } @@ -2853,8 +3182,7 @@ sub select_csv_file { debugprint "PERFORM: $req CSV into $tfile"; - my $start_time = - eval { require Time::HiRes; Time::HiRes::time(); } || time(); + my $start_time = get_time(); my @result = select_array_with_headers($req); my $row_count = scalar(@result); @@ -2886,7 +3214,7 @@ sub select_csv_file { } close $fh; - my $end_time = eval { require Time::HiRes; Time::HiRes::time(); } || time(); + my $end_time = get_time(); my $duration = $end_time - $start_time; if ( $duration > 5.0 ) { @@ -2960,7 +3288,7 @@ sub write_manifest_files { } my $json_content = - "{\n \"version\": \"" . ( $tunerversion // '2.8.45' ) . "\",\n"; + "{\n \"version\": \"" . ( $tunerversion // '2.9.0' ) . "\",\n"; $json_content .= " \"exported_at\": \"" . scalar( gmtime() ) . " UTC\",\n"; $json_content .= " \"total_files\": $total_files,\n"; $json_content .= " \"total_size_bytes\": $total_size,\n"; @@ -2976,7 +3304,7 @@ sub write_manifest_files { my $meta_content = "MySQLTuner Offline Diagnostic Snapshot Metadata\n"; $meta_content .= "================================================\n"; - $meta_content .= "Version: " . ( $tunerversion // '2.8.45' ) . "\n"; + $meta_content .= "Version: " . ( $tunerversion // '2.9.0' ) . "\n"; $meta_content .= "Exported At: " . scalar( gmtime() ) . " UTC\n"; $meta_content .= "Host: " . ( $myvar{'hostname'} // 'unknown' ) . "\n"; $meta_content .= @@ -3436,6 +3764,23 @@ sub get_all_vars { ); } } + + # Calculate if the server is loopback/local-only + $is_local_only = 0; + if ( defined $myvar{'skip_networking'} && $myvar{'skip_networking'} eq 'ON' ) { + $is_local_only = 1; + } + elsif ( defined $myvar{'bind_address'} ) { + my @addrs = split( /\s*,\s*/, $myvar{'bind_address'} ); + my $all_local = 1; + foreach my $addr (@addrs) { + if ( $addr ne '127.0.0.1' && $addr ne '::1' && $addr ne 'localhost' && $addr !~ /\.(?:local|localhost)$/i ) { + $all_local = 0; + last; + } + } + $is_local_only = 1 if ( @addrs && $all_local ); + } } sub remove_cr { @@ -3993,6 +4338,25 @@ sub get_other_process_memory { return $totalMemOther; } +sub get_load_average { + my $prefix = get_transport_prefix(); + return () if $prefix eq '' && is_remote(); + + if ( $prefix eq '' && -f "/proc/loadavg" ) { + my $content = file2string("/proc/loadavg") // ''; + if ( $content =~ /^([\d.]+)\s+([\d.]+)\s+([\d.]+)/ ) { + return ( $1, $2, $3 ); + } + } + my $uptime_output = execute_system_command("uptime") // ''; + if ( $uptime_output =~ + /load average(?:s)?:?\s+([\d.]+),?\s+([\d.]+),?\s+([\d.]+)/i ) + { + return ( $1, $2, $3 ); + } + return (); +} + sub get_os_release { if ( -f "/etc/lsb-release" ) { my @info_release = get_file_contents "/etc/lsb-release"; @@ -4603,6 +4967,10 @@ sub system_recommendations { # --------------------------------------------------------------------------- sub ssl_tls_recommendations { subheaderprint "SSL/TLS Security Recommendations"; + if ($is_local_only) { + infoprint "Skipping SSL/TLS security recommendations: Server is bound to localhost-only (Ref: https://dev.mysql.com/doc/refman/8.0/en/server-options.html#option_mysqld_skip-networking)."; + return; + } my @ssl_csv_rows = ("Variable,Value,IssueType,Description"); @@ -4683,12 +5051,17 @@ sub ssl_tls_recommendations { # Certificate presence and local audit if ( !$myvar{'ssl_cert'} && !$myvar{'ssl_key'} ) { - badprint "No SSL certificates configured (ssl_cert/ssl_key are empty)"; - push_recommendation( 'Security', + if ( mysql_version_ge( 11, 4 ) && $myvar{'version'} =~ /MariaDB/i && ( defined($myvar{'have_ssl'}) && ($myvar{'have_ssl'} eq 'YES' || $myvar{'have_ssl'} eq 'ON') ) ) { + goodprint "TLS is active, but no explicit ssl_cert/ssl_key paths are configured (MariaDB zero-configuration TLS active)"; + } + else { + badprint "No SSL certificates configured (ssl_cert/ssl_key are empty)"; + push_recommendation( 'Security', "Configure SSL certificates (ssl_cert, ssl_key, ssl_ca) to enable encrypted connections." - ); - push @ssl_csv_rows, + ); + push @ssl_csv_rows, "ssl_cert/ssl_key,empty,NoCertificates,Configure SSL certificates (ssl_cert, ssl_key, ssl_ca) to enable encrypted connections."; + } } else { check_local_certificates( \@ssl_csv_rows ); @@ -4843,12 +5216,19 @@ sub check_remote_user_ssl { my @remote_users; if ( mysql_version_ge( 10, 4 ) && $myvar{'version'} =~ /MariaDB/i ) { @remote_users = select_array( -"SELECT CONCAT(QUOTE(user), '\@', QUOTE(host)) FROM mysql.global_priv WHERE host NOT IN ('localhost', '127.0.0.1', '::1') AND JSON_VALUE(Priv, '\$.ssl_type') = ''" +"SELECT CONCAT(QUOTE(user), '\@', QUOTE(host)) FROM mysql.global_priv WHERE host NOT IN ('localhost', '127.0.0.1', '::1') AND JSON_VALUE(Priv, '\$.ssl_type') = '' AND COALESCE(JSON_VALUE(Priv, '\$.is_role'), '') NOT IN ('true', '1')" ); } else { + my $is_role_column = select_one( +"select count(*) from information_schema.columns where TABLE_NAME='user' AND TABLE_SCHEMA='mysql' and COLUMN_NAME='IS_ROLE'" + ); + my $extra_user_condition = ""; + if ( defined($is_role_column) && $is_role_column =~ /^\d+$/ && $is_role_column > 0 ) { + $extra_user_condition = " AND IS_ROLE = 'N'"; + } @remote_users = select_array( -"SELECT CONCAT(QUOTE(user), '\@', QUOTE(host)) FROM mysql.user WHERE host NOT IN ('localhost', '127.0.0.1', '::1') AND (ssl_type = 'NONE' OR ssl_type = '')" +"SELECT CONCAT(QUOTE(user), '\@', QUOTE(host)) FROM mysql.user WHERE host NOT IN ('localhost', '127.0.0.1', '::1') AND (ssl_type = 'NONE' OR ssl_type = '')$extra_user_condition" ); } @@ -5016,7 +5396,8 @@ sub check_auth_plugins { sub security_recommendations { subheaderprint "Security Recommendations"; - infoprint "$myvar{'version_comment'} - $myvar{'version'}"; + infoprint( ( $myvar{'version_comment'} // 'N/A' ) . " - " + . ( $myvar{'version'} // 'N/A' ) ); my $PASS_COLUMN_NAME = get_password_column_name(); @@ -5136,31 +5517,33 @@ sub security_recommendations { } } - @mysqlstatlist = select_array - "SELECT CONCAT(QUOTE(user), '\@', host) FROM mysql.user WHERE HOST='%'"; - if ( scalar(@mysqlstatlist) > 0 ) { - if ( $opt{dumpdir} ) { - select_csv_file( - "$opt{dumpdir}/user_with_general_wildcard.csv", - "SELECT user, host FROM mysql.user WHERE HOST='%'" + unless ($is_local_only) { + @mysqlstatlist = select_array + "SELECT CONCAT(QUOTE(user), '\@', host) FROM mysql.user WHERE HOST='%'"; + if ( scalar(@mysqlstatlist) > 0 ) { + if ( $opt{dumpdir} ) { + select_csv_file( + "$opt{dumpdir}/user_with_general_wildcard.csv", + "SELECT user, host FROM mysql.user WHERE HOST='%'" + ); + } + my $luser = 'user_name'; + if ( scalar(@mysqlstatlist) == 1 ) { + $luser = ( split /@/, $mysqlstatlist[0] )[0]; + } + foreach my $line ( sort @mysqlstatlist ) { + chomp($line); + badprint "User " . $line + . " does not specify hostname restrictions."; + } + push( @generalrec, + "Restrict Host for $luser\@'%' to $luser\@LimitedIPRangeOrLocalhost" ); + push( @generalrec, + "RENAME USER $luser\@'%' TO " + . $luser + . "\@LimitedIPRangeOrLocalhost;" ); } - my $luser = 'user_name'; - if ( scalar(@mysqlstatlist) == 1 ) { - $luser = ( split /@/, $mysqlstatlist[0] )[0]; - } - foreach my $line ( sort @mysqlstatlist ) { - chomp($line); - badprint "User " . $line - . " does not specify hostname restrictions."; - } - push( @generalrec, - "Restrict Host for $luser\@'%' to $luser\@LimitedIPRangeOrLocalhost" - ); - push( @generalrec, - "RENAME USER $luser\@'%' TO " - . $luser - . "\@LimitedIPRangeOrLocalhost;" ); } unless ( -f $basic_password_files ) { @@ -5508,8 +5891,8 @@ sub get_replication_status { } # Parallel replication checks (MariaDB specific) - if ( ( $myvar{'version'} =~ /MariaDB/i ) - or ( $myvar{'version_comment'} =~ /MariaDB/i ) ) + if ( ( ( $myvar{'version'} // '' ) =~ /MariaDB/i ) + or ( ( $myvar{'version_comment'} // '' ) =~ /MariaDB/i ) ) { my $parallel_threads = $myvar{'slave_parallel_threads'} // $myvar{'replica_parallel_threads'} // 0; @@ -5580,15 +5963,33 @@ sub validate_mysql_version { } } +my $cached_version_str; +my ( $cached_v_maj, $cached_v_min, $cached_v_mic ); + +sub _parse_version { + my $ver = $myvar{'version'} // ''; + if ( !defined $cached_version_str || $cached_version_str ne $ver ) { + $cached_version_str = $ver; + if ( $ver =~ /^(\d+)(?:\.(\d+))?(?:\.(\d+))?/ ) { + $cached_v_maj = $1 // 0; + $cached_v_min = $2 // 0; + $cached_v_mic = $3 // 0; + } + else { + $cached_v_maj = 0; + $cached_v_min = 0; + $cached_v_mic = 0; + } + } + return ( $cached_v_maj, $cached_v_min, $cached_v_mic ); +} + # Checks if MySQL version is equal to (major, minor, micro) sub mysql_version_eq { my ( $maj, $min, $mic ) = @_; return 0 unless defined $myvar{'version'}; - my ( $v_maj, $v_min, $v_mic ) = - $myvar{'version'} =~ /^(\d+)(?:\.(\d+))?(?:\.(\d+))?/; - $v_maj //= 0; - $v_min //= 0; - $v_mic //= 0; + $maj //= 0; + my ( $v_maj, $v_min, $v_mic ) = _parse_version(); return int($v_maj) == int($maj) if ( !defined($min) && !defined($mic) ); @@ -5603,13 +6004,10 @@ sub mysql_version_eq { sub mysql_version_ge { my ( $maj, $min, $mic ) = @_; return 0 unless defined $myvar{'version'}; - $min ||= 0; - $mic ||= 0; - my ( $v_maj, $v_min, $v_mic ) = - $myvar{'version'} =~ /^(\d+)(?:\.(\d+))?(?:\.(\d+))?/; - $v_maj //= 0; - $v_min //= 0; - $v_mic //= 0; + $maj //= 0; + $min //= 0; + $mic //= 0; + my ( $v_maj, $v_min, $v_mic ) = _parse_version(); return int($v_maj) > int($maj) @@ -5623,13 +6021,10 @@ sub mysql_version_ge { sub mysql_version_le { my ( $maj, $min, $mic ) = @_; return 0 unless defined $myvar{'version'}; - $min ||= 0; - $mic ||= 0; - my ( $v_maj, $v_min, $v_mic ) = - $myvar{'version'} =~ /^(\d+)(?:\.(\d+))?(?:\.(\d+))?/; - $v_maj //= 0; - $v_min //= 0; - $v_mic //= 0; + $maj //= 0; + $min //= 0; + $mic //= 0; + my ( $v_maj, $v_min, $v_mic ) = _parse_version(); return int($v_maj) < int($maj) @@ -6058,8 +6453,7 @@ sub dump_into_file { my $actual_file = "$opt{dumpdir}/$file"; my $gzip_bin = which('gzip'); my $is_compressed = 0; - my $start_time = - eval { require Time::HiRes; Time::HiRes::time(); } || time(); + my $start_time = get_time(); my $fh; if ( $opt{'compress-dump'} && $gzip_bin && $file =~ /\.csv$/ ) { $actual_file .= '.gz'; @@ -6074,8 +6468,7 @@ sub dump_into_file { } print $fh $content; close $fh; - my $end_time = - eval { require Time::HiRes; Time::HiRes::time(); } || time(); + my $end_time = get_time(); my $duration = $end_time - $start_time; if ( $duration > 5.0 ) { @@ -6154,6 +6547,7 @@ sub calculations { if ( is_int( $mycalc{'max_tmp_table_size'} ) ) { $per_thread_buffers_without_tmp -= $mycalc{'max_tmp_table_size'}; } + $mycalc{'per_thread_buffers_without_tmp'} = $per_thread_buffers_without_tmp; my $internal_tmp_engine = $myvar{'internal_tmp_mem_storage_engine'} // 'TempTable'; @@ -6407,16 +6801,16 @@ sub calculations { $mycalc{'query_cache_efficiency'} = 0; } elsif ( mysql_version_ge(4) ) { - if ( ( $mystat{'Com_select'} || 0 ) + ( $mystat{'Qcache_hits'} || 0 ) > - 0 ) - { - $mycalc{'query_cache_efficiency'} = sprintf( - "%.1f", - ( - $mystat{'Qcache_hits'} / - ( $mystat{'Com_select'} + $mystat{'Qcache_hits'} ) - ) * 100 - ); + + # MDEV-4981: In MariaDB, Com_select includes query cache hits (Qcache_hits) + my $total_selects = + $is_mariadb + ? ( $mystat{'Com_select'} || 0 ) + : ( + ( $mystat{'Com_select'} || 0 ) + ( $mystat{'Qcache_hits'} || 0 ) ); + if ( $total_selects > 0 ) { + $mycalc{'query_cache_efficiency'} = sprintf( "%.1f", + ( ( $mystat{'Qcache_hits'} || 0 ) / $total_selects ) * 100 ); } else { $mycalc{'query_cache_efficiency'} = 0; @@ -6700,11 +7094,28 @@ sub mysql_stats { infoprint "Max MySQL memory : " . hr_bytes( $mycalc{'max_peak_memory'} ); infoprint "Other process memory: " . hr_bytes( get_other_process_memory() ); - infoprint "Total buffers: " - . hr_bytes( $mycalc{'server_buffers'} ) - . " global + " - . hr_bytes( $mycalc{'per_thread_buffers'} ) - . " per thread ($myvar{'max_connections'} max threads)"; + my $is_mariadb = ( $myvar{'version'} // '' ) =~ /mariadb/i; + my $internal_tmp_engine = $myvar{'internal_tmp_mem_storage_engine'} // 'TempTable'; + if ( defined $myvar{'temptable_max_ram'} + && is_int( $myvar{'temptable_max_ram'} ) + && !$is_mariadb + && $internal_tmp_engine eq 'TempTable' ) + { + infoprint "Total buffers: " + . hr_bytes( $mycalc{'server_buffers'} ) + . " global + " + . hr_bytes( $myvar{'temptable_max_ram'} ) + . " temptable + " + . hr_bytes( $mycalc{'per_thread_buffers_without_tmp'} ) + . " per thread ($myvar{'max_connections'} max threads)"; + } + else { + infoprint "Total buffers: " + . hr_bytes( $mycalc{'server_buffers'} ) + . " global + " + . hr_bytes( $mycalc{'per_thread_buffers'} ) + . " per thread ($myvar{'max_connections'} max threads)"; + } infoprint "Performance_schema Max memory usage: " . hr_bytes_rnd( get_pf_memory() ); $result{'Performance_schema'}{'memory'} = get_pf_memory(); @@ -6833,8 +7244,9 @@ sub mysql_stats { if ( $myvar{'long_query_time'} > 10 ) { push( @adjvars, "long_query_time (<= 10)" ); } - if ( defined( $myvar{'log_slow_queries'} ) ) { - if ( $myvar{'log_slow_queries'} eq "OFF" ) { + my $slow_query_log_active = $myvar{'slow_query_log'} // $myvar{'log_slow_queries'}; + if ( defined( $slow_query_log_active ) ) { + if ( $slow_query_log_active eq "OFF" ) { push( @generalrec, "Enable the slow query log to troubleshoot bad queries" ); } @@ -6944,12 +7356,21 @@ sub mysql_stats { "Query cache cannot be analyzed: no SELECT statements executed"; } else { + # MDEV-4981: In MariaDB, Com_select includes query cache hits (Qcache_hits) + my $is_mariadb = ( $myvar{'version'} // '' ) =~ /mariadb/i + || ( $myvar{'version_comment'} // '' ) =~ /mariadb/i; + my $total_selects = + $is_mariadb + ? ( $mystat{'Com_select'} || 0 ) + : ( + ( $mystat{'Com_select'} || 0 ) + ( $mystat{'Qcache_hits'} || 0 ) ); + if ( $mycalc{'query_cache_efficiency'} < 20 ) { badprint "Query cache efficiency: $mycalc{'query_cache_efficiency'}% (" . hr_num( $mystat{'Qcache_hits'} ) . " cached / " - . hr_num( $mystat{'Qcache_hits'} + $mystat{'Com_select'} ) + . hr_num($total_selects) . " selects)"; badprint "Query cache may be disabled by default due to mutex contention."; @@ -6961,7 +7382,7 @@ sub mysql_stats { "Query cache efficiency: $mycalc{'query_cache_efficiency'}% (" . hr_num( $mystat{'Qcache_hits'} ) . " cached / " - . hr_num( $mystat{'Qcache_hits'} + $mystat{'Com_select'} ) + . hr_num($total_selects) . " selects)"; if ( $mycalc{'query_cache_prunes_per_day'} > 98 ) { badprint @@ -8334,6 +8755,7 @@ sub mysql_pfs { schema => $schema, table => $table, index => $index, + sql => "ALTER TABLE $schema.$table DROP INDEX $index", } ); $nbL++; @@ -10503,20 +10925,22 @@ sub mysql_innodb { infoprint " +-- InnoDB Log File Size: " . hr_bytes( $myvar{'innodb_log_file_size'} ); } - if ( defined $myvar{'innodb_log_files_in_group'} ) { + if ( defined $myvar{'innodb_log_files_in_group'} + && defined $myvar{'innodb_log_file_size'} ) + { infoprint " +-- InnoDB Log File In Group: " . $myvar{'innodb_log_files_in_group'}; infoprint " +-- InnoDB Total Log File Size: " . hr_bytes( $myvar{'innodb_log_files_in_group'} * $myvar{'innodb_log_file_size'} ) . "(" - . $mycalc{'innodb_log_size_pct'} + . ( $mycalc{'innodb_log_size_pct'} // 0 ) . " % of buffer pool)"; } - else { + elsif ( defined $myvar{'innodb_log_file_size'} ) { infoprint " +-- InnoDB Total Log File Size: " . hr_bytes( $myvar{'innodb_log_file_size'} ) . "(" - . $mycalc{'innodb_log_size_pct'} + . ( $mycalc{'innodb_log_size_pct'} // 0 ) . " % of buffer pool)"; } } @@ -10968,7 +11392,9 @@ sub mysql_innodb { "InnoDB Write Log efficiency: metrics are not reliable (writes > write requests)"; } elsif ( defined $mycalc{'pct_write_efficiency'} - && $mycalc{'pct_write_efficiency'} < 90 ) + && $mycalc{'pct_write_efficiency'} < 90 + && defined $mystat{'Innodb_log_waits'} + && $mystat{'Innodb_log_waits'} > 0 ) { badprint "InnoDB Write Log efficiency: " . abs( $mycalc{'pct_write_efficiency'} ) . "% (" @@ -11462,6 +11888,8 @@ sub historical_comparison { my $file = $opt{'compare-file'}; return if !$file; + calculate_health_score() unless defined $result{'HealthScore'}; + subheaderprint "Historical Trend Analysis"; if ( !-e $file ) { badprint "Comparison file not found: $file"; @@ -11490,8 +11918,9 @@ sub historical_comparison { return; } - infoprint "Comparing current results with snapshot from: " - . ( $old->{'General'}{'Date'} // 'unknown' ); + my $snapshot_date = $old->{'General'}{'Date'} // 'unknown'; + infoprint "Comparing current results with snapshot from: " . $snapshot_date; + $result{'Trends'}{'SnapshotDate'} = $snapshot_date; # 1. Compare QPS if ( defined $result{'Stats'}{'QPS'} && defined $old->{'Stats'}{'QPS'} ) { @@ -11500,13 +11929,25 @@ sub historical_comparison { ( $old->{'Stats'}{'QPS'} > 0 ) ? ( $diff / $old->{'Stats'}{'QPS'} ) * 100 : 0; - my $trend = ( $diff >= 0 ) ? "+" : ""; - infoprint sprintf( + my $trend = ( $diff >= 0 ) ? "+" : ""; + my $qps_trend = sprintf( "QPS Trend: %.2f -> %.2f (%s%.2f%%)", $old->{'Stats'}{'QPS'}, $result{'Stats'}{'QPS'}, $trend, $pct ); + infoprint $qps_trend; + $result{'Trends'}{'QPS'} = $qps_trend; + } + + # Compare Health Score + if ( defined $result{'HealthScore'} && defined $old->{'HealthScore'} ) { + my $diff = $result{'HealthScore'} - $old->{'HealthScore'}; + my $trend = ( $diff > 0 ) ? "+" : ""; + my $score_trend = sprintf( "Health Score Trend: %d -> %d (%s%d)", + $old->{'HealthScore'}, $result{'HealthScore'}, $trend, $diff ); + infoprint $score_trend; + $result{'Trends'}{'HealthScore'} = $score_trend; } # 2. Compare Total Data Size @@ -11515,10 +11956,30 @@ sub historical_comparison { { my $diff = $result{'Stats'}{'Total Data Size'} - $old->{'Stats'}{'Total Data Size'}; - infoprint "Data Growth: " + my $size_trend = + "Data Growth: " . hr_bytes( $old->{'Stats'}{'Total Data Size'} ) . " -> " . hr_bytes( $result{'Stats'}{'Total Data Size'} ) . " (" . hr_bytes($diff) . ")"; + infoprint $size_trend; + $result{'Trends'}{'TotalDataSize'} = $size_trend; + } + + # Compare sectional scores + foreach + my $sec ( 'General', 'Storage', 'Security', 'Replication', 'Modeling' ) + { + if ( defined $result{'SectionalHealthScore'}{$sec} + && defined $old->{'SectionalHealthScore'}{$sec} ) + { + my $diff = $result{'SectionalHealthScore'}{$sec} - + $old->{'SectionalHealthScore'}{$sec}; + my $trend = ( $diff > 0 ) ? "+" : ""; + $result{'Trends'}{'Sectional'}{$sec} = sprintf( "%d -> %d (%s%d)", + $old->{'SectionalHealthScore'}{$sec}, + $result{'SectionalHealthScore'}{$sec}, + $trend, $diff ); + } } push @adjvars, "Historical comparison performed against " . basename($file); @@ -11654,8 +12115,8 @@ sub check_query_anti_patterns { sub mariadb_query_cache_info { subheaderprint "Query Cache Information"; - unless ( ( $myvar{'version'} =~ /MariaDB/i ) - or ( $myvar{'version_comment'} =~ /MariaDB/i ) ) + unless ( ( ( $myvar{'version'} // '' ) =~ /MariaDB/i ) + or ( ( $myvar{'version_comment'} // '' ) =~ /MariaDB/i ) ) { infoprint "Not a MariaDB server. Skipping Query Cache Info plugin check."; @@ -11821,7 +12282,7 @@ sub mysql_databases { $result{'Databases'}{'All databases'}{'Index Pct'} = percentage( $totaldbinfo[2], $totaldbinfo[3] ) . "%"; $result{'Databases'}{'All databases'}{'Total Size'} = $totaldbinfo[3]; - print "\n" unless ( $opt{'silent'} or $opt{'json'} ); + print "\n" unless ( $opt{'silent'} or $opt{'json'} or $opt{'yaml'} ); my $nbViews = 0; my $nbTables = 0; @@ -12428,6 +12889,7 @@ sub headerprint { . "\t * Major Hayden \n" . " >> Bug reports, feature requests, and downloads at http://mysqltuner.pl/\n" . " >> Run with '--help' for additional options and output filtering"; + prettyprint " >> Started at: " . $tuner_start_datetime if defined $tuner_start_datetime; debugprint( "Debug: " . $opt{debug} ); debugprint( "Experimental: " . $opt{experimental} ); } @@ -12476,7 +12938,8 @@ sub format_recommendation_item { my $schema = $item->{schema} // ''; my $table = $item->{table} // ''; my $index = $item->{index} // ''; - return "Unused index: $schema.$table ($index)"; + my $sql = $item->{sql} // ''; + return "Unused index: $schema.$table ($index) (suggested SQL: $sql)"; } elsif ( $item->{type} eq 'redundant_index' ) { my $schema = $item->{schema} // ''; @@ -12494,7 +12957,157 @@ sub format_recommendation_item { return $item // ''; } +sub _yaml_scalar { + my ( $v, $indent ) = @_; + return "~" unless defined $v; + + if ( $v =~ /\n/ ) { + my $pad = ' ' x ( $indent + 1 ); + $v =~ s/\r\n?/\n/g; + my @lines = split /\n/, $v, -1; + if ( @lines > 1 && $lines[-1] eq '' ) { + pop @lines; + } + my $joined = join( "\n", map { $pad . $_ } @lines ); + return "|\n" . $joined . "\n"; + } + + if ( $v eq '' + || $v =~ /[:\#\'\"\[\]\{\},&*?!|>%-]/ + || $v =~ /^(?:yes|no|true|false|null|on|off|~)$/i ) + { + $v =~ s/'/''/g; + return "'$v'"; + } + return $v; +} + +sub _to_yaml { + my ( $data, $indent ) = @_; + $indent //= 0; + my $spaces = ' ' x $indent; + my $output = ''; + + if ( !defined $data ) { + return "~\n"; + } + elsif ( ref $data eq 'HASH' ) { + $output .= "\n" if $indent > 0; + foreach my $key ( sort keys %$data ) { + my $val = $data->{$key}; + $output .= $spaces . $key . ":"; + if ( ref $val ) { + $output .= _to_yaml( $val, $indent + 1 ); + } + else { + my $yaml_val = _yaml_scalar( $val, $indent ); + if ( $yaml_val =~ /^\|/ ) { + $output .= " " . $yaml_val; + } + else { + $output .= " " . $yaml_val . "\n"; + } + } + } + } + elsif ( ref $data eq 'ARRAY' ) { + $output .= "\n" if $indent > 0; + foreach my $item (@$data) { + $output .= $spaces . "-"; + if ( ref $item ) { + my $inner = _to_yaml( $item, $indent + 1 ); + $inner =~ s/^\n\s*//; + $output .= " " . $inner; + } + else { + my $yaml_val = _yaml_scalar( $item, $indent ); + if ( $yaml_val =~ /^\|/ ) { + $output .= " " . $yaml_val; + } + else { + $output .= " " . $yaml_val . "\n"; + } + } + } + } + else { + my $yaml_val = _yaml_scalar( $data, $indent ); + if ( $yaml_val =~ /^\|/ ) { + $output .= $yaml_val; + } + else { + $output .= $yaml_val . "\n"; + } + } + return $output; +} + +sub _sanitized_result_for_export { + my $orig = shift; + my %copy = %$orig; + + if ( ref $copy{'MySQLTuner'} eq 'HASH' ) { + my %mt_copy = %{ $copy{'MySQLTuner'} }; + if ( ref $mt_copy{'options'} eq 'HASH' ) { + my %opts = %{ $mt_copy{'options'} }; + for my $secret (qw(pass password ssh-password passenv userenv)) { + $opts{$secret} = '[REDACTED]' if defined $opts{$secret}; + } + $mt_copy{'options'} = \%opts; + } + $copy{'MySQLTuner'} = \%mt_copy; + } + + if ( ref $copy{'MySQL Client'} eq 'HASH' ) { + my %mc_copy = %{ $copy{'MySQL Client'} }; + if ( defined $mc_copy{'Authentication Info'} ) { + my $auth = $mc_copy{'Authentication Info'}; + $auth =~ s/-p'[^']*'/-p'[REDACTED]'/g; + $auth =~ s/-p(?!'\[REDACTED\]')\S+/-p[REDACTED]/g; + $mc_copy{'Authentication Info'} = $auth; + } + $copy{'MySQL Client'} = \%mc_copy; + } + + return \%copy; +} + +sub _serialize_to_json { + my ($data) = @_; + if (!defined $data) { + return 'null'; + } + my $ref = ref($data); + if (!$ref) { + if ($data =~ /^-?(?:[0-9]+(?:\.[0-9]+)?)$/) { + return $data; + } + my $escaped = $data; + $escaped =~ s/\\/\\\\/g; + $escaped =~ s/"/\\"/g; + $escaped =~ s/\n/\\n/g; + $escaped =~ s/\r/\\r/g; + $escaped =~ s/\t/\\t/g; + return '"' . $escaped . '"'; + } elsif ($ref eq 'HASH') { + my @pairs; + foreach my $k (sort keys %$data) { + my $v = $data->{$k}; + push @pairs, _serialize_to_json($k) . ':' . _serialize_to_json($v); + } + return '{' . join(',', @pairs) . '}'; + } elsif ($ref eq 'ARRAY') { + my @elems = map { _serialize_to_json($_) } @$data; + return '[' . join(',', @elems) . ']'; + } + return 'null'; +} + sub dump_result { + if ( $opt{'json'} && $opt{'yaml'} ) { + print STDERR "ERROR: --json and --yaml are mutually exclusive\n"; + return 1; + } #debugprint Dumper( \%result ) if ( $opt{'debug'} ); debugprint "HTML REPORT: " . ( $opt{'reportfile'} // 'undef' ); @@ -12520,7 +13133,36 @@ sub dump_result { ( $details->{'res_logs'} // 10 ) + ( $details->{'res_meta'} // 10 ); + my $report_host = $opt{'host'} // $myvar{'hostname'} // 'localhost'; + my $report_port = $opt{'port'} // $myvar{'port'} // '3306'; + # Render lists + my @sys_recs = ( + @sysrec, + grep { /(swap|swappiness|memory|ram|cpu|process|disk|mountpoint|limit|packet|event|tcp|slot|proc|open files)/i } @generalrec + ); + my %seen_sys; + @sys_recs = grep { !$seen_sys{$_}++ } @sys_recs; + + my @sec_recs = ( + @secrec, + grep { /(cve|security|password|authentication|ssl|encrypt|host|grant|privilege|user|transport)/i } @generalrec + ); + my %seen_sec; + @sec_recs = grep { !$seen_sec{$_}++ } @sec_recs; + + my @conn_recs = + grep { /(connection|connect|thread|max_user_connections|max_connections|aborted)/i } + @generalrec; + + my @perf_recs = + grep { /(slow|query|join|sort|cache|temporary|tmp|lock|started)/i } + @generalrec; + + my @repl_recs = + grep { /(replica|sla[v]e|gtid|replication|binlog|relay)/i } + @generalrec; + my $general_rec_html = join( "\n", map { @@ -12557,7 +13199,7 @@ sub dump_result { "
  • ‱" . escape_html( format_recommendation_item($_) ) . "
    • " - } @secrec + } @sec_recs ) || "
  • No security concerns identified.
    • "; @@ -12567,22 +13209,459 @@ sub dump_result { "
  • ‱" . escape_html( format_recommendation_item($_) ) . "
    • " - } @sysrec + } @sys_recs ) || "
  • No OS or system modifications suggested.
    • "; - # Build raw output HTML - my $raw_output_html = - join( "\n", map { escape_html($_) } @raw_output_lines ); + my $connections_rec_html = join( + "\n", + map { +"
  • ‱" + . escape_html( format_recommendation_item($_) ) + . "
    • " + } @conn_recs + ) + || "
  • No connection or network recommendations.
    • "; - # Determine health score color class - my $score_color_class = - $score > 80 - ? 'text-emerald-400' - : ( $score > 50 ? 'text-amber-400' : 'text-rose-500' ); - my $score_bg_class = - $score > 80 ? 'bg-emerald-500/10 border-emerald-500/30' - : ( + my $performance_rec_html = join( + "\n", + map { +"
  • ‱" + . escape_html( format_recommendation_item($_) ) + . "
    • " + } @perf_recs + ) + || "
  • No query or performance recommendations.
    • "; + + my $replication_rec_html = join( + "\n", + map { +"
  • ‱" + . escape_html( format_recommendation_item($_) ) + . "
    • " + } @repl_recs + ) + || "
  • No replication recommendations.
    • "; + + # Build raw output HTML + my $raw_output_html = + join( "\n", map { escape_html($_) } @raw_output_lines ); + + # Phase 13: Calculate Sectional Health Score variables + my $kpi_gen = $result{'SectionalHealthScore'}{'General'} // 100; + my $kpi_stor = $result{'SectionalHealthScore'}{'Storage'} // 100; + my $kpi_sec = $result{'SectionalHealthScore'}{'Security'} // 100; + my $kpi_repl = $result{'SectionalHealthScore'}{'Replication'} // 100; + my $kpi_mode = $result{'SectionalHealthScore'}{'Modeling'} // 100; + + # Serialize metrics to JSON for Javascript download + my $json_myvar = _serialize_to_json( \%myvar ); + my $json_mystat = _serialize_to_json( \%mystat ); + my $json_mycalc = _serialize_to_json( \%mycalc ); + my $json_general = _serialize_to_json( \@generalrec ); + my $json_adjvars = _serialize_to_json( \@adjvars ); + my $json_secrec = _serialize_to_json( \@secrec ); + my $json_sysrec = _serialize_to_json( \@sysrec ); + my $json_modeling = _serialize_to_json( \@modeling ); + + # Additional metrics calculations for HTML visual components + my $bp_reads = $mystat{'Innodb_buffer_pool_reads'} // 0; + my $bp_reqs = $mystat{'Innodb_buffer_pool_read_requests'} // 0; + my $bp_hit_pct = 100.0; + if ($bp_reqs > 0) { + $bp_hit_pct = 100.0 * (1.0 - ($bp_reads / $bp_reqs)); + } + $bp_hit_pct = sprintf("%.2f", $bp_hit_pct); + + my $threads_created = $mystat{'Threads_created'} // 0; + my $connections = $mystat{'Connections'} // 0; + my $thread_cache_hit_pct = 100.0; + if ($connections > 0) { + $thread_cache_hit_pct = 100.0 * (1.0 - ($threads_created / $connections)); + } + $thread_cache_hit_pct = sprintf("%.2f", $thread_cache_hit_pct); + + my $tmp_disk = $mystat{'Created_tmp_disk_tables'} // 0; + my $tmp_total = $mystat{'Created_tmp_tables'} // 0; + my $tmp_mem_pct = 100.0; + my $tmp_disk_pct = 0.0; + if ($tmp_total > 0) { + $tmp_disk_pct = 100.0 * ($tmp_disk / $tmp_total); + $tmp_mem_pct = 100.0 - $tmp_disk_pct; + } + $tmp_mem_pct = sprintf("%.1f", $tmp_mem_pct); + $tmp_disk_pct = sprintf("%.1f", $tmp_disk_pct); + + # Schema and Engine Statistics calculations for Storage & Modeling tabs + my $db_count = exists $result{'Databases'}{'List'} ? scalar @{ $result{'Databases'}{'List'} } : 0; + my $total_tables = 0; + my $engines_table_html = ''; + my $engines_status_html = ''; + + if ( exists $result{'Engine'} ) { + my @badges; + my $total_data_size = 0; + my $total_index_size = 0; + my $total_size = 0; + + foreach my $eng ( sort keys %{$result{'Engine'}} ) { + my $info = $result{'Engine'}{$eng}; + my $tbl_count = $info->{'Table Number'} // 0; + my $d_size = $info->{'Data Size'} // 0; + my $i_size = $info->{'Index Size'} // 0; + my $t_size = $info->{'Total Size'} // 0; + + my $status = $info->{'Enabled'} // ''; + my $color_class = ($status eq 'YES' || $status eq 'DEFAULT' || $status eq 'ACTIVE') + ? 'text-emerald-400 bg-emerald-500/10 border-emerald-500/20' + : 'text-slate-500 bg-slate-500/5 border-slate-500/10'; + my $prefix = ($status eq 'YES' || $status eq 'DEFAULT' || $status eq 'ACTIVE') ? '+' : '-'; + push @badges, "$prefix$eng"; + + next if $tbl_count == 0; + + $total_tables += $tbl_count; + $total_data_size += $d_size; + $total_index_size += $i_size; + $total_size += $t_size; + + $engines_table_html .= sprintf( + "" . + "%s" . + "%d" . + "%s" . + "%s" . + "%s" . + "", + escape_html($eng), + $tbl_count, + hr_bytes($d_size), + hr_bytes($i_size), + hr_bytes($t_size) + ); + } + if ($engines_table_html) { + $engines_table_html .= sprintf( + "" . + "Total" . + "%d" . + "%s" . + "%s" . + "%s" . + "", + $total_tables, + hr_bytes($total_data_size), + hr_bytes($total_index_size), + hr_bytes($total_size) + ); + } + $engines_status_html = join(' ', @badges); + } + + if (!$engines_table_html) { + $engines_table_html = "No storage engine usage statistics available."; + } + if (!$engines_status_html) { + $engines_status_html = "No storage engine status available."; + } + + my $frg_count = $fragtables // 0; + my $no_pk_count = exists $result{'Tables without PK'} ? scalar @{ $result{'Tables without PK'} } : 0; + my $unused_index_count = grep { ref($_) eq 'HASH' && $_->{type} eq 'unused_index' } @modeling; + my $redundant_index_count = grep { ref($_) eq 'HASH' && $_->{type} eq 'redundant_index' } @modeling; + + # InnoDB Page stats & capacity Calculations + my $bp_pages_total = $mystat{'Innodb_buffer_pool_pages_total'} // 0; + my $bp_pages_free = $mystat{'Innodb_buffer_pool_pages_free'} // 0; + my $bp_pages_used = $bp_pages_total - $bp_pages_free; + my $bp_page_size = $myvar{'innodb_page_size'} // 16384; + my $bp_total_bytes = $bp_pages_total * $bp_page_size; + my $bp_free_bytes = $bp_pages_free * $bp_page_size; + my $bp_used_bytes = $bp_pages_used * $bp_page_size; + + my $innodb_log_info_html = ''; + if ( mysql_version_ge( 8, 0, 30 ) ) { + if ( defined $myvar{'innodb_redo_log_capacity'} ) { + $innodb_log_info_html = "innodb_redo_log_capacity" . hr_bytes($myvar{'innodb_redo_log_capacity'}) . ""; + } else { + $innodb_log_info_html = "innodb_redo_log_capacityN/A"; + } + } else { + my $log_file_size = $myvar{'innodb_log_file_size'} // 0; + my $log_files_in_group = $myvar{'innodb_log_files_in_group'} // 0; + my $total_log_size = $log_file_size * $log_files_in_group; + $innodb_log_info_html = "innodb_log_file_size" . hr_bytes($log_file_size) . "" . + "innodb_log_files_in_group$log_files_in_group" . + "Total Log File Size" . hr_bytes($total_log_size) . " (" . ($mycalc{'innodb_log_size_pct'} // 0) . "% of buffer pool)"; + } + + my $chunk_align_html = ''; + if (defined $myvar{'innodb_buffer_pool_chunk_size'} && defined $myvar{'innodb_buffer_pool_instances'} && $myvar{'innodb_buffer_pool_chunk_size'} > 0 && $myvar{'innodb_buffer_pool_instances'} > 0) { + my $num_chunks = int( ($myvar{'innodb_buffer_pool_size'} // 0) / $myvar{'innodb_buffer_pool_chunk_size'} ); + my $expected_size = int($myvar{'innodb_buffer_pool_chunk_size'}) * int($myvar{'innodb_buffer_pool_instances'}); + my $is_aligned = (int($myvar{'innodb_buffer_pool_size'} // 0) % $expected_size == 0); + my $align_status = $is_aligned ? "Aligned" : "Not Aligned"; + $chunk_align_html = "innodb_buffer_pool_chunk_size" . hr_bytes($myvar{'innodb_buffer_pool_chunk_size'}) . "" . + "InnoDB Buffer Pool Chunks$num_chunks (instances: " . $myvar{'innodb_buffer_pool_instances'} . ")" . + "Chunk Alignment Status$align_status"; + } + + my $innodb_write_rate_html = ''; + my $innodb_os_log_written = $mystat{'Innodb_os_log_written'} || 0; + my $uptime = $mystat{'Uptime'} || 1; + if ($uptime > 3600) { + my $hourly_rate = $innodb_os_log_written / ($uptime / 3600); + $innodb_write_rate_html = "Hourly InnoDB Log Write Rate" . hr_bytes($hourly_rate) . "/hour"; + } else { + $innodb_write_rate_html = "Total InnoDB OS Log Written" . hr_bytes($innodb_os_log_written) . " (uptime < 1h)"; + } + + my $db_table_html = ''; + foreach my $db ( sort @dblist ) { + my $info = $result{'Databases'}{$db}; + next unless defined $info; + my $tbls = $info->{'Tables'} // 0; + my $rows = $info->{'Rows'} // 0; + my $d_size = $info->{'Data Size'} // 0; + my $i_size = $info->{'Index Size'} // 0; + my $t_size = $info->{'Total Size'} // 0; + + my $d_size_str = ($d_size =~ /^\d+$/) ? hr_bytes($d_size) : $d_size; + my $i_size_str = ($i_size =~ /^\d+$/) ? hr_bytes($i_size) : $i_size; + my $t_size_str = ($t_size =~ /^\d+$/) ? hr_bytes($t_size) : $t_size; + + $db_table_html .= sprintf( + "" . + "%s" . + "%d" . + "%d" . + "%s" . + "%s" . + "%s" . + "", + escape_html($db), + $tbls, + $rows, + $d_size_str, + $i_size_str, + $t_size_str + ); + } + if (!$db_table_html) { + $db_table_html = "No user database statistics available."; + } + + my $fragmented_tables_html = ''; + if ( exists $result{'Tables'}{'Fragmented tables'} && scalar @{ $result{'Tables'}{'Fragmented tables'} } > 0 ) { + foreach my $table_line ( @{ $result{'Tables'}{'Fragmented tables'} } ) { + my ( $table_schema, $table_name, $engine, $data_free ) = split /\t/msx, $table_line; + my $free_mb = $data_free / 1024 / 1024; + my $free_str = sprintf("%.2f MB", $free_mb); + my $sql = ($engine eq 'InnoDB') + ? "ALTER TABLE `$table_schema`.`$table_name` FORCE;" + : "OPTIMIZE TABLE `$table_schema`.`$table_name`;"; + $fragmented_tables_html .= sprintf( + "" . + "%s.%s" . + "%s" . + "%s" . + "%s" . + "", + escape_html($table_schema), + escape_html($table_name), + escape_html($engine), + $free_str, + escape_html($sql) + ); + } + } + if (!$fragmented_tables_html) { + $fragmented_tables_html = "No fragmented tables found."; + } + + my $no_pk_tables_html = ''; + if ( exists $result{'Tables without PK'} && scalar @{ $result{'Tables without PK'} } > 0 ) { + foreach my $badtable ( @{ $result{'Tables without PK'} } ) { + my ($schema, $table) = split /,/, $badtable, 2; + if (!defined $table) { + $schema = ''; + $table = $badtable; + } + $no_pk_tables_html .= sprintf( + "" . + "%s.%s" . + "Add explicit primary key for performance, maintenance, and replication stability." . + "", + escape_html($schema), + escape_html($table) + ); + } + } + if (!$no_pk_tables_html) { + $no_pk_tables_html = "All base tables have a primary key or unique index."; + } + + my $redundant_indexes_html = ''; + my $unused_indexes_html = ''; + foreach my $item (@modeling) { + if ( ref($item) eq 'HASH' ) { + if ( ($item->{type} // '') eq 'redundant_index' ) { + my $schema = $item->{schema} // ''; + my $table = $item->{table} // ''; + my $index = $item->{index} // ''; + my $dominant = $item->{dominant_index} // ''; + my $sql = $item->{sql} // ''; + $redundant_indexes_html .= sprintf( + "" . + "%s.%s" . + "%s" . + "redundant of %s" . + "%s" . + "", + escape_html($schema), + escape_html($table), + escape_html($index), + escape_html($dominant), + escape_html($sql) + ); + } + elsif ( ($item->{type} // '') eq 'unused_index' ) { + my $schema = $item->{schema} // ''; + my $table = $item->{table} // ''; + my $index = $item->{index} // ''; + my $sql = $item->{sql} // ''; + $unused_indexes_html .= sprintf( + "" . + "%s.%s" . + "%s" . + "%s" . + "", + escape_html($schema), + escape_html($table), + escape_html($index), + escape_html($sql) + ); + } + } + } + if (!$redundant_indexes_html) { + $redundant_indexes_html = "No redundant indexes detected."; + } + if (!$unused_indexes_html) { + $unused_indexes_html = "No unused indexes detected."; + } + + # Prioritized Top Findings lists + my @general_top = get_top_findings( \@generalrec ); + my @storage_top = get_top_findings( \@adjvars ); + my @security_top = get_top_findings( \@secrec ); + my @replication_top = get_top_findings( \@repl_recs ); + my @modeling_top = get_top_findings( \@modeling ); + + my $format_top_findings = sub { + my ( $title, $findings_ref ) = @_; + my @findings = @$findings_ref; + if ( !@findings ) { + return +"
    " + . "

    $title🟱 Optimal

    " + . "

    No critical issues or recommendations detected in this area.

    " + . "
    "; + } + my $items_html = ''; + foreach my $f (@findings) { + my $badge_color = + ( $f->{badge} eq 'Critical' ) + ? 'text-rose-400 bg-rose-500/10 border-rose-500/20' + : 'text-amber-400 bg-amber-500/10 border-amber-500/20'; + $items_html .= +"
  • " + . "" + . $f->{badge} + . "" + . "" + . escape_html( $f->{text} ) + . "" . "
    • "; + } + return +"
    " + . "

    $title⚠ Action Required

    " + . "
      $items_html
    " + . "
    "; + }; + + my $gen_findings_html = + $format_top_findings->( 'General Stats', \@general_top ); + my $store_findings_html = + $format_top_findings->( 'Storage & Performance', \@storage_top ); + my $sec_findings_html = + $format_top_findings->( 'Security & Compliance', \@security_top ); + my $repl_findings_html = + $format_top_findings->( 'Replication & HA', \@replication_top ); + my $model_findings_html = + $format_top_findings->( 'SQL Modeling', \@modeling_top ); + + # Throughput Efficiency Index + my $tei_qps = $result{'ThroughputEfficiency'}{'QPS'} // 0.0; + my $tei_reads = $result{'ThroughputEfficiency'}{'LogicalReadsSec'} + // 0.0; + my $tei_index = $result{'ThroughputEfficiency'}{'Index'} // 0.00000; + + # Resource Saturation Heatmap + my $sat_cpu = $result{'ResourceSaturation'}{'CPU'} // 0; + my $sat_mem = $result{'ResourceSaturation'}{'Memory'} // 0; + my $sat_conn = $result{'ResourceSaturation'}{'Connections'} // 0; + my $sat_io = $result{'ResourceSaturation'}{'IO'} // 0; + + my $get_sat_color = sub { + my $val = shift; + return $val > 85 + ? 'bg-rose-500' + : ( $val > 60 ? 'bg-amber-400' : 'bg-emerald-500' ); + }; + my $cpu_color = $get_sat_color->($sat_cpu); + my $mem_color = $get_sat_color->($sat_mem); + my $conn_color = $get_sat_color->($sat_conn); + my $io_color = $get_sat_color->($sat_io); + + # Historical Trend Deltas + my $historical_deltas_html = ''; + if ( defined $result{'Trends'} ) { + my $qps_delta = $result{'Trends'}{'QPS'} // 'N/A'; + my $score_delta = $result{'Trends'}{'HealthScore'} // 'N/A'; + my $size_delta = $result{'Trends'}{'TotalDataSize'} // 'N/A'; + my $sec_deltas = ''; + foreach my $sec ( 'General', 'Storage', 'Security', 'Replication', + 'Modeling' ) + { + my $d = $result{'Trends'}{'Sectional'}{$sec} // 'N/A'; + $sec_deltas .= +"
    $sec: $d
    "; + } + $historical_deltas_html = <<"HTML"; +
    +

    Historical Performance Deltas

    +
    +
    QPS: $qps_delta
    +
    Health Score: $score_delta
    +
    Data Size: $size_delta
    +
    +

    Sectional Score Trends

    + $sec_deltas +
    +
    +
    +HTML + } + + # Determine health score color class + my $score_color_class = + $score > 80 + ? 'text-emerald-400' + : ( $score > 50 ? 'text-amber-400' : 'text-rose-500' ); + my $score_bg_class = + $score > 80 ? 'bg-emerald-500/10 border-emerald-500/30' + : ( $score > 50 ? 'bg-amber-500/10 border-amber-500/30' : 'bg-rose-500/10 border-rose-500/30' ); @@ -12601,25 +13680,230 @@ sub dump_result { my $sec_width = int( $sec_score * 100 / 30 ); my $res_width = int( $res_score * 100 / 30 ); - # HTML Content template my $html_content = <<"HTML"; - MySQLTuner Report + MySQLTuner Advanced Report - + +
    @@ -12629,7 +13913,7 @@ sub dump_result {

    MySQLTuner Audit Report

    - $db_ver ($db_comment) + $db_ver ($db_comment) at $report_host:$report_port ‱ Tuner $tunerversion

    @@ -12640,164 +13924,1096 @@ sub dump_result {
    - -
    - - -
    - - -
    -

    Overall Health Score

    - - -
    - - - - -
    - $score - out of 100 + + + + + + +
    +
    + +
    +
    +

    Overall Health Score

    +
    + + + + +
    + $score + out of 100 +
    +
    +
    + $badge_text
    - -
    - $badge_text + +
    +

    Category Scores

    +
    +
    + Performance + $perf_score / 40 +
    +
    +
    +
    +
    +
    +
    + Security + $sec_score / 30 +
    +
    +
    +
    +
    +
    +
    + Resilience + $res_score / 30 +
    +
    +
    +
    +
    + $historical_deltas_html
    - -
    -

    Category Scores

    - - -
    -
    - Performance - $perf_score / 40 + +
    +
    +
    +

    Sectional Indicators & KPIs Dashboard

    + Phase 13 Unified View
    -
    -
    +
    +
    + General Stats + $kpi_gen% +
    +
    +
    +
    +
    + Storage & Perf + $kpi_stor% +
    +
    +
    +
    +
    + Security + $kpi_sec% +
    +
    +
    +
    +
    + Replication & HA + $kpi_repl% +
    +
    +
    +
    +
    + SQL Modeling + $kpi_mode% +
    +
    +
    +
    +
    + +
    + +
    +

    Resource Saturation Heatmap

    +
    +
    + CPU Load Saturation + $sat_cpu% +
    +
    +
    +
    +
    +
    +
    + Memory Saturation + $sat_mem% +
    +
    +
    +
    +
    +
    +
    + Connections Saturation + $sat_conn% +
    +
    +
    +
    +
    +
    +
    + Disk I/O Pressure + $sat_io% +
    +
    +
    +
    +
    +
    + + +
    +
    +

    Throughput Efficiency Index

    +
    +
    + Logical Work + $tei_qps QPS +
    +
    + Buffer Logical Reads + $tei_reads/s +
    +
    +
    +
    +
    +
    + Efficiency Ratio + Queries completed per logical page read +
    + $tei_index +
    +
    +
    -
    - -
    -
    - Security - $sec_score / 30 + +
    +

    Prioritized Executive Summary Findings

    +
    + $gen_findings_html + $store_findings_html + $sec_findings_html + $repl_findings_html + $model_findings_html +
    -
    -
    +
    +
    +
    + + +
    +
    + +
    + + +

    Full Console Output Trace

    + Show Logs + +     +
    + 
    $raw_output_html
    +
    +
    +
    - -
    -
    - Resilience - $res_score / 30 + + + + + + + + +
    - -
    -
    +
    +
    +
    -

    Security Findings

    +

    Security Findings & Hardening Advice

    -
    -
      - $secrec_html -
    +
    + +
    -
    +
    +
    +
      + $secrec_html +
    +
    +
    +
    - -
    -
    - -

    System & OS Recommendations

    -
    -
    -
      - $sysrec_html -
    + +
    +
    +
    + +
    +
    + User Databases + $db_count +
    +
    + Total Tables + $total_tables +
    +
    + Fragmented Tables + $frg_count +
    +
    + Tables w/o PK + $no_pk_count +
    +
    + Redundant Indexes + $redundant_index_count +
    +
    + Unused Indexes + $unused_index_count +
    + +
    +

    User Databases Size Distribution

    +
    + + + + + + + + + + + + + $db_table_html + +
    Database NameTablesRowsData SizeIndex SizeTotal Size
    +
    +
    + + +
    +

    Fragmented Tables Details (>100MB data & >10% free space)

    +
    + + + + + + + + + + + $fragmented_tables_html + +
    Table NameEngineFree SpaceSuggested Defragmentation SQL
    +
    +
    + + +
    +

    Tables Without Primary Key Details

    +
    + + + + + + + + + $no_pk_tables_html + +
    Table NameStatus / Hardening Advice
    +
    +
    + + +
    +

    Redundant Indexes Details

    +
    + + + + + + + + + + + $redundant_indexes_html + +
    Table NameRedundant IndexStatusSuggested Drop SQL
    +
    +
    + + +
    +

    Unused Indexes Details

    +
    + + + + + + + + + + $unused_indexes_html + +
    Table NameUnused IndexSuggested Drop SQL
    +
    +
    + +
    +
    +
    + +

    SQL Schema & Modeling Recommendations

    +
    +
    + + +
    +
    +
    +
      + $modeling_html +
    +
    +
    - -
    -
    - + + -
    - 
    $raw_output_html
    -
    -
    +
    +
      + $replication_rec_html +
    +
    + +
    + + + + + + + + + + + + + + + + + +