Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions config-zh.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,9 @@ log_level = "warn"

[script]
# 非 TUI 脚本模式的标准输出渲染方式
# 可选值:"pretty", "plain", "quiet"
# 可选值:"pretty", "plain"
# - pretty: 类似 TUI 的头信息,包含时间戳/PID/TID,并保留缩进后的 payload
# - plain: 只输出脚本 payload 行
# - quiet: 完全不在 stdout 打印事件输出
# 默认值:"pretty"
output = "pretty"

Expand Down Expand Up @@ -241,6 +240,11 @@ mem_dump_cap = 256
# 如需在单个事件中打印较多/较大的变量,可适当增大。
max_trace_event_size = 32768

# 每条 bt/backtrace 指令最多采集的 DWARF unwind 栈帧数。
# 有效范围:1 到 128。
# 默认值:128
backtrace_depth = 128

# 强制使用 PerfEventArray 而不是 RingBuf(仅用于测试)
# 警告:仅用于测试目的。正常情况下系统会自动检测内核能力,
# 并使用 RingBuf(内核 >= 5.8)或回退到 PerfEventArray。
Expand Down
5 changes: 5 additions & 0 deletions config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,11 @@ mem_dump_cap = 256
# Increase if you plan to print many/large variables in one event.
max_trace_event_size = 32768

# Max DWARF-unwound frames captured by each bt/backtrace instruction.
# Valid range: 1 to 128.
# Default: 128
backtrace_depth = 128

# Force use of PerfEventArray instead of RingBuf (for testing only)
# WARNING: This is for testing purposes only. Normally the system auto-detects
# kernel capabilities and uses RingBuf (kernel >= 5.8) or falls back to PerfEventArray.
Expand Down
8 changes: 8 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,14 @@ GhostScope uses an **instruction-based protocol** for flexible trace event repre
| **Backtrace** | 0x10 | Stack backtrace with frame addresses |
| **EndInstruction** | 0xFF | Marks end of instruction sequence |

`Backtrace` is a compact frame stream, not pre-rendered text. The compiler asks
`ghostscope-dwarf` for compact DWARF CFI rows, loads those rows into a BPF array
map, and the uprobe program records module cookies plus module-normalized PCs.
Userspace then resolves raw IPs through the process module map and asks
`ghostscope-dwarf` for function, source line, and inline-chain information.
`bt` always means DWARF unwinding; the script language intentionally does not
expose helper/fp/backend selection.

**Variable Status Tracking**:

Each variable instruction includes a `status` field (u8) indicating data acquisition result:
Expand Down
2 changes: 1 addition & 1 deletion docs/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Background: one motivation for GhostScope was that newer bpftrace versions no lo
| Source line and statement probes | Supported; line-level attachment is a core path | Supported; statement probes can be resolved and attached |
| Variable access (params, locals, globals) | Supported. Build PC-context read plans with gimli-backed DWARF data; render by real types; naturally ASLR and PIE friendly | Supported. DWARF location expressions are lowered through SystemTap's pipeline into eBPF-compatible logic, with verifier and stack constraints |
| DWARF expression handling | Convert DWARF locations into semantic read plans and lower supported plans into eBPF runtime reads | Translate DWARF operations into internal representations and lower them into eBPF instruction sequences |
| Stack unwinding (CFI) | Not supported yet; planned via `.eh_frame` unwinding | Not supported in the eBPF backend |
| Stack unwinding (CFI) | Supported through DWARF-only `bt`/`backtrace` for compact CFI rows that can be executed safely in eBPF | Not supported in the eBPF backend |
| Event transport and formatting | RingBuf (on newer kernels) or PerfEventArray; configurable pages and event size; built-in dump helpers such as `{:x.N}`, `{:s.N}`, and `{:p}` | PERF_EVENT_ARRAY plus userspace formatting/interpreter flow; formatting and string handling are more constrained |
| BTF, CO-RE, linkage | Aya ecosystem, prefer RingBuf; not centered on BTF or CO-RE | No BTF or CO-RE focus; minimal libbpf-style backend |
| eBPF generation pipeline | Rust and Aya loader; focused on reading userspace DWARF variables and presentation | Custom IR and assembler pipeline that emits eBPF bytecode and ELF artifacts |
Expand Down
9 changes: 9 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ Behavior:
| `--script-file <PATH>` | | Script file to execute | None |
| `--script-help` | | Print the embedded script language reference and exit | Off |
| `--script-output <MODE>` | | Script event stdout mode: pretty, plain | pretty |
| `--backtrace-depth <N>` | | Max DWARF-unwound frames captured by each `bt`/`backtrace` instruction (`1..=128`) | 128 |
| `--dry-run` | | Compile the script, resolve trace targets, and exit without attaching uprobes. Requires the same eBPF privileges and kernel capabilities as a real run. | Off |
| `--dry-run-details` | | Include source, inline, and variable diagnostics in dry-run output; requires `--dry-run` | Off |
| `--status` | | Enable interactive DWARF/script/attach stderr status prompts | On |
Expand Down Expand Up @@ -477,6 +478,11 @@ compare_cap = 64
# Maximum size of a single trace event (bytes). Applies to PerfEventArray accumulation buffer.
max_trace_event_size = 32768

# Max DWARF-unwound frames captured by each bt/backtrace instruction.
# Can be overridden for one run with --backtrace-depth.
# Valid range: 1 to 128.
backtrace_depth = 128

# Recommended values:
# - Simple prints: 16384
# - General use: 32768
Expand Down Expand Up @@ -559,6 +565,7 @@ ringbuf_size = 1048576 # 1MB buffer for high event rates
mem_dump_cap = 4096 # Larger per-arg dump
compare_cap = 64 # Max bytes for built-in compares (strncmp/memcmp)
max_trace_event_size = 65536 # Larger event size for big formatted prints
backtrace_depth = 128 # Default full DWARF backtrace cap
proc_module_offsets_max_entries = 8192 # Support many modules

[general]
Expand All @@ -575,6 +582,7 @@ ringbuf_size = 131072 # 128KB minimal buffer
mem_dump_cap = 512
compare_cap = 32 # Smaller compare cap for minimal overhead
max_trace_event_size = 16384
backtrace_depth = 32
proc_module_offsets_max_entries = 1024 # Single process only
enable_sysmon_for_target = false # Disable standalone -t lifecycle tracking

Expand Down Expand Up @@ -674,6 +682,7 @@ GhostScope validates configuration at startup:
- **ringbuf_size**: Must be power of 2, range 4096-16777216 bytes
- **perf_page_count**: Must be power of 2, range 8-1024 pages
- **proc_module_offsets_max_entries**: Must be in range 64-65536
- **backtrace_depth**: Must be in range 1-128 frames
- **mem_dump_cap**, **compare_cap**, and **max_trace_event_size** are runtime caps; `max_trace_event_size` may be clamped by the selected event transport.

Invalid configuration will produce clear error messages with suggestions for fixes.
Expand Down
11 changes: 7 additions & 4 deletions docs/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,16 @@ Primarily tested and validated with DWARF 5 format. Theoretically supports DWARF

GhostScope recognizes `DW_OP_form_tls_address`, but the runtime TLS address resolver currently handles only x86_64 executable static TLS. GhostScope resolves the current thread's TLS base at probe time, so a trace running on different pthreads reads each thread's own TLS instance for that supported executable case. The same DWARF operation is also used for dynamic/shared-library TLS; those cases require DTV/module TLS lookup and are not modeled yet, so GhostScope rejects shared-object TLS instead of guessing an address.

### 7. Highly Optimized Code Support
### 7. Stack Backtrace Coverage
`bt` uses DWARF CFI only. GhostScope does not fall back to kernel stack helpers or frame-pointer walking, and it reports an explicit stop status when CFI is unavailable, not supported by the compact eBPF fast path, or a user-stack memory read fails. Cross-module frames can be symbolized from their raw IPs when the process module map is available, but unwinding continues only while GhostScope has compact DWARF rows it can execute safely in eBPF. Deep DWARF unwinding is split through an eBPF tail-call step program so the default `backtrace_depth = 128` avoids LLVM branch-distance and verifier-size limits; `status=truncated` means the configured depth or the tail-call unwind budget was reached before a natural stop.

### 8. Highly Optimized Code Support
Compiler optimizations (-O2, -O3) can cause variables to be optimized away or generate complex DWARF expressions. GhostScope will attempt to parse them, including inline function support, but some variables may be inaccessible (shown as OptimizedOut) because the compiler optimized them away.

### 8. Dynamically Loaded Libraries (dlopen)
### 9. Dynamically Loaded Libraries (dlopen)
GhostScope scans `/proc/PID/maps` at startup to obtain loaded dynamic library information. As long as GhostScope is started after `dlopen`, tracing works normally. Future plans include dynamically monitoring process `dlopen` behavior for better user experience.

### 9. Global Variables in `-t` Mode
### 10. Global Variables in `-t` Mode

- **Executable targets**: When `-t` points to an executable (`-t /path/to/app`), GhostScope treats that binary as the primary module and globals are supported by default.
- **Shared-library targets (existing processes)**: If GhostScope starts after the library has already been mapped (e.g., tracing a running process that loaded `libfoo.so` earlier), globals work without extra steps.
Expand All @@ -60,7 +63,7 @@ GhostScope scans `/proc/PID/maps` at startup to obtain loaded dynamic library in

> **Note**: The current sysmon pipeline still assumes the library is mapped when the exec event is handled; if a loader pulls it in much later, offsets are not retried yet.

### 10. `-p <pid>` Mode inside Containers or WSL
### 11. `-p <pid>` Mode inside Containers or WSL

- See [Container Environments](container.md) for the full explanation of container / WSL scenarios, PID namespace terminology, the scenario matrix, and current implementation limits.
- See [PID namespaces manual](https://www.man7.org/linux/man-pages/man7/pid_namespaces.7.html), [WSL issue #12408](https://github.com/microsoft/WSL/issues/12408), and [WSL issue #12115](https://github.com/microsoft/WSL/issues/12115) for background.
10 changes: 7 additions & 3 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,13 @@ GhostScope is still evolving quickly. The milestones below are ordered from “s
- Keep compatibility fallbacks for kernels or libbpf/Aya paths that still require regular `uprobe` attachments.

## Stack Unwinding
- Capture full call stacks at each trace point by parsing `.eh_frame`/`.eh_frame_hdr`.
- Surface the stack in the TUI with symbol/source awareness.
Reference: <https://lesenechal.fr/en/linux/unwinding-the-stack-the-hard-way#h5.1-parsing-eh_frame-and-eh_frame_hdr-with-gimli>
- DWARF-only `bt` / `backtrace` is now supported for compact CFI rows that can
be executed safely in eBPF, including deep stacks through a tail-call unwind
step program.
- Continue improving CFI coverage, cross-module accuracy, stop-status
diagnostics, and performance for large debug-info workloads.
- Keep the TUI and CLI renderers source-aware, with structured TUI display and
stable plain output for scripts.

## Stability & accuracy
- Keep fixing defects, hardening error handling, and ensuring data consistency.
Expand Down
60 changes: 54 additions & 6 deletions docs/scripting.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ GhostScope uses a domain‑specific language to define trace points and actions.
7. [Expressions](#expressions)
8. [Built-in Functions](#built-in-functions)
9. [Special Variables](#special-variables)
10. [Examples](#examples)
11. [Limitations](#limitations)
12. [Runtime Expression Failures (ExprError)](#runtime-expression-failures-exprerror)
10. [Stack Backtrace](#stack-backtrace)
11. [Examples](#examples)
12. [Limitations](#limitations)
13. [Runtime Expression Failures (ExprError)](#runtime-expression-failures-exprerror)

## Basic Syntax

Expand All @@ -34,7 +35,7 @@ GhostScope uses a domain‑specific language to define trace points and actions.
GhostScope supports the following statements:
- `trace` — define trace points and their actions
- `print` — output formatted text
- `backtrace` / `bt` — reserved syntax for future stack unwinding
- `backtrace` / `bt` — print a DWARF-unwound stack backtrace
- `if` / `else` — conditional execution
- `let` — script variable declaration
- Expression statements
Expand Down Expand Up @@ -676,9 +677,56 @@ trace foo {
}
```

## Stack Backtrace (not implemented)
## Stack Backtrace

Backtrace printing via `backtrace;` or `bt;` is planned but not implemented yet. The syntax is reserved, and a dedicated section will be added once available.
`backtrace;` and `bt;` emit a source-aware stack backtrace at the probe point. The unwinder uses DWARF CFI directly; there is no `unwind=` option and no helper/fp fallback mode to select.

```ghostscope
trace test_function {
print "before";
bt;
print "after";
}
```

Options:

- `bt raw;` prints raw module cookie, module offset, and runtime IP without source symbolization.
- `bt full;` prints symbolized source-aware frames. Raw IP/cookie debug metadata is kept out of `bt full` and is only shown by `bt raw`.
- `bt inline;` enables inline call-chain rendering. This is the default.
- `bt noinline;` suppresses inline call-chain rendering.

Backtrace depth is configured globally, not in the script. Use `--backtrace-depth <N>` or `[ebpf] backtrace_depth = N` in the config file. Valid range is `1..=128`; the default is `128`.
In `--script-output pretty`, backtrace payload lines are colorized when `[script] color` enables ANSI output. `--script-output plain` always emits the raw payload text without ANSI color.

Examples:

```ghostscope
trace test_function {
bt full;
bt raw noinline;
}
```

Typical output:

```text
backtrace: complete, 4 frames (max 128)
#0 test_function(int argc, char** argv) at sample_program.c:8:5 [sample_program+0x1189]
#1 caller(int value) at sample_program.c:42:9 [sample_program+0x1234]
#2 main(int argc, char** argv) at sample_program.c:88:12 [sample_program+0x13a0]
#3 <unknown function> at ?? [libc.so.6+0x2a1ca]
```

`bt raw;` keeps the same header but prints machine-facing fields for diagnosis:

```text
backtrace: truncated, 2 frames (max 2)
#0 0x1189 [sample_program+0x1189] raw=0x55... cookie=0x...
#1 0x1234 [sample_program+0x1234] raw=0x55... cookie=0x...
```

`status=complete` means DWARF unwinding reached a natural stop before the configured depth cap. `status=truncated` means GhostScope hit the configured depth cap or the eBPF tail-call unwind budget before a natural stop. Other statuses explain where unwinding stopped, for example unsupported CFI, unavailable module offsets, a failed user-memory read, or an invalid next frame. When available, `stopped:` includes a stable reason label and numeric code.

## Examples

Expand Down
6 changes: 6 additions & 0 deletions docs/zh/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,12 @@ GhostScope 使用**基于指令的协议**实现灵活的追踪事件表示:
| **Backtrace** | 0x10 | 带栈帧地址的栈回溯 |
| **EndInstruction** | 0xFF | 标记指令序列结束 |

`Backtrace` 是紧凑的栈帧数据流,不是预先渲染好的文本。compiler 会从
`ghostscope-dwarf` 获取 compact DWARF CFI row,并把这些 row 加载到 BPF
array map;uprobe 程序只记录 module cookie 与模块内标准化 PC。用户态再根据
进程模块映射解析 raw IP,并交给 `ghostscope-dwarf` 查询函数、源码行号和
inline 调用链。`bt` 始终表示 DWARF unwind,脚本语言不会暴露 helper/fp/后端选择。

**变量状态跟踪**:

每个变量指令都包含一个 `status` 字段 (u8) 指示数据获取结果:
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ GhostScope 的目标很明确:**针对带有 DWARF 调试信息的活跃进程
| 源码行/语句级设点 | 支持,行级附着是核心路径 | 支持,statement probe 可以解析后附着 |
| 变量访问(参/局/全) | 支持。基于 gimli 读取 DWARF,并生成 PC 上下文读取计划;按真实类型渲染,天然适配 ASLR 和 PIE | 支持。DWARF 位置表达式会经过 SystemTap 的处理链路降为 eBPF 可执行逻辑,但要受验证器和栈限制 |
| DWARF 表达式处理 | 将 DWARF 位置转换成语义读取计划,并把支持的计划 lower 成 eBPF 运行时读取 | 把 DWARF 操作翻译成内部表示,再继续降成 eBPF 指令序列 |
| 栈回溯(CFI) | 还不支持,计划通过 `.eh_frame` 支持 | eBPF 后端暂不支持 |
| 栈回溯(CFI) | 通过 DWARF-only `bt`/`backtrace` 支持可安全降到 eBPF 的 compact CFI row | eBPF 后端暂不支持 |
| 事件传输/格式化 | 新内核优先 RingBuf,也支持 PerfEventArray;页数和事件大小可配;内置 `{:x.N}`、`{:s.N}`、`{:p}` 等 dump helper | 更偏 PERF_EVENT_ARRAY + 用户态解释/格式化流程,格式和字符串能力更受约束 |
| BTF/CO-RE/链接 | Aya 生态,优先 RingBuf;不以 BTF/CO-RE 为核心 | 不以 BTF/CO-RE 为核心,更接近最小 libbpf 风格后端 |
| eBPF 生成链 | Rust + Aya 装载,重点是用户态 DWARF 变量读取和展示 | 自研 IR / assembler 流水线,产出 eBPF 字节码和 ELF 产物 |
Expand Down
9 changes: 9 additions & 0 deletions docs/zh/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ ghostscope bpffs prune --dry-run --json
| `--script-file <PATH>` | | 要执行的脚本文件 | 无 |
| `--script-help` | | 输出内嵌的脚本语言参考并退出 | 关 |
| `--script-output <MODE>` | | 脚本事件 stdout 模式:pretty, plain | pretty |
| `--backtrace-depth <N>` | | 每条 `bt`/`backtrace` 指令最多采集的 DWARF unwind 栈帧数(`1..=128`) | 128 |
| `--dry-run` | | 编译脚本、解析 trace 目标,然后退出,不 attach uprobe。需要与真实运行相同的 eBPF 权限和内核能力。 | 关 |
| `--dry-run-details` | | 在 dry-run 输出中包含源码、inline 和变量诊断;需要同时使用 `--dry-run` | 关 |
| `--status` | | 启用交互式 DWARF/脚本/attach stderr 状态提示 | 开 |
Expand Down Expand Up @@ -472,6 +473,11 @@ compare_cap = 64
# 单条 trace 事件的最大大小(字节)。适用于 PerfEventArray 累计缓冲区。
max_trace_event_size = 32768

# 每条 bt/backtrace 指令最多采集的 DWARF unwind 栈帧数。
# 可通过命令行 --backtrace-depth 临时覆盖。
# 有效范围:1 到 128。
backtrace_depth = 128

# 推荐值:
# - 简单打印:16384
# - 通用场景:32768
Expand Down Expand Up @@ -552,6 +558,7 @@ ringbuf_size = 1048576 # 1MB 缓冲区用于高事件率
mem_dump_cap = 4096 # 单参数转储上限更高
compare_cap = 64 # 内置比较最大比较字节数(strncmp/memcmp)
max_trace_event_size = 65536 # 大格式化输出需要更大的事件大小
backtrace_depth = 128 # 默认完整 DWARF 回溯上限
proc_module_offsets_max_entries = 8192 # 支持更多模块

[general]
Expand All @@ -568,6 +575,7 @@ ringbuf_size = 131072 # 128KB 最小缓冲区
mem_dump_cap = 512
compare_cap = 32 # 降低内置比较上限以减小开销
max_trace_event_size = 16384
backtrace_depth = 32
proc_module_offsets_max_entries = 1024 # 仅单进程
enable_sysmon_for_target = false # 关闭独立 -t 生命周期跟踪

Expand Down Expand Up @@ -667,6 +675,7 @@ GhostScope 在启动时验证配置:
- **ringbuf_size**:必须是 2 的幂,范围 4096-16777216 字节
- **perf_page_count**:必须是 2 的幂,范围 8-1024 页
- **proc_module_offsets_max_entries**:必须在 64-65536 范围内
- **backtrace_depth**:必须在 1-128 栈帧范围内
- **mem_dump_cap**、**compare_cap** 和 **max_trace_event_size** 是运行时上限;`max_trace_event_size` 可能会根据实际事件传输方式被 clamp。

无效配置将产生清晰的错误消息和修复建议。
Expand Down
Loading
Loading