Skip to content

Commit 6673dc9

Browse files
aiwonderlandaiwonderland
committed
To pypi
1 parent ebabb44 commit 6673dc9

13 files changed

Lines changed: 7528 additions & 4 deletions

File tree

‎.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
11
.venv/
22
__pycache__/
3+
*.py[cod]
4+
*.pyd
5+
*.so
6+
build/
37
dist/
8+
*.egg-info/
9+
.pytest_cache/
10+

‎MANIFEST.in‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
include README.md
2+
include LICENSE
3+
include pyproject.toml
4+
include setup.py
5+
include _remote_debugging/remote_debug.h
6+
include _remote_debugging/remote_debug_314.h
7+
include _remote_debugging/remote_debugging_314.c
8+
include _remote_debugging/clinic/_remote_debugging_module_314.c.h
9+
recursive-include _remote_debugging *.c *.h
10+
recursive-include profiling *.py *.css *.html *.js *.png
11+
recursive-include .github *.md *.yml *.yaml
12+
prune build
13+
prune dist
14+
prune profiling.egg-info
15+
global-exclude *.py[cod] __pycache__

‎README.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# python-profiling
2+
3+
`python-profiling` backports the [`profiling`](https://docs.python.org/3.15/library/profiling.html)
4+
package introduced in Python 3.15 to CPython 3.14. It provides both a deterministic tracing profiler and a
5+
low-overhead statistical sampling profiler named **Tachyon**.
6+
7+
> **Status: Alpha.** This project depends on CPython private C APIs and the native `_remote_debugging`
8+
> extension. It does not support PyPy or the Python limited ABI. The profiler and target process must use
9+
> the same CPython major and minor version.
10+
11+
## Features
12+
13+
- `profiling.tracing`: deterministic profiling that records every call and return and produces `pstats`-compatible output.
14+
- `profiling.sampling`: external statistical sampling of a target process's Python call stacks.
15+
- Run a script or module, attach to a PID, or capture a one-shot stack dump.
16+
- Export pstats, collapsed stacks, interactive flame graphs, differential flame graphs, Firefox Profiler data,
17+
source heatmaps, JSONL, and raw binary profiles.
18+
- Wall-clock, CPU, GIL, and exception modes, plus all-thread sampling, async awareness, GC/native synthetic frames,
19+
opcode collection, subprocess profiling, and a live TUI.
20+
21+
## Requirements
22+
23+
- **CPython 3.14 or 3.15** (the current release range is `>=3.14,<3.16`).
24+
- Windows, Linux, or macOS.
25+
- Building from source requires a C compiler and CPython development headers:
26+
- Windows: Visual Studio 2022 Build Tools with **Desktop development with C++**.
27+
- Debian/Ubuntu: `build-essential python3.14-dev`.
28+
- macOS: Xcode Command Line Tools.
29+
- Attaching to another process may require additional privileges. See [Permissions and platform notes](#permissions-and-platform-notes).
30+
31+
## Installation
32+
33+
```bash
34+
pip install python-profiling
35+
```
36+
37+
For an editable development installation:
38+
39+
```bash
40+
python -m pip install -e .
41+
```
42+
43+
Raw binary profiles are uncompressed by default. If the zstd development library is installed, enable it at build time:
44+
45+
```bash
46+
# Linux/macOS
47+
PYTHON_PROFILING_WITH_ZSTD=1 python -m pip install .
48+
49+
# PowerShell
50+
$env:PYTHON_PROFILING_WITH_ZSTD = "1"
51+
python -m pip install .
52+
```
53+
54+
Custom zstd include and library directories can be supplied with `PYTHON_PROFILING_ZSTD_INCLUDE` and
55+
`PYTHON_PROFILING_ZSTD_LIB`, respectively.
56+
57+
## Quick start
58+
59+
### Statistical sampling
60+
61+
```bash
62+
# Profile a script; pstats output is printed to the terminal by default
63+
python -m profiling.sampling run examples/workload.py
64+
65+
# Profile a module
66+
python -m profiling.sampling run -m http.server 8000
67+
68+
# Generate a self-contained interactive HTML flame graph
69+
python -m profiling.sampling run --flamegraph -o profile.html examples/workload.py
70+
71+
# Attach to a running CPython process
72+
python -m profiling.sampling attach -d 10 -a --flamegraph -o profile.html 12345
73+
74+
# Display a running process's current stacks
75+
python -m profiling.sampling dump -a 12345
76+
```
77+
78+
Run `python -m profiling.sampling --help` or
79+
`python -m profiling.sampling <run|attach|dump|replay> --help` for the complete command-line reference.
80+
81+
### Deterministic tracing
82+
83+
```bash
84+
python -m profiling.tracing examples/workload.py
85+
python -m profiling.tracing -o trace.pstats examples/workload.py
86+
python -m profiling.tracing -m examples.workload
87+
```
88+
89+
The tracing profiler can also be used programmatically:
90+
91+
```python
92+
from profiling import tracing
93+
94+
tracing.run("sum(i * i for i in range(100_000))")
95+
```
96+
97+
## Output formats and raw data
98+
99+
| Option | Output | Purpose |
100+
| --- | --- | --- |
101+
| `--pstats` | Terminal table, or binary pstats with `-o` | General analysis and use with `pstats` |
102+
| `--collapsed` | `.txt` | Brendan Gregg-style collapsed stacks |
103+
| `--flamegraph` | Self-contained `.html` | Interactive call-stack flame graph |
104+
| `--diff-flamegraph BASELINE` | `.html` | Comparison against a baseline binary profile |
105+
| `--gecko` | `.json` | Import into [Firefox Profiler](https://profiler.firefox.com/) |
106+
| `--heatmap` | HTML directory | Source-line and opcode heatmap |
107+
| `--jsonl` | `.jsonl` | Aggregated raw data for programs, scripts, and agents |
108+
| `--binary` | `.bin` | High-throughput raw samples that can be replayed later |
109+
110+
The binary format stores sample timestamps, string and frame tables, and encoded stack changes. It is intended for
111+
capture-first, analyze-later workflows:
112+
113+
```bash
114+
python -m profiling.sampling run --binary -o raw-profile.bin examples/workload.py
115+
python -m profiling.sampling replay --flamegraph -o profile.html raw-profile.bin
116+
python -m profiling.sampling replay --jsonl -o profile.jsonl raw-profile.bin
117+
```
118+
119+
JSONL records appear in the fixed order `meta`, `string_table`, `frame_table`, `agg`, and `end`.
120+
Every line includes a schema version (`v`) and a per-run `run_id`. Consumers should ignore unknown record types and
121+
fields for forward compatibility. See the module documentation in `profiling/sampling/jsonl_collector.py` for the
122+
complete schema.
123+
124+
## Permissions and platform notes
125+
126+
- Linux access is controlled by `ptrace_scope` and `CAP_SYS_PTRACE`; containers commonly require `--cap-add=SYS_PTRACE`.
127+
- macOS may require `sudo`, and System Integrity Protection may prevent access to system Python processes.
128+
- On Windows, attaching to a process owned by another user may require an Administrator terminal.
129+
- The profiler and target must use the same CPython major and minor version. Pre-release builds may require an exact version match.
130+
- `--live` requires `curses`, which is usually unavailable in the official Windows Python distribution.
131+
- Profilers are intended to locate performance bottlenecks; use `timeit` instead for microbenchmarks.
132+
133+
## Building distributions
134+
135+
```bash
136+
python -m pip install build
137+
python -m build
138+
```
139+
140+
`setup.py` describes only the platform-specific `_remote_debugging` C extension. Project metadata, Python package
141+
discovery, and resource declarations are maintained in `pyproject.toml`. Because the extension depends on CPython
142+
private APIs, wheels must be built separately for every supported Python minor version and platform.
143+
144+
## License and origin
145+
146+
The code is backported from CPython and distributed under the Python Software Foundation License Version 2 and the
147+
third-party licenses listed in the repository's [`LICENSE`](LICENSE) file. Bundled frontend assets such as D3 and
148+
d3-flame-graph retain their respective licenses.
149+
150+
This project is not an official Python Software Foundation distribution.

‎_remote_debugging/_remote_debugging.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ extern "C" {
3232
#include "internal/pycore_pyerrors.h" // _PyErr_FormatFromCause
3333
#include "internal/pycore_pyhash.h" // _Py_HashPointerRaw
3434
#include "internal/pycore_stackref.h" // Py_TAG_BITS
35-
#include "../../Python/remote_debug.h"
35+
#include "remote_debug.h"
3636

3737
#include <assert.h>
3838
#include <errno.h>

0 commit comments

Comments
 (0)