Skip to content

Conversation

@pablogsal
Copy link
Member

@pablogsalpablogsal commented Dec 7, 2025

This PR restructures the profiling documentation to match the new profiling
package introduced by PEP 799.

The existing monolithic profile.rst is split into focused pages:

  • profiling.rst: Entry point that helps users choose between statistical
    and deterministic profiling, with a comparison table and guidance on when to
    use each approach.

  • profiling-tracing.rst: Documents the deterministic profiler (formerly
    documented under cProfile). Notes that cProfile remains as a
    backward-compatible alias.

  • profiling-sampling.rst: Documents the new statistical sampling
    profiler. Includes quick examples upfront, explains the key concepts, covers
    all output formats (pstats, flamegraph, gecko, heatmap), live mode, and
    async-aware profiling. A defaults table shows what happens without any flags.

  • pstats.rst: Separated out since it's used by both profiler types.

  • profile.rst: Reduced to a deprecation stub with migration guidance for
    users of the pure Python profiler.

The whatsnew entry is updated to link to the new documentation and reference
profiling.tracing instead of cProfile.


📚 Documentation preview 📚: https://cpython-previews--142373.org.readthedocs.build/en/142373/library/profiling.html

PEP 799 introduces a new `profiling` package that reorganizes Python's profiling tools under a unified namespace. This commit adds the documentation structure to match: a main entry point (profiling.rst) that helps users choose between profilers, detailed docs for the tracing profiler (profiling-tracing.rst), and separated pstats documentation. The tracing profiler docs note that cProfile remains as a backward-compatible alias, so existing code continues to work. The pstats module gets its own page since it's used by both profiler types and deserves focused documentation.
The sampling profiler is new in Python 3.15 and works fundamentally differently from the tracing profiler. It observes programs from outside by periodically capturing stack snapshots, which means zero overhead on the profiled code. This makes it practical for production use where you can attach to live servers. The docs explain the key concepts (statistical vs deterministic profiling), provide quick examples upfront, document all output formats (pstats, flamegraph, gecko, heatmap), and cover the live TUI mode. The defaults table helps users understand what happens without any flags.
Add the new profiling module pages to the Debugging and Profiling toctree. The order places the main profiling.rst entry point first, followed by the two profiler implementations, then pstats, and finally the deprecated profile module last.
The pure Python profile module is deprecated in 3.15 and scheduled for removal in 3.17. Users should migrate to profiling.tracing (or use the cProfile alias which continues to work). The page now focuses on helping existing users migrate: it shows the old vs new import style, keeps the shared API reference since both modules have the same interface, and preserves the calibration docs for anyone still using the pure Python implementation during the transition period.
Point cProfile to profiling.tracing docs and add profiling.sampling to the list of modules with CLI interfaces. The old profile-cli label no longer exists after the documentation restructure.
Enable cross-references to the new profiling module documentation and update the CLI examples to use the current syntax with the attach subcommand. Also reference profiling.tracing instead of cProfile since that's the new canonical name.
@pablogsal
Copy link
MemberAuthor

Most of this PR is moving existing documentation into separate files rather than writing new content. The profiling-tracing.rst and pstats.rst pages are largely relocated from the old profile.rst, and profile.rst itself is reduced to a deprecation stub with migration guidance.

The actual new writing is in profiling-sampling.rst, which documents the new statistical sampling profiler. This is where review attention is most valuable. The profiling.rst entry point also has new prose explaining when to use each profiler type.

:mod:`threading` module or implicitly through libraries that manage thread
pools.

By default, the profiler samples only the main thread. The ``--all-threads``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Threading feels quite common. It feels like the option even existing implies there is some consequence of enabling it? State why someone might NOT want to sample all threads in the docs.

Perhaps out of scope for this docs PR, but do we actually want this default or should it be the opposite? Does the option then even need to exist?

Copy link
MemberAuthor

@pablogsalpablogsalDec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Threading feels quite common. It feels like the option even existing implies there is some consequence of enabling it? State why someone might NOT want to sample all threads in the docs.
Perhaps out of scope for this docs PR, but do we actually want this default or should it be the opposite? Does the option then even need to exist?

We can discuss this elsewhere, but the reason is that is more expensive. You are basically dividing the sampling rate by the number of threads. There are also some of important optimizations if you know that you only want to sample the main thread.

This also generates a lot more data so the result files are a bit bigger. We can chat about what we think the default should be but the option should exist so people can decide to dedicate all sampling budget to the main thread if they wish. This can make a big difference.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

State why someone might NOT want to sample all threads in the docs.

Good point 👍 Will do

@pablogsal
Copy link
MemberAuthor

pablogsal commented Dec 7, 2025

Tachyon logo by @maigimenez ❤️ (she did the 3.11 and 3.10 cool release logos)

Copy link
Member

@hugovkhugovk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started adding suggestions via the GitHub UI but it was taking 4s to click the + and 6s to click "Add review comment" so I gave up and opened pablogsal#108 instead :/

@pablogsal
Copy link
MemberAuthor

I started adding suggestions via the GitHub UI but it was taking 4s to click the + and 6s to click "Add review comment" so I gave up and opened pablogsal#108 instead :/

giphy

pablogsaland others added 2 commits December 8, 2025 15:01
@hugovkhugovk mentioned this pull request Dec 9, 2025
Copy link
Member

@hugovkhugovk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@pablogsalpablogsal merged commit 4279785 into python:mainDec 9, 2025
50 checks passed
@pablogsalpablogsal deleted the gh-140727 branch December 9, 2025 12:55
@github-project-automationgithub-project-automationbot moved this from Todo to Done in Docs PRsDec 9, 2025
@pablogsal
Copy link
MemberAuthor

Thanks everyone for the review! ❤️

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docsDocumentation in the Doc dirskip news

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants

@pablogsal@gpshead@hugovk@savannahostrowski@StanFromIreland