Skip to content

Conversation

@eendebakpt
Copy link
Contributor

@eendebakpteendebakpt commented Jul 21, 2023

We improve the performance of methodcaller by using the vector protocol. The performance improvement depends on the use case, but for simple calls the performance doubles. Creation is a bit slower (because the vector call arguments have to be constructed), but the net gain is positive already after a single invocation of the methodcaller

call: Mean +- std dev: [main] 138 ns +- 1 ns -> [patch] 55.5 ns +- 1.6 ns: 2.49x faster creation+call: Mean +- std dev: [main] 257 ns +- 10 ns -> [patch] 190 ns +- 2 ns: 1.35x faster call kwarg: Mean +- std dev: [main] 222 ns +- 8 ns -> [patch] 81.4 ns +- 0.4 ns: 2.72x faster creation+call kwarg: Mean +- std dev: [main] 387 ns +- 11 ns -> [patch] 334 ns +- 5 ns: 1.16x faster Geometric mean: 1.80x faster 
Benchmark script
import pyperf setup = """ from operator import methodcaller as mc arr = [] call = mc('sort') call_kwarg = mc('sort', reverse=True) """ runner = pyperf.Runner() runner.timeit(name="call", stmt="call(arr)", setup=setup) runner.timeit(name="creation+call", stmt="call = mc('sort'); call(arr)", setup=setup) runner.timeit(name="call kwarg", stmt="call_kwarg(arr)", setup=setup) runner.timeit(name="creation+call kwarg", stmt="call = mc('sort', reverse=True); call(arr)", setup=setup) 

Note: this is a continuation of #27782. Changes with respect to that PR

  • The methodcaller_call was removed in favor of the vector call, but it is still needed for some builds. It is now restored, but less efficient
  • Store the full positional arguments as xargs instead of of a slice to avoid creation of an additional tuple
  • Additional storage in the methodcallerobject compared to main is now 2 variables PyObject* and a tuple holding the keyword argument names.
  • To avoid construction of the vector call arguments during the definition of the methodcaller, one can also defer the construction to the first invocation of the methodcaller. A branch with this approach is main...eendebakpt:cpython:fastmethodcaller_lazy_vectorcall

@eendebakpteendebakpt changed the title Draft: gh-27782: Improve the performance of methodcallerDraft: gh-89013: Improve the performance of methodcallerJul 21, 2023
@eendebakpteendebakpt changed the title Draft: gh-89013: Improve the performance of methodcallergh-89013: Improve the performance of methodcallerJul 23, 2023
@eendebakpteendebakpt changed the title gh-89013: Improve the performance of methodcallerDraft: gh-89013: Improve the performance of methodcallerJul 23, 2023
@eendebakpteendebakpt changed the title Draft: gh-89013: Improve the performance of methodcallergh-89013: Improve the performance of methodcallerJul 24, 2023
@corona10
Copy link
Member

close the PR since #107201 is merged.

@corona10corona10 closed this Aug 1, 2023
@eendebakpteendebakpt deleted the fastmethodcaller branch March 16, 2025 20:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@eendebakpt@corona10@bedevere-bot@anntzer