GH-132554: "Virtual" iterators

[markshannon]

Just a draft PR until I have performance numbers.

• Issue: Use tagged ints for faster iteration #132554

[markshannon]

Performance is good. Nothing amazing, but a small speedup.

Stats show no significant changes

[Fidget-Spinner]

@markshannon I don't think it matters here, but you didn't update in https://github.com/python/cpython/pull/132545/filesPy_TAG_BITS to 3. It's still 1. I know it "technically" doesn't matter because we use a bitshift to zero it out, but we should still update it to be consistent.

[markshannon]

@markshannon I don't think it matters here, but you didn't update in https://github.com/python/cpython/pull/132545/filesPy_TAG_BITS to 3. It's still 1. I know it "technically" doesn't matter because we use a bitshift to zero it out, but we should still update it to be consistent.

Fixed in #134244

[iritkatriel]

I don't understand why you use this condition. Should it not be assert(ref.bits + 4 > ref.bits) or something like that?

[iritkatriel]

Would it help to define these via a macro? Something like

#define STACKREF_CHECK_FUNC(T) \ static inline bool \ PyStackRef_ ## T ## Check(_PyStackRef stackref) \ if (PyStackRef_IsTaggedInt(stackref)){\ return false; \ } \ return Py ## T ## _Check(PyStackRef_AsPyObjectBorrow(stackref)); \ } ... STACKREF_CHECK_FUNC(Exception); STACKREF_CHECK_FUNC(Code); STACKREF_CHECK_FUNC(Function); 

[Fidget-Spinner]

I think it might not work when want to define variants of Check and CheckExact, though we can always define two macros for that if we go down this route.

[iritkatriel]

I'd add this to make sure the test is testing what the comment is saying.
assert(sys.getrefcount(x) == 1)

[iritkatriel]

Do here same as in d9dc597 ?