Python 3.13.3 runs __iter__ twice in a list comprehension

[fb2078]

Bug report

Bug description:

I can't upload comprehensionTestbed2.py because I am told the .py type is not allowed so I here is the script I used to reproduce this issue:

===========================================

# TestClass: Class that interacts with the DVD data file.

class TestClass:
 # Initialize the file timestamp.
 def __init__(self):
 pass
 # The iterator method.
 def __iter__(self):
 print('S1')
 self._index = 0
 return self
 # The next method.
 def __next__(self):
 print('S5')
 if self._index == 5:
 raise StopIteration
 value = self._index
 self._index += 1
 return value


# Test code.

print('#1')
for n in TestClass():
 print(n)

print('#2')
numbers = [n for n in TestClass()]
print(numbers)

==============================================

Here is the output of the Python script above:

D:\Files\Testing> comprehensionTestbed2.py 
#1 
S1 
S5 
0 
S5 
1 
S5 
2 
S5 
3 
S5 
4 
S5 
#2 
S1 
S1 <----- The __iter__ method is invoked twice in the list comprehension when using 3.13.3 (and 3.13.2). 
S5 
S5 
S5 
S5 
S5 
S5 
[0, 1, 2, 3, 4] 

This does not happen with python 3.12.9 and is a real problem when the iter method creates a mutex.

Edward

CPython versions tested on:

3.13

Operating systems tested on:

Windows

[picnixz]

cc @carljm (I don't know if you're the one to ping but IIRC, you worked on list comprehensions recently)

[picnixz]

Ah, could it be a duplicate of #127682?

[efimov-mikhail]

Ah, could it be a duplicate of #127682?

I think so, this is a duplicate.

[fb2078]

This was fixed in 3.13.4 but is back in 3.13.5: D:\Files\Testing> more comprehensionTestbed2.py # TestClass: Class that interacts with the DVD data file. class TestClass: # Initialize the file timestamp. def __init__(self): pass # The iterator method. def __iter__(self): print('S1') self._index = 0 return self # The next method. def __next__(self): print('S5') if self._index == 5: raise StopIteration value = self._index self._index += 1 return value # Test code. print('#1') for n in TestClass(): print(n) print('#2') numbers = [n for n in TestClass()] print(numbers) D:\Files\Testing> comprehensionTestbed2.py #1 S1 S5 0 S5 1 S5 2 S5 3 S5 4 S5 #2 S1 S1 S5 S5 S5 S5 S5 S5 [0, 1, 2, 3, 4] I was in the process of switching from 3.12.9 to 3.13.4 but then I noticed 3.13.5 had been released so I tested it. This is important to me because in production code the __iter__ method creates a lock and the script will hang when __iter__ is called the second time. Of course, I could get around that by using a counter but why is __iter__ being called more than once when I does not need to be. Edward

…

________________________________ From: Bénédikt Tran ***@***.***> Sent: Saturday, April 19, 2025 2:11 AM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***> Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) Closed #132711<#132711> as duplicate. — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2AWY5MU4LAPD2A4LUL22IHKNAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJXGMZDQMJTGI3DONA>. You are receiving this because you authored the thread.

[efimov-mikhail]

Unfortunately, we can't fix this bug without producing other bigger problems. So, fix in 3.13.4 was reverted.

You can read this topic for more information: https://discuss.python.org/t/python-3-13-5-is-now-available-yes-really/95211

[fb2078]

Understood. For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension? Edward

…

________________________________ From: Mikhail Efimov ***@***.***> Sent: Thursday, June 19, 2025 12:58 PM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***> Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) [https://avatars.githubusercontent.com/u/5574866?s=20&v=4]efimov-mikhail left a comment (python/cpython#132711)<#132711 (comment)> Unfortunately, we can't fix this bug without producing other bigger problems. So, fix in 3.13.4 was reverted. You can read this topic for more information: https://discuss.python.org/t/python-3-13-5-is-now-available-yes-really/95211 — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2GBD5NGOSNBHBRNPNT3EMI4ZAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOBZGA2TCMJTGE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[efimov-mikhail]

For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension?

Only in comprehensions.

JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way:

def __iter__(self):
 return self

[fb2078]

That works but it seems that the __iter__ method is not really supposed do anything but just return self. Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version? Edward

…

________________________________ From: Mikhail Efimov ***@***.***> Sent: Thursday, June 19, 2025 11:53 PM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***> Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) [https://avatars.githubusercontent.com/u/5574866?s=20&v=4]efimov-mikhail left a comment (python/cpython#132711)<#132711 (comment)> For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension? Only in comprehensions. JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way: def __iter__(self): return self — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2DXTJJHYAPKHC34Y4L3EOVXTAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJQGAYDANJXGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[fb2078]

Sorry, I pushed the Send too soon. I wanted to thank you for your responses and all who were involved with this issue. Edward

…

________________________________ From: Edward Waugh ***@***.***> Sent: Friday, June 20, 2025 6:30 PM To: python/cpython ***@***.***> python/cpython ***@***.***> Cc: Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) That works but it seems that the __iter__ method is not really supposed do anything but just return self. Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version? Edward

________________________________ From: Mikhail Efimov ***@***.***> Sent: Thursday, June 19, 2025 11:53 PM To: python/cpython ***@***.***> Cc: fb2078 ***@***.***> Author ***@***.***> Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711) [https://avatars.githubusercontent.com/u/5574866?s=20&v=4]efimov-mikhail left a comment (python/cpython#132711)<#132711 (comment)> For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension? Only in comprehensions. JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way: def __iter__(self): return self — Reply to this email directly, view it on GitHub<#132711 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BRUQY2DXTJJHYAPKHC34Y4L3EOVXTAVCNFSM6AAAAAB3NZ2A3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJQGAYDANJXGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>