gh-129005: Fix buffer expansion in _pyio.FileIO.readall

[cmaloney]

Move to a linear slice append with an iterator which has a length hint. This is more expensive then PyByteArray_Resize, but I think as efficient as can get without a new bytearray Python API to resize.

The previous code didn't append as I had intended:

a=bytearray() >>>a[0:5] =b'\0'>>>abytearray(b'\x00') >>>a[5:16] =b'\01'>>>abytearray(b'\x00\x01') >>>len(a) 2

This should get the buildbots which are failing on test_io.test_io.PyMiscIOTest back to green. (A number have independent failures on test_getpath still)

What bytearray.resize looks like (Wrapping the C API): https://github.com/python/cpython/compare/main...cmaloney:cpython:resize_fixup?expand=0 (Currently that returns an int, but probably would make sense to return None)

• Issue: Reduce copies when reading files in pyio, match behavior of _io #129005

[cmaloney]

@vstinner this should get remaining bots failing on test_io.test_io.PyMiscIOTest back to green. This patch fixes the underlying issue: readall didn't actually readall, as the buffer isn't as big as the code thought, it slices it to bytes_read resulting in a 0-size buffer being passed to osreadinto, which returns 0, which ends the readall.

[vstinner]

Why not just b'\0' * addend?

[cmaloney]

That uses a lot more memory (And overall goal is to get _io and _pyio to use the same). result is expanded in length by addend bytes, a new bytes is constructed which is of length addend containing just null, then there's a memcpy from the new temporary bytes object to result... That means get 2x addend memory usage, rather than just expanding the buffer by addend bytes.

In this case, bytearray.extend also doesn't prefer working in place / creates a PyByteArray_FromStringAndSize that it copies the data out of a iterator on the object passed to it as an argument...

My preference is definitely bytearray.resize which does what I'd need here, but didn't want that to be a blocker for getting bots back to green (#129560). Reading through the slice, extend, append, etc. code in bytearray haven't found any other efficient ways to just "expand capacity, in place if possible without multiplying existing space" and (ideally) without requiring a by-byte write to every byte (the read loop is about to do that anyways).

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @encukou for commit 755b385 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F30617%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

[cmaloney]

will open a new PR on top of the reverts + gh-129559