`netrc` emits syntax errors for comments after blank lines

[LukeShu]

Bug report

netrc.netrc() emits a syntax error if it encounters a comment after a blank line. Prior to the 3.11, it only did this if the comment was the first non-whitespace thing in the file, but now with 3.11 it does it for all comments.

For example:

import netrc

filename = 'netrctest'

with open(filename, 'w') as fl:
 print("""
# HTTP
machine www.example.com login myuser password mypass

# FTP
machine ftp.example.com login myuser password mypass
""", file=fl)

print(netrc.netrc(filename))

produces

$ python netrc-test.py 
Traceback (most recent call last):
 File "/home/lukeshu/netrc-test.py", line 14, in <module>
 print(netrc.netrc(filename))
 ^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/netrc.py", line 31, in __init__
 self._parse(file, fp, default_netrc)
 File "/usr/lib/python3.11/netrc.py", line 66, in _parse
 raise NetrcParseError(
netrc.NetrcParseError: bad toplevel token 'HTTP' (netrctest, line 3)

This issue was made worse by #26330. Prior to that change (prior to 3.11), it only triggered on the if the comment was the first thing in the file. For example, if we remove the leading newline:

import netrc

filename = 'netrctest'

with open(filename, 'w') as fl:
 print("""# HTTP
machine www.example.com login myuser password mypass

# FTP
machine ftp.example.com login myuser password mypass
""", file=fl)

print(netrc.netrc(filename))

Before we got:

$ python netrc-test.py 
machine www.example.com
 login myuser
 password mypass
machine ftp.example.com
 login myuser
 password mypass

but now in 3.11 we get:

$ python netrc-test.py 
Traceback (most recent call last):
 File "/home/lukeshu/netrc-test.py", line 13, in <module>
 print(netrc.netrc(filename))
 ^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/netrc.py", line 75, in __init__
 self._parse(file, fp, default_netrc)
 File "/usr/lib/python3.11/netrc.py", line 140, in _parse
 raise NetrcParseError("bad follower token %r" % tt,
netrc.NetrcParseError: bad follower token 'FTP' (netrctest, line 4)

Your environment

• CPython versions tested on: 3.11.3
• Operating system and architecture: Parabola GNU/Linux-libre (like Arch Linux) on x86_64.

Linked PRs

• gh-104306: Fix incorrect comment handling in the netrc module, minor refactor #104511

[ssbarnea]

I recently found out that ~/.netrc files do not officially support comments and even worse, if you add # prefix to a line, netrc will still load that line and attempt to use that entry!

In other cased it might produce a syntax error but the current behavior is clearly messed badly.

Curl itself seems to treat lines prefixed with # as commented, but not python. Python will somehow load them incorrectly. I tried with a github.com entry and while the entry is correct by default and used by github/python to successfully download, once I add # prefix, curl continues to work (using non authenticated calls), python will fail, likely because is sending an incorrect request to github.

I suspect, it might be fixed by #104511