ConfigParser does not handle files without sections

[kernc]

BPO	22253
Nosy	@terryjreedy, @pfmoore, @ambv, @vadmium, @serhiy-storchaka, @kernc, @pslacerda, @johnlinp
PRs	gh-66449: Add support to unnamed sections in ConfigParser #2735
Files	nosection.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ambv'
closed_at = None
created_at = <Date 2014-08-22.18:22:48.077>
labels = ['type-feature', 'library']
title = 'ConfigParser does not handle files without sections'
updated_at = <Date 2019-11-05.20:39:34.585>
user = 'https://github.com/kernc'

bugs.python.org fields:

activity = <Date 2019-11-05.20:39:34.585>
actor = 'Pedro Lacerda'
assignee = 'lukasz.langa'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-08-22.18:22:48.077>
creator = 'kernc'
dependencies = []
files = ['43345']
hgrepos = []
issue_num = 22253
keywords = ['patch']
message_count = 15.0
messages = ['225693', '225696', '225715', '226587', '226593', '226621', '226827', '226835', '226909', '226936', '227749', '268271', '298339', '298448', '356060']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'paul.moore', 'lukasz.langa', 'tshepang', 'martin.panter', 'serhiy.storchaka', 'kernc', 'Pedro Lacerda', 'johnlinp']
pr_nums = ['2735']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22253'
versions = []

Linked PRs

• gh-66449: configparser: Add support for unnamed sections #117273
• gh-66449: remove duplicate configparser section in 3.13 whatsnew #123874
• [3.13] gh-66449: remove duplicate configparser section in 3.13 whatsnew (GH-123874) #123879

[kernc]

ConfigParser does not handle files that specify options in the "global" section (first option specified before any section is). This configuration "behavior" is present in the configuration files in the wild [1, 2], and while the naive workaround is simple, ... really?!

The support for section-less configuration would also play nice with various "POSIX-compatible config files" (think /etc/default/*, /etc/os-release, ...).

Ideally, the support could be enabled by default or only when strict=False constructor argument is supplied. The options in this global section could likely be accessed in the empty ('') section, e.g. config.get('', 'option').

Thus, ConfigParser could finally really replace the venerable ConfigObj 3.

[ambv]

That's an interesting feature request. Parsing it only while strict=False sounds like a good plan.

[kernc]

I, for one, would actually prefer if global options were parsed by default and MissingSectionHeaderError was deprecated instead.
From what little specification available, INI format does **not** require options be in sections [4, 5].

Additionally, "Linux and Unix systems also use a similar file format for system configuration" 6 and allowing global options being a (very sane) default would nicely fill this use case as well.

In general, the format is not well defined 6, so choice of name strict for an argument is kind of odd too. What is it conforming to?

It may be my sole opinion that parsing global options by default into a '' (or appropriate) section and deprecating MissingSectionHeaderError would benefit everyone [2, 9] and hinder few if any one at all [8, 9]. YMMV.

[gvanrossum]

It looks like this feature request tries to change an existing (ancient) module into something it isn't. At the very least can you point to a spec for the syntax of "POSIX" config files? I always thought they were essentially shell scripts, which suggests that they might have a more complex (and different) quoting syntax than ConfigParser, so there might be cases where the interpretation of a line in a POSIX config file would be different than the interpretation of the same line in a .ini file.

[pfmoore]

It's not unreasonable as a new feature, but the default behaviour shouldn't change. It matches ini files (like it or not, ConfigParser parses ini-style files - the docs even say so), and sectionless values are not standard ini format.

I'd suggest a new __init__ option, allow_unnamed_section (default False) that permits variables to be placed before the first section header. I'd further suggest that the names be treated as if they were in a section with name '', for consistency of access with other sections.

It's plausible that people might want the defaults section to be the unnamed section. If so, that could be another option, default_is_unnamed (default False, if True this implies allow_unnamed_section). But I'm not sure the additional complexity is worth it.

[terryjreedy]

The MS function GetPrivateProfileString appears to require sections.
http://msdn.microsoft.com/en-us/library/ms724353.aspx
On the other hand, it does not appear to do interpolation, so we have already not restricted ourselves to the MS function.

In looking through the .ini files in my game directory, which includes some old games, I found a couple with no section header. So such files do exist in the wild. I am dubious that there are any with a mixture of both sections and additional option lines at the top without a section.

Anyone writing an app and planning to parse a .ini file can add [Start] or [Setup] at the top. So there is only an issue for 3rd party software parsing a file without.

I think a more useful new configparser feature would be to keep comment lines and write them back out after a configuration is changed.

[kernc]

I am dubious that there are any with a mixture of both sections and
additional option lines at the top without a section.

rsyncd.conf [1] is one such example, and I wouldn't say there aren't
countless more in the wild.

Anyone writing an app and planning to parse a .ini file can add [Start] or
[Setup] at the top.

Indeed. Here lies the problem of this unfortunate issue:
MissingSectionHeaderError is only ever caught [9] to mitigate this **awful
default behavior** and attach a dummy section at the top, as you say. Or
can anyone care to propose another relevant use case for this poorly (un-)
thought through exception?

I think a more useful new configparser feature would be to keep comment
lines and write them back out after a configuration is changed.

While this is very much off-topic, configobj [3] does too seem to have done
so since ages.

[serhiy-storchaka]

Microsoft Windows INI files, "POSIX-compatible config files", and other formats (e.g. Java properties files) use different methods for escaping, quoting, line continuing, interpolations, etc. Actually there are more differences than similarity between them.

[ambv]

I don't like the idea to magically introduce a '' section since this behaviour would be confusing for interpolation and not particularly discoverable by programmers. Let alone bikeshedding if this should rather be a None section.

Using DEFAULTSECT for this purpose is equally wrong since it would silently inject default values to every section, which may or may not be desirable.

All in all, it comes down to the question whether the programmer expects section-less configuration. If not, the '' section will not be helpful anyway. If yes, then it's desirable to be able to specify a section name for global options at *read time*. Symmetrically, the user could specify which section name to omit during configuration writing. I like that since it's explicit and more composable than a blanket global section. It would also be 100% backwards compatible.

I'll prepare a patch for this idea so we can see how good this API looks like in practice.

[terryjreedy]

I read the rsyncd.conf doc at http://linux.die.net/man/5/rsyncd.conf (linked from the StackOverflow question). These files are not .ini files. However, I believe the parsing rules are mostly compatible with RawConfigParser, or could be made so by using existing customization options, including subclassing.

The sectionless options are called 'global options' for one of two different reasons. Some, selected from a predefined list of startup options, act as if they were in a [STARTUP] section. Others, selected from a predefined list of module options, act as if they were in a [DEFAULT] section. The latter provide alternate default value for options in the various [<module>] sections.

We clearly should not directly support the specialized rules of rsyncd.conf. But if, as kernc requests, RawConfigParser gathered sectionless options into a '' section, users could probably work with these files. The details would be up to them, or a config_unix module writer. The same comment applies to other files, including .ini files with sectionless options.

Łukasz, the only one bikeshedding '' is you. I do not see any need for a new API and I think it just confuses this issue. Reading and writing sectionless options via a '' section should be sufficient to work with .ini files.

To me, the remaining question is whether to retain configparser.MissingSectionHeaderError. The problem with piggy-backing its optional use on the 'strict' parameter is that someone might want to reject duplicates while reading sectionless options. But it ia a plausible idea.

As an aside, the documentation for MissingSectionHeaderError is currently a bit confused. The docstring correctly says "Raised when a key-value pair is found before any section header." The doc incorrectly says "Exception raised when attempting to parse a file which has no section headers." I tested and it is indeed raised even when there is a section header after an initial option line. The exception message is also wrong: "File contains no section headers." The latter two should really say that the files does not *start* with a section header (ignoring comment lines), or use the wording of the docstring.