-m option to run a module as a script

[ncoghlan]

BPO	1035498
Nosy	@arigo, @theller, @rhettinger, @ncoghlan
Files	run_module_6.diff: Should fix Windows compile bug run_module_7.diff: Sets argv[0] correctly run_module_docs.diff: Update tutorial section on Invoking the Interpreter run_module_complete.diff: All code and documentation changes

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2004-10-08.22:53:28.000>
created_at = <Date 2004-09-27.14:22:45.000>
labels = ['interpreter-core']
title = '-m option to run a module as a script'
updated_at = <Date 2004-10-08.22:53:28.000>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2004-10-08.22:53:28.000>
actor = 'ncoghlan'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2004-09-27.14:22:45.000>
creator = 'ncoghlan'
dependencies = []
files = ['6268', '6269', '6270', '6271']
hgrepos = []
issue_num = 1035498
keywords = ['patch']
message_count = 22.0
messages = ['46957', '46958', '46959', '46960', '46961', '46962', '46963', '46964', '46965', '46966', '46967', '46968', '46969', '46970', '46971', '46972', '46973', '46974', '46975', '46976', '46977', '46978']
nosy_count = 4.0
nosy_names = ['arigo', 'theller', 'rhettinger', 'ncoghlan']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1035498'
versions = ['Python 2.4']

[ncoghlan]

Implements the '-m' option recently discussed on
python-dev.

Runs a module 'as if' it had been invoked from the
command line.

E.g., for a build directory, the following two commands
are equivalent:
./python -m pdb
./python Lib/pdb.py

Note that neither of these does the same thing as
"./python -c "import pdb". (In this latter case, any
"if __name__ == "__main__" code would not be executed,
whereas it will be invoked in the first two cases).

Given the vagaries of sys.path, this is quite handy -
if a module can be imported, it can be easily used as a
script too. Not that all modules will necessarily do
anything _interesting_ when used as a script. . .

The current implementation makes changes to main.c
(handle the new argument), pythonrun.[ch] (new flavour
of PyRun_) and import.[ch] (expose a
PyImport_FindModule C API)

Current limitations / to-do if this is pursued:

• print sensible errors for non-Python files (e.g.
  hotshot)
• allow 'dotted' names
• decide what to do with packages (e.g. run __init__.py)
• check usage messages in main.c are all still under
  512 characters (I assume that limit was set for a reason)
• the 1K limit on absolute file paths seems rather
  arbitrary.
• this is code written in the wee hours of the
  morning, so it probably has a few other lurking 'features'.

[ncoghlan]

Logged In: YES
user_id=1038590

New version of patch attached (run_module_2.diff).

This version allows modules within packages to be executed.
It does this *without* loading the packages first (just as
if the script were specified directly on the command line).

It's also a little more responsive when it comes to error
message, and I've confirmed that the usage message
components are now under 512 bytes each.

The FindModule method is now marked as private (i.e. with
leading underscore).

The limit on file paths in pythonrun.c is now taken from
osdefs.h (the same limit that import.c already uses)

The only bug I know of is that it misbehaves if you supply a
name like "a.b.module.d". You will get an ImportError saying
that d was not found, and the file pointer for 'module'
won't get closed properly.

[ncoghlan]

Logged In: YES
user_id=1038590

Barry, kicking this in your direction solely because you
were one of the people to comment on it on py-dev. Feel free
to say 'too busy' :)

[ncoghlan]

Logged In: YES
user_id=1038590

New version (run_module_3.diff)

Takes more account of the file description reported by
PyImport. Explictly restricts itself to PY_SOURCE and
PY_COMPILED scripts.

Behaves correctly when a non-package is encountered while
parsing a dotted name (closes the file descriptor and
reports an error).

I also realised that the '-m' option doesn't currently
comprehend "__path__" variables in packages. So maybe we
*should* be loading the packages as we find them, and then
querying them for their __path__ variable.

[ncoghlan]

Logged In: YES
user_id=1038590

New version (#4)

Trivial fix - tell PyRun_SimpleFile to close the file when
it's done with it.

[ncoghlan]

Logged In: YES
user_id=1038590

Relevant documentation locations that I can find:
Tutorial section 2.1 (Invoking the Interpreter)
Add a very brief description of the '-m' option

Python/C API section 2 (The Very High Level Layer)
Document PyRun_SimpleModule

What's New sections 12 & 13
I'll do this as a separate patch from the previous two,
as I can't recall if Andrew prefers to write these himself.

[ncoghlan]

Logged In: YES
user_id=1038590

5th version of patch attached

• restricts option to top-level modules only. The correct
  semantics for a module inside a package just aren't clear
  enough at this stage. Support for dotted names can be added
  later if this patch gets accepted, and it still seems like a
  good idea.

• adds a missing check from main.c (if we got the module
  option, don't try to open the file).

[ncoghlan]

Logged In: YES
user_id=1038590

Doc patch attached. Take it with a grain of salt, since this
is the first real doc patch I've made.

This doesn't cover anything for 'What's New' - that can wait
until the patch gets accepted (if it does).

[ncoghlan]

Logged In: YES
user_id=1038590

Raymond,

Passing to you to see what you think of the code, rather
than just the feature in the abstract.

Guido settled for not vetoing it, but the discussion with
him allowed me to articulate why I think this is a
worthwhile feature.
(Details at
http://mail.python.org/pipermail/python-dev/2004-October/049236.html)

Extension to allow packages and zip imports can wait until
there is any demand for either feature. I don't really
expect to see any, since I think the main uses will be
"python -m pdb", "python -m profile" and "python -m timeit".

[rhettinger]

Logged In: YES
user_id=80475

Armin, do you have time for this one. I'm buried for a few
days.

[arigo]

Logged In: YES
user_id=4771

Sorry, I didn't follow any of this discussion, neither here
nor in python-dev, so I cannot really comment.