Executable permissions of Python source files

[serhiy-storchaka]

BPO	19936
Nosy	@doko42, @gpshead, @skrah, @serhiy-storchaka
Files	executable_scripts.patch findnoshebang.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-01-23.14:37:44.083>
created_at = <Date 2013-12-09.11:23:21.357>
labels = ['type-bug', 'library']
title = 'Executable permissions of Python source files'
updated_at = <Date 2014-01-23.14:37:44.082>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2014-01-23.14:37:44.082>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-01-23.14:37:44.083>
closer = 'serhiy.storchaka'
components = ['Demos and Tools', 'Library (Lib)']
creation = <Date 2013-12-09.11:23:21.357>
creator = 'serhiy.storchaka'
dependencies = []
files = ['33056', '33498']
hgrepos = []
issue_num = 19936
keywords = ['patch']
message_count = 14.0
messages = ['205677', '208276', '208278', '208282', '208284', '208285', '208287', '208289', '208332', '208335', '208694', '208724', '208933', '208944']
nosy_count = 6.0
nosy_names = ['doko', 'gregory.p.smith', 'skrah', 'tshepang', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19936'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

[serhiy-storchaka]

Following files have executable permission bit but have no shebang ("#!"):

Lib/test/ssltests.py
Lib/test/test_pathlib.py
Lib/token.py
Tools/scripts/analyze_dxp.py
Tools/scripts/run_tests.py
Tools/scripts/win_add2path.py
Tools/stringbench/stringbench.py

I think executable bit should be cleared on them, because they can't be ran
without correct shebang.

Following files have shebang but have no executable permission bit:

Doc/includes/email-alternative.py
Doc/includes/email-dir.py
Doc/includes/email-unpack.py
Lib/difflib.py
Lib/http/cookies.py
Lib/idlelib/PyShell.py
Lib/lib2to3/tests/data/different_encoding.py
Lib/lib2to3/tests/data/false_encoding.py
Lib/mailbox.py
Lib/operator.py
Lib/smtplib.py
Lib/tarfile.py
Lib/test/_test_multiprocessing.py
Lib/test/crashers/recursive_call.py
Lib/test/curses_tests.py
Lib/test/multibytecodec_support.py
Lib/test/test___future__.py
Lib/test/test_bz2.py
Lib/test/test_cmd.py
Lib/test/test_codecencodings_cn.py
Lib/test/test_codecencodings_hk.py
Lib/test/test_codecencodings_iso2022.py
Lib/test/test_codecencodings_jp.py
Lib/test/test_codecencodings_kr.py
Lib/test/test_codecencodings_tw.py
Lib/test/test_codecmaps_cn.py
Lib/test/test_codecmaps_hk.py
Lib/test/test_codecmaps_jp.py
Lib/test/test_codecmaps_kr.py
Lib/test/test_codecmaps_tw.py
Lib/test/test_dbm.py
Lib/test/test_dbm_dumb.py
Lib/test/test_eof.py
Lib/test/test_gzip.py
Lib/test/test_keywordonlyarg.py
Lib/test/test_logging.py
Lib/test/test_marshal.py
Lib/test/test_multibytecodec.py
Lib/test/test_popen.py
Lib/test/test_random.py
Lib/test/test_sched.py
Lib/test/test_smtpnet.py
Lib/test/test_socket.py
Lib/test/test_support.py
Lib/test/test_tcl.py
Lib/test/test_urllib2_localnet.py
Lib/test/test_urllib2net.py
Lib/test/test_urllibnet.py
Lib/test/test_with.py
Lib/test/test_xmlrpc_net.py
Lib/timeit.py
Lib/trace.py
Lib/turtledemo/bytedesign.py
Lib/turtledemo/clock.py
Lib/turtledemo/forest.py
Lib/turtledemo/fractalcurves.py
Lib/turtledemo/lindenmayer.py
Lib/turtledemo/minimal_hanoi.py
Lib/turtledemo/paint.py
Lib/turtledemo/peace.py
Lib/turtledemo/penrose.py
Lib/turtledemo/planet_and_moon.py
Lib/turtledemo/tree.py
Lib/turtledemo/two_canvases.py
Lib/turtledemo/yinyang.py
Lib/webbrowser.py
Mac/Tools/bundlebuilder.py
Mac/Tools/plistlib_generate_testdata.py
Modules/_ctypes/libffi/generate-ios-source-and-headers.py
Modules/_ctypes/libffi/generate-osx-source-and-headers.py
Modules/_decimal/tests/bench.py
Modules/_decimal/tests/deccheck.py
Modules/_decimal/tests/randdec.py
Objects/typeslots.py
Tools/clinic/clinic_test.py
Tools/gdb/libpython.py
Tools/i18n/makelocalealias.py
Tools/pybench/Setup.py
Tools/pybench/clockres.py
Tools/pybench/systimes.py
Tools/scripts/checkpip.py
Tools/ssl/make_ssl_data.py
Tools/test2to3/maintest.py
Tools/unicode/comparecodecs.py
Tools/unittestgui/unittestgui.py

I think most of them should got executable bit.

Exceptions are:

Doc/includes/*.py -- they are just examples, it can be dangerous set
executable bit on files which can be exposed via http server.

Lib/test/test_*.py -- they don't purposed to ran with with default system
Python. Shebangs should be removed.

Lib/test/_test_multiprocessing.py, Lib/test/multibytecodec_support.py -- they
are just auxiliary modules for other tests. Shebangs should be removed.

Lib/http/cookies.py, Lib/mailbox.py, Lib/operator.py, Modules/_decimal/tests/randdec.py -- they don't do anything. Shebangs should be removed.

Tools/test2to3/maintest.py -- this is Python2 script with Python3 shebang.

Lib/difflib.py -- this just runs doctests. Shebang should be removed.

Following files have shebang but have no executable permission bit. Both should be removed.

Lib/test/test_array.py
Lib/test/test_binhex.py
Lib/test/test_errno.py
Lib/test/test_urlparse.py
Lib/test/test_userstring.py

In Tools/unittestgui/unittestgui.py "python" should be changed to "python3" in the shebang.

Any thoughts?

[doko42]

I like this, and I'm doing this in the Debian/Ubuntu packaging anyway. But I would like to see some check script which maybe can be run before a release, that points out regressions or wrong permissions for newly introduced files.

[python-dev]

New changeset 9062a8695c89 by Stefan Krah in branch '3.3':
Issue bpo-19936: Disable shebang lines in order to prevent using a random
http://hg.python.org/cpython/rev/9062a8695c89

[python-dev]

New changeset 92fd0df03823 by Serhiy Storchaka in branch '3.3':
Issue bpo-19936: Added executable bits or shebang lines to Python scripts which
http://hg.python.org/cpython/rev/92fd0df03823

New changeset 83009d70bc9c by Serhiy Storchaka in branch 'default':
Issue bpo-19936: Added executable bits or shebang lines to Python scripts which
http://hg.python.org/cpython/rev/83009d70bc9c

[serhiy-storchaka]

Here is a script. It lists text files with executable bits and without shebangs and files with .py, .pyw, or .sh suffix which have shebang but have not with executable bits.

As far as it searches not only Python files, now it found such files:

• ./python-gdb.py
• ./Doc/includes/email-unpack.py
• ./Doc/includes/email-alternative.py
• ./Doc/includes/email-dir.py
• ./Misc/python-config.sh
  x ./Mac/PythonLauncher/MyDocument.m
  x ./Mac/PythonLauncher/MyDocument.h
  x ./Mac/PythonLauncher/FileSettings.m
  x ./Mac/PythonLauncher/main.m
  x ./Mac/PythonLauncher/FileSettings.h
• ./Tools/test2to3/maintest.py
  x ./Tools/scripts/pydocgui.pyw
  x ./Lib/idlelib/idle.bat
• ./Modules/_ctypes/libffi/build-ios.sh
  x ./Modules/_ctypes/libffi/ltmain.sh
• ./Modules/_ctypes/libffi/msvcc.sh
• ./Modules/_ctypes/libffi/src/arm/gentramp.sh
  x ./Modules/_decimal/tests/deccheck.py
  x ./Modules/_decimal/tests/runall.bat
  x ./Modules/_decimal/tests/bench.py
  x ./PC/msvcrtmodule.c

[skrah]

Somehow Modules/_decimal/tests/bench.py and Modules/_decimal/tests/deccheck.py are now executable, which (I think) they should not be.

[python-dev]

New changeset 085a0ea2015f by Serhiy Storchaka in branch '3.3':
Issue bpo-19936: Remove executable bits from C source files and several forgotten
http://hg.python.org/cpython/rev/085a0ea2015f

New changeset 6baeee39195b by Serhiy Storchaka in branch 'default':
Issue bpo-19936: Remove executable bits from C source files and several forgotten
http://hg.python.org/cpython/rev/6baeee39195b

[python-dev]

New changeset 334116bb2939 by Serhiy Storchaka in branch '2.7':
Issue bpo-19936: Added executable bits or shebang lines to Python scripts which
http://hg.python.org/cpython/rev/334116bb2939

[serhiy-storchaka]

Left files with mismatched executable bits and shebangs:

Python 3.4:

• ./Doc/includes/email-unpack.py
• ./Doc/includes/email-alternative.py
• ./Doc/includes/email-dir.py
  x ./Tools/scripts/pydocgui.pyw
  x ./Lib/idlelib/idle.bat
• ./Modules/_ctypes/libffi/build-ios.sh
  x ./Modules/_ctypes/libffi/ltmain.sh
• ./Modules/_ctypes/libffi/msvcc.sh
• ./Modules/_ctypes/libffi/src/arm/gentramp.sh
  x ./Modules/_decimal/tests/runall.bat

Python 3.3:

• ./Doc/includes/email-unpack.py
• ./Doc/includes/email-alternative.py
• ./Doc/includes/email-dir.py
  x ./Lib/idlelib/idle.bat
• ./Modules/_ctypes/libffi/build-ios.sh
  x ./Modules/_ctypes/libffi/ltmain.sh
• ./Modules/_ctypes/libffi/msvcc.sh
• ./Modules/_ctypes/libffi/src/arm/gentramp.sh
  x ./Modules/_decimal/tests/runall.bat
  x ./PC/VS7.1/rt.bat
  x ./PC/VC6/rt.bat

Python 2.7:

• ./Doc/includes/email-unpack.py
• ./Doc/includes/email-alternative.py
• ./Doc/includes/email-dir.py
  x ./Demo/rpc/test
  x ./Lib/idlelib/idle.bat
  x ./PC/VS7.1/rt.bat
  x ./PC/VC6/rt.bat

[skrah]

IMO the .bat files should have the executable bit as a reminder that they're
executable on Windows.

[serhiy-storchaka]

Gregory, is it intentional that Modules/_ctypes/libffi/build-ios.sh, Modules/_ctypes/libffi/libtool-ldflags, Modules/_ctypes/libffi/msvcc.sh and Modules/_ctypes/libffi/src/arm/gentramp.sh have no executable bits? They all have executable bits in mainstream libffi-3.0.13 distribution.

[gpshead]

For those four libffi files in msg208964, do what you want with them. I ignored executable bits entirely (didn't set or unset) when importing the code as there is zero need to execute any of them.

IMNSHO, I think executable bits should _not_ be set on most files in the Python source tree. *Especially* not on things under Lib and Lib/test/. Afterall, any #! on those is *guaranteed* to not refer to the Python interpreter the source tree represents.