Accessing a tkinter object's string representation converts the object to a string on Windows

[daniilS]

Introduced in #16545
Affects Python 3.7+ on Windows only

---

The FromObj function in _tkinter.c attempt to convert a Tcl_Obj to an equivalent Python object if possible, and otherwise returns a _tkinter.Tcl_Obj with the typename attribute set to the original object's type.

However, on Windows, accessing the resulting object's string representation calls Tcl_GetUnicodeFromObj, which converts the Tcl_Obj to a String. This side effect isn't mentioned in the Tcl documentation, but is in the Tcl source code. As a result, retrieving the same tk property afterwards will return a Python string instead.

Minimal example:

import tkinter as tk

root = tk.Tk()
print(type(root.cget("padx")))
_ = str(root.cget("padx")) # should really not cause any side effects
print(type(root.cget("padx")))

# Windows:
# <class '_tkinter.Tcl_Obj'>
# <class 'str'>
# Other platforms:
# <class '_tkinter.Tcl_Obj'>
# <class '_tkinter.Tcl_Obj'>

---

Possible solutions: unicodeFromTclObj should copy the object before passing calling Tcl_GetUnicodeFromObj, or handle Unicode without it like on other platforms.

Linked PRs

• gh-101830: Fix Tcl_Obj to string conversion #120884
• [3.13] gh-101830: Fix Tcl_Obj to string conversion (GH-120884) #120905
• [3.12] gh-101830: Fix Tcl_Obj to string conversion (GH-120884) #120913

[chrstphrchvz]

To offer some background: Tcl usually calls this side effect “shimmering”. The Tcl syntax documentation does not seem to explain it, but the C API documentation gives some hints: https://www.tcl-lang.org/man/tcl8.6/TclLib/Object.htm explains that a Tcl value has a “string representation” (modified UTF-8 bytes) and up to one “internal representation” depending on its type; and in https://www.tcl-lang.org/man/tcl8.6/TclLib/StringObj.htm there is a distinction between a “Unicode representation” (a sequence of 16-bit or 32-bit codepoints) which is the internal representation of a value with the “string” type, versus the “string representation” for values of any type. So Tcl_GetUnicodeFromObj() can cause a value to generate a string representation (if it does not already have one) from its current internal representation, and then become the “string” type by discarding the existing internal representation and replacing it with a Unicode representation generated from the string representation.

[serhiy-storchaka]

Thank you, I overlook this effect.

The reason of using Tcl_GetUnicodeFromObj/Tcl_NewUnicodeObj on Windows and Tcl_GetStringFromObj/Tcl_NewStringObj on other platforms is that by default Tcl does not support Unicode outside of the BMP, but the underlying graphic system may support if the data is present in native multibyte encoding (UTF16 on Windows, UTF8 on other platforms). Tcl cannot correctly convert between UTF16 and UTF8 representations of non-BMP characters, so we need to create Tcl objects in the appropriate representation.

Using Tcl_GetUnicodeFromObj only for Tcl objects with the “Unicode representation” should fix most issues. It will create a new issue: if the Tcl object is compound and contains strings with non-BMP characters, Tcl_GetStringFromObj will produce result containing broken UTF8. This is uncommon case, because Tcl "list" objects are represented as tuples in Python, but Tcl "dict" objects are represented as Tcl_Objs. I do not know whether Tcl "dict" objects are used in Tk. Copying the object before passing calling Tcl_GetUnicodeFromObj could solve this issue, but this is very complex solution.

I think that we should fix regression in common cases and leave possible complicated solution of more subtle problem until we encounter it in real world.