The main reason test_pretty_print.py falls over seems to be that the log it
receives has a mixture of latin1 and utf8 encoding so nothing can convert
it to unicode properly.

The log comes from test_cmdline_tool.py and that also falls over in
python2.7 due to this line:
https://github.com/openscad/openscad/blob/master/tests/test_cmdline_tool.py#L245
which adds an ascii string to a unicode string. Under Python3 both strings
are unicode so it will work.

So it looks like since I last used it mods have been made to make it
Python3 compatible. Is it still supposed to work on Python2?

One of the problems is sys.argv[] are ascii strings with latin encoding in
Python2 on Win7. Presumably they are already unicode in Python3 so I think
it would need code condition on the version just to handle the args. And on
Linux Python7 are they utf8?

Should I chase down the Python2 bugs or try to upgrade MSYS2 to Python3?
Having recently upgraded my native Python to 3 this is a whole world of
pain, manly because they changed the string type to unicode and you can no
longer use 8 bit ascii without encoding it.

On 23 August 2018 at 17:31, nop head nop.head@gmail.com wrote:

On 08/25/2018 01:06 PM, nop head wrote:

I don't know if all the build environments that are in use
will support Python3. So Python2 support might still be
needed at this point.

I guess going forward by getting rid of Python2 is the
better option. Making the MSYS2 env work with Python3 would
be a step into that direction.

We don't have any automated builds for MSYS2, so issues
tend to be detected late. Are there any CI providers that
would allow running MSYS2 builds?

ciao,
Torsten.

So I played around with this for a few days and came to the conclusion it
isn't practical to make the same code work on Python2 and Python3 once
unicode with non ascii characters is involved.

The way to do it in Python2 is with 8 bit str strings. The program then
doesn't care about encoding, it just handles 8 bits transparently. The OS
handles which characters the 8 bit codes represent. That all works fine
with Western languages as long we all sing from the same codepage. It
doesn't work well if you want more than one language at the same time or
more than 256 glyphs.

Python3 strings are unicode and it seems to automatically promote the args
from the OS encoding and be able to print unicode, automatically doing the
output encoding to match the output device.

In Python2 the args come in in latin1 and need to be presented to the
subprocess in latin1. This presumable is platform and language dependent.
You can reliably print unicode without encoding it to bytes but if you do
that in Python3 it prints the b" prefix. So the only way to make a program
work on both is with lots of conditional code, which is ugly and not easy
to maintain.

I took the plunge and updated my MSYS2 to Python3 with pacman -S python3.
The only issue with that is it installs it to /usr/bin but Python2 is in
/mingw64/bin and that is higher in the search order. I had to manually nuke
Python2, I don't know if there is a better way to do it.

After upgrading I found some bugs in test_pretty_print.py that I don't
think are anything to do with my platform. I have submitted a PR to fix
that: https://github.com/openscad/openscad/pull/2467.

On 25 August 2018 at 13:55, Torsten Paul Torsten.Paul@gmx.de wrote:

4th paragraph should say "You can't reliably print unicode without
encoding it to bytes"

On 29 August 2018 at 11:58, nop head nop.head@gmail.com wrote: