discuss@lists.openscad.org

OpenSCAD general discussion

View all threads

Static Code Analysis for OpenSCAD

J
julianstirling
Fri, Nov 20, 2020 12:21 AM

Hi Everyone,

I am part of the OpenFlexure project. An OpenSCAD based 3D printed
laboratory grade microscope (openflexure.org).

As part of trying to tidy up the OpenFlexure Microscope code base I have
started to write a linter/static code analyser for OpenSCAD. The project is
written in Python, you can install it with pip install sca2d or clone it
from
https://gitlab.com/bath_open_instrumentation_group/sca2d

SCA2D is only 4 days old so don't expect the world. I have concentrated on
trying to lex and parse the scad properly so I can check for things like
variables being redefined from parent scopes or other such code that can
cause confusing issues. I am yet to write proper unit tests for the code,
but it seems to parse the microscope repository (which is quite complicated)
with no problems. I have defined the OpenSCAD grammar in ENBF format. I did
a bit of a check and couldn't find and ENBF or BFN file for OpenSCAD in the
main repo?

It would be great if others could try it on a range of files and let me know
if it fails to parse valid files or claims invalid files are fine. I would
also be interested to hear the OpenSCAD community's perspectives on good
code style checks to include.

I am planning to follow a pylint ethos where the linting is configurable,
because we all have different needs. I am far from being able to do this yet
as the messaging system for the warnings is pretty ad-hoc. As we are hoping
to move the microscope towards medical certification we will probably opt
for very strict code style which may not be appropriate for all projects.

If you have any thoughts contact me here or open an issue on the SCA2D
GitLab page.

--
Sent from: http://forum.openscad.org/

Hi Everyone, I am part of the OpenFlexure project. An OpenSCAD based 3D printed laboratory grade microscope (openflexure.org). As part of trying to tidy up the OpenFlexure Microscope code base I have started to write a linter/static code analyser for OpenSCAD. The project is written in Python, you can install it with `pip install sca2d` or clone it from https://gitlab.com/bath_open_instrumentation_group/sca2d SCA2D is only 4 days old so don't expect the world. I have concentrated on trying to lex and parse the scad properly so I can check for things like variables being redefined from parent scopes or other such code that can cause confusing issues. I am yet to write proper unit tests for the code, but it seems to parse the microscope repository (which is quite complicated) with no problems. I have defined the OpenSCAD grammar in ENBF format. I did a bit of a check and couldn't find and ENBF or BFN file for OpenSCAD in the main repo? It would be great if others could try it on a range of files and let me know if it fails to parse valid files or claims invalid files are fine. I would also be interested to hear the OpenSCAD community's perspectives on good code style checks to include. I am planning to follow a pylint ethos where the linting is configurable, because we all have different needs. I am far from being able to do this yet as the messaging system for the warnings is pretty ad-hoc. As we are hoping to move the microscope towards medical certification we will probably opt for very strict code style which may not be appropriate for all projects. If you have any thoughts contact me here or open an issue on the SCA2D GitLab page. -- Sent from: http://forum.openscad.org/
L
lar3ry
Fri, Nov 20, 2020 4:06 AM

Sounds great!
Unfortunately, I am pretty much unfamiliar with python, so please bear with
me if I am asking the obvious.
I see a setup.py program. Do I run that to install it on a Windows machine?
If so, will it install the main program and libraries in the appropriate
places?

If so, I may have done something wrong, as I do not see even the sca2d.py
program in any of my python directories. I have three directories:
python37, python37-32, and python38-32

If I have to put the programs/libs in particular places,  where would they
be?
And should I get rid of some of these versions of python?

--
Sent from: http://forum.openscad.org/

Sounds great! Unfortunately, I am pretty much unfamiliar with python, so please bear with me if I am asking the obvious. I see a setup.py program. Do I run that to install it on a Windows machine? If so, will it install the main program and libraries in the appropriate places? If so, I may have done something wrong, as I do not see even the sca2d.py program in any of my python directories. I have three directories: python37, python37-32, and python38-32 If I have to put the programs/libs in particular places, where would they be? And should I get rid of some of these versions of python? -- Sent from: http://forum.openscad.org/
J
julianstirling
Fri, Nov 20, 2020 10:57 AM

Hi lar3ry,

Sorry for the confusion. I am not really much of a windows user so I am not
sure how much of a problem the multiple versions will be. I should have
explained how to install it. You can run from the setup.py but it is much
easier to get the releases from PyPi.

Hopefully one of your pythons is on the PATH. If you open up powershell and
type:
pip install sca2d
Then hopefully it will install sca2d. You should then just be able to run:
sca2d filename.scad
without having put a py file through python. It does all depend on python
being on the PATH. If you have problems then maybe removing the current
versions of python and starting again from the python website, but make sure
during install that you select the check box to add python to the path.

Hope this helps

--
Sent from: http://forum.openscad.org/

Hi lar3ry, Sorry for the confusion. I am not really much of a windows user so I am not sure how much of a problem the multiple versions will be. I should have explained how to install it. You can run from the setup.py but it is much easier to get the releases from PyPi. Hopefully one of your pythons is on the PATH. If you open up powershell and type: pip install sca2d Then hopefully it will install sca2d. You should then just be able to run: sca2d filename.scad without having put a py file through python. It does all depend on python being on the PATH. If you have problems then maybe removing the current versions of python and starting again from the python website, but make sure during install that you select the check box to add python to the path. Hope this helps -- Sent from: http://forum.openscad.org/
TP
Torsten Paul
Fri, Nov 20, 2020 12:26 PM

On 20.11.20 01:21, julianstirling wrote:

I did a bit of a check and couldn't find and ENBF or BFN file
for OpenSCAD in the main repo?

No separate EBNF, but there's a bison parser definition in
https://github.com/openscad/openscad/blob/master/src/parser.y

ciao,
Torsten.

On 20.11.20 01:21, julianstirling wrote: > I did a bit of a check and couldn't find and ENBF or BFN file > for OpenSCAD in the main repo? No separate EBNF, but there's a bison parser definition in https://github.com/openscad/openscad/blob/master/src/parser.y ciao, Torsten.
NH
nop head
Fri, Nov 20, 2020 1:12 PM

I installed it with pip3 and I do have python38-32 in my path but it comes
up with this:

Traceback (most recent call last):
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py",
line 192, in _run_module_as_main
return run_code(code, main_globals, None,
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py",
line 85, in run_code
exec(code, run_globals)
File
"C:\Users\ChrisP\AppData\Local\Programs\Python\Python38-32\Scripts\sca2d.exe_main
.py",
line 7, in <module>
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d_main
.py",
line 29, in main
analyser = Analyser()
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py",
line 41, in init
self._parser = ScadParser()
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py",
line 17, in init
self._parser = self._create_parser()
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py",
line 24, in _create_parser
with open(scad_lark_filename, 'r') as scad_lark_file:
FileNotFoundError: [Errno 2] No such file or directory:
'c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\lark\scad.lark'

On Fri, 20 Nov 2020 at 12:27, Torsten Paul Torsten.Paul@gmx.de wrote:

On 20.11.20 01:21, julianstirling wrote:

I did a bit of a check and couldn't find and ENBF or BFN file
for OpenSCAD in the main repo?

I installed it with pip3 and I do have python38-32 in my path but it comes up with this: Traceback (most recent call last): File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\ChrisP\AppData\Local\Programs\Python\Python38-32\Scripts\sca2d.exe\__main__.py", line 7, in <module> File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\__main__.py", line 29, in main analyser = Analyser() File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py", line 41, in __init__ self._parser = ScadParser() File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py", line 17, in __init__ self._parser = self._create_parser() File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py", line 24, in _create_parser with open(scad_lark_filename, 'r') as scad_lark_file: FileNotFoundError: [Errno 2] No such file or directory: 'c:\\users\\chrisp\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages\\sca2d\\lark\\scad.lark' On Fri, 20 Nov 2020 at 12:27, Torsten Paul <Torsten.Paul@gmx.de> wrote: > On 20.11.20 01:21, julianstirling wrote: > > I did a bit of a check and couldn't find and ENBF or BFN file > > for OpenSCAD in the main repo? > > No separate EBNF, but there's a bison parser definition in > https://github.com/openscad/openscad/blob/master/src/parser.y > > ciao, > Torsten. > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
J
julianstirling
Fri, Nov 20, 2020 1:30 PM

@Nophead Totally my fault, I never tested the wheel, I included the lark file
incorrectly. I have patched it now. Can you try running:
pip install sca2d --upgrade
and then trying again.

@tp3 - I did have a look at that file but it confused me, I will look into
bison. I will have a think about if I want to jump ship from ENBF to bison.
I quite like the ENBF style. So as long as I keep them equivalent it should
be fine.

--
Sent from: http://forum.openscad.org/

@Nophead Totally my fault, I never tested the wheel, I included the lark file incorrectly. I have patched it now. Can you try running: pip install sca2d --upgrade and then trying again. @tp3 - I did have a look at that file but it confused me, I will look into bison. I will have a think about if I want to jump ship from ENBF to bison. I quite like the ENBF style. So as long as I keep them equivalent it should be fine. -- Sent from: http://forum.openscad.org/
NH
nop head
Fri, Nov 20, 2020 4:32 PM

Yes that installs correctly, thanks.

I ran it on my library test file
https://github.com/nophead/NopSCADlib/blob/master/libtest.scad which
includes all of my library but it doesn't like a lot of the syntax and
eventually falls over with a Python error.

Traceback (most recent call last):
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py",
line 192, in _run_module_as_main
return run_code(code, main_globals, None,
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py",
line 85, in run_code
exec(code, run_globals)
File
"C:\Users\ChrisP\AppData\Local\Programs\Python\Python38-32\Scripts\sca2d.exe_main
.py",
line 7, in <module>
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d_main
.py",
line 30, in main
parsed = analyser.analyse_file(args.filename,
output_tree=args.output_tree)
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py",
line 86, in analyse_file
scope.analyse_tree(self._parsed_files)
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py",
line 112, in analyse_tree
self._check_defintions(parsed_files)
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py",
line 132, in _check_defintions
[var_defs, mod_defs, func_defs] =
inc_scope.get_include_defintions(parsed_files,
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py",
line 71, in get_include_defintions
def_var, def_mod, def_func =
inc_scope.get_include_defintions(parsed_files,
File
"c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py",
line 62, in get_include_defintions
if filename in breadcrumbs:
TypeError: argument of type 'NoneType' is not iterable

Valid OpenSCAD syntax it does not like :
each in a list comprehension. It doesn't seem to like list comprehensions
in general, or let. They are departures for C like syntax.

vitamins\screws.scad:0:0: F0001: Cannot read file due to syntax error:

  • No terminal defined for 'e' at line 122 col 35

  • screws = [for(list = screw_lists) each list];

  •                               ^
    
  • Expecting: {'__ANON_2', '__ANON_5', 'MORETHAN', 'QMARK', 'SLASH',
    '__ANON_1',

If you belive this is a bug in SCA2D please report it to us.

Variables can begin with numbers in OpenSCAD.

parsing vitamins\pin_headers.scad
vitamins\pin_headers.scad:0:0: F0001: Cannot read file due to syntax
error:

  • No terminal defined for '2' at line 27 col 1
  • 2p54header    = ["2p54header",  2.54, 1
  • ^
  • Expecting: {'INCLUDE', 'FOR', 'IF', 'LET', 'CALL_NAME', 'VARIABLE',
    'USE', '__ANON_0', 'LBRACE', 'MODULE', 'TERMINATION', 'FUNCTION'}

If you belive this is a bug in SCA2D please report it to us.

Arrays can be indexed by field names .x, .y, ,.z and some more I can't
remember.

vitamins\led_meter.scad:0:0: F0001: Cannot read file due to syntax error:

  • No terminal defined for '.' at line 39 col 49
  • meter_pos(type) = (meter_pcb_size(type).y - meter_size(type).y) *
    meter_offset(
  •                                     ^
    
  • Expecting: {'__ANON_2', 'RPAR', '__ANON_5', 'MORETHAN', 'QMARK',
    'SLASH', '__ANON_1', '__ANON_4', 'PERCENT', 'STAR', 'LSQB', '__ANON_6',
    'MINUS', '__ANON_3', 'LESSTHAN', 'PLUS'}

If you belive this is a bug in SCA2D please report it to us.

Plus many many more syntax errors. You might want to clone NopSCADlib and
test it yourself because there are certainly no syntax errors in it because
you can load that file in OpenSCAD and it generates the cover picture for
the library.

On Fri, 20 Nov 2020 at 13:31, julianstirling julian@julianstirling.co.uk
wrote:

@Nophead Totally my fault, I never tested the wheel, I included the lark
file
incorrectly. I have patched it now. Can you try running:
pip install sca2d --upgrade
and then trying again.

@tp3 - I did have a look at that file but it confused me, I will look into
bison. I will have a think about if I want to jump ship from ENBF to bison.
I quite like the ENBF style. So as long as I keep them equivalent it should
be fine.

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

Yes that installs correctly, thanks. I ran it on my library test file <https://github.com/nophead/NopSCADlib/blob/master/libtest.scad> which includes all of my library but it doesn't like a lot of the syntax and eventually falls over with a Python error. Traceback (most recent call last): File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\ChrisP\AppData\Local\Programs\Python\Python38-32\Scripts\sca2d.exe\__main__.py", line 7, in <module> File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\__main__.py", line 30, in main parsed = analyser.analyse_file(args.filename, output_tree=args.output_tree) File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\sca2d.py", line 86, in analyse_file scope.analyse_tree(self._parsed_files) File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py", line 112, in analyse_tree self._check_defintions(parsed_files) File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py", line 132, in _check_defintions [var_defs, mod_defs, func_defs] = inc_scope.get_include_defintions(parsed_files, File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py", line 71, in get_include_defintions def_var, def_mod, def_func = inc_scope.get_include_defintions(parsed_files, File "c:\users\chrisp\appdata\local\programs\python\python38-32\lib\site-packages\sca2d\outerscope.py", line 62, in get_include_defintions if filename in breadcrumbs: TypeError: argument of type 'NoneType' is not iterable Valid OpenSCAD syntax it does not like : each in a list comprehension. It doesn't seem to like list comprehensions in general, or let. They are departures for C like syntax. `vitamins\screws.scad`:0:0: F0001: Cannot read file due to syntax error: - No terminal defined for 'e' at line 122 col 35 - - screws = [for(list = screw_lists) each list]; - ^ - - Expecting: {'__ANON_2', '__ANON_5', 'MORETHAN', 'QMARK', 'SLASH', '__ANON_1', - If you belive this is a bug in SCA2D please report it to us. Variables can begin with numbers in OpenSCAD. parsing vitamins\pin_headers.scad `vitamins\pin_headers.scad`:0:0: F0001: Cannot read file due to syntax error: - No terminal defined for '2' at line 27 col 1 - - 2p54header = ["2p54header", 2.54, 1 - ^ - - Expecting: {'INCLUDE', 'FOR', 'IF', 'LET', 'CALL_NAME', 'VARIABLE', 'USE', '__ANON_0', 'LBRACE', 'MODULE', 'TERMINATION', 'FUNCTION'} - If you belive this is a bug in SCA2D please report it to us. Arrays can be indexed by field names .x, .y, ,.z and some more I can't remember. `vitamins\led_meter.scad`:0:0: F0001: Cannot read file due to syntax error: - No terminal defined for '.' at line 39 col 49 - - meter_pos(type) = (meter_pcb_size(type).y - meter_size(type).y) * meter_offset( - ^ - - Expecting: {'__ANON_2', 'RPAR', '__ANON_5', 'MORETHAN', 'QMARK', 'SLASH', '__ANON_1', '__ANON_4', 'PERCENT', 'STAR', 'LSQB', '__ANON_6', 'MINUS', '__ANON_3', 'LESSTHAN', 'PLUS'} - If you belive this is a bug in SCA2D please report it to us. Plus many many more syntax errors. You might want to clone NopSCADlib and test it yourself because there are certainly no syntax errors in it because you can load that file in OpenSCAD and it generates the cover picture for the library. On Fri, 20 Nov 2020 at 13:31, julianstirling <julian@julianstirling.co.uk> wrote: > @Nophead Totally my fault, I never tested the wheel, I included the lark > file > incorrectly. I have patched it now. Can you try running: > pip install sca2d --upgrade > and then trying again. > > @tp3 - I did have a look at that file but it confused me, I will look into > bison. I will have a think about if I want to jump ship from ENBF to bison. > I quite like the ENBF style. So as long as I keep them equivalent it should > be fine. > > > > -- > Sent from: http://forum.openscad.org/ > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
L
lar3ry
Fri, Nov 20, 2020 5:46 PM

Python37 is in PATH, and .py is in PATHEXT.
I cd to sca2d-master\sca2d and run
pip install sca2d
I get an error:

Fatal error in launcher: Unable to create process using
'"c:\users\admin\appdata\local\programs\python\python37\python.exe"
"C:\Users\Admin\AppData\Local\Programs\Python\Python37\Scripts\pip.exe"
install sca2d'

But not to worry. I'll wait until someone has successfully installed it on
Windows.

--
Sent from: http://forum.openscad.org/

Python37 is in PATH, and .py is in PATHEXT. I cd to sca2d-master\sca2d and run pip install sca2d I get an error: Fatal error in launcher: Unable to create process using '"c:\users\admin\appdata\local\programs\python\python37\python.exe" "C:\Users\Admin\AppData\Local\Programs\Python\Python37\Scripts\pip.exe" install sca2d' But not to worry. I'll wait until someone has successfully installed it on Windows. -- Sent from: http://forum.openscad.org/
J
julianstirling
Fri, Nov 20, 2020 6:46 PM

@nophead Thanks again.

I had no idea about the .x .y .z indexing, and yes I missed the generators.
I suppose I was biased to the way we write SCAD. I will make sure that I can
get it all running on of NopSCAD. It is a fantastic resource, thank you!

The other error is related to how I am handling imports. I will look into
the problem in my code that caused this error and make sure it is fixed.

@lar3ry Hmm that is a strange message. installing with pip should not need
you to CD anywhere as it will pull the package down from the internet. I
wonder if the problem is with pip on your machine? What happens if you run:
pip install pyyaml
Pyyaml is a pretty well established package, so if that won't work then it
is a python installation issue.

--
Sent from: http://forum.openscad.org/

@nophead Thanks again. I had no idea about the .x .y .z indexing, and yes I missed the generators. I suppose I was biased to the way we write SCAD. I will make sure that I can get it all running on of NopSCAD. It is a fantastic resource, thank you! The other error is related to how I am handling imports. I will look into the problem in my code that caused this error and make sure it is fixed. @lar3ry Hmm that is a strange message. installing with pip should not need you to CD anywhere as it will pull the package down from the internet. I wonder if the problem is with pip on your machine? What happens if you run: pip install pyyaml Pyyaml is a pretty well established package, so if that won't work then it is a python installation issue. -- Sent from: http://forum.openscad.org/
L
lar3ry
Fri, Nov 20, 2020 7:11 PM

Please reread my previous, which I have edited.
It should not have posted.

--
Sent from: http://forum.openscad.org/

Please reread my previous, which I have edited. It should not have posted. -- Sent from: http://forum.openscad.org/