discuss@lists.openscad.org

OpenSCAD general discussion Mailing-list

View all threads

command line and multiple mesh export

R
rmi
Wed, Nov 15, 2017 8:15 AM

Hi,

(sorry for the long introduction, but I think it helps to understand my
point)

I'm creating a plugin for Cura, such that Cura can open .scad files and
selectively (controlled
by carefully crafted comments) export meshes from it. The focus is on
improving the workflow for
parts that make use of multi extrusion. They require multiple meshes, often
produced from the same
.scad file, and so I find myself juggling with * and ! to control what will
be rendered; a
tiresome and error prone process.

The plugin would ease this process as it allows to

  • export multiple meshes
  • track slicer settings per mesh
  • groups several meshes into a single part
  • track changes to the slicer profile

To select which mesh is exported, the plugin writes a small that contains a
single statement and
includes the original .scad file, eg.

!whatever you want to render() #! to make sure that this only this will be
rendered
include <file.scad>                  # use <file.scad> wouldn't set the
global variables

The downside of this approach; it requires several invocations of openscad
which makes it slow as
the CGAL cache isn't preserved. I had several ideas how to solve, or at
least speed it up, and I'm
willing to try to implement it.

(A) cache persistance, eg. give a filename and size limit on the command
line to save the cache
between invocations. This has been discussed a couple of times in the past
and the general consens
sounded like 'would be great'. Forcing the CGAL cache to allocate it's
memroy from a memory mapped
file region could be the way to go. Some care has to be taken on reload, the
cache should be
treated like hostile user input. Integrating it into the UI shouldn't be to
difficult, eg. by
allowing the UI to construct a cache file name based on the loaded file.

(B) specify several source/destination file tuples on the command line and
have them all exported in
one go, sharing the CGAL cache. I've seen discussion about adding an export
statement to the language,
which would create all kind of problems down the road, up to allow arbitrary
files written when files
are downloaded from thingiverse. That's why I'm suggesting a repeatable
command line argument, which
wouldn't have this issue.

(C) 'export' a hash of the normalized object tree, such that the plugin can
decide if a previous output
can be reused or not, basically pushing the caching issue one layer up.

What's your opinion?

--
Sent from: http://forum.openscad.org/

Hi, (sorry for the long introduction, but I think it helps to understand my point) I'm creating a plugin for Cura, such that Cura can open .scad files and selectively (controlled by carefully crafted comments) export meshes from it. The focus is on improving the workflow for parts that make use of multi extrusion. They require multiple meshes, often produced from the same .scad file, and so I find myself juggling with * and ! to control what will be rendered; a tiresome and error prone process. The plugin would ease this process as it allows to - export multiple meshes - track slicer settings per mesh - groups several meshes into a single part - track changes to the slicer profile To select which mesh is exported, the plugin writes a small that contains a single statement and includes the original .scad file, eg. !whatever you want to render() #! to make sure that this only this will be rendered include <file.scad> # use <file.scad> wouldn't set the global variables The downside of this approach; it requires several invocations of openscad which makes it slow as the CGAL cache isn't preserved. I had several ideas how to solve, or at least speed it up, and I'm willing to try to implement it. (A) cache persistance, eg. give a filename and size limit on the command line to save the cache between invocations. This has been discussed a couple of times in the past and the general consens sounded like 'would be great'. Forcing the CGAL cache to allocate it's memroy from a memory mapped file region could be the way to go. Some care has to be taken on reload, the cache should be treated like hostile user input. Integrating it into the UI shouldn't be to difficult, eg. by allowing the UI to construct a cache file name based on the loaded file. (B) specify several source/destination file tuples on the command line and have them all exported in one go, sharing the CGAL cache. I've seen discussion about adding an export statement to the language, which would create all kind of problems down the road, up to allow arbitrary files written when files are downloaded from thingiverse. That's why I'm suggesting a repeatable command line argument, which wouldn't have this issue. (C) 'export' a hash of the normalized object tree, such that the plugin can decide if a previous output can be reused or not, basically pushing the caching issue one layer up. What's your opinion? -- Sent from: http://forum.openscad.org/
JD
Jerry Davis
Wed, Nov 15, 2017 2:00 PM

There would have to be a way to determine if the cache saved is valid or
not, for the given .scad file.

It would seem to me that one way would be to put a timestamp in the cache
that matches the modification timestamp of the .scad file. If the
timestamps differ then the saved cache would be invalid.

--
Extra Ham Operator: K7AZJ
Registered Linux User: 275424
Raspberry Pi and Openscad developer

The most exciting phrase to hear in science - the one that heralds new
discoveries - is not "Eureka!" but "That's funny...".
- Isaac. Asimov

On Wed, Nov 15, 2017 at 1:15 AM, rmi rmie@gmx.de wrote:

Hi,

(sorry for the long introduction, but I think it helps to understand my
point)

I'm creating a plugin for Cura, such that Cura can open .scad files and
selectively (controlled
by carefully crafted comments) export meshes from it. The focus is on
improving the workflow for
parts that make use of multi extrusion. They require multiple meshes, often
produced from the same
.scad file, and so I find myself juggling with * and ! to control what will
be rendered; a
tiresome and error prone process.

The plugin would ease this process as it allows to

  • export multiple meshes
  • track slicer settings per mesh
  • groups several meshes into a single part
  • track changes to the slicer profile

To select which mesh is exported, the plugin writes a small that contains a
single statement and
includes the original .scad file, eg.

!whatever you want to render() #! to make sure that this only this will be
rendered
include <file.scad>                  # use <file.scad> wouldn't set the
global variables

The downside of this approach; it requires several invocations of openscad
which makes it slow as
the CGAL cache isn't preserved. I had several ideas how to solve, or at
least speed it up, and I'm
willing to try to implement it.

(A) cache persistance, eg. give a filename and size limit on the command
line to save the cache
between invocations. This has been discussed a couple of times in the past
and the general consens
sounded like 'would be great'. Forcing the CGAL cache to allocate it's
memroy from a memory mapped
file region could be the way to go. Some care has to be taken on reload,
the
cache should be
treated like hostile user input. Integrating it into the UI shouldn't be to
difficult, eg. by
allowing the UI to construct a cache file name based on the loaded file.

(B) specify several source/destination file tuples on the command line and
have them all exported in
one go, sharing the CGAL cache. I've seen discussion about adding an export
statement to the language,
which would create all kind of problems down the road, up to allow
arbitrary
files written when files
are downloaded from thingiverse. That's why I'm suggesting a repeatable
command line argument, which
wouldn't have this issue.

(C) 'export' a hash of the normalized object tree, such that the plugin can
decide if a previous output
can be reused or not, basically pushing the caching issue one layer up.

What's your opinion?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

There would have to be a way to determine if the cache saved is valid or not, for the given .scad file. It would seem to me that one way would be to put a timestamp in the cache that matches the modification timestamp of the .scad file. If the timestamps differ then the saved cache would be invalid. -- Extra Ham Operator: K7AZJ Registered Linux User: 275424 Raspberry Pi and Openscad developer *The most exciting phrase to hear in science - the one that heralds new discoveries - is not "Eureka!" but "That's funny...".*- Isaac. Asimov On Wed, Nov 15, 2017 at 1:15 AM, rmi <rmie@gmx.de> wrote: > Hi, > > (sorry for the long introduction, but I think it helps to understand my > point) > > I'm creating a plugin for Cura, such that Cura can open .scad files and > selectively (controlled > by carefully crafted comments) export meshes from it. The focus is on > improving the workflow for > parts that make use of multi extrusion. They require multiple meshes, often > produced from the same > .scad file, and so I find myself juggling with * and ! to control what will > be rendered; a > tiresome and error prone process. > > The plugin would ease this process as it allows to > - export multiple meshes > - track slicer settings per mesh > - groups several meshes into a single part > - track changes to the slicer profile > > To select which mesh is exported, the plugin writes a small that contains a > single statement and > includes the original .scad file, eg. > > !whatever you want to render() #! to make sure that this only this will be > rendered > include <file.scad> # use <file.scad> wouldn't set the > global variables > > The downside of this approach; it requires several invocations of openscad > which makes it slow as > the CGAL cache isn't preserved. I had several ideas how to solve, or at > least speed it up, and I'm > willing to try to implement it. > > (A) cache persistance, eg. give a filename and size limit on the command > line to save the cache > between invocations. This has been discussed a couple of times in the past > and the general consens > sounded like 'would be great'. Forcing the CGAL cache to allocate it's > memroy from a memory mapped > file region could be the way to go. Some care has to be taken on reload, > the > cache should be > treated like hostile user input. Integrating it into the UI shouldn't be to > difficult, eg. by > allowing the UI to construct a cache file name based on the loaded file. > > (B) specify several source/destination file tuples on the command line and > have them all exported in > one go, sharing the CGAL cache. I've seen discussion about adding an export > statement to the language, > which would create all kind of problems down the road, up to allow > arbitrary > files written when files > are downloaded from thingiverse. That's why I'm suggesting a repeatable > command line argument, which > wouldn't have this issue. > > (C) 'export' a hash of the normalized object tree, such that the plugin can > decide if a previous output > can be reused or not, basically pushing the caching issue one layer up. > > What's your opinion? > > > > -- > Sent from: http://forum.openscad.org/ > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
RM
Roland Mieslinger
Thu, Nov 16, 2017 9:35 AM

<jdawgaz@gmail.com>:

There would have to be a way to determine if the cache saved is valid or not, for the given .scad file.

Are you sure? Worst case your hit ratio would be 0, but the CGALCache will never return wrong information as the key is the normalized object tree.

JD
Jerry Davis
Thu, Nov 16, 2017 1:06 PM

well, take for example: python.

I believe how it works, is that when a given python script file is run, it
determines if there is a .pyc file for it, and if so, checks to see if it
is up-to-date WRT to the .py file. And if so, uses the .pyc file. If not,
then it re-creates the .pyc file and then runs.

That would be what I am referring to. Otherwise, how could you make sure
your persisted cache is the same as the source it came from?

I know I would want it that way.

--
Extra Ham Operator: K7AZJ
Registered Linux User: 275424
Raspberry Pi and Openscad developer

The most exciting phrase to hear in science - the one that heralds new
discoveries - is not "Eureka!" but "That's funny...".
- Isaac. Asimov

On Thu, Nov 16, 2017 at 2:35 AM, Roland Mieslinger rmie@gmx.de wrote:

jdawgaz@gmail.com:

There would have to be a way to determine if the cache saved is valid or
not, for the given .scad file.

Are you sure? Worst case your hit ratio would be 0, but the CGALCache will
never return wrong information as the key is the normalized object tree.


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

well, take for example: python. I believe how it works, is that when a given python script file is run, it determines if there is a .pyc file for it, and if so, checks to see if it is up-to-date WRT to the .py file. And if so, uses the .pyc file. If not, then it re-creates the .pyc file and then runs. That would be what I am referring to. Otherwise, how could you make sure your persisted cache is the same as the source it came from? I know I would want it that way. -- Extra Ham Operator: K7AZJ Registered Linux User: 275424 Raspberry Pi and Openscad developer *The most exciting phrase to hear in science - the one that heralds new discoveries - is not "Eureka!" but "That's funny...".*- Isaac. Asimov On Thu, Nov 16, 2017 at 2:35 AM, Roland Mieslinger <rmie@gmx.de> wrote: > > <jdawgaz@gmail.com>: > > There would have to be a way to determine if the cache saved is valid or > not, for the given .scad file. > > > Are you sure? Worst case your hit ratio would be 0, but the CGALCache will > never return wrong information as the key is the normalized object tree. > > > > > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org > >
RM
Roland Mieslinger
Fri, Nov 17, 2017 7:04 PM

> well, take for example: python.

>

> I believe how it works, is that when a given python script file is run, it determines if there is

> a .pyc file for it, and if so, checks to see if it is up-to-date WRT to the .py file. And if so, uses

> the .pyc file. If not, then it re-creates the .pyc file and then runs.

well, taking a procederal languag (python) as an example for a functional (OpenSCAD) ... but

let's try. The difference is that python caches code (methods, functions) while the CGAL cache

maps OpenSCAD code to points, edges and surfaces (results). It's the same difference as in

'how to add two numbers' vs. '1 + 1 = 2'.

> That would be what I am referring to. Otherwise, how could you make sure your persisted

> cache is the same as the source it came from?

It doesn't matter. If you create the same model in two places, it is still the same model and

can be used; eg. have a design with multiple files, but each requires the same 20 tooth

bevel gear.

> I know I would want it that way.

Still?

btw. I have a prototype (nothing beats running code) working. Lot more work needed

  • the cache file is hardcoded

  • unused objects are always discarded

  • not even started to touch the GUI

'Lazy' (or deferred) loading seems to be important, as deserializing CGAL_Nef_polyhedron

seems to be expensiv.

first execution, takes 36s

time ../src/openscad/openscad --render -o Extruder.stl Extruder.scad
...

CGAL cache saved 51 entries.

real 0m36.602s
user 0m35.138s
sys 0m0.918s

second execution, takes 3s

time ../src/openscad/openscad --render -o Extruder.stl Extruder.scad
CGAL lazy loaded %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1,:
...
CGAL lazy loaded 52 entries
CGAL lazy cache load: %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1,
CGAL Cache insert: %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1, (3966872 bytes)
CGAL cache saved %multmatrix([[1,0,0,0],[0,1,0,0],[0,0,1,
CGAL cache saved 1 entries.

real 0m2.951s
user 0m2.692s
sys 0m0.188s

--

Roland