discuss@lists.openscad.org

OpenSCAD general discussion Mailing-list

View all threads

Module caching

HL
Hans L
Mon, Jan 28, 2019 10:29 AM

Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
key storing got reworked significantly in the past year.

With the example you gave I get 7s preview and "Geometries in cache: 1". I
see that yours is showing 395 geometries so either your posted example
simplified away the cause of the issue, or something different between with
the build you are running.

On Mon, Jan 28, 2019 at 2:59 AM nop head nop.head@gmail.com wrote:

Here is a boiled down version:
https://gist.github.com/nophead/fd4ca58b5f6b19dc25bb98f7edbae90f

I captured the points and face lists generated by sweep and hard coded
them to remove all of my library. Each spring is 4576 points, 9122 faces
but they are all the same polyhedron repeated 100 times. Given that it
doesn't cache polyhedra it has to process the same lists to a Polyset 100
times. Taking about 1/4 second to process 9000 faces seems very slow to me
on a 64 bit 2.5GHz core I7 with 8 GB memory.

On Mon, 28 Jan 2019 at 02:23, Hans L thehans@gmail.com wrote:

Hi nophead,

Could you post a sample script for others to reproduce the slowness you
experience?

Thanks,
Hans

On Sun, Jan 27, 2019 at 4:59 PM nop head nop.head@gmail.com wrote:

It spends most of the 22 seconds stuck at Compiling design (CSG Tree
generation)...

After that it does all the rest in about 1 second, including a very fast
progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds

It would be a problem if I was designing a mattress with PLA springs but
fortunately only a battery box with four, but it would be nice if
polyhedrons were cached like other geometry seems to be. I clearly don't
fully understand OpenSCADs caching system.

On Sun, 27 Jan 2019 at 22:33, Parkinbot rudolf@digitaldocument.de
wrote:

nophead wrote

Is it perhaps that the lists are stored as strings when the

polyhedron is

instantiated and then parsed to vectors every time the geometry is
created.

I doubt that. But you are right, the code really looks like nothing very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after
the
geometry has been created?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

Are you using a somewhat recent dev snapshot? The CSG Tree generation and key storing got reworked significantly in the past year. With the example you gave I get 7s preview and "Geometries in cache: 1". I see that yours is showing 395 geometries so either your posted example simplified away the cause of the issue, or something different between with the build you are running. On Mon, Jan 28, 2019 at 2:59 AM nop head <nop.head@gmail.com> wrote: > Here is a boiled down version: > https://gist.github.com/nophead/fd4ca58b5f6b19dc25bb98f7edbae90f > > I captured the points and face lists generated by sweep and hard coded > them to remove all of my library. Each spring is 4576 points, 9122 faces > but they are all the same polyhedron repeated 100 times. Given that it > doesn't cache polyhedra it has to process the same lists to a Polyset 100 > times. Taking about 1/4 second to process 9000 faces seems very slow to me > on a 64 bit 2.5GHz core I7 with 8 GB memory. > > On Mon, 28 Jan 2019 at 02:23, Hans L <thehans@gmail.com> wrote: > >> Hi nophead, >> >> Could you post a sample script for others to reproduce the slowness you >> experience? >> >> Thanks, >> Hans >> >> On Sun, Jan 27, 2019 at 4:59 PM nop head <nop.head@gmail.com> wrote: >> >>> It spends most of the 22 seconds stuck at Compiling design (CSG Tree >>> generation)... >>> >>> After that it does all the rest in about 1 second, including a very fast >>> progress bar. >>> >>> Compiling design (CSG Products generation)... >>> >>> Geometries in cache: 395 >>> >>> Geometry cache size in bytes: 20331104 >>> >>> CGAL Polyhedrons in cache: 0 >>> >>> CGAL cache size in bytes: 0 >>> >>> Compiling design (CSG Products normalization)... >>> >>> Normalized CSG tree has 100 elements >>> >>> Compile and preview finished. >>> >>> Total rendering time: 0 hours, 0 minutes, 22 seconds >>> >>> >>> It would be a problem if I was designing a mattress with PLA springs but >>> fortunately only a battery box with four, but it would be nice if >>> polyhedrons were cached like other geometry seems to be. I clearly don't >>> fully understand OpenSCADs caching system. >>> >>> On Sun, 27 Jan 2019 at 22:33, Parkinbot <rudolf@digitaldocument.de> >>> wrote: >>> >>>> nophead wrote >>>> > Is it perhaps that the lists are stored as strings when the >>>> polyhedron is >>>> > instantiated and then parsed to vectors every time the geometry is >>>> > created. >>>> >>>> I doubt that. But you are right, the code really looks like nothing very >>>> time consuming is happening. >>>> Is the triangulation of the faces done by Open GL? What happens after >>>> the >>>> geometry has been created? >>>> >>>> >>>> >>>> -- >>>> Sent from: http://forum.openscad.org/ >>>> >>>> _______________________________________________ >>>> OpenSCAD mailing list >>>> Discuss@lists.openscad.org >>>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>>> >>> _______________________________________________ >>> OpenSCAD mailing list >>> Discuss@lists.openscad.org >>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>> >> _______________________________________________ >> OpenSCAD mailing list >> Discuss@lists.openscad.org >> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >> > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
NH
nop head
Mon, Jan 28, 2019 10:52 AM

I am using a snapshot from yesterday.

I only get one geometry if that is the only file loaded. I had more MDI
windows open when I posted yesterday.

You might have a machine that is three times faster than this laptop but
even so 7 seconds seems slow to me. It has to read about 1 million faces
from a list and convert them into Polysets which are just vectors of
vectors of vertices. Perhaps I have just lost touch with how inefficient
modern software is.

OpenSCAD 2019.01.27.ci1306
https://www.openscad.org/

Copyright (C) 2009-2019 The OpenSCAD Developers

This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option)
any later version.

Loaded design 'C:/Users/ChrisP/Documents/springs.csg'.

Compiling design (CSG Tree generation)...

Compiling design (CSG Products generation)...

Geometries in cache: 1

Geometry cache size in bytes: 657560

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 21 seconds

On Mon, 28 Jan 2019 at 10:31, Hans L thehans@gmail.com wrote:

Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
key storing got reworked significantly in the past year.

With the example you gave I get 7s preview and "Geometries in cache: 1".
I see that yours is showing 395 geometries so either your posted example
simplified away the cause of the issue, or something different between with
the build you are running.

On Mon, Jan 28, 2019 at 2:59 AM nop head nop.head@gmail.com wrote:

Here is a boiled down version:
https://gist.github.com/nophead/fd4ca58b5f6b19dc25bb98f7edbae90f

I captured the points and face lists generated by sweep and hard coded
them to remove all of my library. Each spring is 4576 points, 9122 faces
but they are all the same polyhedron repeated 100 times. Given that it
doesn't cache polyhedra it has to process the same lists to a Polyset 100
times. Taking about 1/4 second to process 9000 faces seems very slow to me
on a 64 bit 2.5GHz core I7 with 8 GB memory.

On Mon, 28 Jan 2019 at 02:23, Hans L thehans@gmail.com wrote:

Hi nophead,

Could you post a sample script for others to reproduce the slowness you
experience?

Thanks,
Hans

On Sun, Jan 27, 2019 at 4:59 PM nop head nop.head@gmail.com wrote:

It spends most of the 22 seconds stuck at Compiling design (CSG Tree
generation)...

After that it does all the rest in about 1 second, including a very
fast progress bar.

Compiling design (CSG Products generation)...

Geometries in cache: 395

Geometry cache size in bytes: 20331104

CGAL Polyhedrons in cache: 0

CGAL cache size in bytes: 0

Compiling design (CSG Products normalization)...

Normalized CSG tree has 100 elements

Compile and preview finished.

Total rendering time: 0 hours, 0 minutes, 22 seconds

It would be a problem if I was designing a mattress with PLA springs
but fortunately only a battery box with four, but it would be nice if
polyhedrons were cached like other geometry seems to be. I clearly don't
fully understand OpenSCADs caching system.

On Sun, 27 Jan 2019 at 22:33, Parkinbot rudolf@digitaldocument.de
wrote:

nophead wrote

Is it perhaps that the lists are stored as strings when the

polyhedron is

instantiated and then parsed to vectors every time the geometry is
created.

I doubt that. But you are right, the code really looks like nothing
very
time consuming is happening.
Is the triangulation of the faces done by Open GL? What happens after
the
geometry has been created?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

I am using a snapshot from yesterday. I only get one geometry if that is the only file loaded. I had more MDI windows open when I posted yesterday. You might have a machine that is three times faster than this laptop but even so 7 seconds seems slow to me. It has to read about 1 million faces from a list and convert them into Polysets which are just vectors of vectors of vertices. Perhaps I have just lost touch with how inefficient modern software is. OpenSCAD 2019.01.27.ci1306 https://www.openscad.org/ Copyright (C) 2009-2019 The OpenSCAD Developers This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. Loaded design 'C:/Users/ChrisP/Documents/springs.csg'. Compiling design (CSG Tree generation)... Compiling design (CSG Products generation)... Geometries in cache: 1 Geometry cache size in bytes: 657560 CGAL Polyhedrons in cache: 0 CGAL cache size in bytes: 0 Compiling design (CSG Products normalization)... Normalized CSG tree has 100 elements Compile and preview finished. Total rendering time: 0 hours, 0 minutes, 21 seconds On Mon, 28 Jan 2019 at 10:31, Hans L <thehans@gmail.com> wrote: > Are you using a somewhat recent dev snapshot? The CSG Tree generation and > key storing got reworked significantly in the past year. > > With the example you gave I get 7s preview and "Geometries in cache: 1". > I see that yours is showing 395 geometries so either your posted example > simplified away the cause of the issue, or something different between with > the build you are running. > > On Mon, Jan 28, 2019 at 2:59 AM nop head <nop.head@gmail.com> wrote: > >> Here is a boiled down version: >> https://gist.github.com/nophead/fd4ca58b5f6b19dc25bb98f7edbae90f >> >> I captured the points and face lists generated by sweep and hard coded >> them to remove all of my library. Each spring is 4576 points, 9122 faces >> but they are all the same polyhedron repeated 100 times. Given that it >> doesn't cache polyhedra it has to process the same lists to a Polyset 100 >> times. Taking about 1/4 second to process 9000 faces seems very slow to me >> on a 64 bit 2.5GHz core I7 with 8 GB memory. >> >> On Mon, 28 Jan 2019 at 02:23, Hans L <thehans@gmail.com> wrote: >> >>> Hi nophead, >>> >>> Could you post a sample script for others to reproduce the slowness you >>> experience? >>> >>> Thanks, >>> Hans >>> >>> On Sun, Jan 27, 2019 at 4:59 PM nop head <nop.head@gmail.com> wrote: >>> >>>> It spends most of the 22 seconds stuck at Compiling design (CSG Tree >>>> generation)... >>>> >>>> After that it does all the rest in about 1 second, including a very >>>> fast progress bar. >>>> >>>> Compiling design (CSG Products generation)... >>>> >>>> Geometries in cache: 395 >>>> >>>> Geometry cache size in bytes: 20331104 >>>> >>>> CGAL Polyhedrons in cache: 0 >>>> >>>> CGAL cache size in bytes: 0 >>>> >>>> Compiling design (CSG Products normalization)... >>>> >>>> Normalized CSG tree has 100 elements >>>> >>>> Compile and preview finished. >>>> >>>> Total rendering time: 0 hours, 0 minutes, 22 seconds >>>> >>>> >>>> It would be a problem if I was designing a mattress with PLA springs >>>> but fortunately only a battery box with four, but it would be nice if >>>> polyhedrons were cached like other geometry seems to be. I clearly don't >>>> fully understand OpenSCADs caching system. >>>> >>>> On Sun, 27 Jan 2019 at 22:33, Parkinbot <rudolf@digitaldocument.de> >>>> wrote: >>>> >>>>> nophead wrote >>>>> > Is it perhaps that the lists are stored as strings when the >>>>> polyhedron is >>>>> > instantiated and then parsed to vectors every time the geometry is >>>>> > created. >>>>> >>>>> I doubt that. But you are right, the code really looks like nothing >>>>> very >>>>> time consuming is happening. >>>>> Is the triangulation of the faces done by Open GL? What happens after >>>>> the >>>>> geometry has been created? >>>>> >>>>> >>>>> >>>>> -- >>>>> Sent from: http://forum.openscad.org/ >>>>> >>>>> _______________________________________________ >>>>> OpenSCAD mailing list >>>>> Discuss@lists.openscad.org >>>>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>>>> >>>> _______________________________________________ >>>> OpenSCAD mailing list >>>> Discuss@lists.openscad.org >>>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>>> >>> _______________________________________________ >>> OpenSCAD mailing list >>> Discuss@lists.openscad.org >>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>> >> _______________________________________________ >> OpenSCAD mailing list >> Discuss@lists.openscad.org >> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >> > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
P
Parkinbot
Mon, Jan 28, 2019 2:38 PM

thehans wrote

Are you using a somewhat recent dev snapshot?  The CSG Tree generation and
key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was some progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too much
effort to set it up), I don't know the code well enough to be able to follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?

--
Sent from: http://forum.openscad.org/

thehans wrote > Are you using a somewhat recent dev snapshot? The CSG Tree generation and > key storing got reworked significantly in the past year. I tried some older snaps with 100 copies of a sweep object, each containing 19002 facets and no buffering of the points and faces: one from 2016 (1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019 (1':00"). So there was *some* progress. With buffering of points and faces the 2019 version needed 0'42". As compiling an own fork of OpenSCAD is not recommended on Windows (too much effort to set it up), I don't know the code well enough to be able to follow what is happening in an F5 context. But I am curious. Can anybody provide a sketch of how the execution goes on after createGeometry()? -- Sent from: http://forum.openscad.org/
NH
nop head
Mon, Jan 28, 2019 4:59 PM

I think the majority of the time is taken in createGeometry, not after it
it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only
takes 1 second, presumably because it is cached, or it is 20 times quicker
to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot rudolf@digitaldocument.de wrote:

thehans wrote

Are you using a somewhat recent dev snapshot?  The CSG Tree generation

and

key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was some progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too
much
effort to set it up), I don't know the code well enough to be able to
follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

I think the majority of the time is taken in createGeometry, not after it it. 100 spheres with $fn set to 150, which gives 11250 vertices each, only takes 1 second, presumably because it is cached, or it is 20 times quicker to calculate vertices with trig than it is to read them from a list. On Mon, 28 Jan 2019 at 14:40, Parkinbot <rudolf@digitaldocument.de> wrote: > thehans wrote > > Are you using a somewhat recent dev snapshot? The CSG Tree generation > and > > key storing got reworked significantly in the past year. > > I tried some older snaps with 100 copies of a sweep object, each containing > 19002 facets and no buffering of the points and faces: one from 2016 > (1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019 > (1':00"). So there was *some* progress. > With buffering of points and faces the 2019 version needed 0'42". > > As compiling an own fork of OpenSCAD is not recommended on Windows (too > much > effort to set it up), I don't know the code well enough to be able to > follow > what is happening in an F5 context. But I am curious. > > Can anybody provide a sketch of how the execution goes on after > createGeometry()? > > > > -- > Sent from: http://forum.openscad.org/ > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
HL
Hans L
Mon, Jan 28, 2019 6:06 PM

Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, I
wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference. I
wonder if Windows snapshots are not getting the same -O2 compiler
optimizations?

But anyways, the main thing about caching is that it doesn't simply cache a
call to a module.  OpenSCAD first converts the entire script into csg
format which is just a static representation of primitives, multmatrix
transformations and boolean operations, so it has to evaluate all lists
comprehensions, control structures etc, creating scope for each node etc;
essentially building up the entire output that you see in .csg export, in
memory (about 31.6MB of text for nopheads example, although in memory it
should be a bit less due to whitespace/indentation stripping).  Then it
caches Polysets for matching subtrees (substrings basically of any node and
its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example,
that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head nop.head@gmail.com wrote:

I think the majority of the time is taken in createGeometry, not after it
it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only
takes 1 second, presumably because it is cached, or it is 20 times quicker
to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot rudolf@digitaldocument.de wrote:

thehans wrote

Are you using a somewhat recent dev snapshot?  The CSG Tree generation

and

key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each
containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was some progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too
much
effort to set it up), I don't know the code well enough to be able to
follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, I wouldn't expect it to be 3x faster than a 2.5GHz core i7.. I am testing on Linux, not Windows, so maybe that is the main difference. I wonder if Windows snapshots are not getting the same -O2 compiler optimizations? But anyways, the main thing about caching is that it doesn't simply cache a *call* to a module. OpenSCAD first converts the entire script into csg format which is just a static representation of primitives, multmatrix transformations and boolean operations, so it has to evaluate all lists comprehensions, control structures etc, creating scope for each node etc; essentially building up the entire output that you see in .csg export, in memory (about 31.6MB of text for nopheads example, although in memory it should be a bit less due to whitespace/indentation stripping). *Then* it caches Polysets for matching subtrees (substrings basically of any node and its children) from this .csg representation. Compare the 31.6MB to the size of the .csg output of your sphere example, that's gonna be like a couple kb of text processing. On Mon, Jan 28, 2019 at 11:01 AM nop head <nop.head@gmail.com> wrote: > I think the majority of the time is taken in createGeometry, not after it > it. > > 100 spheres with $fn set to 150, which gives 11250 vertices each, only > takes 1 second, presumably because it is cached, or it is 20 times quicker > to calculate vertices with trig than it is to read them from a list. > > On Mon, 28 Jan 2019 at 14:40, Parkinbot <rudolf@digitaldocument.de> wrote: > >> thehans wrote >> > Are you using a somewhat recent dev snapshot? The CSG Tree generation >> and >> > key storing got reworked significantly in the past year. >> >> I tried some older snaps with 100 copies of a sweep object, each >> containing >> 19002 facets and no buffering of the points and faces: one from 2016 >> (1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019 >> (1':00"). So there was *some* progress. >> With buffering of points and faces the 2019 version needed 0'42". >> >> As compiling an own fork of OpenSCAD is not recommended on Windows (too >> much >> effort to set it up), I don't know the code well enough to be able to >> follow >> what is happening in an F5 context. But I am curious. >> >> Can anybody provide a sketch of how the execution goes on after >> createGeometry()? >> >> >> >> -- >> Sent from: http://forum.openscad.org/ >> >> _______________________________________________ >> OpenSCAD mailing list >> Discuss@lists.openscad.org >> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >> > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
NH
nop head
Mon, Jan 28, 2019 6:40 PM

It is only 31.6MB in the csg because the lists are text and are repeated
over and over again. But at runtime there should only be one copy of the
lists in binary. Does it really generate a textual csg representation in
memory or is that just when you dump it? And does it actually make 100
copies of the lists by value? That might explain why it is so slow.

On Mon, 28 Jan 2019 at 18:08, Hans L thehans@gmail.com wrote:

Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz,
I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference.
I wonder if Windows snapshots are not getting the same -O2 compiler
optimizations?

But anyways, the main thing about caching is that it doesn't simply cache
a call to a module.  OpenSCAD first converts the entire script into csg
format which is just a static representation of primitives, multmatrix
transformations and boolean operations, so it has to evaluate all lists
comprehensions, control structures etc, creating scope for each node etc;
essentially building up the entire output that you see in .csg export, in
memory (about 31.6MB of text for nopheads example, although in memory it
should be a bit less due to whitespace/indentation stripping).  Then it
caches Polysets for matching subtrees (substrings basically of any node and
its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example,
that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head nop.head@gmail.com wrote:

I think the majority of the time is taken in createGeometry, not after it
it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only
takes 1 second, presumably because it is cached, or it is 20 times quicker
to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot rudolf@digitaldocument.de
wrote:

thehans wrote

Are you using a somewhat recent dev snapshot?  The CSG Tree generation

and

key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each
containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was some progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too
much
effort to set it up), I don't know the code well enough to be able to
follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

It is only 31.6MB in the csg because the lists are text and are repeated over and over again. But at runtime there should only be one copy of the lists in binary. Does it really generate a textual csg representation in memory or is that just when you dump it? And does it actually make 100 copies of the lists by value? That might explain why it is so slow. On Mon, 28 Jan 2019 at 18:08, Hans L <thehans@gmail.com> wrote: > Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, > I wouldn't expect it to be 3x faster than a 2.5GHz core i7.. > > I am testing on Linux, not Windows, so maybe that is the main difference. > I wonder if Windows snapshots are not getting the same -O2 compiler > optimizations? > > But anyways, the main thing about caching is that it doesn't simply cache > a *call* to a module. OpenSCAD first converts the entire script into csg > format which is just a static representation of primitives, multmatrix > transformations and boolean operations, so it has to evaluate all lists > comprehensions, control structures etc, creating scope for each node etc; > essentially building up the entire output that you see in .csg export, in > memory (about 31.6MB of text for nopheads example, although in memory it > should be a bit less due to whitespace/indentation stripping). *Then* it > caches Polysets for matching subtrees (substrings basically of any node and > its children) from this .csg representation. > > Compare the 31.6MB to the size of the .csg output of your sphere example, > that's gonna be like a couple kb of text processing. > > On Mon, Jan 28, 2019 at 11:01 AM nop head <nop.head@gmail.com> wrote: > >> I think the majority of the time is taken in createGeometry, not after it >> it. >> >> 100 spheres with $fn set to 150, which gives 11250 vertices each, only >> takes 1 second, presumably because it is cached, or it is 20 times quicker >> to calculate vertices with trig than it is to read them from a list. >> >> On Mon, 28 Jan 2019 at 14:40, Parkinbot <rudolf@digitaldocument.de> >> wrote: >> >>> thehans wrote >>> > Are you using a somewhat recent dev snapshot? The CSG Tree generation >>> and >>> > key storing got reworked significantly in the past year. >>> >>> I tried some older snaps with 100 copies of a sweep object, each >>> containing >>> 19002 facets and no buffering of the points and faces: one from 2016 >>> (1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019 >>> (1':00"). So there was *some* progress. >>> With buffering of points and faces the 2019 version needed 0'42". >>> >>> As compiling an own fork of OpenSCAD is not recommended on Windows (too >>> much >>> effort to set it up), I don't know the code well enough to be able to >>> follow >>> what is happening in an F5 context. But I am curious. >>> >>> Can anybody provide a sketch of how the execution goes on after >>> createGeometry()? >>> >>> >>> >>> -- >>> Sent from: http://forum.openscad.org/ >>> >>> _______________________________________________ >>> OpenSCAD mailing list >>> Discuss@lists.openscad.org >>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>> >> _______________________________________________ >> OpenSCAD mailing list >> Discuss@lists.openscad.org >> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >> > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
HL
Hans L
Mon, Jan 28, 2019 6:47 PM

Yes it really does.  The csg text representation of a node is its cache
key.  And the csg format doesn't really do individual variable (list)
caching.

On Mon, Jan 28, 2019 at 12:41 PM nop head nop.head@gmail.com wrote:

It is only 31.6MB in the csg because the lists are text and are repeated
over and over again. But at runtime there should only be one copy of the
lists in binary. Does it really generate a textual csg representation in
memory or is that just when you dump it? And does it actually make 100
copies of the lists by value? That might explain why it is so slow.

On Mon, 28 Jan 2019 at 18:08, Hans L thehans@gmail.com wrote:

Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz,
I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main difference.
I wonder if Windows snapshots are not getting the same -O2 compiler
optimizations?

But anyways, the main thing about caching is that it doesn't simply cache
a call to a module.  OpenSCAD first converts the entire script into csg
format which is just a static representation of primitives, multmatrix
transformations and boolean operations, so it has to evaluate all lists
comprehensions, control structures etc, creating scope for each node etc;
essentially building up the entire output that you see in .csg export, in
memory (about 31.6MB of text for nopheads example, although in memory it
should be a bit less due to whitespace/indentation stripping).  Then it
caches Polysets for matching subtrees (substrings basically of any node and
its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere example,
that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head nop.head@gmail.com wrote:

I think the majority of the time is taken in createGeometry, not after
it it.

100 spheres with $fn set to 150, which gives  11250 vertices each, only
takes 1 second, presumably because it is cached, or it is 20 times quicker
to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot rudolf@digitaldocument.de
wrote:

thehans wrote

Are you using a somewhat recent dev snapshot?  The CSG Tree

generation and

key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each
containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was some progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows (too
much
effort to set it up), I don't know the code well enough to be able to
follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

Yes it really does. The csg text representation of a node *is* its cache key. And the csg format doesn't really do individual variable (list) caching. On Mon, Jan 28, 2019 at 12:41 PM nop head <nop.head@gmail.com> wrote: > It is only 31.6MB in the csg because the lists are text and are repeated > over and over again. But at runtime there should only be one copy of the > lists in binary. Does it really generate a textual csg representation in > memory or is that just when you dump it? And does it actually make 100 > copies of the lists by value? That might explain why it is so slow. > > On Mon, 28 Jan 2019 at 18:08, Hans L <thehans@gmail.com> wrote: > >> Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, >> I wouldn't expect it to be 3x faster than a 2.5GHz core i7.. >> >> I am testing on Linux, not Windows, so maybe that is the main difference. >> I wonder if Windows snapshots are not getting the same -O2 compiler >> optimizations? >> >> But anyways, the main thing about caching is that it doesn't simply cache >> a *call* to a module. OpenSCAD first converts the entire script into csg >> format which is just a static representation of primitives, multmatrix >> transformations and boolean operations, so it has to evaluate all lists >> comprehensions, control structures etc, creating scope for each node etc; >> essentially building up the entire output that you see in .csg export, in >> memory (about 31.6MB of text for nopheads example, although in memory it >> should be a bit less due to whitespace/indentation stripping). *Then* it >> caches Polysets for matching subtrees (substrings basically of any node and >> its children) from this .csg representation. >> >> Compare the 31.6MB to the size of the .csg output of your sphere example, >> that's gonna be like a couple kb of text processing. >> >> On Mon, Jan 28, 2019 at 11:01 AM nop head <nop.head@gmail.com> wrote: >> >>> I think the majority of the time is taken in createGeometry, not after >>> it it. >>> >>> 100 spheres with $fn set to 150, which gives 11250 vertices each, only >>> takes 1 second, presumably because it is cached, or it is 20 times quicker >>> to calculate vertices with trig than it is to read them from a list. >>> >>> On Mon, 28 Jan 2019 at 14:40, Parkinbot <rudolf@digitaldocument.de> >>> wrote: >>> >>>> thehans wrote >>>> > Are you using a somewhat recent dev snapshot? The CSG Tree >>>> generation and >>>> > key storing got reworked significantly in the past year. >>>> >>>> I tried some older snaps with 100 copies of a sweep object, each >>>> containing >>>> 19002 facets and no buffering of the points and faces: one from 2016 >>>> (1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019 >>>> (1':00"). So there was *some* progress. >>>> With buffering of points and faces the 2019 version needed 0'42". >>>> >>>> As compiling an own fork of OpenSCAD is not recommended on Windows (too >>>> much >>>> effort to set it up), I don't know the code well enough to be able to >>>> follow >>>> what is happening in an F5 context. But I am curious. >>>> >>>> Can anybody provide a sketch of how the execution goes on after >>>> createGeometry()? >>>> >>>> >>>> >>>> -- >>>> Sent from: http://forum.openscad.org/ >>>> >>>> _______________________________________________ >>>> OpenSCAD mailing list >>>> Discuss@lists.openscad.org >>>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>>> >>> _______________________________________________ >>> OpenSCAD mailing list >>> Discuss@lists.openscad.org >>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>> >> _______________________________________________ >> OpenSCAD mailing list >> Discuss@lists.openscad.org >> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >> > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
HL
Hans L
Mon, Jan 28, 2019 8:17 PM

I did some profiling and found that its spending much of its time in
PolysetUtils::tessellate_faces, because its re-tessellating each geometry
after pulling from geometry cache.  Ideally it should tesselate only once
before putting into the cache.  Pull request incoming. :)

On Mon, Jan 28, 2019 at 12:47 PM Hans L thehans@gmail.com wrote:

Yes it really does.  The csg text representation of a node is its cache
key.  And the csg format doesn't really do individual variable (list)
caching.

On Mon, Jan 28, 2019 at 12:41 PM nop head nop.head@gmail.com wrote:

It is only 31.6MB in the csg because the lists are text and are repeated
over and over again. But at runtime there should only be one copy of the
lists in binary. Does it really generate a textual csg representation in
memory or is that just when you dump it? And does it actually make 100
copies of the lists by value? That might explain why it is so slow.

On Mon, 28 Jan 2019 at 18:08, Hans L thehans@gmail.com wrote:

Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz,
I  wouldn't expect it to be 3x faster than a 2.5GHz core i7..

I am testing on Linux, not Windows, so maybe that is the main
difference. I wonder if Windows snapshots are not getting the same -O2
compiler optimizations?

But anyways, the main thing about caching is that it doesn't simply
cache a call to a module.  OpenSCAD first converts the entire script into
csg format which is just a static representation of primitives, multmatrix
transformations and boolean operations, so it has to evaluate all lists
comprehensions, control structures etc, creating scope for each node etc;
essentially building up the entire output that you see in .csg export, in
memory (about 31.6MB of text for nopheads example, although in memory it
should be a bit less due to whitespace/indentation stripping).  Then it
caches Polysets for matching subtrees (substrings basically of any node and
its children) from this .csg representation.

Compare the 31.6MB to the size of the .csg output of your sphere
example, that's gonna be like a couple kb of text processing.

On Mon, Jan 28, 2019 at 11:01 AM nop head nop.head@gmail.com wrote:

I think the majority of the time is taken in createGeometry, not after
it it.

100 spheres with $fn set to 150, which gives  11250 vertices each,
only takes 1 second, presumably because it is cached, or it is 20 times
quicker to calculate vertices with trig than it is to read them from a list.

On Mon, 28 Jan 2019 at 14:40, Parkinbot rudolf@digitaldocument.de
wrote:

thehans wrote

Are you using a somewhat recent dev snapshot?  The CSG Tree

generation and

key storing got reworked significantly in the past year.

I tried some older snaps with 100 copies of a sweep object, each
containing
19002 facets and no buffering of the points and faces: one from 2016
(1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019
(1':00"). So there was some progress.
With buffering of points and faces the 2019 version needed 0'42".

As compiling an own fork of OpenSCAD is not recommended on Windows
(too much
effort to set it up), I don't know the code well enough to be able to
follow
what is happening in an F5 context. But I am curious.

Can anybody provide a sketch of how the execution goes on after
createGeometry()?

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

I did some profiling and found that its spending much of its time in PolysetUtils::tessellate_faces, because its re-tessellating each geometry after pulling from geometry cache. Ideally it should tesselate only once *before* putting into the cache. Pull request incoming. :) On Mon, Jan 28, 2019 at 12:47 PM Hans L <thehans@gmail.com> wrote: > Yes it really does. The csg text representation of a node *is* its cache > key. And the csg format doesn't really do individual variable (list) > caching. > > On Mon, Jan 28, 2019 at 12:41 PM nop head <nop.head@gmail.com> wrote: > >> It is only 31.6MB in the csg because the lists are text and are repeated >> over and over again. But at runtime there should only be one copy of the >> lists in binary. Does it really generate a textual csg representation in >> memory or is that just when you dump it? And does it actually make 100 >> copies of the lists by value? That might explain why it is so slow. >> >> On Mon, 28 Jan 2019 at 18:08, Hans L <thehans@gmail.com> wrote: >> >>> Well for reference my cpu is: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz, >>> I wouldn't expect it to be 3x faster than a 2.5GHz core i7.. >>> >>> I am testing on Linux, not Windows, so maybe that is the main >>> difference. I wonder if Windows snapshots are not getting the same -O2 >>> compiler optimizations? >>> >>> But anyways, the main thing about caching is that it doesn't simply >>> cache a *call* to a module. OpenSCAD first converts the entire script into >>> csg format which is just a static representation of primitives, multmatrix >>> transformations and boolean operations, so it has to evaluate all lists >>> comprehensions, control structures etc, creating scope for each node etc; >>> essentially building up the entire output that you see in .csg export, in >>> memory (about 31.6MB of text for nopheads example, although in memory it >>> should be a bit less due to whitespace/indentation stripping). *Then* it >>> caches Polysets for matching subtrees (substrings basically of any node and >>> its children) from this .csg representation. >>> >>> Compare the 31.6MB to the size of the .csg output of your sphere >>> example, that's gonna be like a couple kb of text processing. >>> >>> On Mon, Jan 28, 2019 at 11:01 AM nop head <nop.head@gmail.com> wrote: >>> >>>> I think the majority of the time is taken in createGeometry, not after >>>> it it. >>>> >>>> 100 spheres with $fn set to 150, which gives 11250 vertices each, >>>> only takes 1 second, presumably because it is cached, or it is 20 times >>>> quicker to calculate vertices with trig than it is to read them from a list. >>>> >>>> On Mon, 28 Jan 2019 at 14:40, Parkinbot <rudolf@digitaldocument.de> >>>> wrote: >>>> >>>>> thehans wrote >>>>> > Are you using a somewhat recent dev snapshot? The CSG Tree >>>>> generation and >>>>> > key storing got reworked significantly in the past year. >>>>> >>>>> I tried some older snaps with 100 copies of a sweep object, each >>>>> containing >>>>> 19002 facets and no buffering of the points and faces: one from 2016 >>>>> (1'36"), one from 2017 (1'16"), one from 2018 (1'00") and a recent 2019 >>>>> (1':00"). So there was *some* progress. >>>>> With buffering of points and faces the 2019 version needed 0'42". >>>>> >>>>> As compiling an own fork of OpenSCAD is not recommended on Windows >>>>> (too much >>>>> effort to set it up), I don't know the code well enough to be able to >>>>> follow >>>>> what is happening in an F5 context. But I am curious. >>>>> >>>>> Can anybody provide a sketch of how the execution goes on after >>>>> createGeometry()? >>>>> >>>>> >>>>> >>>>> -- >>>>> Sent from: http://forum.openscad.org/ >>>>> >>>>> _______________________________________________ >>>>> OpenSCAD mailing list >>>>> Discuss@lists.openscad.org >>>>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>>>> >>>> _______________________________________________ >>>> OpenSCAD mailing list >>>> Discuss@lists.openscad.org >>>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>>> >>> _______________________________________________ >>> OpenSCAD mailing list >>> Discuss@lists.openscad.org >>> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >>> >> _______________________________________________ >> OpenSCAD mailing list >> Discuss@lists.openscad.org >> http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >> >
P
Parkinbot
Mon, Jan 28, 2019 9:28 PM

Ah I see. Thanks for your information. That explains a lot and it looks like
values are indeed parsed from string representation as nophead had supposed.

--
Sent from: http://forum.openscad.org/

Ah I see. Thanks for your information. That explains a lot and it looks like values are indeed parsed from string representation as nophead had supposed. -- Sent from: http://forum.openscad.org/
NH
nop head
Mon, Jan 28, 2019 10:20 PM

I am totally confused because in the source code it is just passed as a
variable that happens to be a list. Where does it get converted to back a
string again and parsed? Or are OpenSCAD lists just vectors of strings?

And as for re-tessellating. How long does it take to tessellate something
that is already mostly triangles? Should I manually tessellate the end caps
with a triangle fan? The other 9000+ facets are already triangles.

On Mon, 28 Jan 2019 at 21:30, Parkinbot rudolf@digitaldocument.de wrote:

Ah I see. Thanks for your information. That explains a lot and it looks
like
values are indeed parsed from string representation as nophead had
supposed.

--
Sent from: http://forum.openscad.org/


OpenSCAD mailing list
Discuss@lists.openscad.org
http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org

I am totally confused because in the source code it is just passed as a variable that happens to be a list. Where does it get converted to back a string again and parsed? Or are OpenSCAD lists just vectors of strings? And as for re-tessellating. How long does it take to tessellate something that is already mostly triangles? Should I manually tessellate the end caps with a triangle fan? The other 9000+ facets are already triangles. On Mon, 28 Jan 2019 at 21:30, Parkinbot <rudolf@digitaldocument.de> wrote: > Ah I see. Thanks for your information. That explains a lot and it looks > like > values are indeed parsed from string representation as nophead had > supposed. > > > > > -- > Sent from: http://forum.openscad.org/ > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >