discuss@lists.openscad.org

OpenSCAD general discussion Mailing-list

View all threads

Passing function literals to builtin modules.

A
adrianv
Fri, Jan 8, 2021 11:36 PM

Why does it matter when the render happens, exactly?

Regarding execution time, there are a lot of 2d cases where rendering will
have negligible time.  This approach would make it possible to use geometry
as input to a userspace sweep operation instead of forcing the user to
maintain everything as point lists.  I tried hull() of two spheres and it
seems to render very quickly (and lazy union will help with this).  If the
alternative is running hull() in user space (to maintain access to geometry
data) I think this approach will make preview faster rather than slower.
I looked into using hull() to produce rounded polyhedra in a polyhedron
library and user space hull was prohibitively expensive (takes many
minutes).

JordanBrown wrote

[ pass geometry to arbitrary function ]

Does render() run at the right time to be helpful?

I don't know beans about the actual implementation, but my mental model
and experimentation says that the "execution" of the program is a
distinct step earlier than any rendering.  The render() module doesn't
cause its children to be rendered right then and there.  Rather, it's a
node in the CSG tree that causes its children to be rendered using CGAL
rather than using CSG, even when doing a preview.

Here's a test program.  It takes about 10 seconds on my (older and low
end) desktop, so long enough to be clear and short enough not to be
annoying.  All of the echo output comes out instantly, before any of the
rendering starts.  (Unsurprisingly, with the render()s, it's about the
same amount of time for F5 or F6.)

 module teardrop(k) {
     junk = echo("1", k);
     echo("2", k);
     for (i=[1:0.5:5]) translate([i,0,0]*k) sphere(i);
     junk2 = echo("3", k);
     echo("4", k);
 }

 render() teardrop(3);
 render() translate([0,20,0]) teardrop(4);

It also makes an interesting shape.  That's just a happy accident; I
wanted something simple that would multiply up into easily visible
render times.

It seems like the proposed transform module would have to move the CGAL
step much earlier in the processing pipeline.


OpenSCAD mailing list

Discuss@.openscad

Why does it matter when the render happens, exactly? Regarding execution time, there are a lot of 2d cases where rendering will have negligible time. This approach would make it possible to use geometry as input to a userspace sweep operation instead of forcing the user to maintain everything as point lists. I tried hull() of two spheres and it seems to render very quickly (and lazy union will help with this). If the alternative is running hull() in user space (to maintain access to geometry data) I think this approach will make preview *faster* rather than slower. I looked into using hull() to produce rounded polyhedra in a polyhedron library and user space hull was prohibitively expensive (takes many minutes). JordanBrown wrote >> [ pass geometry to arbitrary function ] > > Does render() run at the right time to be helpful? > > I don't know beans about the actual implementation, but my mental model > and experimentation says that the "execution" of the program is a > distinct step earlier than any rendering.  The render() module doesn't > cause its children to be rendered right then and there.  Rather, it's a > node in the CSG tree that causes its children to be rendered using CGAL > rather than using CSG, even when doing a preview. > > Here's a test program.  It takes about 10 seconds on my (older and low > end) desktop, so long enough to be clear and short enough not to be > annoying.  All of the echo output comes out instantly, before any of the > rendering starts.  (Unsurprisingly, with the render()s, it's about the > same amount of time for F5 or F6.) > > module teardrop(k) { > junk = echo("1", k); > echo("2", k); > for (i=[1:0.5:5]) translate([i,0,0]*k) sphere(i); > junk2 = echo("3", k); > echo("4", k); > } > > render() teardrop(3); > render() translate([0,20,0]) teardrop(4); > > It also makes an interesting shape.  That's just a happy accident; I > wanted something simple that would multiply up into easily visible > render times. > > > It seems like the proposed transform module would have to move the CGAL > step much earlier in the processing pipeline. > > > _______________________________________________ > OpenSCAD mailing list > Discuss@.openscad > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org -- Sent from: http://forum.openscad.org/
HL
Hans L
Sat, Jan 9, 2021 1:19 AM

Jordan raises a point which I hadn't thought through yet.

The main thing about the order in which scripts are evaluated, is that we
first generate the CSG Tree (menu: "Design-> Display CSG Tree...", or "File
-> Export as CSG"  to see this), then evaluate geometry from that.

This CSG text should be functionally equivalent to the original script, so
you can Export CSG, then load it up in a new tab or instance of OpenSCAD,
and preview/render of that CSG should behave the same.
Also the text of this tree is important for being able to identify cache
hits correctly.

So far, making this CSG tree basically meant that all expressions and
control statements were fully evaluated, leaving only builtin
module instantiations with their arguments expressed as literal values.
Now, if modules are going to start accepting function-literal arguments,
then those arguments will necessarily contain their own expressions as part
of the CSG tree.
In some way, this changes the meaning of what a CSG file can contain, but I
don't think that in itself is much of a problem.

The main issue I see now is in cases where a function-literal forms a
closure, capturing external variables.
If you echo() a function literal, the original expressions are shown, but
no information about the value of those captured variables.

A quick example:
answer = 42;
is_answer = function(x) x==answer;
echo(is_answer);        // ECHO: function(x) (x == answer)

So I think those captured variables will need to be replaced with their
corresponding literal values to resolve this issue.

Hans

On Fri, Jan 8, 2021 at 5:37 PM adrianv avm4@cornell.edu wrote:

Why does it matter when the render happens, exactly?

Regarding execution time, there are a lot of 2d cases where rendering will
have negligible time.  This approach would make it possible to use
geometry
as input to a userspace sweep operation instead of forcing the user to
maintain everything as point lists.  I tried hull() of two spheres and it
seems to render very quickly (and lazy union will help with this).  If the
alternative is running hull() in user space (to maintain access to geometry
data) I think this approach will make preview faster rather than slower.
I looked into using hull() to produce rounded polyhedra in a polyhedron
library and user space hull was prohibitively expensive (takes many
minutes).

JordanBrown wrote

[ pass geometry to arbitrary function ]

Does render() run at the right time to be helpful?

I don't know beans about the actual implementation, but my mental model
and experimentation says that the "execution" of the program is a
distinct step earlier than any rendering.  The render() module doesn't
cause its children to be rendered right then and there.  Rather, it's a
node in the CSG tree that causes its children to be rendered using CGAL
rather than using CSG, even when doing a preview.

Here's a test program.  It takes about 10 seconds on my (older and low
end) desktop, so long enough to be clear and short enough not to be
annoying.  All of the echo output comes out instantly, before any of the
rendering starts.  (Unsurprisingly, with the render()s, it's about the
same amount of time for F5 or F6.)

 module teardrop(k) {
     junk = echo("1", k);
     echo("2", k);
     for (i=[1:0.5:5]) translate([i,0,0]*k) sphere(i);
     junk2 = echo("3", k);
     echo("4", k);
 }

 render() teardrop(3);
 render() translate([0,20,0]) teardrop(4);

It also makes an interesting shape.  That's just a happy accident; I
wanted something simple that would multiply up into easily visible
render times.

It seems like the proposed transform module would have to move the CGAL
step much earlier in the processing pipeline.


OpenSCAD mailing list

Discuss@.openscad

Jordan raises a point which I hadn't thought through yet. The main thing about the order in which scripts are evaluated, is that we first generate the CSG Tree (menu: "Design-> Display CSG Tree...", or "File -> Export as CSG" to see this), then evaluate geometry from that. This CSG text should be functionally equivalent to the original script, so you can Export CSG, then load it up in a new tab or instance of OpenSCAD, and preview/render of that CSG should behave the same. Also the text of this tree is important for being able to identify cache hits correctly. So far, making this CSG tree basically meant that all expressions and control statements were fully evaluated, leaving only builtin module instantiations with their arguments expressed as literal values. Now, if modules are going to start accepting function-literal arguments, then those arguments will necessarily contain their own expressions as part of the CSG tree. In some way, this changes the meaning of what a CSG file can contain, but I don't think that in itself is much of a problem. The main issue I see now is in cases where a function-literal forms a closure, capturing external variables. If you echo() a function literal, the original expressions are shown, but no information about the value of those captured variables. A quick example: answer = 42; is_answer = function(x) x==answer; echo(is_answer); // ECHO: function(x) (x == answer) So I think those captured variables will need to be replaced with their corresponding literal values to resolve this issue. Hans On Fri, Jan 8, 2021 at 5:37 PM adrianv <avm4@cornell.edu> wrote: > Why does it matter when the render happens, exactly? > > Regarding execution time, there are a lot of 2d cases where rendering will > have negligible time. This approach would make it possible to use > geometry > as input to a userspace sweep operation instead of forcing the user to > maintain everything as point lists. I tried hull() of two spheres and it > seems to render very quickly (and lazy union will help with this). If the > alternative is running hull() in user space (to maintain access to geometry > data) I think this approach will make preview *faster* rather than slower. > I looked into using hull() to produce rounded polyhedra in a polyhedron > library and user space hull was prohibitively expensive (takes many > minutes). > > > JordanBrown wrote > >> [ pass geometry to arbitrary function ] > > > > Does render() run at the right time to be helpful? > > > > I don't know beans about the actual implementation, but my mental model > > and experimentation says that the "execution" of the program is a > > distinct step earlier than any rendering. The render() module doesn't > > cause its children to be rendered right then and there. Rather, it's a > > node in the CSG tree that causes its children to be rendered using CGAL > > rather than using CSG, even when doing a preview. > > > > Here's a test program. It takes about 10 seconds on my (older and low > > end) desktop, so long enough to be clear and short enough not to be > > annoying. All of the echo output comes out instantly, before any of the > > rendering starts. (Unsurprisingly, with the render()s, it's about the > > same amount of time for F5 or F6.) > > > > module teardrop(k) { > > junk = echo("1", k); > > echo("2", k); > > for (i=[1:0.5:5]) translate([i,0,0]*k) sphere(i); > > junk2 = echo("3", k); > > echo("4", k); > > } > > > > render() teardrop(3); > > render() translate([0,20,0]) teardrop(4); > > > > It also makes an interesting shape. That's just a happy accident; I > > wanted something simple that would multiply up into easily visible > > render times. > > > > > > It seems like the proposed transform module would have to move the CGAL > > step much earlier in the processing pipeline. > > > > > > _______________________________________________ > > OpenSCAD mailing list > > > Discuss@.openscad > > > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org > > > > > > -- > Sent from: http://forum.openscad.org/ > > _______________________________________________ > OpenSCAD mailing list > Discuss@lists.openscad.org > http://lists.openscad.org/mailman/listinfo/discuss_lists.openscad.org >
HL
Hans L
Sat, Jan 9, 2021 1:44 AM

On Fri, Jan 8, 2021 at 5:14 PM Jordan Brown openscad@jordan.maileater.net
wrote:

It seems like the proposed transform module would have to move the CGAL
step much earlier in the processing pipeline.

Just to clarify, more to your point: I wanted to say that it's not really
that CGAL needs to be called earlier, but that some user-defined
functions/expressions would be evaluated later (after CSG Tree
generation) which would be the main change here.
And besides the closure issue mentioned above, I don't think there's any
other problems with doing that.

On Fri, Jan 8, 2021 at 5:14 PM Jordan Brown <openscad@jordan.maileater.net> wrote: > It seems like the proposed transform module would have to move the CGAL > step much earlier in the processing pipeline. > Just to clarify, more to your point: I wanted to say that it's not really that CGAL needs to be called earlier, but that some user-defined functions/expressions would be evaluated *later* (after CSG Tree generation) which would be the main change here. And besides the closure issue mentioned above, I don't think there's any other problems with doing that.
T
TLC123
Sat, Jan 9, 2021 4:35 PM

Q:Maybe the render() children() could be optional?

Great ideas from all of you.

I very much like the proposed general transform module.
Complicated to say the least, even more so if the passed function-literal
contains further function calls, be it recursive, external or literal.

Even in the case those function-literals have to have only hard coded
values, very useful.

Its a little scary, this concept of having function-literals reaching into
the afterlife of evaluation but also thrilling.
I make some stupid assumption that most of the infrastructure that normally
run the evaluations could be reached from there to.

The proposed transform module that take function-literal arguments could
bring-
User defined:
-transformation,e.g by functions of position or connectivity
-deformation, e.g by noise or patterns
-geometry subset manipulation, e.g make these here co-planar/co-linear
-echo analysis and measurements to console returning nothing e.g
caliper,volume surface.
-subdivision / refinement e.g Catmull-Clark
-simplification e.g edge collapse
-(unsafer) smooth / sharpen / inflate / contract / melt / erode
-replace input e.g  by bounding box /sphere
-generate new geometry along surface/paths // (but no boolean here) Just
return new geometry and
boolean result with copy of original.
-3d that return 2d , e.g cut, project, unfold,    f=function(points,
faces)[points, paths];
-2d that return 3d , e.g sweep, loft, roof, soap bubble, f=function(points,
paths) [points, faces] ;
-re-meshing  e.g for the brave implement delaunay triangulation with only
list
-by analysis transformation e.g balancing / rotation to stable configuration
-reinforcing thin parts
-imported mesh repair?
-engineering optimizations e.g minimum curvature/ stress points
-retopology
-on boolean result: do a hole slew of discrete differential geometry math.
-find genus

caveats like polyhedron/polygon  user is responsible to check for:

  • self intersection
  • face orientation
  • open geometry
  • non manifold

What most probably cannot be done:
-collision / interference test, since its only 1 child
-stack / attach / pack / layout / optimize mutual position , since its only
1 child
-compare homo-topic properites
-comparing analysis in general , since its only 1 child
(unlikely without full closed loop geometry-evaluation rework)
-intermediate boolean operation with generated geometry by cgal/clipper,
wrong namespace.

Q:Maybe the render children could be opted out of or left to user space?
/transform3d(f) {render(){ ... }} as the safe way to do it
transform3d(f) {obj();obj();obj();}  and function f([[points, faces],
[points, faces], [points, faces]])
/

--
Sent from: http://forum.openscad.org/

Q:Maybe the render() children() could be optional? Great ideas from all of you. I very much like the proposed general transform module. Complicated to say the least, even more so if the passed function-literal contains further function calls, be it recursive, external or literal. Even in the case those function-literals have to have only hard coded values, very useful. Its a little scary, this concept of having function-literals reaching into the afterlife of evaluation but also thrilling. I make some stupid assumption that most of the infrastructure that normally run the evaluations could be reached from there to. The proposed transform module that take function-literal arguments could bring- User defined: -transformation,e.g by functions of position or connectivity -deformation, e.g by noise or patterns -geometry subset manipulation, e.g make these here co-planar/co-linear -echo analysis and measurements to console returning nothing e.g caliper,volume surface. -subdivision / refinement e.g Catmull-Clark -simplification e.g edge collapse -(unsafer) smooth / sharpen / inflate / contract / melt / erode -replace input e.g by bounding box /sphere -generate new geometry along surface/paths // (but no boolean here) Just return new geometry and boolean result with copy of original. -3d that return 2d , e.g cut, project, unfold, f=function(points, faces)[points, paths]; -2d that return 3d , e.g sweep, loft, roof, soap bubble, f=function(points, paths) [points, faces] ; -re-meshing e.g for the brave implement delaunay triangulation with only list -by analysis transformation e.g balancing / rotation to stable configuration -reinforcing thin parts -imported mesh repair? -engineering optimizations e.g minimum curvature/ stress points -retopology -on boolean result: do a hole slew of discrete differential geometry math. -find genus caveats like polyhedron/polygon user is responsible to check for: - self intersection - face orientation - open geometry - non manifold What most probably cannot be done: -collision / interference test, since its only 1 child -stack / attach / pack / layout / optimize mutual position , since its only 1 child -compare homo-topic properites -comparing analysis in general , since its only 1 child (unlikely without full closed loop geometry-evaluation rework) -intermediate boolean operation with generated geometry by cgal/clipper, wrong namespace. Q:Maybe the render children could be opted out of or left to user space? /transform3d(f) {render(){ ... }} as the safe way to do it transform3d(f) {obj();obj();obj();} and function f([[points, faces], [points, faces], [points, faces]]) / -- Sent from: http://forum.openscad.org/
JB
Jordan Brown
Sat, Jan 9, 2021 4:43 PM

[ Only Adrian will notice this, but... sigh, I did it again.  Maybe for
the last time.  I'd previously used a Thunderbird add-on to manage
"From" addresses, but a Thunderbird upgrade broke it.  I just found a
new version that seems to work.  Fingers crossed. ]

On 1/8/2021 3:36 PM, adrianv wrote:

Why does it matter when the render happens, exactly?

Abstractly, it doesn't.

Concretely, OpenSCAD has an existing pipeline from source files to mesh,
and moving the rendering up into the "execution" phase where functions
could operate on the render (or, equivalently, moving function execution
down into the "render" phase) results seems like it might be major surgery.

Regarding execution time,

I was using execution time only as a way to see when things happened. 
If the render() module caused things to be rendered right then and
there, before proceeding to evaluate the next module, you'd see it in
the timing of the echo outputs.

I agree that for some applications adding render time to a preview might
be negligible or worth the wait.

[ Only Adrian will notice this, but... sigh, I did it again.  Maybe for the last time.  I'd previously used a Thunderbird add-on to manage "From" addresses, but a Thunderbird upgrade broke it.  I just found a new version that seems to work.  Fingers crossed. ] On 1/8/2021 3:36 PM, adrianv wrote: > Why does it matter when the render happens, exactly? Abstractly, it doesn't. Concretely, OpenSCAD has an existing pipeline from source files to mesh, and moving the rendering up into the "execution" phase where functions could operate on the render (or, equivalently, moving function execution down into the "render" phase) results seems like it might be major surgery. > Regarding execution time, I was using execution time only as a way to see when things happened.  If the render() module caused things to be rendered right then and there, before proceeding to evaluate the next module, you'd see it in the timing of the echo outputs. I agree that for some applications adding render time to a preview might be negligible or worth the wait.
A
adrianv
Sat, Jan 9, 2021 5:01 PM

TLC123 wrote

Q:Maybe the render() children() could be optional?

Great ideas from all of you.

I very much like the proposed general transform module.
Complicated to say the least, even more so if the passed function-literal
contains further function calls, be it recursive, external or literal.

Even in the case those function-literals have to have only hard coded
values, very useful.

I assume that

What do you mean by "hard coded"?

If I want to write path_sweep() then using this mechanism I need a way to
pass user parameters into the transformation, or it isn't very useful.  I'm
imagining the implementation of a path sweep method looking sort of like
this, where I need to specify a path, select an algorithm (the method) and
indicate of the path closes on itself.

module path_sweep(path, method, closed)  // path = path to sweep along,
method=a sweep algorithm choice
{
transform2d( function (geometry)
path_sweep_function(geometry,path,method,closed))
children();
}

There are some ambitious ideas on your list.  The limitations of the
OpenSCAD language probably make some of those more ambitious ideas
impractical.  This transformation idea doesn't exactly make new things
possible.  It makes things easier and makes new things possible that are
better unified with the base language---rather than having to carry around
point lists you can work with geometry.  It should make it simpler for users
to use a library that implements operations like sweep because they don't
need to learn a new paradigm.  But doing big calculations on large polyhedra
is likely to be prohibitive.  There's a "bend" module in BOSL2 but it's
really slow, for example.  That's fundamental, not a limitation due to
access to geometry.

Q:Maybe the render children could be opted out of or left to user space?
/transform3d(f) {render(){ ... }} as the safe way to do it
transform3d(f) {obj();obj();obj();}  and function f([[points, faces],
[points, faces], [points, faces]])

Don't you still need implicit renders on each object?  That is,

transform3d(f) {render()obj(); render()obj(); ...}

It seems like the most flexibility is created by passing children separately
and having the function expect a list of objects.  But the render is going
to be required on each object passed, isn't it?  Is there harm in putting
render on something that didn't need it, like a single object that didn't
need CGAL?  Requiring explicit renders is also bad because it breaks the
unified paradigm I was talking about above.  If a user is using a library
how do they know when to stick in a render and when not to?  It will be a
point of confusion.  (And what happens if you skip it.)

--
Sent from: http://forum.openscad.org/

TLC123 wrote > Q:Maybe the render() children() could be optional? > > Great ideas from all of you. > > I very much like the proposed general transform module. > Complicated to say the least, even more so if the passed function-literal > contains further function calls, be it recursive, external or literal. > > Even in the case those function-literals have to have only hard coded > values, very useful. I assume that What do you mean by "hard coded"? If I want to write path_sweep() then using this mechanism I need a way to pass user parameters into the transformation, or it isn't very useful. I'm imagining the implementation of a path sweep method looking sort of like this, where I need to specify a path, select an algorithm (the method) and indicate of the path closes on itself. module path_sweep(path, method, closed) // path = path to sweep along, method=a sweep algorithm choice { transform2d( function (geometry) path_sweep_function(geometry,path,method,closed)) children(); } There are some ambitious ideas on your list. The limitations of the OpenSCAD language probably make some of those more ambitious ideas impractical. This transformation idea doesn't exactly make new things possible. It makes things easier and makes new things possible that are better unified with the base language---rather than having to carry around point lists you can work with geometry. It should make it simpler for users to use a library that implements operations like sweep because they don't need to learn a new paradigm. But doing big calculations on large polyhedra is likely to be prohibitive. There's a "bend" module in BOSL2 but it's really slow, for example. That's fundamental, not a limitation due to access to geometry. > Q:Maybe the render children could be opted out of or left to user space? > /transform3d(f) {render(){ ... }} as the safe way to do it > transform3d(f) {obj();obj();obj();} and function f([[points, faces], > [points, faces], [points, faces]]) Don't you still need implicit renders on each object? That is, transform3d(f) {render()obj(); render()obj(); ...} It seems like the most flexibility is created by passing children separately and having the function expect a list of objects. But the render is going to be required on each object passed, isn't it? Is there harm in putting render on something that didn't need it, like a single object that didn't need CGAL? Requiring explicit renders is also bad because it breaks the unified paradigm I was talking about above. If a user is using a library how do they know when to stick in a render and when not to? It will be a point of confusion. (And what happens if you skip it.) -- Sent from: http://forum.openscad.org/