Instances + octrees

Hi list!

First, I am still using v3.5 - I have a Mac and no X-sources, so it's hard to compile Head.

To improve the performance during rendering a very large model, I atomized it into LOTS of small parts. These are assembled by replmarks, using octrees generated with oconv -f. I thought this was the most efficient solution, and I want to save memory. This leads to a model which is assembled from hundreds of octrees (instances).

Now, after importing one more (and big) part into the model, I am not able to oconv the file which is to assemble all these instances any more. Oconv is running without end, I stop it when it takes about 300 MB memory. When I use xform to insert my elements (the same model, just replaced the -i by -x and *.oct by *.rad in the replmarks command line), not instances, everything works fine (except of a very large octree).

I heard about some "overlapping octree" problems, however, I never understood how they appear. May this be one such problem? Actually, the geometry is not overlapping, but the small octrees bounding boxes certainly are (as they are all cubes, right?). What did you experience with such instanced models?

Is the oconv tool in the 3.5 buggy? Any known bugs like that?

Is it my error (should I check my scripts ten more times :wink: ?

I really would like to keep using octree-instances, as the model is growing quite fast at the moment, and I won't be able to finally render it on my machine without instances. So I am very interested in why this problem appears and how to resolve it.

TIA+CU, Lars.

···

--
Lars O. Grobe
[email protected]

Hi again!

If it is about octrees, to add the question: Should I

a) increase the resolution (-r)?
b) decrease the resolution?
c) change my model's structure (will be hard...)?

TIA+CU Lars.

···

--
Lars O. Grobe
[email protected]

Lars O. Grobe wrote:

Hi list!

First, I am still using v3.5 - I have a Mac and no X-sources, so it's hard to compile Head.

To improve the performance during rendering a very large model, I atomized it into LOTS of small parts. These are assembled by replmarks, using octrees generated with oconv -f. I thought this was the most efficient solution, and I want to save memory. This leads to a model which is assembled from hundreds of octrees (instances).

Now, after importing one more (and big) part into the model, I am not able to oconv the file which is to assemble all these instances any more. Oconv is running without end, I stop it when it takes about 300 MB memory. When I use xform to insert my elements (the same model, just replaced the -i by -x and *.oct by *.rad in the replmarks command line), not instances, everything works fine (except of a very large octree).

I heard about some "overlapping octree" problems, however, I never understood how they appear. May this be one such problem? Actually, the geometry is not overlapping, but the small octrees bounding boxes certainly are (as they are all cubes, right?). What did you experience with such instanced models?

saving Greg the answer: You're exactly right. non-cubic objects (thin rods etc.) generate cubic octrees, which do overlap, whereas the geometry does not. The final oconv than tries very hard to seperate the small octree cubes again, which it can't if there are more overlapping things (small octrees in this case) at any point in space then allowed by the -n option. Either allow more objects per (final, large) octree by increasing -n at the fnal oconv run (which may slow down the raytracer), go back to 'xform' without small octrees or pay Greg to rewrite the code so that the sides of octree-cubicles may have different length.

Is the oconv tool in the 3.5 buggy? Any known bugs like that?

not to my knowledge, although R3.5 is pretty buggy in other respects compared with the current release

_Peter

···

--
pab-opto, Freiburg, Germany, http://www.pab-opto.de
[see web page to check digital email signature]

Hi,

while I am still not sure if I am on a secure path with my model's structure, setting oconv's resolution to 256 fixed the problem for now...

CU Lars.

···

--
Lars O. Grobe
[email protected]

Lars O. Grobe wrote:

Hi,

while I am still not sure if I am on a secure path with my model's
structure, setting oconv's resolution to 256 fixed the problem for
now...

CU Lars.

A more efficient approach *might* be to create a set of nested instances. Each super-instance would include eight sub-instances (-n 8), and each of the eight sub-instance further instances until the entire scene is enclosed in one bounding box. If there are 8 or fewer "objects" per octree (instance) then oconv does not bother to futher subdivide, thereby allowing oconv to do its job. Greg would know how many times you can nest instances, but if I remember correctly, it is not infinite. An instance counts as a single object, of course.

It is debateable as to whether this approach will result in a "better" octree. The maxres (-r 256) approach could introduce errors in the intersection calculation in very large scenes with very small pieces, but increasing the object limit (-n >7) slows down the ray-intersection calculation because it has more objects to deal with (a higher-order equation to solve for the zeroes). In my experience, you can double the object limit with only a small increase in rendering time, sometimes not even noticeable.

However, when it comes to render the scene, each ray that goes between one instance and another must go through a ray transformation operation to align it with the "object space" of the next instance. The transformation ops are applied to *every* ray including ambient, shadow, etc. All other things being equal, the resulting octree will not render as quickly as one generated without instances. For a given maxres, the optimum solution would be a balance between the object count per instance (-n N) and the total number of sub-instances.

And, be sure the boundaries of each of the instances are world-axis aligned. If there is any structure to the underlying geometry, be sure to align the octree boundaries with the "structure" of the underlying geometry (the other way around, actually.) For example, a digital elevation model is made up of a uniform pattern of triangles forming a square when projected onto a plane. It is much easier to oconv when the "grid" of the DEM is world axis-aligned. For non-converging scenes it is better to oconv the object and then rotate and transform it into the appropriate location in the scene as an instance rather than the other way around.

The DEM scene also is a good example of carefully selecting the maxobj parameter. A cubic "chuck" of such a scene can either contain 2, 8, or 32 triangles. Eight is the best value for -n because it is closest to Radiance's optimum value of 7.

Obviously, if oconv simply will never converge on a solution (so to speak) then you *must* use the nested instance approach. But, the success of this approach depends upon the modeler's ability to divide up the scene into nice, semi-cubic, non-overlapping chunks. Some scenes simply do not allow this to happen easily. In one case, I used something akin to a fancy "sed" script to arbitrarily divide the scene into separate pieces which were then oconv'd, instanced, and assembled into a coherent scene. With Desktop Radiance, if you create an AutoCAD "block" out of some surfaces and then "insert" it into the drawing, that object will be treated as an instance. (I had to get a plug in there somewhere.)

I apologize if this is old news. I do not keep up with this list as much as I'd like.

-Chas

Hi Charles,

thank You very much for this information. It seams to be a bit like the distilled information about oconv, octrees and instances over the years ;-), and that was just what I need... Maybe one should start a Radiance-FAQ on radiance-online.org...

I will try to use the nested-instances method you describes. Is the number of sub-instances fixed or just resulting from the -n 8 option? Nested instances represent very well the hierarchic structure of my model. So far, I used instances for the smaller elements (with replmarks) and xformed whole building blocks containg all the instances. If I replace the xform transformations by the useage of "super-instances", I arrive at the nested instance model, however, I would have about 40 instances in a typical "super-instance". So should I change the -n to 40?

So far, I could not see any errors cause by the -r 256 option, I just guessed that I could reduce memory needs by reducing -r, as the default was much lower in old versions of radiance and I remember quite well what happened when I had to increase -r some time ago...

Anyway... in the worst case, I will have to give up instancing and do the brute-force method with a lot of geometry, a very large octree and high memory needs. I just don't want to as I don't know how much geometry will be added in the future, and also the excessive application of colorpicts makes me think about memory...

Thanks, CU Lars.

···

--
Lars O. Grobe
[email protected]

Hi Lars,

The -n parameter is the maximum allowed number of objects per "voxel". A voxel in Radiance lingo would be cubic piece of space. The oconv program sub-divides the scene until all voxels contain no more than -n number of objects--be they spheres, polygons, whatever. Since an instance counts as one object, the fact that it (the instance) contains sub-instances is irrelevant. So no, keep -n at 8.

The number of times that an instance can be recursively instanced inside of super-instances is, I imagine, a fixed number based upon the size of a varriable--of which Greg would be the best to advise.

Perhaps the default value for -n should be 8 considering the likelihood that architectural scenes will contain square-ish and cubical entities. The impact on rendering time should be negligible, I presume.

Greg boasted that using sub-instances it is possible to model the entire planet earth down to a resolution of one inch without any loss of geometric precision in the ray-intersection calculation.

-Chas

···

Date: Wed, 7 Jul 2004 17:19:49 +0300
From: Lars O. Grobe <[email protected]>
Subject: Re: [Radiance-general] Re: Instances + octrees
To: Radiance general discussion <[email protected]>
Message-ID: <[email protected]>
Content-Type: text/plain; charset=US-ASCII; format=flowed

Hi Charles,

thank You very much for this information. It seams to be a bit like the
distilled information about oconv, octrees and instances over the years
;-), and that was just what I need... Maybe one should start a
Radiance-FAQ on radiance-online.org...

I will try to use the nested-instances method you describes. Is the
number of sub-instances fixed or just resulting from the -n 8 option?
Nested instances represent very well the hierarchic structure of my
model. So far, I used instances for the smaller elements (with
replmarks) and xformed whole building blocks containg all the
instances. If I replace the xform transformations by the useage of
"super-instances", I arrive at the nested instance model, however, I
would have about 40 instances in a typical "super-instance". So should
I change the -n to 40?

So far, I could not see any errors cause by the -r 256 option, I just
guessed that I could reduce memory needs by reducing -r, as the default
was much lower in old versions of radiance and I remember quite well
what happened when I had to increase -r some time ago...

Anyway... in the worst case, I will have to give up instancing and do
the brute-force method with a lot of geometry, a very large octree and
high memory needs. I just don't want to as I don't know how much
geometry will be added in the future, and also the excessive
application of colorpicts makes me think about memory...

Thanks, CU Lars.
--
Lars O. Grobe
[email protected]