rvu: too many modifiers in ambient list

Hi!

I have a problem with the ambient exclude options. I am currently trying to make rendering times shorter, excluding detail geometry from the ambient calculation. Using -ae, I got the "too many modifiers" message, and switched to -aE with a file containing the modifier names. However this did not help. So how many modifiers can I exclude with a standard-compile radiance (I guess it is defined somewhere in the sources)?

TIA+CU Lars.

Hi Lars,

I'm not sure what the limit is in the official release, as this is something I changed recently, but it's 512 if the latest head. If you'd like to change it, you can either edit AMBLLEN in src/rt/ray.h or set a -DAMBLLEN=1024 (or whatever) in your optimization flags in rmake and recompile.

If you are excluding more modifiers than you are including, you can use the -ai or -aI option instead.

-Greg

···

From: Lars O. Grobe <[email protected]>
Date: July 8, 2005 3:46:21 AM PDT

Hi!

I have a problem with the ambient exclude options. I am currently trying to make rendering times shorter, excluding detail geometry from the ambient calculation. Using -ae, I got the "too many modifiers" message, and switched to -aE with a file containing the modifier names. However this did not help. So how many modifiers can I exclude with a standard-compile radiance (I guess it is defined somewhere in the sources)?

TIA+CU Lars.

Hi Greg!

I'm not sure what the limit is in the official release, as this is something I changed recently, but it's 512 if the latest head.

Is this the number of modifiers or the total string length? If it's the first, there must have been a real change, cause I could exclude around 6 or seven, not more in my old release (which is something after 3.6).

  If you'd like to change it, you can either edit AMBLLEN in src/rt/ray.h or set a -DAMBLLEN=1024 (or whatever) in your optimization flags in rmake and recompile.

I'll take a look at this and compile a new head this week-end.

Thank you, have a nice week-end, Lars.

Hi Lars,

It's not total number of characters, but list length. The fewest I ever allowed was 128, I think, so something else must be amiss with your version, or the way you're using it.

-G

···

From: Lars O. Grobe <[email protected]>
Date: July 8, 2005 6:59:53 AM PDT

Hi Greg!

I'm not sure what the limit is in the official release, as this is something I changed recently, but it's 512 if the latest head.

Is this the number of modifiers or the total string length? If it's the first, there must have been a real change, cause I could exclude around 6 or seven, not more in my old release (which is something after 3.6).

  If you'd like to change it, you can either edit AMBLLEN in src/rt/ray.h or set a -DAMBLLEN=1024 (or whatever) in your optimization flags in rmake and recompile.

I'll take a look at this and compile a new head this week-end.

Thank you, have a nice week-end, Lars.

Hi Greg!

I guess the problem arises somewhere else. I get this problem even with only on -ae modifier now. I think the problem is that I simply have too many materials defined. The check in ambient.c is done if I use the -ae switch, but I don't think it is the number of -ae modifiers.

The background is that I do something stupid at the first look: I include my global material library with every object of my scene, than assemble the scene with replmarks. The reason for that (including into each object instead using the library at the final oconv run) is that my materials have to undergo all the transformations I apply to the objects (because I use image maps massively). Before, I had precompiled all these objects and used them as instances, so there was no need in having the material definitions in all objects, but this gave a bad overhead (the point is that a rotated instances still has its mapping, but a rotated rad-scene has wrong mappings, cause the mapping is rotated only if the material definition is IN the scene).

So I guess I will have to find a way to filter the materials better (so that only the materials actually used are included into an object file) or live with the instance overhead.

Thanks, CU Lars.

Hi Lars,

That explains the problem. Because each new inclusion of a material gets a new id, and this id needs to go into the ambient include/exclude list for that modifier name, you will quickly overflow your ambient exclude set if you incorporate a copy of your materials for every object.

This is bad practice in general because it means that most of your memory is occupied with redundant material definitions rather than geometry. A better approach is to include any materials that need to undergo transformations as part of the geometric object file. If you just have a map you wish to apply, possibly substituting the underlying material, you can use an alias appropriately, e.g.:

# The map for this material:

void colorpict thisMap
7 ...
0

# origMat is the name of a predefined material in my main library, included once

thisMap alias thisMat origMat

thisMat {geometry ...}

···

-----
This allows you to include the main material definitions only once, and control them in a central library. You can even control the maps, because they are named by the colorpict rather than defined directly in your geometry files.

Hope this helps.
-Greg

From: Lars O. Grobe <[email protected]>
Date: July 11, 2005 2:50:16 AM PDT

Hi Greg!

I guess the problem arises somewhere else. I get this problem even with only on -ae modifier now. I think the problem is that I simply have too many materials defined. The check in ambient.c is done if I use the -ae switch, but I don't think it is the number of -ae modifiers.

The background is that I do something stupid at the first look: I include my global material library with every object of my scene, than assemble the scene with replmarks. The reason for that (including into each object instead using the library at the final oconv run) is that my materials have to undergo all the transformations I apply to the objects (because I use image maps massively). Before, I had precompiled all these objects and used them as instances, so there was no need in having the material definitions in all objects, but this gave a bad overhead (the point is that a rotated instances still has its mapping, but a rotated rad-scene has wrong mappings, cause the mapping is rotated only if the material definition is IN the scene).

So I guess I will have to find a way to filter the materials better (so that only the materials actually used are included into an object file) or live with the instance overhead.

Thanks, CU Lars.

Hi Greg!

That explains the problem. Because each new inclusion of a material gets a new id, and this id needs to go into the ambient include/exclude list for that modifier name, you will quickly overflow your ambient exclude set if you incorporate a copy of your materials for every object.

Ok, so if I define aliases, no new ids are created, right? I changed my scripts so that only a file (let's call it material.aliases) is cat'ed into the rad-scene-file, while the actual definitions of materials (material.mat) and modifying patterns (material.map) are passed only to oconv in the final processing step. I have about 20 aliases in my material.aliases file, and a lot of objects (radiance scene files), so I hope that these aliases won't need to have their own ids (if I calculate 200 objects, 200*20=4000 id's would already massively overrun all limits).

The next question arises with the use of obj2mesh. I use

obj2mesh -a material.mat -a material.map -a material.alias object.obj > object.rad

for each of my objects. I have to pass the mat and map files to obj2mesh, which behaves more like oconv and can't use just the aliases (and having the actual definitions linked later by oconv). So here I still get all these definitions for each object, will this count in my ambient list?

This is bad practice in general because it means that most of your memory is occupied with redundant material definitions rather than geometry.

Yes. I guess I could solve all this if using only obj as cad input, using u/v-coordinates with obj2mesh to position and transform the maps and don't include any mappings into the scene files at all. However, I have a mixed model with dxf and obj, which developed before obj2mesh was released, and I am a bit suspicious concerning modeling apps and u/v.

I am currently trying to reduce the material definitions, but as you can see from my questions, I won't be able to change a lot if some of the points I mentioned are like I guess. In the worst case, I'd have to give up counting on speed-up by -ae.

Thank you for your answers, CU Lars.

Hi!

Once again, maybe I misunderstood, maybe I just wrote a misleading description what I wanted to achieve by including the material definitions (at least the colorpict modifier's definition) into each object file.

I have radiance-files describing geometrical objects, like e.g. a plane of marble. This "object" is placed by replmarks, which brings it to its place moving and rotating it in space. Now, as the object is made of marble in my example, it also has a surface structure which is to be represented by a picture, used by the colorpict modifier. Both, geometry and image, are placed at (0;0;0) on the x-y-plane originally, and translated by replmarks.

So far, I either used instances with replmarks or included the object geometry with the colorpict modifier's definition. As such, the xform operations performed by replmarks were also applied to the colorpict.

Now, having only the alias definition left in the object file, the image map is not translated together with the object itself any more. So the object moves somewhere in space, while the image map, staying at (0;0;0) on xy, visually disappears because it no longer touches its geometry.

Does that mean that I have to stay with the redundant definition of colorpict modifiers? Using only instances gives bad overhead with such simple geometry, objects with only the alias included don't show the mapped image when moved or rotated. Using object geometry with included colorpict definition gives my the "ambient list" message that I initially described when beginning this thread. For future projects: using u/v with mesh primitives would prevent all that trouble, right?

So for now, I think I will have to stay with the redundant material definitions included in object (geometry) files.

Thank you for all the help, CU Lars.

Hi Lars,

If you don't use a material or set of materials for an object, then including the unused material is a waste of space. Thus, including a whole material library with each object, which uses at most a few of these materials, is a huge tax on memory during rendering. Don't do it. Find another way, even if it's less convenient.

My earlier suggestion was to include the colorpict in each object file as needed to move coordinates around with your geometry, and alias'ing the material the colorpict is applied to. That way, you duplicate only what is needed for correct rendering, and can still use a library. The alias also takes up space, but not as much as a whole library of materials. It is about the same as putting just the materials used in each geometry file. Giving each alias a unique name also avoids the generation of excess id's with the -ae option you are currently experiencing. Each unique modifier name generates as many id's as there are appearances in your scene. (Instancing a mesh or octree counts as a single appearance, where xform'ing counts for multiple appearances.)

If you can avoid duplicating your entire material library with each object, this should solve all your problems.

-Greg

Hi Greg.

I followed your suggestion and used sed to replace all the modifiers in my cad files. So marble_grey in object plane_1 is renamed to plane_1_marble_grey now. However, there is one question left (in fact I didn't really understand the overflow condition I am causing). If I reduce the modifiers that will be included in all the object files to a lower number (in fact, only the mappings are included into each, as they are used in most cases). I wonder if I will get the "too many modifiers" if I have e.g. three modifiers defined in all object files. Is the overrun depending on the total number of modifier definitions, or is it the limit that ONE modifier can't be defined more than the fixed number? The latter means that I have to change a lot, as I would have to examine the cad files also for the most common colorpict modifiers (which I include into each object file so far). From what I understood, I am afraid that will be necessary, but before spending some hours on this, I would like to understand the nature of this limit of modifier ids.

TIA+CU. Lars.

If you can avoid duplicating your entire material library with each object, this should solve all your problems.

(This is the key point - I don't duplicate the entire material library with each object any more now, but still some modifiers of the library.)

Hi Lars,

As long as your modifiers are named differently, you can have as many as you like and use the -ae option successfully. There is no limit to the number of modifiers or the reuse of modifier names in Radiance, but if a modifier appears hundreds of times with the same name, the -ae option takes the one name and translates it into a hundred instances of that name as necessary to form the exclusion set.

Hopefully, what you have done already should suffice.

-Greg

P.S. I will be out of town and out of contact for the next week, so if you write with a question and get no response, I'm not ignoring you. I'm just gone.

···

From: Lars O. Grobe <[email protected]>
Date: July 15, 2005 7:12:54 AM PDT

Hi Greg.

I followed your suggestion and used sed to replace all the modifiers in my cad files. So marble_grey in object plane_1 is renamed to plane_1_marble_grey now. However, there is one question left (in fact I didn't really understand the overflow condition I am causing). If I reduce the modifiers that will be included in all the object files to a lower number (in fact, only the mappings are included into each, as they are used in most cases). I wonder if I will get the "too many modifiers" if I have e.g. three modifiers defined in all object files. Is the overrun depending on the total number of modifier definitions, or is it the limit that ONE modifier can't be defined more than the fixed number? The latter means that I have to change a lot, as I would have to examine the cad files also for the most common colorpict modifiers (which I include into each object file so far). From what I understood, I am afraid that will be necessary, but before spending some hours on this, I would like to understand the nature of this limit of modifier ids.

TIA+CU. Lars.

If you can avoid duplicating your entire material library with each object, this should solve all your problems.

(This is the key point - I don't duplicate the entire material library with each object any more now, but still some modifiers of the library.)

Hi,

this is a really late follow-up...

I am having still these problems from time to time, and while I can debug that, I am curious about the consequences of this overflow.

When one modifier has been "instanced" too many times, what effect does this have for the exclude list? Will all the "instances" of the modifier up to the limit still be excluded? Will the modifier who has been instanced too often be not excluded at all, but other modifiers which are in the range still get excluded when on the exclude list? Or will the whole exclude list be ignored if one modifier causes the overrun?

TIA+CU, Lars.

···

As long as your modifiers are named differently, you can have as many as you like and use the -ae option successfully. There is no limit to the number of modifiers or the reuse of modifier names in Radiance, but if a modifier appears hundreds of times with the same name, the -ae option takes the one name and translates it into a hundred instances of that name as necessary to form the exclusion set.

Hi Lars,

I had to check the code, myself. Up to the limit (2047 in the current release), you can add modifiers to include or exclude, counting each once per appearance on the input. (Actual instances containing materials count only once per, but repeating the same material name over and over does count.) After the limit is reached, a warning is issued and further modifiers that should be on the list are silently ignored. It still counts the ones up until that point.

As for Terry's question, your answer is correct about the #@mkillum lines on the input. You cannot have different rtrace settings on a per-illum basis, however. You can have different ones for mkillum than the rest of the process, by using a different setting for render= and mkillum= in the rad input file.

Hope this clears things up.
-Greg

···

From: "Lars O. Grobe" <[email protected]>
Date: February 20, 2007 1:30:19 AM PST

Hi,

this is a really late follow-up...

I am having still these problems from time to time, and while I can debug that, I am curious about the consequences of this overflow.

When one modifier has been "instanced" too many times, what effect does this have for the exclude list? Will all the "instances" of the modifier up to the limit still be excluded? Will the modifier who has been instanced too often be not excluded at all, but other modifiers which are in the range still get excluded when on the exclude list? Or will the whole exclude list be ignored if one modifier causes the overrun?

TIA+CU, Lars.

As long as your modifiers are named differently, you can have as many as you like and use the -ae option successfully. There is no limit to the number of modifiers or the reuse of modifier names in Radiance, but if a modifier appears hundreds of times with the same name, the -ae option takes the one name and translates it into a hundred instances of that name as necessary to form the exclusion set.