alias handling - the infernal internals

I must admit that I'm still not completely certain about what the
actual proposal really is. We just jumped into the middle of an
ongoing discussion here. Peter, could you give a short and
complete outline for everybody?

ok. I really should have been more lengthy at my first posting, sorry for
that. Maybe I'll start with internal bits (it's a developer list, after
all. Please correct me if I'm astray) to sharpen my description ( "use the
source, Luke!" ) :

All Radiance objects are fed into an internal structure defined in
common/object.h :

     typedef struct {
             OBJECT omod; /* modifier number */
             short otype; /* object type number */
             char *oname; /* object name */
             FUNARGS oargs; /* object specification
     */
             char *os; /* object structure */
     } OBJREC;

where an OBJECT is an integer, which is converted by the macro objptr()
to an OBJREC pointer.
By starting with the OBJREC struture of the geometric primitive and
following omod the material description is decoded. There may be
branches (mixfunc et al), and both leaves could be followed by
modifier(org->oargs.sarg[0]) . Of course it can,- that's what happens at
render time. Other programs can do that too, e.g. displaying the full tree
of materials for one geometric primitive.

Alias handling is exceptional, as "alias" is decoded only once while the
rad file is read ( common/readobj.c , around line 117):
It looks up an existing primitive with the target name of the alias, and
fills the already allocated OBJREC structure with the type and arguments of
the target primitive. The name of the target primitive is lost. Effectively
we have a copy of the target primitive in memory, under a new name.
Assuming that's exactly what Greg wanted, I would call this a fairly well
working , flawless, solid tested, straight forward implementation. No
doubt.
But the information about the name of the target material for the alias is
gone for good.
Which leaves us with an octree ready for rendering, but no way of telling
what material had been used from a library of materials. Judging by Jack's
comments too, that is not an uncommon situation.

If we would have an alias primitive in the sense of common/otypes.h , we
could put the alias information itself in an OBJREC strcture and follow
the omod link as we do with other materials (e.g. textures and patterns).
That would work regardless of the different "alias" syntax in the rad file.

Advantage: full information of original rad file is conserved and clearity
of what is used in the octree is gained.
Disadvantage: One omod lookup more. Contrary to my first posting, there
wouldn't even be a memory increase.

Currently rtrace/rview return the first modifier of the modifier chain,
which is the name of the alias as of now. With a new option to rtrace, it
could return the first non-alias modifier of the chain, in case there are
aliases. So the user could get both the alias name and the original name.
The above assumes that the material library doesn't use alias in itself and
would allow the user recursive aliases (for whatever reasons) in his scene
file. Alternatively, we could output the first modifier after the alias.
Whatever we output, we have the chance of displaying the information since
it would be in the octree. Whereas now, it isn't.

-Peter

PS: I'll sit back, have some wine and wait how many postings I've missed
while typing this one.....

···

--
pab-opto, Freiburg, Germany, www.pab-opto.de

Peter A-B writes:

If we would have an alias primitive in the sense of common/otypes.h , we
could put the alias information itself in an OBJREC strcture and follow
the omod link as we do with other materials (e.g. textures and patterns).
That would work regardless of the different "alias" syntax in the rad file.

If we would have an alias primitive in the sense of common/otypes.h , we
could put the alias information itself in an OBJREC strcture and follow
the omod link as we do with other materials (e.g. textures and patterns).
That would work regardless of the different "alias" syntax in the rad file.

We would still have to store the target ID in a separate string argument, or else we'd lose the ability to change the modifier chain, which was one of the principal purposes I had in mind for aliases. This then requires a lastmod() to identify the target material at runtime, which can take some time if the target was a long ways back in the input stream.

This gets a bit convoluted, but we could use a string argument in the case when we're replacing the modifier in the change, and a straight link with the OBJREC omod member in the more common case where we are not. That way, the common case wouldn't require any extra memory or take any extra time during rendering. Following the chain in rshow then requires an extra check for the presence of string arguments to determine which modifier was aliased, but this also makes it explicit when the modifier chain was altered or not.

-Greg

Greg Ward wrote:

Peter A-B writes:
> If we would have an alias primitive in the sense of common/otypes.h
> , we
> could put the alias information itself in an OBJREC strcture and follow
> the omod link as we do with other materials (e.g. textures and
> patterns).
> That would work regardless of the different "alias" syntax in the rad
> file.

We would still have to store the target ID in a separate string
argument, or else we'd lose the ability to change the modifier chain,

why not put the modifier number in iargs ?

     oldmod_string ALIAS newmod_string oldbase_string

is converted to

     newmod_orec->omod = oldbase_number
     newmod_orec->oname = newmod_string
     newmod_orec->oargs->iargs[0] = oldmod_id

so in the common case of oldmod_string="void" we set niargs=0 during octree
generation and then, during rendering, there's just the test for niargs==0 .

If niargs==1, follow oldmod_id without string lookup.

Peter

···

--
pab-opto, Freiburg, Germany, www.pab-opto.de

Peter A-B writes:

why not put the modifier number in iargs ?

     oldmod_string ALIAS newmod_string oldbase_string

is converted to

     newmod_orec->omod = oldbase_number
     newmod_orec->oname = newmod_string
     newmod_orec->oargs->iargs[0] = oldmod_id

This would indeed be handy if IARGS were defined in the source, which it is not. Defining it would not only create a problem with octree format compatibility, it would increase scene storage size by 8 bytes/object, whether alias'es are present or not. I don't think it's worth it.

The solution I suggested, where the modifier chain is used in the normal case where we are not changing it with the alias, won't take any extra memory and won't incur a lookup during rendering. Only if the modifier was changed by the alias will we have to call lastmod(), and this should be a small minority of the cases. I don't see why a CAD translator would ever use this feature, for example. In cases where the user is playing with modifier chains, they would hopefully do so all in the same materials file, in which case the cost of calling lastmod() would be minimal, as the modifier is only a few objects away from the alias in the input. We could add a note of advice along these lines to the reference manual when we document the new "inherit" alias keyword.

I do not believe that octrees will be invalidated by these changes -- we should be able to maintain backward compatibility. Frozen octrees will certainly not be affected. Since the old aliases occupied an object record just as the new alias will, the object count won't change in a standard octree, so they should be fine as well. Adding new primitive types never affects backward compatibility, since we build a type lookup when reading in an octree. The only thing that causes problems is removing a primitive type that is referenced in an octree, as one might expect.

Jack de Valpine writes:

Another area where alias handling can be a problem is in the specification of a material for exclusion in the ambient calculation. As things currently stand you must pass the name/id that is actually used to modify the geometry, when in fact it might be preferable in some cases to pass the name of the alias, e.g., if I have a material "black" defined I would rather pass this name for exclusion than some exporter derived name applied to the geometry.

Regrettably, such a change would require some extensive coding. Also, at what point in the modifier chain do you stop checking? When you get to a material? What about mixtures of two or more materials? Some people might be using the alias name as a way of controlling what to include or exclude from the ambient calculation. I wouldn't want to mess them up in the process, either.

Unless there are any objections, I'm going to go ahead and implement these changes as I have outlined. I'm also working on the new mesh primitive, which I should be able to finish this week if all goes well. Then, we can do a little testing and package a new release.

-Greg

Greg Ward wrote:

The solution I suggested, where the modifier chain is used in the
normal case where we are not changing it with the alias, won't take any
extra memory and won't incur a lookup during rendering. Only if the
modifier was changed by the alias will we have to call lastmod(), and
this should be a small minority of the cases. I don't see why a CAD
translator would ever use this feature, for example.

The translator as such probably not, but Rayfront uses this all
the time to manipulate modifier trees. As it is now, Rayfront
stores all modifier definitions in one file per project, which
gets loaded first. The primitives in that file all have no
modifier of their own. The modifier trees that are then assigned
to each geometry "object" are composed exclusively through
recursive aliases (I posted a simple example earlier in this
thread).

This system was the only way that I could figure out that gives
the user complete freedom in combining materials and modifiers in
a GUI environment, so I'm not really prepared for any fundamental
changes in how it works.

In cases where
the user is playing with modifier chains, they would hopefully do so
all in the same materials file, in which case the cost of calling
lastmod() would be minimal, as the modifier is only a few objects away
from the alias in the input.

Right now, Rayfront writes the !xform for each scene file and its
associated modifier/alias tree together, which would mean that
the aliases for the last scene file would have the complete scene
ahead of them to search through. I could change this, so that the
aliases come first, and the !xforms get combined together at the
end in one block, which would move the geometry out of the way.
That's not quite as nice for someone looking at the file in an
editor, although nobody should really need to do that...

But I'm still not happy with generating additional name lookups
at runtime at all. Somehow I'm starting to think that using the
same data structure for geometry and modifier primitives might
not be quite such a good idea after all.

In an object oriented architecture, this problem would be solved
by polyformism, which means that the aliases would get a
different brand of oargs, that includes niargs and *iarg. This
is also possible to do in C, just that the language doesn't
explicitly help us to get it right. The disadvantage is that the
routines allocating and freeing those structures need to know
about the difference and treat each instance accordingly.

I actually think that this could still be done without backwards
incompatible changes to the octree format. It should be possible
to resolve the names once when loading the octree (which is a lot
more acceptable than repeatedly during a simulation), so that
niargs and *iarg will only ever exist in memory.

All that probably sounds a lot more complicated than it actually
is, and it would give us the best of both worlds. We could even
go further, and use the same principle to save more memory.
Assume that polygons (and most other primitives) used a version
of oargs that doesn't carry nsargs and **sarg around. That's an
easy gain of several MB of RAM with non-trivial scenes. Ok, so I
said that I'm against adding complexity just for relatively small
memory gains, but it could easily be that all it takes would be
two more typedefs and a few carefully placed casts.

Thoughts? Counter arguments? Flames? :wink:

-schorsch

···

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Schorsch writes:

Only if the
modifier was changed by the alias will we have to call lastmod(), and
this should be a small minority of the cases. I don't see why a CAD
translator would ever use this feature, for example.

The translator as such probably not, but Rayfront uses this all
the time to manipulate modifier trees. As it is now, Rayfront
stores all modifier definitions in one file per project, which
gets loaded first. The primitives in that file all have no
modifier of their own. The modifier trees that are then assigned
to each geometry "object" are composed exclusively through
recursive aliases (I posted a simple example earlier in this
thread).

Unless you are reusing modifier names, the cost of calling lastmod() is quite modest. If you check the code, it first does a hash lookup, which is extremely fast, and only if the modifier has been redefined since the current primitive (in this context, our alias) was loaded will it resort to a linear search. So, I really don't think you'll see much of a performance hit with the suggested solution.

Regarding the questionable wisdom of having a single data structure for modifiers and objects, I tend to agree with you. However, it's really late in the game to making these sorts of fundamental "adjustments" to the source code. If we're going to do that, we may as well chuck the whole scene description language in favor of something more MGF-like. I've given this a lot of thought, though, and I really don't think it would be worth the effort. Besides the nasty compatibility issues, there's just not that much to be gained by moving to a more sensible input structure, and much time to be lost. That's a whole can of worms I don't want to open right now. "Never" would be a good time.

-Greg

I just checked in my change to Radiance alias handling, which also fixed that memory bug I mentioned in freeobjects(). The new module, rt/m_alias.c does the runtime handling, which is essentially free for aliased materials that have the same modifier as the original (either by coincidence or using the explicit "inherit" keyword). The change touched a few other files besides the obvious ones -- it even cleaned up the code a bit in gen/xform.c.

I also checked in the documentation in ray/doc -- the reference manual and man pages. I figure we should start tracking changes on those again as well. I don't think we need to bother about recovering the SCCS revisions, so I won't even ask Peter for that.

-Greg