time needed for octree reading (was Re: setting buffer size on octree reads (setvbuf) )

Hi all,

The observed slow reading of frozen octrees happens if and only if:
- the geometric object (polygons in this case) each have their own, different modifier. This is probably rare, but may happen if each polygon uses a texfunc to interpolate the normal.
- each of these modifiers have a unique name
It is only relevant if there are millions of such objects (e.g. tessellated surfaces imported from CAD).

Then the octree reading spends 96% of its time in otndx() (readoct.c), which handles a hash table. I would *guess* (not be field of expertise anyway) that the hash algorithm is slow under these extreme circumstances, since these unique modifier names are only inserted, but never ever re-used. Effectively this exotic scenario screws up the idea of having a hash table.

The solution (thanks Greg) was to use a modifier name for the texfunc that applies to all polygons with the same material, although each texfunc is different. This works since the texfunc is specified just prior to its polygon.
The decrease in loading time is noticeable: 18 versus 148 seconds for 320MB of frozen octrees.

It is worth to note that, in a wider context and from a different perspective, this specific use of texfunc/polygons might be better served with the Radiance mesh primitive, resulting in less file and memory use and with faster reading and rendering.

As a general hint, reading of non-frozen octrees may be slower. Use of frozen octrees is recommended for speed (by Greg).

Regarding my suggestion to increase the stream buffer size for octree readings: The above timings use a 2MB buffer. Likely the 4k standard buffer size doesn't matter much for local discs, although I'm inclined to stick with the 2MB, because of the reduced. number of system calls. It may have a larger influence on NFS mounts, depending on the mount options. The only reason not to change the source is that *any* non-urgent change may introduce unforeseen side effects on some systems. Even if setvbuf seems to be a pretty much standardized call.

Thanks to Greg and the gprof profiler



pab-opto, Freiburg, Germany, http://www.pab-opto.de
[see web page to check digital email signature]