Instances

Dear users,
Using instance I noticed that if I put -a into the transform field it
doesn`t work: how is it possible?

void instance ceiling
15 oct/namefile.oct -a 10 -t 1 0 0 -a 10 -t 0 1 0 -i 1
0
0

Thanks a lot

Roberto

Supera i limiti: raddoppia la velocità da 10 a 20 Mega! Risparmia con Tutto Incluso: telefono + adsl 20 mega a soli 29,95 € al mese per due anni!SCONTO DI 240 EURO!http://abbonati.tiscali.it/telefono-adsl/prodotti/tc/tuttoincluso/?WT.mc_id=01fw

Hi Roberto,

I do not think that you can do that. Here is one approach that would work:

    assume an octree of material/geometry that is to be used as an
    instance: object.oct
    NOTE this probably should be a "frozen" octree

    define: instance.rad

        void instance object
        4 object.oct -t 0 0 0
        0

    then you can do one of the following:

    define: array.rad

        !xform -a 10 -t 1 0 0 -a 10 -t 0 1 0 instance.rad

    OR

    do: xform -a 10 -t 1 0 0 -a 10 -t 0 1 0 instance.rad > array.rad

Note also that you could use markers and replmarks to achieve deploying an instance object repeatedly.

Regards,

-Jack

···

--
# Jack de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

On 11/23/2010 3:03 PM, [email protected] wrote:

Dear users,
Using instance I noticed that if I put -a into the transform field it
doesn`t work: how is it possible?

void instance ceiling
15 oct/namefile.oct -a 10 -t 1 0 0 -a 10 -t 0 1 0 -i 1
0

Thanks a lot

Roberto

Supera i limiti: raddoppia la velocità da 10 a 20 Mega! Risparmia con Tutto Incluso: telefono + adsl 20 mega a soli 29,95 € al mese per due anni!SCONTO DI 240 EURO!http://abbonati.tiscali.it/telefono-adsl/prodotti/tc/tuttoincluso/?WT.mc_id=01fw

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Dera Roberto and Jack...

I assume this was done to reduce the memory usage...

Jack, you were talking about a frozen octree... would this help saving memory? What would be, memory wise the difference between a frozen octree and a non-frozen one? Or, in other words, why do you suggest it should be a frozen one?

Sorry for so many questions :wink:

Lucio

···

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

Hi Lucio!

Jack, you were talking about a frozen octree... would this help
saving memory? What would be, memory wise the difference between a
frozen octree and a non-frozen one? Or, in other words, why do you
suggest it should be a frozen one?

The difference between a frozen octree and a "regular" octree is, that the frozen one contains all geometry that is only referenced by a regular octree. Thus if you compile a scene before running rtrace or rpict on it, a regular octree is fine, while, if you want to build up a library of octree-objects, you would probably prefer this library to be independent of any other files and thus use frozen octrees. Another advantage of frozen octrees here is that speed.

Cheers, Lars.

Hi Lucio,

Yes, instancing is a way to deploy an object multiple times for the relative memory price of one of the objects. We do this all the time for repetitive objects such as furniture or trees (for example).

It is possible to use xform or replmarks to deploy copies of geometry also (eg files that contain material and geometry definitions). This however will not result in memory saving. To get the memory saving the object needs to be an octree. A frozen octree is self contained whereas a regular (unfrozen octree) contains references to other data/files that must be read in at load time. I believe from an "in-memory" point of view (eg when running) the memory size would be the same. I think that one difference is that frozen octrees will load faster in addition they can be treated as library objects and moved around from project to project without having to move around a lot of other source files.

Another option from a memory saving point of view is to use mesh object (obj2mesh) which is another compiled representation that may even be more compact and can also be treated like an octree instance.

Hope that helps.

Regards,

-Jack

···

--
# Jack de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

On 11/24/2010 4:37 AM, Lucio Boscolo wrote:

Dera Roberto and Jack...
  I assume this was done to reduce the memory usage...
  Jack, you were talking about a frozen octree... would this help saving memory? What would be, memory wise the difference between a frozen octree and a non-frozen one? Or, in other words, why do you suggest it should be a frozen one?
  Sorry for so many questions :wink:
  Lucio
____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Thank you for all the replies sent through and sorry if didn’t thank yet.

I reckon then that my memory issues were due to the ambient file... I always thought it was read from the HD (and this was slowing the process so that a balance between calculation time and disk access had to be made to optimize the calculation time) but apparently it is loaded entirely in memory, which makes sense speed wise and teaches me to always simplify my models.

Can you please confirm my understanding is right?

Thank you once again for your great help!

Lucio Boscolo Mezzopan

···

From: [email protected] [mailto:[email protected]] On Behalf Of Jack de Valpine
Sent: 24 November 2010 12:43
To: Radiance general discussion
Subject: Re: [Radiance-general] Instances

Hi Lucio,

Yes, instancing is a way to deploy an object multiple times for the relative memory price of one of the objects. We do this all the time for repetitive objects such as furniture or trees (for example).

It is possible to use xform or replmarks to deploy copies of geometry also (eg files that contain material and geometry definitions). This however will not result in memory saving. To get the memory saving the object needs to be an octree. A frozen octree is self contained whereas a regular (unfrozen octree) contains references to other data/files that must be read in at load time. I believe from an "in-memory" point of view (eg when running) the memory size would be the same. I think that one difference is that frozen octrees will load faster in addition they can be treated as library objects and moved around from project to project without having to move around a lot of other source files.

Another option from a memory saving point of view is to use mesh object (obj2mesh) which is another compiled representation that may even be more compact and can also be treated like an octree instance.

Hope that helps.

Regards,

-Jack

--

# Jack de Valpine

# president

#

# visarc incorporated

# http://www.visarc.com

#

# channeling technology for superior design and construction

On 11/24/2010 4:37 AM, Lucio Boscolo wrote:

Dera Roberto and Jack...

I assume this was done to reduce the memory usage...

Jack, you were talking about a frozen octree... would this help saving memory? What would be, memory wise the difference between a frozen octree and a non-frozen one? Or, in other words, why do you suggest it should be a frozen one?

Sorry for so many questions :wink:

Lucio
____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________

Radiance-general mailing list

[email protected]<mailto:[email protected]>

http://www.radiance-online.org/mailman/listinfo/radiance-general

Hi Lucio,

I have never heard about a case where ambient data became critical memory-wise... Why do you think that there is a problem with memory in your case? How much memory does the rpict process occupy?

If you have a lot of detail geometry that does not contribute significantly to the diffuse-indirect illumination of your scene, you can use the ambient exclude parameters of rpict/rtrace (-ae and -aE).

If you use too many instances, or if you create them in a wrong way, they will slow down the rendering. Each time a ray hits an instance (passes the bounding cube it fits into), it has to check for collisions with any surface inside that instance. So if you have a lot of overlapping instances with little geometry contained in each, or instances with a bad ration of width/length/height, a lot of unnecessary intersection checks will happen. Instances improve memory efficiency at the cost of rendering time. It is important to balance the two. So only pack compact, high-res geometry into instances.

Cheers, Lars.

13.12.2010 10:49, Lucio Boscolo wrote:

···

Thank you for all the replies sent through and sorry if didn’t thank yet.

I reckon then that my memory issues were due to the ambient file... I
always thought it was read from the HD (and this was slowing the process
so that a balance between calculation time and disk access had to be
made to optimize the calculation time) but apparently it is loaded
entirely in memory, which makes sense speed wise and teaches me to
always simplify my models.

Can you please confirm my understanding is right?

Thank you once again for your great help!

*Lucio Boscolo Mezzopan*

Hi Lucio,

When a Radiance job is running (rpict, rtrace, rvu, etc) the whole scene (octree) is indeed in memory. It is not being loaded from disk as needed. Depending on how your model is made and with what level of detail this can get very big. Where possible and convenient in makes sense to try to put your scenes together in a thoughtful manner. On the other hand scene complexity/size does not necessarily mean longer simulation times (eg, complex exterior scene under sun/sky only lighting).

-Jack

···

--
# Jack de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

On 12/13/2010 5:40 AM, Lars O. Grobe wrote:

Hi Lucio,

I have never heard about a case where ambient data became critical memory-wise... Why do you think that there is a problem with memory in your case? How much memory does the rpict process occupy?

If you have a lot of detail geometry that does not contribute significantly to the diffuse-indirect illumination of your scene, you can use the ambient exclude parameters of rpict/rtrace (-ae and -aE).

If you use too many instances, or if you create them in a wrong way, they will slow down the rendering. Each time a ray hits an instance (passes the bounding cube it fits into), it has to check for collisions with any surface inside that instance. So if you have a lot of overlapping instances with little geometry contained in each, or instances with a bad ration of width/length/height, a lot of unnecessary intersection checks will happen. Instances improve memory efficiency at the cost of rendering time. It is important to balance the two. So only pack compact, high-res geometry into instances.

Cheers, Lars.

13.12.2010 10:49, Lucio Boscolo wrote:

Thank you for all the replies sent through and sorry if didn’t thank yet.

I reckon then that my memory issues were due to the ambient file... I
always thought it was read from the HD (and this was slowing the process
so that a balance between calculation time and disk access had to be
made to optimize the calculation time) but apparently it is loaded
entirely in memory, which makes sense speed wise and teaches me to
always simplify my models.

Can you please confirm my understanding is right?

Thank you once again for your great help!

*Lucio Boscolo Mezzopan*

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Hi Lars,

Sorry again for late answer.

I think it is a memory problem because I monitored it with top and memory is quickly reducing and as soon as it goes under 100 MB or so (when it probably starts swapping to the HD) processors use drops to 0.something %. I've got 8 GB memory and ambient files are big up to 1.5 GB (very high settings). In the worst case the octree is 600 MB.

Unfortunately the detail is all in a shading device (it is actually a transmittance study) so, no chance for using ambient exclude. And there are no overlapping geometries in the single instances and the instances replication is done by xform using the bounding box for spacing. Also care has been taken in taking away vertical faces at the boundaries to avoid overlapping surfaces.

So what I understand from this is that if rpict loads ambient files in memory as well as octrees, in the worst case I will have for each rpict calculation 1.5 + 0.5 + some work memory in use (I don't know how to quantify this). As I cannot run more than 2 processes, the work memory is probably bigger than [7.5(available ram memory) / 3 (max processes before it runs out of memory)] -[1.5+0.5] = 0.5 GB, which makes sense to me.

What do you think?

As Jack was saying I should have organized the model better, but still is good to understand what happened for future use.

Thanks for all your help and Merry Christmas to you both and to all the radiance community :slight_smile:

Lucio Boscolo Mezzopan

···

-----Original Message-----
From: Lars O. Grobe [mailto:[email protected]]
Sent: 13 December 2010 10:41
To: Radiance general discussion
Subject: Re: [Radiance-general] Instances

Hi Lucio,

I have never heard about a case where ambient data became critical memory-wise... Why do you think that there is a problem with memory in your case? How much memory does the rpict process occupy?

If you have a lot of detail geometry that does not contribute significantly to the diffuse-indirect illumination of your scene, you can use the ambient exclude parameters of rpict/rtrace (-ae and -aE).

If you use too many instances, or if you create them in a wrong way, they will slow down the rendering. Each time a ray hits an instance (passes the bounding cube it fits into), it has to check for collisions with any surface inside that instance. So if you have a lot of overlapping instances with little geometry contained in each, or instances with a bad ration of width/length/height, a lot of unnecessary intersection checks will happen. Instances improve memory efficiency at the cost of rendering time. It is important to balance the two. So only pack compact, high-res geometry into instances.

Cheers, Lars.

13.12.2010 10:49, Lucio Boscolo wrote:

Thank you for all the replies sent through and sorry if didn’t thank yet.

I reckon then that my memory issues were due to the ambient file... I
always thought it was read from the HD (and this was slowing the
process so that a balance between calculation time and disk access had
to be made to optimize the calculation time) but apparently it is
loaded entirely in memory, which makes sense speed wise and teaches me
to always simplify my models.

Can you please confirm my understanding is right?

Thank you once again for your great help!

*Lucio Boscolo Mezzopan*

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

Hi Lucio,

If you are executing in parallel on a shared memory machine, are you using the -PP option to rtrace (or rpict)? This can help to reduce memory, especially on jobs where you are loading in a substantial ambient file to start.

Another approach is to abandon ambient caching entirely, which makes sense in scenes where you have a lot of important detail, but what's most important about it is the aggregate behavior. I think your problem falls into that category. Unfortunately, whatever -a* settings you have worked out will have to be rethought when you do this. I should have a good guide for how to set these parameters, but you basically start by setting -aa 0, at which point -af, -ar, and -aw settings become irrelevant (no caching), and the -ad and -as settings should be changed. You also should look at the -lw and -lr settings, because they will have a strong influence over how deep the ray tree is explored.

Axel Jacobs offers a brief treatment of these settings on page 12 of his rtcontrib tutorial, currently linked at:

  http://luminance.londonmet.ac.uk/learnix/docs/rtcontrib_lesson.pdf

It's probably worth a try. I use -aa 0 routinely when modeling blind systems in mkillum, for example.

Best,
-Greg

···

From: Lucio Boscolo <[email protected]>
Date: December 23, 2010 4:18:25 AM PST

Hi Lars,

Sorry again for late answer.

I think it is a memory problem because I monitored it with top and memory is quickly reducing and as soon as it goes under 100 MB or so (when it probably starts swapping to the HD) processors use drops to 0.something %. I've got 8 GB memory and ambient files are big up to 1.5 GB (very high settings). In the worst case the octree is 600 MB.

Unfortunately the detail is all in a shading device (it is actually a transmittance study) so, no chance for using ambient exclude. And there are no overlapping geometries in the single instances and the instances replication is done by xform using the bounding box for spacing. Also care has been taken in taking away vertical faces at the boundaries to avoid overlapping surfaces.

So what I understand from this is that if rpict loads ambient files in memory as well as octrees, in the worst case I will have for each rpict calculation 1.5 + 0.5 + some work memory in use (I don't know how to quantify this). As I cannot run more than 2 processes, the work memory is probably bigger than [7.5(available ram memory) / 3 (max processes before it runs out of memory)] -[1.5+0.5] = 0.5 GB, which makes sense to me.

What do you think?

As Jack was saying I should have organized the model better, but still is good to understand what happened for future use.

Thanks for all your help and Merry Christmas to you both and to all the radiance community :slight_smile:

Lucio Boscolo Mezzopan

Hi Greg,

Thank you so much for your detailed answer (and sorry for being late again exploring it).

-aa method is a good one, but I am a little concerned this will impact a lot on rendering time in a scene like mine in which many ab are required and a very high number of rays (it is a comparison between shading systems to work out transmittances).

The -PP option seems to be what I need, but I am not sure I understand it. Please tell me if the following is correct.

I start an rpict (or rtrace) with a -PP namefile option.

I start other rpict or rtrace wit -PP namefile (the same) and they will share memory (ram?)

Ambient file instead will be shared by using -af and the same ambient file for all the processes.

Is this correct?

I believe -S is not needed with the -PP option, am I right?

Can I share with -PP between rpict and rtrace processes?

Thank you for your help, one of my colleagues run into the same issue today, so it would be very good if we manage to solve this!

Have a nice day!!!

Lucio

···

-----Original Message-----
From: Greg Ward [mailto:[email protected]]
Sent: 23 December 2010 17:33
To: Radiance general discussion
Subject: Re: [Radiance-general] Instances

Hi Lucio,

If you are executing in parallel on a shared memory machine, are you using the -PP option to rtrace (or rpict)? This can help to reduce memory, especially on jobs where you are loading in a substantial ambient file to start.

Another approach is to abandon ambient caching entirely, which makes sense in scenes where you have a lot of important detail, but what's most important about it is the aggregate behavior. I think your problem falls into that category. Unfortunately, whatever -a* settings you have worked out will have to be rethought when you do this. I should have a good guide for how to set these parameters, but you basically start by setting -aa 0, at which point -af, -ar, and -aw settings become irrelevant (no caching), and the -ad and -as settings should be changed. You also should look at the -lw and -lr settings, because they will have a strong influence over how deep the ray tree is explored.

Axel Jacobs offers a brief treatment of these settings on page 12 of his rtcontrib tutorial, currently linked at:

  http://luminance.londonmet.ac.uk/learnix/docs/rtcontrib_lesson.pdf

It's probably worth a try. I use -aa 0 routinely when modeling blind systems in mkillum, for example.

Best,
-Greg

From: Lucio Boscolo <[email protected]>
Date: December 23, 2010 4:18:25 AM PST

Hi Lars,

Sorry again for late answer.

I think it is a memory problem because I monitored it with top and memory is quickly reducing and as soon as it goes under 100 MB or so (when it probably starts swapping to the HD) processors use drops to 0.something %. I've got 8 GB memory and ambient files are big up to 1.5 GB (very high settings). In the worst case the octree is 600 MB.

Unfortunately the detail is all in a shading device (it is actually a transmittance study) so, no chance for using ambient exclude. And there are no overlapping geometries in the single instances and the instances replication is done by xform using the bounding box for spacing. Also care has been taken in taking away vertical faces at the boundaries to avoid overlapping surfaces.

So what I understand from this is that if rpict loads ambient files in memory as well as octrees, in the worst case I will have for each rpict calculation 1.5 + 0.5 + some work memory in use (I don't know how to quantify this). As I cannot run more than 2 processes, the work memory is probably bigger than [7.5(available ram memory) / 3 (max processes before it runs out of memory)] -[1.5+0.5] = 0.5 GB, which makes sense to me.

What do you think?

As Jack was saying I should have organized the model better, but still is good to understand what happened for future use.

Thanks for all your help and Merry Christmas to you both and to all the radiance community :slight_smile:

Lucio Boscolo Mezzopan

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general
____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

Hi Lucio,

Thank you so much for your detailed answer (and sorry for being late again exploring it).

-aa method is a good one, but I am a little concerned this will impact a lot on rendering time in a scene like mine in which many ab are required and a very high number of rays (it is a comparison between shading systems to work out transmittances).

It doesn't necessarily take as long as you might expect, because the ray tree is very sparse after the first bounce (depending on the -lw and -ld settings).

The -PP option seems to be what I need, but I am not sure I understand it. Please tell me if the following is correct.

I start an rpict (or rtrace) with a -PP namefile option.

I start other rpict or rtrace wit -PP namefile (the same) and they will share memory (ram?)

Correct, although new ambient values will not share RAM.

Ambient file instead will be shared by using -af and the same ambient file for all the processes.

This will share the values themselves, which is critical for speed if -aa is not 0, but they will take additional memory in each process.

I believe -S is not needed with the -PP option, am I right?

With rpict, you need to couple -PP with -S. The rpiece program takes care of dividing a single image into pieces if that is your desire.

Can I share with -PP between rpict and rtrace processes?

Regrettably no, you need a separate syncfile for rpict and rtrace, as they are different executables.

Cheers,
-Greg

Sorry Greg,

I feel a bit dumb, but still I'd like to understand this.

If -PP is not sharing RAM, what memory is it sharing?

Thanks for your help again.

Lucio Boscolo Mezzopan
Assistant Designer | Lighting
       
Arup
13 Fitzroy Street London W1T 4BQ United Kingdom
t +44 20 7636 1531 d +44 20 7755 6311
f +44 20 7755 9012
www.arup.com

···

-----Original Message-----
From: Greg Ward [mailto:[email protected]]
Sent: 07 January 2011 23:22
To: Radiance general discussion
Subject: Re: [Radiance-general] Instances

Hi Lucio,

Thank you so much for your detailed answer (and sorry for being late again exploring it).

-aa method is a good one, but I am a little concerned this will impact a lot on rendering time in a scene like mine in which many ab are required and a very high number of rays (it is a comparison between shading systems to work out transmittances).

It doesn't necessarily take as long as you might expect, because the ray tree is very sparse after the first bounce (depending on the -lw and -ld settings).

The -PP option seems to be what I need, but I am not sure I understand it. Please tell me if the following is correct.

I start an rpict (or rtrace) with a -PP namefile option.

I start other rpict or rtrace wit -PP namefile (the same) and they will share memory (ram?)

Correct, although new ambient values will not share RAM.

Ambient file instead will be shared by using -af and the same ambient file for all the processes.

This will share the values themselves, which is critical for speed if -aa is not 0, but they will take additional memory in each process.

I believe -S is not needed with the -PP option, am I right?

With rpict, you need to couple -PP with -S. The rpiece program takes care of dividing a single image into pieces if that is your desire.

Can I share with -PP between rpict and rtrace processes?

Regrettably no, you need a separate syncfile for rpict and rtrace, as they are different executables.

Cheers,
-Greg
_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general
____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

Hi Lucio,

This is somewhat confusing. -PP enables multiple process on the same machine to share memory. What is shared is memory space taken up by the octree. Ambient values are shared via the ambient file (-af), so they are not shared in RAM between processes (eg there is not a message passing mechanism for communicating back and forth between processes). At given points processes will write out the ambient data that has been calculated (by the given process) and held in memory to the ambient file. As processes do this they also read in the ambient values from the file (hope I get this right), so this is the method of sharing ambient terms.

-Jack

···

--
# Jack de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

On 1/10/2011 10:06 AM, Lucio Boscolo wrote:

Sorry Greg,

I feel a bit dumb, but still I'd like to understand this.

If -PP is not sharing RAM, what memory is it sharing?

Thanks for your help again.

Lucio Boscolo Mezzopan
Assistant Designer | Lighting

Arup
13 Fitzroy Street London W1T 4BQ United Kingdom
t +44 20 7636 1531 d +44 20 7755 6311
f +44 20 7755 9012
www.arup.com

-----Original Message-----
From: Greg Ward [mailto:[email protected]]
Sent: 07 January 2011 23:22
To: Radiance general discussion
Subject: Re: [Radiance-general] Instances

Hi Lucio,

Thank you so much for your detailed answer (and sorry for being late again exploring it).

-aa method is a good one, but I am a little concerned this will impact a lot on rendering time in a scene like mine in which many ab are required and a very high number of rays (it is a comparison between shading systems to work out transmittances).

It doesn't necessarily take as long as you might expect, because the ray tree is very sparse after the first bounce (depending on the -lw and -ld settings).

The -PP option seems to be what I need, but I am not sure I understand it. Please tell me if the following is correct.

I start an rpict (or rtrace) with a -PP namefile option.

I start other rpict or rtrace wit -PP namefile (the same) and they will share memory (ram?)

Correct, although new ambient values will not share RAM.

Ambient file instead will be shared by using -af and the same ambient file for all the processes.

This will share the values themselves, which is critical for speed if -aa is not 0, but they will take additional memory in each process.

I believe -S is not needed with the -PP option, am I right?

With rpict, you need to couple -PP with -S. The rpiece program takes care of dividing a single image into pieces if that is your desire.

Can I share with -PP between rpict and rtrace processes?

Regrettably no, you need a separate syncfile for rpict and rtrace, as they are different executables.

Cheers,
-Greg
_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general
____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Hi Jack,

Your description of the behavior of rpict and rtrace with the -PP option is correct, and you essentially captured the reason behind the lack of memory sharing of ambient values. Doing so would require locking and coordination of memory sharing across processes, which is implemented a little differently on different systems. I chose to avoid the differences by taking advantage of the "copy-on-write" behavior of most Unix systems when fork() is called.

When you use the -PP option, rpict (or rtrace) loads in the octree and all the associated, static data structures needed for rendering. It also loads in any indirect irradiance values that have been previously computed and cached in the ambient file. Once that is done, it calls fork() to make a copy of the process and waits for additional connections by other rpict/rtrace commands using the same -PP syncfile. It's a bit awkward, but it works on all modern Unix systems, and requires no further communication among processes once started.

The newer -n option in rvu works similarly, but through a different mechanism. Rather than making multiple copies of the same process that expects input from stdin to tell it what to do, it has a single parent process that communicates via pipes to all its children, which perform the actual ray tracing. It still uses the copy-on-write trick, loading the scene and ambient data prior to forking the child processes, but the whole thing is hidden from the user.

I subsequently implemented the -n option in rtrace as well, which is a bit confusing. In most cases, it's a little more efficient to use rtrace -n than rtrace -PP, and you have to use one or the other. The times you would still choose rtrace -PP are when you need to start the parallel processes at different times, or take the ray input from different sources.

I don't expect to add a -n option to rpict in the forseeable future. There are certain things about the way rpict samples the image plane that make such an option problematic, so we're kind of stuck with -PP for now.

Best,
-Greg

···

From: Jack de Valpine <[email protected]>
Date: January 10, 2011 7:44:50 AM PST

Hi Lucio,

This is somewhat confusing. -PP enables multiple process on the same machine to share memory. What is shared is memory space taken up by the octree. Ambient values are shared via the ambient file (-af), so they are not shared in RAM between processes (eg there is not a message passing mechanism for communicating back and forth between processes). At given points processes will write out the ambient data that has been calculated (by the given process) and held in memory to the ambient file. As processes do this they also read in the ambient values from the file (hope I get this right), so this is the method of sharing ambient terms.

-Jack

Greg, Jack,

Thanks for your explanations, I think I've got it now!

So the RAM memory I can save via -PP is the one taken by the oct.

(hope I haven't misunderstood it again!)

Thanks again! You are great!

Lucio Boscolo Mezzopan
Assistant Designer | Lighting
       
Arup
13 Fitzroy Street London W1T 4BQ United Kingdom
t +44 20 7636 1531 d +44 20 7755 6311
f +44 20 7755 9012
www.arup.com

···

-----Original Message-----
From: Greg Ward [mailto:[email protected]]
Sent: 10 January 2011 16:47
To: Radiance general discussion
Subject: Re: [Radiance-general] Instances

Hi Jack,

Your description of the behavior of rpict and rtrace with the -PP option is correct, and you essentially captured the reason behind the lack of memory sharing of ambient values. Doing so would require locking and coordination of memory sharing across processes, which is implemented a little differently on different systems. I chose to avoid the differences by taking advantage of the "copy-on-write" behavior of most Unix systems when fork() is called.

When you use the -PP option, rpict (or rtrace) loads in the octree and all the associated, static data structures needed for rendering. It also loads in any indirect irradiance values that have been previously computed and cached in the ambient file. Once that is done, it calls fork() to make a copy of the process and waits for additional connections by other rpict/rtrace commands using the same -PP syncfile. It's a bit awkward, but it works on all modern Unix systems, and requires no further communication among processes once started.

The newer -n option in rvu works similarly, but through a different mechanism. Rather than making multiple copies of the same process that expects input from stdin to tell it what to do, it has a single parent process that communicates via pipes to all its children, which perform the actual ray tracing. It still uses the copy-on-write trick, loading the scene and ambient data prior to forking the child processes, but the whole thing is hidden from the user.

I subsequently implemented the -n option in rtrace as well, which is a bit confusing. In most cases, it's a little more efficient to use rtrace -n than rtrace -PP, and you have to use one or the other. The times you would still choose rtrace -PP are when you need to start the parallel processes at different times, or take the ray input from different sources.

I don't expect to add a -n option to rpict in the forseeable future. There are certain things about the way rpict samples the image plane that make such an option problematic, so we're kind of stuck with -PP for now.

Best,
-Greg

From: Jack de Valpine <[email protected]>
Date: January 10, 2011 7:44:50 AM PST

Hi Lucio,

This is somewhat confusing. -PP enables multiple process on the same machine to share memory. What is shared is memory space taken up by the octree. Ambient values are shared via the ambient file (-af), so they are not shared in RAM between processes (eg there is not a message passing mechanism for communicating back and forth between processes). At given points processes will write out the ambient data that has been calculated (by the given process) and held in memory to the ambient file. As processes do this they also read in the ambient values from the file (hope I get this right), so this is the method of sharing ambient terms.

-Jack

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general
____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses