multi-processor rendering

Several years ago I tried out rpiece but found it wasn't always reliable - at least not on the system I had been using. I found this following vwrays and rtrace command sequence to work well to get around this. This is for creating a few renderings of a single static scene (for example electric lighting, or single daylight condition). Of course there would be additional ambient parameters on the command line. So I have two questions in addition to opening this to general critique:

1. Has rpeice changed much over the past few years, and would it have any processing speed benefits if I were to try it again?
2. Would using rcontrib offer any processing speed benefits? (I started that way but then realized I should gain speed using the ambient file for rtrace)

Example successful command sequence using 6 processors:

view1="-x 4096 -y 2160 -vf view1.vf"
vwrays -ff $view | rtrace -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1.hdr
vwrays -ff $view | rtrace -i -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1-i.hdr

Thanks for all feedback.
Chris

Hi Chris,

If you would use -ps 1 on the command line of rpiece because you are doing specular sampling or setting -dj .7 (or whatever) to calculate penumbras, then your rtrace command will be just as fast as rpiece. The main advantage of rpiece on large images is the image plane sampling it does. Setting -ps 4 (the default) samples only 1 in 16 pixels to start, then subsamples in areas where it detects something going on. This can be a big time-saver, especially in scenes with a lot of smoothly shaded areas.

The rad command will run rpiece for you if you set the -N option to the number of processes and the -v option to specify a single view you want rendered. This is a much easier way to operate, and should be entirely reliable under Mac OS X and most versions of Linux. You can run your own comparison against rtrace with vwrays and report back. I'd be interested to hear what you find.

One more thing about rtrace is that it doesn't put your view in the header of the output, so if you want to use that output with programs that need the view, you will have to re-insert it. The latest version of getinfo makes this a little easier, e.g.:

  vwrays ... | rtrace ... | getinfo -a "VIEW= $view" > view1.hdr

Cheers,
-Greg

···

From: Christopher Rush <[email protected]>
Date: October 6, 2016 9:08:37 AM PDT

Several years ago I tried out rpiece but found it wasn’t always reliable – at least not on the system I had been using. I found this following vwrays and rtrace command sequence to work well to get around this. This is for creating a few renderings of a single static scene (for example electric lighting, or single daylight condition). Of course there would be additional ambient parameters on the command line. So I have two questions in addition to opening this to general critique:

1. Has rpeice changed much over the past few years, and would it have any processing speed benefits if I were to try it again?
2. Would using rcontrib offer any processing speed benefits? (I started that way but then realized I should gain speed using the ambient file for rtrace)

Example successful command sequence using 6 processors:

view1="-x 4096 -y 2160 -vf view1.vf"
vwrays -ff $view | rtrace -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1.hdr
vwrays -ff $view | rtrace -i -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1-i.hdr

Thanks for all feedback.
Chris

Great suggestion. I will test. I think I had tried the -N option with rad but didn't find it using multiple processors when just one view is in the .rif file (if my memory serves correctly). I didn't think to try specifically calling for just the one view along with the -N option.

-Chris

···

From: Greg Ward [mailto:[email protected]]
Sent: Thursday, October 06, 2016 12:32 PM
To: Radiance general discussion
Subject: Re: [Radiance-general] multi-processor rendering

Hi Chris,

If you would use -ps 1 on the command line of rpiece because you are doing specular sampling or setting -dj .7 (or whatever) to calculate penumbras, then your rtrace command will be just as fast as rpiece. The main advantage of rpiece on large images is the image plane sampling it does. Setting -ps 4 (the default) samples only 1 in 16 pixels to start, then subsamples in areas where it detects something going on. This can be a big time-saver, especially in scenes with a lot of smoothly shaded areas.

The rad command will run rpiece for you if you set the -N option to the number of processes and the -v option to specify a single view you want rendered. This is a much easier way to operate, and should be entirely reliable under Mac OS X and most versions of Linux. You can run your own comparison against rtrace with vwrays and report back. I'd be interested to hear what you find.

One more thing about rtrace is that it doesn't put your view in the header of the output, so if you want to use that output with programs that need the view, you will have to re-insert it. The latest version of getinfo makes this a little easier, e.g.:

                vwrays ... | rtrace ... | getinfo -a "VIEW= $view" > view1.hdr

Cheers,
-Greg

From: Christopher Rush <[email protected]<mailto:[email protected]>>

Date: October 6, 2016 9:08:37 AM PDT

Several years ago I tried out rpiece but found it wasn't always reliable - at least not on the system I had been using. I found this following vwrays and rtrace command sequence to work well to get around this. This is for creating a few renderings of a single static scene (for example electric lighting, or single daylight condition). Of course there would be additional ambient parameters on the command line. So I have two questions in addition to opening this to general critique:

1. Has rpeice changed much over the past few years, and would it have any processing speed benefits if I were to try it again?
2. Would using rcontrib offer any processing speed benefits? (I started that way but then realized I should gain speed using the ambient file for rtrace)

Example successful command sequence using 6 processors:

view1="-x 4096 -y 2160 -vf view1.vf"
vwrays -ff $view | rtrace -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1.hdr
vwrays -ff $view | rtrace -i -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1-i.hdr

Thanks for all feedback.
Chris

Well, this feature is not exactly "new" anymore, but rad didn't always manage rpiece for you. I added this facility back in 2011. It should work the same if there is only one view in the file as when you use the -v option to specify a single view.

Cheers,
-Greg

···

From: Christopher Rush <[email protected]>
Date: October 7, 2016 6:20:36 AM PDT

Great suggestion. I will test. I think I had tried the -N option with rad but didn’t find it using multiple processors when just one view is in the .rif file (if my memory serves correctly). I didn’t think to try specifically calling for just the one view along with the -N option.

-Chris

From: Greg Ward [mailto:[email protected]]
Sent: Thursday, October 06, 2016 12:32 PM
To: Radiance general discussion
Subject: Re: [Radiance-general] multi-processor rendering

Hi Chris,

If you would use -ps 1 on the command line of rpiece because you are doing specular sampling or setting -dj .7 (or whatever) to calculate penumbras, then your rtrace command will be just as fast as rpiece. The main advantage of rpiece on large images is the image plane sampling it does. Setting -ps 4 (the default) samples only 1 in 16 pixels to start, then subsamples in areas where it detects something going on. This can be a big time-saver, especially in scenes with a lot of smoothly shaded areas.

The rad command will run rpiece for you if you set the -N option to the number of processes and the -v option to specify a single view you want rendered. This is a much easier way to operate, and should be entirely reliable under Mac OS X and most versions of Linux. You can run your own comparison against rtrace with vwrays and report back. I'd be interested to hear what you find.

One more thing about rtrace is that it doesn't put your view in the header of the output, so if you want to use that output with programs that need the view, you will have to re-insert it. The latest version of getinfo makes this a little easier, e.g.:

                vwrays ... | rtrace ... | getinfo -a "VIEW= $view" > view1.hdr

Cheers,
-Greg

From: Christopher Rush <[email protected]>
Date: October 6, 2016 9:08:37 AM PDT

Several years ago I tried out rpiece but found it wasn’t always reliable – at least not on the system I had been using. I found this following vwrays and rtrace command sequence to work well to get around this. This is for creating a few renderings of a single static scene (for example electric lighting, or single daylight condition). Of course there would be additional ambient parameters on the command line. So I have two questions in addition to opening this to general critique:

1. Has rpeice changed much over the past few years, and would it have any processing speed benefits if I were to try it again?
2. Would using rcontrib offer any processing speed benefits? (I started that way but then realized I should gain speed using the ambient file for rtrace)

Example successful command sequence using 6 processors:

view1="-x 4096 -y 2160 -vf view1.vf"
vwrays -ff $view | rtrace -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1.hdr
vwrays -ff $view | rtrace -i -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1-i.hdr

Thanks for all feedback.
Chris
_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Reporting back to all:

Greg is correct, but with some caveats in my opinion. My preferred rendering parameters did include -dj 0.7 or whatever. Actually rad set -dj 0.9 without me noticing, and I applied -dj 0.6 in my script using rtrace. The rad command set -ps 2 and saved 5% of the calculation time using 3 processors compared to nearly the same settings with vwrays and rtrace. The rad command also did 2x oversampling in that 5% less time, so I admit it's not an apples-to-apples comparison. However I found much better rendering quality (in my opinion) using vwrays with rtrace for the slightly extra time, despite the lack of oversampling. You can see a partial comparison in the attached cropped partial image close-ups - which were processed with a mix of pcond options before compressing and cropping.

The scene is a complicated curving opaque reflective ceiling geometry with IES files placed amongst it. I've cropped that part of the image out for the sake of small files and not having to ask anyone for permission to share in-process design images. On the old machine I was using, took a little under 4 days when limiting to only 3 processors (to run both tests simultaneously).

In the vwrays and rtrace result there's none of the subdivision artifacts that I saw from rpiece. And I saw better ambient calculation consistency with vwrays-rtrace even though I tweaked the rad settings until the ambient parameters were approximately the same as my script. The rad command gives a crisper image within that rendering time because of the extra oversampling. The aliasing and slight blur isn't objectionable in the full rendering in my opinion, although I admit hard to judge from the snippets I'm sharing.

I tested also without the -dj setting (penumbras = false) for sake of time comparison even though I already knew the results weren't up to the quality I was looking for. In this case, rad and rpiece set -ps 4 and the calculation time was 20% less than with vwrays and rtrace. Obviously the shadow results were less accurate, but I also saw some of the same rendering artifacts as in the attached snippets.

My guess is that the time savings of rad might increase with additional processors due to compounding of the pixel processing efficiency. But I'm not which settings I'd need to tweak to get an equal rendering quality, and might always be able to see hints of the rpiece segments.

By the way, I had noted in a different email on this thread that I wasn't getting the expected rpiece behavior from rad. I discovered that was due to an outdated copy of rad from 2010 accidentally mixed into the set of binaries on that machine.

-Chris

···

From: Greg Ward [mailto:[email protected]]
Sent: Thursday, October 06, 2016 12:32 PM
To: Radiance general discussion
Subject: Re: [Radiance-general] multi-processor rendering

Hi Chris,

If you would use -ps 1 on the command line of rpiece because you are doing specular sampling or setting -dj .7 (or whatever) to calculate penumbras, then your rtrace command will be just as fast as rpiece. The main advantage of rpiece on large images is the image plane sampling it does. Setting -ps 4 (the default) samples only 1 in 16 pixels to start, then subsamples in areas where it detects something going on. This can be a big time-saver, especially in scenes with a lot of smoothly shaded areas.

The rad command will run rpiece for you if you set the -N option to the number of processes and the -v option to specify a single view you want rendered. This is a much easier way to operate, and should be entirely reliable under Mac OS X and most versions of Linux. You can run your own comparison against rtrace with vwrays and report back. I'd be interested to hear what you find.

One more thing about rtrace is that it doesn't put your view in the header of the output, so if you want to use that output with programs that need the view, you will have to re-insert it. The latest version of getinfo makes this a little easier, e.g.:

                vwrays ... | rtrace ... | getinfo -a "VIEW= $view" > view1.hdr

Cheers,
-Greg

From: Christopher Rush <[email protected]<mailto:[email protected]>>

Date: October 6, 2016 9:08:37 AM PDT

Several years ago I tried out rpiece but found it wasn't always reliable - at least not on the system I had been using. I found this following vwrays and rtrace command sequence to work well to get around this. This is for creating a few renderings of a single static scene (for example electric lighting, or single daylight condition). Of course there would be additional ambient parameters on the command line. So I have two questions in addition to opening this to general critique:

1. Has rpeice changed much over the past few years, and would it have any processing speed benefits if I were to try it again?
2. Would using rcontrib offer any processing speed benefits? (I started that way but then realized I should gain speed using the ambient file for rtrace)

Example successful command sequence using 6 processors:

view1="-x 4096 -y 2160 -vf view1.vf"
vwrays -ff $view | rtrace -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1.hdr
vwrays -ff $view | rtrace -i -w -n 6 -af scene.amb -ffc $(vwrays -d $view) scene.oct > view1-i.hdr

Thanks for all feedback.
Chris