Depth-of-field hack

Radiance users,

Recently, I have been playing around with several Radiance
programs that I never would ordinarily use, and I devised a new
way to create depth-of-field blur. My primary motivation is
for creating very large images of very complex scenes. I
want to thank Greg for making time to meet me at SIGGRAPH,
and for getting me started on this line of thinking.

I have been creating images by using -aa 0 with -ps 1 for
over a year now. In a little previous work
(http://mark.technolope.org/radmisc/aa0_ps1_test/final.html)
I found that the fastest renderings could be had with
very low -ad but high oversampling. I wanted to find a
method for depth-of-field blur that worked similarly,
and would be compatible with this method.

I first tried pdfblur, which outputs a series of views, each
of which can be rendered with rpict or created with pinterp.
The pinterp method is definitely faster, but, as expected,
the pixels that were in focus were never recalculated, and,
thus, remained spotty:
http://mark.technolope.org/radiance/dof_tests/img06_30.png
(-ad 4, pdfblur 0.003 0.4 30, reduced 4:1)

Using pdfblur with rpict, though, forced resampling of each
pixel, so it took longer, but I didn't have to reduce the
final image at all to get this:
http://mark.technolope.org/radiance/dof_tests/img03_30.png
(-ad 4, pdfblur 0.003 0.4 30, not reduced)

The problem that I see, though, is that some large images will
need hundreds, if not thousands, of images blurred together
to eliminate the sharp lines in the highly-blurred regions,
while the oversampling technique rarely needs more that 36-64
original pixels to each output one (6-8x oversampling).

That's when I started reading lots of man pages. I wanted a way
to simulate DOF within a single image. What I came up with could
be used as an option to rpict, where it could be implemented more
accurately, I presume, (and which I may just do sometime). It
consists of a rather long command-line using vwrays (to create
each pixels' view ray), vwright (to calculate numerous view
parameters), rcalc (to adjust the view point and direction of
each), and rtrace (to take that information and compute a pixel
value).

My first try only supported aperture specification, but set
the focus distance to infinity. I then added a slightly-
incorrect hack to point the view direction back toward the
original view-at point. This is the script:

vwrays -fd -vf vp -x 512 -y 512 | rcalc -id6 -e 'a:0.003;d:0.4'
-e `vwright i < vp`
-e 'theta=2*PI*rand(2*recno-1);r=0.5*a*sqrt(rand(2*recno))'
-e 'r1=r*cos(theta);r2=r*sin(theta)'
-e 'dx=r1*ihx+r2*ivx;dy=r1*ihy+r2*ivy;dz=r1*ihz+r2*ivz'
-e '$1=ipx+dx;$2=ipy+dy;$3=ipz+dz'
-e '$4=$4-dx/d;$5=$5-dy/d;$6=$6-dz/d'
-od | rtrace -fdc -x 512 -y 512 @opts scene5f.oct > img04.pic

The "a:0.003;d:0.4" line specifies aperture and focus distance.
The "vp" file looks like this:
rview -vta -vp 0.51 0.52 0.34 -vd -0.43427 -0.86854 0.238848 -vu
0 0 1 -vh 70 -vv 70 -vo 0 -va 0 -vs 0 -vl 0
The opts file looks like this:
-ab 2 -ad 4 -aa 0 -as 0 -dj 0.7 -ms 0.03
The resolution gets specified in two places on the command-line.

At 512x512, this produced the following image:
http://mark.technolope.org/radiance/dof_program/img04.png

And at 4096x4096, reduced to 512x512 (aperture was not
halved in radius calculation, thus appears more blurry):
http://mark.technolope.org/radiance/dof_program/img05q.png

And that's what I wanted to share. I hope someone finds it
useful. I especially hope that someone improves on it. Right
now it is less useful to me, as I cannot incorporate it
into an rpiece batch job, or restart it if it dies.

Mark
[email protected]

Hi Mark

...

I first tried pdfblur, which outputs a series of views, each
of which can be rendered with rpict or created with pinterp.

I'm not quite sure I see the light here yet, but I'm interested, as rshow had some experimental features of generating the views (actually before realizing there are some cmdline tools buried deeply somewhere), that's how the lower image at http://www.pab-opto.de/radiance/render_vergleich/intro.htx was rendered with 16 single views.

Before decompiling the rcalc script, could you briefly say why your method is faster ?

To give a hint of the little bits I do understand: Out-of-focus pixels average over a larger solid angle as in-focus ones. Even the solid-angle of in-focus pixels is truly not zero, that's why we need oversampling to avoid aliasing. Traditional methods calculate anti-aliased images with shifted view points and directions which are than added to give the blurred final result. Do I understand correctly that your method incorporates the depth-of-field information into the per-pixel averaging over the per-pixel-solid-angle ? Speed advantage would than come by avoiding unnecessary rays. Is there an easy way to understand how ?
With the old method, each of the images has the same size, so one pixel is effectively oversampled by the number of images. Whether in-focus or not. Are you saying that the out-of-focus ones need more sampling than the in-focus ones ? And the in-focus don't gain much from the higher sampling ?

TIA
Peter

PS: nice Radiance webpages of yours

···

--
pab-opto, Freiburg, Germany, http://www.pab-opto.de
[see web page to check digital email signature]

Peter,

Thank you for your comments. I hope the following response
will be helpful.

>
>I first tried pdfblur, which outputs a series of views, each
>of which can be rendered with rpict or created with pinterp.
>
I'm not quite sure I see the light here yet, but I'm interested, as
rshow had some experimental features of generating the views (actually
before realizing there are some cmdline tools buried deeply somewhere),
that's how the lower image at
http://www.pab-opto.de/radiance/render_vergleich/intro.htx was rendered
with 16 single views.

Right, that seems to be the method that pdfblur uses: creates a
number of different views, each of which you need to create using
either rpict or pinterp.

Before decompiling the rcalc script, could you briefly say why your
method is faster ?

It is faster for a certain class of problems: very
high-resolution images with significant depth-of-field effects.
Its advantage lies in the fact that no sharp renderings are ever
created---those sharp renderings cause the wispy sharp lines in
the image you refer to, and one image that I linked to.

The rcalc script borrows heavily from the rcalc expression
inside of the pdfblur script. Where the pdfblur script applies
depth-of-field blur on an image-wise basis, my command-line
version applies it on a pixel-by-pixel basis.

To give a hint of the little bits I do understand: Out-of-focus pixels
average over a larger solid angle as in-focus ones. Even the solid-angle
of in-focus pixels is truly not zero, that's why we need oversampling to
avoid aliasing. Traditional methods calculate anti-aliased images with
shifted view points and directions which are than added to give the
blurred final result. Do I understand correctly that your method
incorporates the depth-of-field information into the per-pixel averaging
over the per-pixel-solid-angle ? Speed advantage would than come by
avoiding unnecessary rays. Is there an easy way to understand how ?

Using pdfblur with pinterp, you *do* only resample pixels in
areas where the blur effect is significant, thus saving time.
This method, though, is not compatible with the "-aa 0 -ps 1"
method that I presented earlier. This is because using -aa 0
with a very low -ad value (4 or 8) gives very noisy images,
and the entire image must always be oversampled in order to
obtain smooth final images. With the understanding that the
entire image needed to be resampled many times, I wanted a
DOF method that would be noisy on a pixel-basis, but that would
smooth out when the image size was filtered down.

With the old method, each of the images has the same size, so one pixel
is effectively oversampled by the number of images. Whether in-focus or
not. Are you saying that the out-of-focus ones need more sampling than
the in-focus ones ? And the in-focus don't gain much from the higher
sampling ?

This is true for non-noisy images (with high settings for
-ad and -as), and it the aim of pdfblur with pinterp, but
is not the case for the very-high-resolution, noisy images
that dominate my recent work. The most recent production
image that I made was at 24,000^2 resolution, and used
rather low-quality ambient value settings. After appropriate
size reductions, a very fine quality 7200^2 image resulted.

Note that because every pixel is resampled, I could very
easily have rendered a larger number of images at the
final resolution (7200^2) instead of one very large one.
I may shift to that method in the future.

PS: nice Radiance webpages of yours

Thanks! It's taken me a bit of time to assemble the material that
is there, but there is ever more that I'd like to share. Within
the next half-year, I hope to have a website up for my digital
visual art, most of it created with Radiance. I'll be sure to
notify the list when that happens.

Mark

···

On Mon, 16 Aug 2004, Peter Apian-Bennewitz wrote:

PS: nice Radiance webpages of yours

Thanks! It's taken me a bit of time to assemble the material that
is there, but there is ever more that I'd like to share. Within
the next half-year, I hope to have a website up for my digital
visual art, most of it created with Radiance. I'll be sure to
notify the list when that happens.

Something to look forward to... The image you had hanging at SIGGRAPH was amazing!

-Greg