Neutral filter correction

Dear list,
It is recommended to use the neutral filter when sun occurs in the view of the camera. Recently I have performed this experiment to correct the chromatic shift. I have gotten some HDR images captured with and without the Kodak Wratten 3.0 ND filter under an overcast sky. I used the pvalue command in Radiance to get the RGB components of each color patch of the x-rite color chart. However, when I compare the RGB values, it is easily to find their relationship is unreasonable as the magnitude is much smaller than 1000. I haven’t find the reason of this error as far as now. I would really appreciate it if anybody can share any advice or guidance.

pvalue file.hdr > file.txt


Did you use the “-o” option to pvalue, to undo any exposure change in the inputs?

Hi Greg,
I tried the command again with “-o” and compared the results of the RGB. However, their difference can be nearly ignored. And it is weird to get the RGB values with the magnitude of 1/10~1/1000 when not using the filter.
the following are the RGB values of 24 color patches without the filter:
image
Processing: IMG_8726.CR2…
without the filter:

with the filter:

It would help to have some details about how you converted the captures to HDR images. Did you use Photosphere, or hdrgen, or some other method? If the camera response function and luminance calibration are not assigned correctly, this can certainly lead to errors. Remember also that the values reported by “pvalue -o” are in watts/sr/m^2, not cd/m^2. There is a factor of 179 involved in converting from the former to the latter.

Hi, Greg
Well, I used the perl script raw2hdr to merge the raw files to HDR images. 
I captured 7 CR2 images with automatic exposure bracketing.
There is no need to get the response curves of the camera, right?

Besides, I hope to get the correction functions like the following plot:

 the figure is from this article:
 https://doi.org/10.1080/15502724.2019.1684319
------------------ 原始邮件 ------------------

Hi Lee,

I would also expect higher RGB values (in W/sr/m2) for the 24 color patches of the HDR image taken outside without the filter. I guess you did, but did you make sure that the coordinates (x,y) of your color patches are the correct ones in Radiance?

Also, from your first graph, it seems that there is a factor 10 between the RGB of the HDR with the ND filter and those of the HDR without the ND filter. Therefore it seems that you use an ND1 filter instead of an ND3, could that be?

Clotilde

Hi Clotilde,
I am sure that the coordinates of the color patches are right. Cause I know that the origin point in the Radiance is in the bottom left corner.

[quote=“Clotilde_Pierson, post:6, topic:5643”]
Also, from your first graph, it seems that there is a factor 10 between the RGB of the HDR with the ND filter and those of the HDR without the ND filter.
[/quote] That’s where I am puzzled about since I just bought the ND 3.0 filter.

I suppose a quick check is to see if the camera’s automatic exposure changes by the expected 10 EVs (stops) with versus without the filter. (You can use “aperture priority” mode for this test.) If the automatic exposure is only changing by 3 stops, then you definitely have a ND 1 filter rather than an ND 3.

Hi Greg,
I captured two images with and without the filter outside with the aperture of f/11 in the Av mode. As for their shutter speed, one is 4’’ and the other one is 1/320. There are 10 EVs between them so the filter is ND 3.0.

OK, we are getting somewhere. Can you upload your pair of exposure sequences for me to examine more closely? It may be that raw2hdr (via dcraw) is not extracting the correct metadata, somehow. Even a middle exposure from each sequence should be enough for me to determine where things are going wrong.

Hi Greg,
I checked the images captured with the filter and there are 3 pictures whose exposure time is 30 s which maybe the longest time of the camera in the AEB mode. I guess this may have some impact on the result. Besides, I captured 7 images from -3 EV to +3 EV.
The pictures are CR2 format so I don’t know how to upload them. :thinking:

You have to put them on DropBox or Google Drive or similar, I suppose. You can put them in a tar or zip archive if you want to make life easier.

Hi Greg,
I have sent the zip archive by email. Did you receive it?

No, but it’s probably too large for e-mail. You can try sending me the middle exposure from each sequence in separate e-mails. That may be enough for me to figure out the problem if you don’t have access to web storage.

You are still replying to the thread, so your CR2 images were elided. You’ll need to e-mail these to my account directly, which is “gregoryjward@gmail.com”.

-G

OK, I did look at your files, which made it to me this time. I think the links you attempted to upload to Discourse were removed for whatever reason.

Anyway, I don’t see anything wrong with your image files. They are separated by the expected exposure of 1000, and all the included Exif information is extracted correctly by dcraw. Speaking of which, you should make sure your copy of dcraw is working as expected. I am using v9.28, which I believe is the last available version of dcraw. If I run “dcraw -v -i” on your files, I get:

Filename: IMG_8719.CR2
Timestamp: Tue Aug 24 15:45:38 2021
Camera: Canon EOS 6D
ISO speed: 100
Shutter: 20.7 sec
Aperture: f/11.3
Focal length: 8.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size:  5472 x 3648
Full size:   5568 x 3708
Image size:  5496 x 3670
Output size: 5496 x 3670
Raw colors: 3
Filter pattern: RG/GB
Daylight multipliers: 2.093140 0.943072 1.341524
Camera multipliers: 2203.000000 1024.000000 1596.000000 1024.000000

Filename: IMG_8726.CR2
Timestamp: Tue Aug 24 15:48:26 2021
Camera: Canon EOS 6D
ISO speed: 100
Shutter: 1/41.5 sec
Aperture: f/11.3
Focal length: 8.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size:  5472 x 3648
Full size:   5568 x 3708
Image size:  5496 x 3670
Output size: 5496 x 3670
Raw colors: 3
Filter pattern: RG/GB
Daylight multipliers: 2.093140 0.943072 1.341524
Camera multipliers: 2203.000000 1024.000000 1596.000000 1024.000000

This shows that all of the necessary information is present and as expected. This leaves us to wonder what is going wrong with raw2hdr. If I run raw2hdr twice, giving it just the one exposure for each run, I get images that have a 1000:1 exposure ratio between them as expected. Unless there is something wrong with your merge operation, you should get the same from all the exposures.

If not, try the “raw2hdr -x” option to see if skipping over/under-exposed frames helps at all.

Hi Greg,

I checked my version of "dcraw " and is the latest.
image
Besides, I upgraded other commands like exiftool. However, when I ran this script twice giving 1 raw image the same as the two you used and got the RGB values using the command “pvalue”. They are still too small.


for example, the following is the RGB values of the first color patch.
image
And I tried the “raw2hdr -x” option but the RGB value is still small with the magnitude of 1/100.

root@lee-virtual-machine:/home/lee/raw2hdr# perl raw2hdr -o out1.hdr -x IMG_8723.CR2 IMG_8724.CR2 IMG_8725.CR2 IMG_8726.CR2 IMG_8727.CR2 IMG_8728.CR2 IMG_8729.CR2 
Loading Canon EOS 6D image from IMG_8723.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Loading Canon EOS 6D image from IMG_8724.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Loading Canon EOS 6D image from IMG_8725.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Loading Canon EOS 6D image from IMG_8726.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Loading Canon EOS 6D image from IMG_8727.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Loading Canon EOS 6D image from IMG_8728.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Loading Canon EOS 6D image from IMG_8729.CR2 ...
Scaling with darkness 2047, saturation 15490, and
multipliers 2.219491 1.000000 1.422504 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to standard output ...
    1 image files updated
Executing: hdrgen -m 400 -e -a -r /tmp/raw2hdr.tYG77K/sqr.rsp -o out1.hdr -x -c sRGB /tmp/raw2hdr.tYG77K/IMG_8723.tif /tmp/raw2hdr.tYG77K/IMG_8724.tif /tmp/raw2hdr.tYG77K/IMG_8725.tif /tmp/raw2hdr.tYG77K/IMG_8726.tif /tmp/raw2hdr.tYG77K/IMG_8727.tif /tmp/raw2hdr.tYG77K/IMG_8728.tif /tmp/raw2hdr.tYG77K/IMG_8729.tif
     Cleaning Bad Pixels ******************************************************
root@lee-virtual-machine:/home/lee/raw2hdr# pcompos -x 3600 -y 3600 -h out1.hdr -942.7 -25.7 > out1_1.hdr
root@lee-virtual-machine:/home/lee/raw2hdr# pfilt -1 -x 1000 -y 1000 out1_1.hdr > out1_2.hdr
root@lee-virtual-machine:/home/lee/raw2hdr# pvalue -o out1_2.hdr > out1.txt

The problem is that I can’t get the reasonable RGB values from the HDR images. Is there something wrong with the “pvalue” ?
image

I tried your same process and I get a different result. The images are 5496x3670 pixels, and the upper-left red patch is centered near (2604,2084) in Radiance picture coordinates. Using just raw2hdr on the two images you gave me, the pvalue -o output for this pixel is:

IMG_8719:   2604    2084       5.285e-03       8.638e-04       4.538e-04
IMG_8726:   2604    2084       5.534e+00       7.438e-01       6.681e-01

This is more or less what we expect to see, with a roughly 1000:1 ratio between pixel values plus some color cast.

Hi Greg,
Thanks a lot! :+1:
I have found this problem and correct this error.


The reason why I got their false relationship is that I cropped and resized this original HDR images using “pcompos” and “pfilt” after I merged those raw images. This behavior has made the RGB values by “pvalue” too small.
However, I have another question about this error: Why cropping and resizing the original HDR image will bring this problem? :thinking:

This is why it’s important to explain your whole process. You are right that pcompos should not remove exposure information on a single image, but it does. It is a flaw that I will look into correcting. This tool was designed to create composites and overlays of multiple images, and cannot in general maintain metadata for the result. However, there is no good reason it does not keep the exposure during simple cropping, or even when combining of images with matching exposure levels.