Invalid exposure entry

Hi all,
I am performing the overflow correction by refering to this document by @J_Alstan_Jakubiec.
https://www.researchgate.net/publication/305703083
There occurred errors when getting vertical illuminance of the final corrected image 5_corc.hdr.


The exposure value is the same as the original image 5.hdr.

The image mask.hdr is obtained from Photoshop which is different from the appendix.
I once successfully performed this correction process on other images and this time is hindered with the above error. I hope some friends can supply any advice especially the one who has used the same procedures. Thanks!

1 Like

It seems you are missing again the
ra_xyze -r -o
command directly after the image generation to remove the exposure. please see the other post where this was solved.

For pixel-oveflow you can use actually a single evalglare command, when you know where the overflow is (typically sun, use ximage and click on center of sun and press p, then you get x,y).
The you use the -N option in evalglare, see manual page (also slides at radiance workshop 2016) :

" -N xpos ypos angle Ev fname
Pixel replacement in case of pixel overflow in hdr image and measured Ev (in lux) is available. Writes the modified image to fname and exists immediately (without glare evaluation).
Replaces pixels in a circular zone to match Ev. The center of the zone is given by xpos and ypos. The opening angle of the zone is specified in radians. This option should be applied very carefully and only exceptionally. Pixel overflow should be avoided from the beginning by applying shorter exposure times and/or neutral filters."

Jan

I know I keep posting on these threads, but I don’t know how else to inform users of the new “rcrop” command in the latest HEAD, which we should start using to replace pcompos in these conversions. It avoids the need for ra_xyze and the complaints from evalglare about improper exposures. (See this thread for details.)

Best,
-Greg

Hi Greg,

Actually it is not only pcompos causing problems, also pcomb.

Simple example: you have an image with an exposure generated by hdrgen.
Then you apply pcomb -s 1.0 to the image, which scales the image by 1, so all should stay the same. Then the exposure entry gets marked invalid by adding a tab. If you open in ximage, you get “wrong” values displayed. (evalglare would use wrong values as well in that case if it would not stop).

If you apply the same command with -o, you end up exactly with the same header information (exposure is marked invalid by a tab), but now the values are correctly displayed in ximage (because the exposure was applied to the values by pcomb).

So “nearly” the same command leads to exactly the same header, but in one case with wrong values.

People are applying multiple pcomb commands (and also pcompos commands, sometimes with and without -o option). So several tabs are added, depending how often people applied these tools. Without parsing all commands that have been applied to the image, one does not know if the exposure is correct in the image or not. Of course, people should always use the -o option, then you can simply ignore the invalid exposure entry, but not all users do that.
For that reason, to protect people having wrong exposures, evalglare checks if there is an invalid marked exposure in the header and stops.

One solution could be that whenever you apply pcomb/pcompos, the -o is applied automatically and the exposure entry in the header is simply removed or set to 1. And maybe adding a new option if someone wants to keep the exposure.

cheers
Jan

Yes, I did make a change earlier to pcompos to preserve the exposure setting in the final header when all the images shared the same exposure value, but as you point out, this does not satisfy evalglare’s requirement that there be no previous exposure values (other than 1.0) on constituent images.

I do understand how you are trying to protect users from making mistakes, but as you know, that is not the general philosophy in Radiance(!) If I were to change pcomb such that -o was the new default behavior, which I agree is a reasonable suggestion, then any existing scripts that rely on its historical behavior will be immediately broken with the next release, and that would be against our general philosophy of being a reliable package. Such “breaking changes” are only introduced when either (a) we don’t expect anyone to have been aware of the previous behavior and we are only correcting things, or (b) there is a compelling need to support a new feature that is more important than past behavior.

One thing I could do is remove all EXPOSURE= lines from indented headers where the ‘-o’ option is active in pcomb. I believe this would resolve the issue without requiring any breaking changes. I cannot do the same thing in pcompos, but since rcrop is a better tool in this case, anyway, that problem should be solved.

-Greg

yes that is a good solution.
but… what if someone uses pcomb without -o first and then after with -o ?
In that case the invalid exposure marking would be lost and evalglare would treat the image exposure as valid, but exposure might be wrong…

So maybe removing only if the exposure entry is valid would be the safer option, if that is possible. Otherwise I would rather keep things as it is, evalglare with its safety feature is at least preventing people using it with invalid header.

Another suggestion would be to add also some documentation in the manpage of pcomb/pcompos to make clear what it does to the exposure-entry of an image.

cheers
Jan

Hi Jan,

There’s an old saying, that you can make things foolproof, but you can’t make them damnfoolproof. The meaning being that a damn fool will mess things up no matter what you do.

I already made the change to pcomb to elide the indented EXPOSURE= line with the -o option. So long as evalglare looks for EXPOSURE= lines with any number of tabs in front of them (not just one), you should be just as safe as you were before.

However, there is nothing to prevent a damn fool from using the -s or -c option, which foils calibration, or applying a correction that is not properly normalized. For example, the command:

pcond -e 'lo=5*li(1)' -o input.hdr > output.hdr

will have no EXPOSURE= lines in it, but will be off by a factor of 5. After all, pcomb is designed to perform arbitrary operations on image pixels, so there’s not real way to guarantee exposure or even linearity in general.

I did as you suggested and added a few words to the pcomb and pcompos man pages to suggest what happens with picture exposure.

Cheers,
-Greg

P.S. Just to clarify – it seems that the current evalglare code that tests for EXPOSURE= lines with tabs will still trigger if someone uses a pcomb command with -o after another pcomb command without it. Only the incorporated EXPOSURE= lines are absorbed by the updated version of pcomb.

Hi @Jan_Wienold,
Yes I forgot to handle the image with ra_xyze and pcompos after that correction. At last I have obtained the vertical illuminance with evalglare.

As for this -N option, I am going to try it with the same image. But I have a question that is how to get the opening angle of the sun area. For example, how to get the opening angle of the zone 0.9 in the following picture?

image
Besides, the original HDR image(5496X3670 pixels) is too large to display all the content just a corner in the screen so I can’t use ximage to get the location of the center of the sun. I think using pfilt to resize the image will not retain the original imformation of pixels and may cause other problems for analysis. I am puzzled about this limit of ximage for a long time and I hope there exists some solutions to this problem.

P.S.The pixel overflow part of the pdf document from 2016 workshop is not completed.

Hi @Greg_Ward,
I have not tried the new tool “rcrop” yet. You means that if rcrop is used then ra_xyze is avoided before cropping. But we still remember to use the -o option with each use of pcomb for other calibration procedures, right?

Yes, but as Jan reminded me, there is still an issue with pcomb keeping exposure values with -o. I only uploaded a correction to that problem yesterday, and it doesn’t look like packages have been updated since the 13th. (I’ll have to look into that – they’re supposed to build every weekend.) You can still build it yourself from the Radiance HEAD, but it’s a bit more work.

Cheers,
-Greg

P.S. to last message – we just updated the packages containing the new rcrop tool and updated version of pcomb here.

Hi @Greg_Ward
I am using the newset version 05eb231c and rcrop is a good substitute for pcompos.
https://discourse.radiance-online.org/t/calibration-of-hdr-image/5848/4?u=lee
But there maybe a minor mistake about the coordinates of the beginning point. It should be like this:

pcompos -x $DIM_CROP -y $DIM_CROP -h - -$OFFSET_X -$OFFSET_Y

which is equal to

rcrop $OFFSET_R $OFFSET_X $DIM_CROP $DIM_CROP

And $OFFSET_R is $ORIG_Y_DIM - $OFFSET_Y - $DIM_CROP.
Besides, another problem occured when I tried the following commands which is related to this post:

This command can not calculate the illuminance of the HDR image.

pcomb -e "lo=L*Sang*cosCor;L=179*li(1);Sang=S(1);cosCor=Dy(1);" -o ${q}_mc.hdr | pvalue  -d -b -h -H | total

image
However, the same script can be carried out in other versions of Radiance like 1af5566f and before it. So I installed two versions of Radiance and used them separately as ordinary user or root user. It is not convenient and easy to make errors.
I hope you can find and fix some unknown problems about the updated version 05eb231c. Thanks a lot!

Thanks for pointing out both of these errors. My apologies.

I have corrected my post regarding arguments to rcrop, and just checked in a repaired version of pcomb.

Cheers,
-Greg