# rounding problems? when writing colored images

Hi Greg,

I'm not sure if I asked you this already... at least I could not find it in the discussion group...
While modifying evalglare I encountered some slight deviations when writing colored pixels in an hdr image using fwritescan (in common/color.c).

I 'm coloring parts of the image and try to keep the luminance value. Within evalglare this transformation is correct (I'm using the libraries in color.h/color.c). But as soon as I write the image, the color channels are changed. So the values of each channel differ slightly:

e.g. out of r=263.133942 g=0.000000 b=1076.202148 (within evalglare) it gets (in the image) r=2.600e+02 g=4.000e+00 b=1.076e+03
so the luminance changes from 24970.5 to 25285.6
The deviation in that case is 1.25% of the corresponding luminance - I also saw more than 2% in other cases.

I guess it comes from the conversion into mantisse and exponent in combination with large differences between the channels, or? Strangely a value of 0 is getting 4 for the green channel.

Is there any possibility to avoid this or to reduce this deviation?

thanks!

best,

Jan

···

--
Dr.-Ing. Jan Wienold
Ecole Polytechnique F�d�rale de Lausanne (EPFL)
EPFL ENAC IA LIPID

http://people.epfl.ch/jan.wienold
LE 1 111 (Office)
Phone +41 21 69 30849

Hi Jan,

This is as you surmised due to inherent limitations in the accuracy of the 8-bit mantissas used in the RGBE format. Even a value of 0 in one channel will be "rounded off" to a value of 1/512 (roughly) times 2^Exponent. There isn't much you can do to improve the roughly +/- 1% luminance accuracy besides going to an IEEE floating-point representation. Also, the Y-channel error will get worse for certain colors, mostly blue to magenta (where the green channel is small).

Some programs understand the XYZE format, which should maintain slightly better accuracy in the Y channel even with saturated colors. I haven't done an extensive comparison, however.

-Greg

···

From: Jan Wienold <[email protected]>
Subject: [Radiance-dev] rounding problems? when writing colored images
Date: December 8, 2015 11:24:26 AM PST

Hi Greg,

I'm not sure if I asked you this already... at least I could not find it in the discussion group...
While modifying evalglare I encountered some slight deviations when writing colored pixels in an hdr image using fwritescan (in common/color.c).

I 'm coloring parts of the image and try to keep the luminance value. Within evalglare this transformation is correct (I'm using the libraries in color.h/color.c). But as soon as I write the image, the color channels are changed. So the values of each channel differ slightly:

e.g. out of r=263.133942 g=0.000000 b=1076.202148 (within evalglare) it gets (in the image) r=2.600e+02 g=4.000e+00 b=1.076e+03
so the luminance changes from 24970.5 to 25285.6
The deviation in that case is 1.25% of the corresponding luminance - I also saw more than 2% in other cases.

I guess it comes from the conversion into mantisse and exponent in combination with large differences between the channels, or? Strangely a value of 0 is getting 4 for the green channel.

Is there any possibility to avoid this or to reduce this deviation?

thanks!

best,

Jan