Hi it looks like my thesys target is changing direction, but it is not decided yet, anyway, my teacher and I were curios is someone did already the tests I'm going to write about. Else I would do that, and it could be interesting for the whole community.
So ... it exists a tachnical report from CIE which is titled "Test cases to asses the accuracy of lighting computer programs". Briefly, it contains some sets os cenes well described to be reproduced and computed in any software..
If nobody did those tests using radiance, it could be a nice goal for my thesys to complete them or some of them at least.. If somebody already did them, I'd like to look at them, if possible.
And.. well.. do you think it could be interesting for everybody ?
Thank you
Lucio Boscolo
.. Caminante no hay camino, solo se hace camino al andar ..
Dear Lucio,
So ... it exists a tachnical report from CIE which is titled "Test cases to asses the
accuracy of lighting computer programs". Briefly, it contains some sets of
scenes well described to be reproduced and computed in any software..
If nobody did those tests using radiance, it could be a nice goal for my thesys
to complete them or some of them at least.. If somebody already did them,
I'd like to look at them, if possible.
Funnily enough, I was actually going to give this very task to one of
my students this year. I think it's well worth doing as an exercise.
There is a validation of Relux (and I think one other software
package) based on the CIE technical report. If I remember correctly,
Relux worked all right, but they discovered that the CIE overcast sky
definition was implemented incorrectly in Relux, which it's now fixed.
If you cant' find the report on the Net, get back to me, and I'll try
to dig it out.
There are two things that you should be aware of with such a validation:
a) The test cases are all VERY basic. This is because the reference
results are derived analytically (pen on paper). This is to make them
100% correct, but the downside is that the scenarios can't be very
complex. Radiance, however, excels exactly in such cases difficult
cases which can only be validated though elaborate measurements in
real spaces (See J. Mardaljevic, C. Reinhard)
b) Much of Radiance's accuracy is based on the operator's experience
and him setting the ambient parameters correctly. I feel this would
have to be quite a large part of this assessment: How to get them
right, and the penalty of not getting them spot-on.
Those are my thoughts on the subject. I am looking forward to reading
your report, and wish you best of luck with it.
Axel
I thought some of them were actually based on physical scenarios, in fact on Mardaljevic's scenarios, but then I only skimmed the report.
Randolph
···
On Jan 14, 2008, at 2:57 AM, Axel Jacobs wrote:
a) The test cases are all VERY basic. This is because the reference
results are derived analytically (pen on paper). This is to make them
100% correct, but the downside is that the scenarios can't be very
complex. Radiance, however, excels exactly in such cases difficult
cases which can only be validated though elaborate measurements in
real spaces (See J. Mardaljevic, C. Reinhard)
Thank you very much for your reply Axel..
I did thought as well some cases were based on real measurement but I did read too it quite fastly.
According to Antonutto someone else already did that validation.. So, there's no need to repeat it (even if actually I'm not able to find this research on the web).
Anyway.. in an hour I will talk with my teacher to plan something..
As about Relux I didn't look for report, 'couse I don't think it could be interesting to me.. if you see it differently, please, tell me why !
Thank you again !!
Lucio
.Dear Lucio,
.> So ... it exists a tachnical report from CIE which is titled "Test cases to asses the
.> accuracy of lighting computer programs". Briefly, it contains some sets of
.> scenes well described to be reproduced and computed in any software..
.
.> If nobody did those tests using radiance, it could be a nice goal for my thesys
.> to complete them or some of them at least.. If somebody already did them,
.> I'd like to look at them, if possible.
.
.Funnily enough, I was actually going to give this very task to one of
.my students this year. I think it's well worth doing as an exercise.
.
.There is a validation of Relux (and I think one other software
.package) based on the CIE technical report. If I remember correctly,
.Relux worked all right, but they discovered that the CIE overcast sky
.definition was implemented incorrectly in Relux, which it's now fixed.
.If you cant' find the report on the Net, get back to me, and I'll try
.to dig it out.
.
.There are two things that you should be aware of with such a validation:
.
.a) The test cases are all VERY basic. This is because the reference
.results are derived analytically (pen on paper). This is to make them
.100% correct, but the downside is that the scenarios can't be very
.complex. Radiance, however, excels exactly in such cases difficult
.cases which can only be validated though elaborate measurements in
.real spaces (See J. Mardaljevic, C. Reinhard)
.
.b) Much of Radiance's accuracy is based on the operator's experience
.and him setting the ambient parameters correctly. I feel this would
.have to be quite a large part of this assessment: How to get them
.right, and the penalty of not getting them spot-on.
.
.Those are my thoughts on the subject. I am looking forward to reading
.your report, and wish you best of luck with it.
.
.Axel
.
._______________________________________________
.Radiance-general mailing list
[email protected]
.http://www.radiance-online.org/mailman/listinfo/radiance-general
.
.. Caminante no hay camino, solo se hace camino al andar ..
Hi Lucio,
According to Antonutto someone else already did that validation.
I have never heard of this validation being carried out for Radiance.
Does anybody know anything about it? I'd be very interested...
As about Relux I didn't look for report, 'couse I don't think it could be
interesting to me.
That's up to you.
Here is the link:
http://www.talisys.de/pdf/16_Maamari_IEAReport_%20Benchmarks_2005.pdf
The validation was carried out for Relux and Lightscape.
The actual CIE benchmark paper is only available for money.
Axel
Hi Lucio,
Related to your earlier post, I found the following message from Ian Ashdown (author of AGi32) while digging through my InBox:
···
From: "Ian Ashdown" <[email protected]>
Date: July 22, 2007 10:09:57 AM PDT
To: "'Greg Ward'" <[email protected]>
Subject: CIE 171:2006 validation
If you or any of your legion of experienced users ever feel inclined to validate Radiance against CIE 171:2006, “Test cases to assess the accuracy of lighting computer programs,” be sure to read this document from Lighting Analysts’ Web site beforehand. There are two test cases that are incorrectly formulated (confirmed by David DiLaura), and a few errors in the reference data.
- Ian
================
From: "loscotec\@libero\.it" <[email protected]>
Date: January 15, 2008 6:51:05 AM PST
Thank you very much for your reply Axel..
I did thought as well some cases were based on real measurement but I did read too it quite fastly.
According to Antonutto someone else already did that validation.. So, there's no need to repeat it (even if actually I'm not able to find this research on the web).
Anyway.. in an hour I will talk with my teacher to plan something..
As about Relux I didn't look for report, 'couse I don't think it could be interesting to me.. if you see it differently, please, tell me why !
Thank you again !!
Lucio