Help analyzing/calibrating a Radiance model to match simulated and measured illuminance

radiance-general
1

Hello everyone!

May I request you to please give some insight on the study I did to match
simulated and measured illuminance.

Draft report: http://bit.ly/1nCpjU3

Please have a look at the report.

In this draft report I have tried to explain the model, and results using
different statistics. Also I have highlighted some areas in the time-series
graph, where the model is systematically (occurring during the same time)
under-estimating the illuminance. I don't know why?

Do you think, there is still some scope of fine-tuning the model, or the
systematic error is uncertain to hypothesize?

Do you think the results correlate well enough with other similar benchmark
daylight studies that used Radiance?

I observed that the model correlates better in the case of "observed sunny
sky" as compared to "observed cloudy sky." But I couldn't understand the
reason behind this. ?

Request you to please ask me if I missed to provide any info. about the
model.

Thank you in anticipation.

Best regards,
Vaib

2

Dear Vaib,

thank you for sharing the presenatation with us!

After a quick look at it, one potential source for the observed mismatch may be the sky luminance distributions. You use a model of the sky, which has a rather complex luminance distribution, based on only diffuse horizontal and direct normal illuminance. These to values tell you little about the luminance distribution not within the narrow angle obtained by the sun, instead you use a theoretical model to reconstruct the distribution. This means that for sunny sky conditions, when you know where a huge fraction of luminous flux entering your scene is coming from (you measure it and you know the sun position), you have a good estimate on the sky distribution - while especially for sky conditions where the distribution is far from uniform, but mostly diffuse, you really do guess-work. E.g. clouds tend to give you a rather high variance in real world, but get approximated as a smooth distribution with the sky models you use.

The sky models you use with the measured illuminance readings were ment to be used for annual simulations. The generated distributions match the average over a year. For single time-steps, which is what you compare, I would expect deviations.

Cheers, Lars.

Hello everyone!

May I request you to please give some insight on the study I did to match simulated and measured illuminance.

Draft report: http://bit.ly/1nCpjU3

Please have a look at the report.

In this draft report I have tried to explain the model, and results using different statistics. Also I have highlighted some areas in the time-series graph, where the model is systematically (occurring during the same time) under-estimating the illuminance. I don’t know why?

Do you think, there is still some scope of fine-tuning the model, or the systematic error is uncertain to hypothesize?

Do you think the results correlate well enough with other similar benchmark daylight studies that used Radiance?

I observed that the model correlates better in the case of “observed sunny sky” as compared to “observed cloudy sky.” But I couldn’t understand the reason behind this. ?

Request you to please ask me if I missed to provide any info. about the model.

Thank you in anticipation.

Best regards,

Vaib

_______________________________________________ Radiance-general mailing list Radiance-general@radiance-online.org http://www.radiance-online.org/mailman/listinfo/radiance-general

3

Dear Vaib,

I will prepare sky files based on sky scanner measurements for the period
you mentioned in your presentation.

The reason you get acceptable values for horizontal irradiance outside is
that gendaylit is based on direct and diffuse data therefore you will get
similar values in simulation. As Lars also mentioned distribution of
diffuse is an-isotropic specially in case of breaking clouds. So after I
send you the files please re-simulate again and let us know if you see any
improvement in interior sensors values in comparison to the measured values.

Cheers,
Ehsan

···

On Tue, May 13, 2014 at 11:52 AM, Lars Grobe <grobe@gmx.net> wrote:

Dear Vaib,

thank you for sharing the presenatation with us!

After a quick look at it, one potential source for the observed mismatch
may be the sky luminance distributions. You use a model of the sky, which
has a rather complex luminance distribution, based on only diffuse
horizontal and direct normal illuminance. These to values tell you little
about the luminance distribution not within the narrow angle obtained by
the sun, instead you use a theoretical model to reconstruct the
distribution. This means that for sunny sky conditions, when you know where
a huge fraction of luminous flux entering your scene is coming from (you
measure it and you know the sun position), you have a good estimate on the
sky distribution - while especially for sky conditions where the
distribution is far from uniform, but mostly diffuse, you really do
guess-work. E.g. clouds tend to give you a rather high variance in real
world, but get approximated as a smooth distribution with the sky models
you use.

The sky models you use with the measured illuminance readings were ment to
be used for annual simulations. The generated distributions match the
average over a year. For single time-steps, which is what you compare, I
would expect deviations.

Cheers, Lars.

  Hello everyone!

May I request you to please give some insight on the study I did to match
simulated and measured illuminance.

Draft report: http://bit.ly/1nCpjU3

Please have a look at the report.

In this draft report I have tried to explain the model, and results using
different statistics. Also I have highlighted some areas in the time-series
graph, where the model is systematically (occurring during the same time)
under-estimating the illuminance. I don't know why?

Do you think, there is still some scope of fine-tuning the model, or the
systematic error is uncertain to hypothesize?

Do you think the results correlate well enough with other similar
benchmark daylight studies that used Radiance?

I observed that the model correlates better in the case of "observed sunny
sky" as compared to "observed cloudy sky." But I couldn't understand the
reason behind this. ?

Request you to please ask me if I missed to provide any info. about the
model.

Thank you in anticipation.

Best regards,
Vaib
_______________________________________________ Radiance-general mailing
list Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

4

Hi Vaib,

There is a simple solution also to check if the errors caused by the
distribution of the diffuse or not. pick up the data for vertical
irradiance sensor and compare them to virtual sensors in Radiance. then if
there is similar trends as your result, you can infer that using gendaylit
model you cant get better performance as you already have.

Cheers,
Ehsan

···

On Tue, May 13, 2014 at 12:31 PM, Ehsan M.Vazifeh <em.vazifeh@gmail.com>wrote:

Dear Vaib,

I will prepare sky files based on sky scanner measurements for the period
you mentioned in your presentation.

The reason you get acceptable values for horizontal irradiance outside is
that gendaylit is based on direct and diffuse data therefore you will get
similar values in simulation. As Lars also mentioned distribution of
diffuse is an-isotropic specially in case of breaking clouds. So after I
send you the files please re-simulate again and let us know if you see any
improvement in interior sensors values in comparison to the measured values.

Cheers,
Ehsan

On Tue, May 13, 2014 at 11:52 AM, Lars Grobe <grobe@gmx.net> wrote:

Dear Vaib,

thank you for sharing the presenatation with us!

After a quick look at it, one potential source for the observed mismatch
may be the sky luminance distributions. You use a model of the sky, which
has a rather complex luminance distribution, based on only diffuse
horizontal and direct normal illuminance. These to values tell you little
about the luminance distribution not within the narrow angle obtained by
the sun, instead you use a theoretical model to reconstruct the
distribution. This means that for sunny sky conditions, when you know where
a huge fraction of luminous flux entering your scene is coming from (you
measure it and you know the sun position), you have a good estimate on the
sky distribution - while especially for sky conditions where the
distribution is far from uniform, but mostly diffuse, you really do
guess-work. E.g. clouds tend to give you a rather high variance in real
world, but get approximated as a smooth distribution with the sky models
you use.

The sky models you use with the measured illuminance readings were ment
to be used for annual simulations. The generated distributions match the
average over a year. For single time-steps, which is what you compare, I
would expect deviations.

Cheers, Lars.

  Hello everyone!

May I request you to please give some insight on the study I did to match
simulated and measured illuminance.

Draft report: http://bit.ly/1nCpjU3

Please have a look at the report.

In this draft report I have tried to explain the model, and results using
different statistics. Also I have highlighted some areas in the time-series
graph, where the model is systematically (occurring during the same time)
under-estimating the illuminance. I don't know why?

Do you think, there is still some scope of fine-tuning the model, or the
systematic error is uncertain to hypothesize?

Do you think the results correlate well enough with other similar
benchmark daylight studies that used Radiance?

I observed that the model correlates better in the case of "observed
sunny sky" as compared to "observed cloudy sky." But I couldn't understand
the reason behind this. ?

Request you to please ask me if I missed to provide any info. about the
model.

Thank you in anticipation.

Best regards,
Vaib
_______________________________________________ Radiance-general mailing
list Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

5

Thank you Ehsan, Lars, Prof.Mardaljevic !!

Now, the reason behind weak correlation for cloudy sky conditions (with
measured values) makes sense.
Apart from that I couldn't think of any other modeling parameter (surface
property, geometry, or ambient parameter etc.) that can be tweaked to
further improve the results.

I have also followed all the modeling best practices mentioned in Building
Simulation for Design and
Operation<http://www.amazon.com/Building-Performance-Simulation-Design-Operation/dp/0415474140>
book.

I will go ahead and conclude the thesis research with these results. Better
results can be expected when sky-scanner is used to give better/precise sky
luminance distribution for the sky model.

Ehsan, Thanks!

It would be great if you can provide me sky-scanner's data, so that I can
have a quick test to support the conclusion. Will discuss more about the
vertical irrad. sensor that you said.

Best regards,
Vaib

···

On 14 May 2014 13:21, Ehsan M.Vazifeh <em.vazifeh@gmail.com> wrote:

Hi Vaib,

There is a simple solution also to check if the errors caused by the
distribution of the diffuse or not. pick up the data for vertical
irradiance sensor and compare them to virtual sensors in Radiance. then if
there is similar trends as your result, you can infer that using gendaylit
model you cant get better performance as you already have.

Cheers,
Ehsan

On Tue, May 13, 2014 at 12:31 PM, Ehsan M.Vazifeh <em.vazifeh@gmail.com>wrote:

Dear Vaib,

I will prepare sky files based on sky scanner measurements for the period
you mentioned in your presentation.

The reason you get acceptable values for horizontal irradiance outside is
that gendaylit is based on direct and diffuse data therefore you will get
similar values in simulation. As Lars also mentioned distribution of
diffuse is an-isotropic specially in case of breaking clouds. So after I
send you the files please re-simulate again and let us know if you see any
improvement in interior sensors values in comparison to the measured values.

Cheers,
Ehsan

On Tue, May 13, 2014 at 11:52 AM, Lars Grobe <grobe@gmx.net> wrote:

Dear Vaib,

thank you for sharing the presenatation with us!

After a quick look at it, one potential source for the observed mismatch
may be the sky luminance distributions. You use a model of the sky, which
has a rather complex luminance distribution, based on only diffuse
horizontal and direct normal illuminance. These to values tell you little
about the luminance distribution not within the narrow angle obtained by
the sun, instead you use a theoretical model to reconstruct the
distribution. This means that for sunny sky conditions, when you know where
a huge fraction of luminous flux entering your scene is coming from (you
measure it and you know the sun position), you have a good estimate on the
sky distribution - while especially for sky conditions where the
distribution is far from uniform, but mostly diffuse, you really do
guess-work. E.g. clouds tend to give you a rather high variance in real
world, but get approximated as a smooth distribution with the sky models
you use.

The sky models you use with the measured illuminance readings were ment
to be used for annual simulations. The generated distributions match the
average over a year. For single time-steps, which is what you compare, I
would expect deviations.

Cheers, Lars.

  Hello everyone!

May I request you to please give some insight on the study I did to
match simulated and measured illuminance.

Draft report: http://bit.ly/1nCpjU3

Please have a look at the report.

In this draft report I have tried to explain the model, and results
using different statistics. Also I have highlighted some areas in the
time-series graph, where the model is systematically (occurring during the
same time) under-estimating the illuminance. I don't know why?

Do you think, there is still some scope of fine-tuning the model, or the
systematic error is uncertain to hypothesize?

Do you think the results correlate well enough with other similar
benchmark daylight studies that used Radiance?

I observed that the model correlates better in the case of "observed
sunny sky" as compared to "observed cloudy sky." But I couldn't understand
the reason behind this. ?

Request you to please ask me if I missed to provide any info. about the
model.

Thank you in anticipation.

Best regards,
Vaib
_______________________________________________ Radiance-general
mailing list Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

6

Dear Lars, Prof.Mardaljevic, Ehsan,

Thank you for your last replies. It gave some ground to the conundrum I had
since long.

Could you please suggest me some reference papers etc. that further
describes the difference between sky luminance distributions of sky models
from Perez (gendaylit) and sky-scanner? Or something similar. So that I can
refer that in my thesis report. Thank you!

Best regards,
Vaib

···

On 14 May 2014 21:16, Vaib <vaibhavjain.co@gmail.com> wrote:

Thank you Ehsan, Lars, Prof.Mardaljevic !!

Now, the reason behind weak correlation for cloudy sky conditions (with
measured values) makes sense.
Apart from that I couldn't think of any other modeling parameter (surface
property, geometry, or ambient parameter etc.) that can be tweaked to
further improve the results.

I have also followed all the modeling best practices mentioned in Building
Simulation for Design and Operation<http://www.amazon.com/Building-Performance-Simulation-Design-Operation/dp/0415474140> book.

I will go ahead and conclude the thesis research with these results.
Better results can be expected when sky-scanner is used to give
better/precise sky luminance distribution for the sky model.

Ehsan, Thanks!

It would be great if you can provide me sky-scanner's data, so that I can
have a quick test to support the conclusion. Will discuss more about the
vertical irrad. sensor that you said.

Best regards,
Vaib

On 14 May 2014 13:21, Ehsan M.Vazifeh <em.vazifeh@gmail.com> wrote:

Hi Vaib,

There is a simple solution also to check if the errors caused by the
distribution of the diffuse or not. pick up the data for vertical
irradiance sensor and compare them to virtual sensors in Radiance. then if
there is similar trends as your result, you can infer that using gendaylit
model you cant get better performance as you already have.

Cheers,
Ehsan

On Tue, May 13, 2014 at 12:31 PM, Ehsan M.Vazifeh <em.vazifeh@gmail.com>wrote:

Dear Vaib,

I will prepare sky files based on sky scanner measurements for the
period you mentioned in your presentation.

The reason you get acceptable values for horizontal irradiance outside
is that gendaylit is based on direct and diffuse data therefore you will
get similar values in simulation. As Lars also mentioned distribution of
diffuse is an-isotropic specially in case of breaking clouds. So after I
send you the files please re-simulate again and let us know if you see any
improvement in interior sensors values in comparison to the measured values.

Cheers,
Ehsan

On Tue, May 13, 2014 at 11:52 AM, Lars Grobe <grobe@gmx.net> wrote:

Dear Vaib,

thank you for sharing the presenatation with us!

After a quick look at it, one potential source for the observed
mismatch may be the sky luminance distributions. You use a model of the
sky, which has a rather complex luminance distribution, based on only
diffuse horizontal and direct normal illuminance. These to values tell you
little about the luminance distribution not within the narrow angle
obtained by the sun, instead you use a theoretical model to reconstruct the
distribution. This means that for sunny sky conditions, when you know where
a huge fraction of luminous flux entering your scene is coming from (you
measure it and you know the sun position), you have a good estimate on the
sky distribution - while especially for sky conditions where the
distribution is far from uniform, but mostly diffuse, you really do
guess-work. E.g. clouds tend to give you a rather high variance in real
world, but get approximated as a smooth distribution with the sky models
you use.

The sky models you use with the measured illuminance readings were ment
to be used for annual simulations. The generated distributions match the
average over a year. For single time-steps, which is what you compare, I
would expect deviations.

Cheers, Lars.

  Hello everyone!

May I request you to please give some insight on the study I did to
match simulated and measured illuminance.

Draft report: http://bit.ly/1nCpjU3

Please have a look at the report.

In this draft report I have tried to explain the model, and results
using different statistics. Also I have highlighted some areas in the
time-series graph, where the model is systematically (occurring during the
same time) under-estimating the illuminance. I don't know why?

Do you think, there is still some scope of fine-tuning the model, or
the systematic error is uncertain to hypothesize?

Do you think the results correlate well enough with other similar
benchmark daylight studies that used Radiance?

I observed that the model correlates better in the case of "observed
sunny sky" as compared to "observed cloudy sky." But I couldn't understand
the reason behind this. ?

Request you to please ask me if I missed to provide any info. about the
model.

Thank you in anticipation.

Best regards,
Vaib
_______________________________________________ Radiance-general
mailing list Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
Radiance-general@radiance-online.org
http://www.radiance-online.org/mailman/listinfo/radiance-general

7

Hi Vaib,

Here's a (partial) list of references:

[1] D. Enarun and P. Littlefair. Luminance models for overcast skies: Assessment using measured data. Lighting Research and Technology, 27(1):53–58, 1995.

[2] P. Ineichen, B. Molineaux, and R. Perez. Sky luminance data validation: Comparison of seven models with four data banks. Solar Energy, 52(4):337–346, 1994.

[3] P. Littlefair. A comparison of sky luminance models with measured data from garston, united kingdom. Solar Energy, 53(4):315–322, 1994.

[4] J. Mardaljevic. Validation of a lighting simulation program under real sky conditions. Lighting Research and Technology, 27(4):181–188, 12 1995.

[5] J. Mardaljevic. The BRE-IDMP dataset: a new benchmark for the validation of illuminance prediction techniques. Lighting Research and Technology, 33(2):117–134, 2001.

[6] J. Mardaljevic. Verification of program accuracy for illuminance modelling: Assumptions, methodology and an examination of conflicting findings. Lighting Research and Technology, 36(3):217–239, 2004.

[7] J. Mardaljevic. Sky model blends for predicting internal illuminance: a comparison founded on the BRE-IDMP dataset. Journal of Building Performance Simulation, 1(3):163–173, 2008.

And a shed-load of the nitty-gritty on comparing sky models with measured sky luminance scans in Chapter 5 here:

http://climate-based-daylighting.com/doku.php?id=resources:thesis

Best
John

John Mardaljevic PhD FSLL
Professor of Building Daylight Modelling
School of Civil & Building Engineering
Loughborough University
Loughborough
Leicestershire
LE11 3TU, UK

Tel: +44 1509 222630 (Direct)
Tel: +44 1509 228529 (Pam Allen, secretary)

j.mardaljevic@lboro.ac.uk

http://www.lboro.ac.uk/departments/civil-building/staff/mardaljevicjohn

Personal daylighting website:
http://climate-based-daylighting.com

8

Thank you very much Prof.Mardaljevic!

Best regards,

Vaib

···

On May 20, 2014 2:55 AM, "John Mardaljevic" <J.Mardaljevic@lboro.ac.uk> wrote:

Hi Vaib,

Here's a (partial) list of references:

[1] D. Enarun and P. Littlefair. Luminance models for overcast skies:
Assessment using measured data. Lighting Research and Technology,
27(1):53–58, 1995.

[2] P. Ineichen, B. Molineaux, and R. Perez. Sky luminance data
validation: Comparison of seven models with four data banks. Solar Energy,
52(4):337–346, 1994.

[3] P. Littlefair. A comparison of sky luminance models with measured data
from garston, united kingdom. Solar Energy, 53(4):315–322, 1994.

[4] J. Mardaljevic. Validation of a lighting simulation program under real
sky conditions. Lighting Research and Technology, 27(4):181–188, 12 1995.

[5] J. Mardaljevic. The BRE-IDMP dataset: a new benchmark for the
validation of illuminance prediction techniques. Lighting Research and
Technology, 33(2):117–134, 2001.

[6] J. Mardaljevic. Verification of program accuracy for illuminance
modelling: Assumptions, methodology and an examination of conflicting
findings. Lighting Research and Technology, 36(3):217–239, 2004.

[7] J. Mardaljevic. Sky model blends for predicting internal illuminance:
a comparison founded on the BRE-IDMP dataset. Journal of Building
Performance Simulation, 1(3):163–173, 2008.

And a shed-load of the nitty-gritty on comparing sky models with measured
sky luminance scans in Chapter 5 here:

http://climate-based-daylighting.com/doku.php?id=resources:thesis

Best
John

John Mardaljevic PhD FSLL
Professor of Building Daylight Modelling
School of Civil & Building Engineering
Loughborough University
Loughborough
Leicestershire
LE11 3TU, UK

Tel: +44 1509 222630 (Direct)
Tel: +44 1509 228529 (Pam Allen, secretary)

j.mardaljevic@lboro.ac.uk

http://www.lboro.ac.uk/departments/civil-building/staff/mardaljevicjohn

Personal daylighting website:
http://climate-based-daylighting.com