rcontrib malloc error

I'm running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system - out of memory in samp_hemi: Cannot allocate memory
rcontrib: system - read error from render process
Killed: 9

2)
rcontrib: system - read error from render process
Killed: 9

I can't establish a pattern for why/when these two errors occur.

Oddly, I don't always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don't understand why grid size would influence memory though...?

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with Yosemite installed
Latest Radiance HEAD release; although the error repeats on older versions from 6 months ago
rcontrib settings are: -n 12 -ab 12 -ad 100000 -lw 1e-5 -I+

Any clues on why I'm getting these errors?

Thank you!

Galen

···

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

I'm currently struggling with memory issues -- of the volatile and non-volatile kind -- trying to get Radiance to play nicely on a server-based optimization framework, and making it Windoze-friendly. As a result of that last bit, I'm writing view/daylight matrices in ASCII, and they get large in a hurry. 7500 points worth of matrix, for a presumably annual (8760) input, is a BAF (big a** file).

I'm not seeing your errors specifically Galen, mine are all downstream; I either run out of disk space or hit memory errors when rfluxmtx tries to read these BAFs into memory for merging. But maybe we're both barking up the same tree in a sense? I dunno. But now you know why I have a headache today.

···

On 3/17/16, 2:39 PM, "Galen Burrell" <[email protected]<mailto:[email protected]>> wrote:

I'm running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system - out of memory in samp_hemi: Cannot allocate memory
rcontrib: system - read error from render process
Killed: 9

2)
rcontrib: system - read error from render process
Killed: 9

I can't establish a pattern for why/when these two errors occur.

Oddly, I don't always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don't understand why grid size would influence memory though...?

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with Yosemite installed
Latest Radiance HEAD release; although the error repeats on older versions from 6 months ago
rcontrib settings are: -n 12 -ab 12 -ad 100000 -lw 1e-5 -I+

Any clues on why I'm getting these errors?

Thank you!

Galen

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

Hi Rob,

I don't think rfluxmtx reads large matrices into memory. It could write a too-large matrix to disk as the output of rcontrib, but neither process holds the matrices in memory. Maybe you meant rmtxop? That will hold two double (64-bit) color (x3) matrices in memory at a time, which could be a problem for certain very large matrices. You might try dctimestep for matrix multiplication. It uses float (32-bit) color (x3) arrays, so half as much memory is required. Also, it's more clever about skipping zero rows and columns in your matrix, which is good for daylight calculations that include nighttime hours.

-Greg

···

From: "Guglielmetti, Robert" <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:57:36 PM PDT

I'm currently struggling with memory issues -- of the volatile and non-volatile kind -- trying to get Radiance to play nicely on a server-based optimization framework, and making it Windoze-friendly. As a result of that last bit, I'm writing view/daylight matrices in ASCII, and they get large in a hurry. 7500 points worth of matrix, for a presumably annual (8760) input, is a BAF (big a** file).

I'm not seeing your errors specifically Galen, mine are all downstream; I either run out of disk space or hit memory errors when rfluxmtx tries to read these BAFs into memory for merging. But maybe we're both barking up the same tree in a sense? I dunno. But now you know why I have a headache today.

On 3/17/16, 2:39 PM, "Galen Burrell" <[email protected]<mailto:[email protected]>> wrote:

I'm running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system - out of memory in samp_hemi: Cannot allocate memory
rcontrib: system - read error from render process
Killed: 9

2)
rcontrib: system - read error from render process
Killed: 9

I can't establish a pattern for why/when these two errors occur.

Oddly, I don't always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don't understand why grid size would influence memory though...?

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with Yosemite installed
Latest Radiance HEAD release; although the error repeats on older versions from 6 months ago
rcontrib settings are: -n 12 -ab 12 -ad 100000 -lw 1e-5 -I+

Any clues on why I'm getting these errors?

Thank you!

Galen

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Hi Galen,

I've never seen rcontrib run out of memory in this way before. What are your rendering options for rcontrib? Do any of your surfaces have greater than 100% reflectance or transmittance? This could cause the number of rays to increase rather than decrease on multiple bounces, which could lead to exponential memory growth during ray tree evaluation.

-Greg

···

From: Galen Burrell <[email protected]>
Subject: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:39:45 PM PDT

I’m running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed (error code=12)
*** error: can’t allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system – out of memory in samp_hemi: Cannot allocate memory
rcontrib: system – read error from render process
Killed: 9

2)
rcontrib: system – read error from render process
Killed: 9

I can’t establish a pattern for why/when these two errors occur.

Oddly, I don’t always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don’t understand why grid size would influence memory though…?

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with Yosemite installed
Latest Radiance HEAD release; although the error repeats on older versions from 6 months ago
rcontrib settings are: –n 12 –ab 12 –ad 100000 –lw 1e-5 –I+

Any clues on why I’m getting these errors?

Thank you!

Galen

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Thanks man! I did mean rmtxop, and I will give that dctimestep idea a go!

- Rob

···

On Mar 17, 2016, at 3:45 PM, Greg Ward <[email protected]> wrote:

Hi Rob,

I don't think rfluxmtx reads large matrices into memory. It could write a too-large matrix to disk as the output of rcontrib, but neither process holds the matrices in memory. Maybe you meant rmtxop? That will hold two double (64-bit) color (x3) matrices in memory at a time, which could be a problem for certain very large matrices. You might try dctimestep for matrix multiplication. It uses float (32-bit) color (x3) arrays, so half as much memory is required. Also, it's more clever about skipping zero rows and columns in your matrix, which is good for daylight calculations that include nighttime hours.

-Greg

From: "Guglielmetti, Robert" <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:57:36 PM PDT

I'm currently struggling with memory issues -- of the volatile and non-volatile kind -- trying to get Radiance to play nicely on a server-based optimization framework, and making it Windoze-friendly. As a result of that last bit, I'm writing view/daylight matrices in ASCII, and they get large in a hurry. 7500 points worth of matrix, for a presumably annual (8760) input, is a BAF (big a** file).

I'm not seeing your errors specifically Galen, mine are all downstream; I either run out of disk space or hit memory errors when rfluxmtx tries to read these BAFs into memory for merging. But maybe we're both barking up the same tree in a sense? I dunno. But now you know why I have a headache today.

On 3/17/16, 2:39 PM, "Galen Burrell" <[email protected]<mailto:[email protected]>> wrote:

I'm running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system - out of memory in samp_hemi: Cannot allocate memory
rcontrib: system - read error from render process
Killed: 9

2)
rcontrib: system - read error from render process
Killed: 9

I can't establish a pattern for why/when these two errors occur.

Oddly, I don't always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don't understand why grid size would influence memory though...?

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with Yosemite installed
Latest Radiance HEAD release; although the error repeats on older versions from 6 months ago
rcontrib settings are: -n 12 -ab 12 -ad 100000 -lw 1e-5 -I+

Any clues on why I'm getting these errors?

Thank you!

Galen

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Just an update on this error: After searching through my materials definitions per Greg's suggestion, I found that I had a glass definition of 95% light transmission, which resulted in a transmissivity of > 1. Doh! After reducing the light transmission to something more sensible I stopped getting the error. Thanks Greg!

···

Date: Thu, 17 Mar 2016 14:49:01 -0700
From: Greg Ward <[email protected]>
To: Radiance general discussion <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error
Message-ID: <[email protected]>
Content-Type: text/plain; charset="windows-1252"

Hi Galen,

I've never seen rcontrib run out of memory in this way before. What are your rendering options for rcontrib? Do any of your surfaces have greater than 100% reflectance or transmittance? This could cause the number of rays to increase rather than decrease on multiple bounces, which could lead to exponential memory growth during ray tree evaluation.

-Greg

From: Galen Burrell <[email protected]>
Subject: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:39:45 PM PDT

I?m running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed
(error code=12)
*** error: can?t allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system ? out of memory in samp_hemi: Cannot allocate memory
rcontrib: system ? read error from render process
Killed: 9

2)
rcontrib: system ? read error from render process
Killed: 9

I can?t establish a pattern for why/when these two errors occur.

Oddly, I don?t always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don?t understand why grid size would influence memory though??

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with
Yosemite installed Latest Radiance HEAD release; although the error
repeats on older versions from 6 months ago rcontrib settings are: ?n
12 ?ab 12 ?ad 100000 ?lw 1e-5 ?I+

Any clues on why I?m getting these errors?

Thank you!

Galen

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

Thanks for the update; I figured it was something like that since I saw
this commit right around the time of this thread:


63250

=)

···

On 3/23/16, 6:48 AM, "Galen Burrell" <[email protected]> wrote:

Just an update on this error: After searching through my materials
definitions per Greg's suggestion, I found that I had a glass definition
of 95% light transmission, which resulted in a transmissivity of > 1.
Doh! After reducing the light transmission to something more sensible I
stopped getting the error. Thanks Greg!

Date: Thu, 17 Mar 2016 14:49:01 -0700
From: Greg Ward <[email protected]>
To: Radiance general discussion <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error
Message-ID: <[email protected]>
Content-Type: text/plain; charset="windows-1252"

Hi Galen,

I've never seen rcontrib run out of memory in this way before. What are
your rendering options for rcontrib? Do any of your surfaces have
greater than 100% reflectance or transmittance? This could cause the
number of rays to increase rather than decrease on multiple bounces,
which could lead to exponential memory growth during ray tree evaluation.

-Greg

From: Galen Burrell <[email protected]>
Subject: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:39:45 PM PDT

I?m running view matrix calculations using rfluxmtx/rcontrib for the
3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed
(error code=12)
*** error: can?t allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system ? out of memory in samp_hemi: Cannot allocate memory
rcontrib: system ? read error from render process
Killed: 9

2)
rcontrib: system ? read error from render process
Killed: 9

I can?t establish a pattern for why/when these two errors occur.

Oddly, I don?t always get the error with some gridpoint files (with
5900 points), but I consistently get it with another gridpoint file
(with 7500 points). I don?t understand why grid size would influence
memory though??

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with
Yosemite installed Latest Radiance HEAD release; although the error
repeats on older versions from 6 months ago rcontrib settings are: ?n
12 ?ab 12 ?ad 100000 ?lw 1e-5 ?I+

Any clues on why I?m getting these errors?

Thank you!

Galen

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Good catch, Galen! I'm glad you found the explanation. Meanwhile, I changed the code so that (hopefully) we won't get runaway ray counts even if there are surfaces with >100% reflectance or transmittance.

Cheers!
-Greg

···

From: Galen Burrell <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error
Date: March 23, 2016 5:48:33 AM PDT

Just an update on this error: After searching through my materials definitions per Greg's suggestion, I found that I had a glass definition of 95% light transmission, which resulted in a transmissivity of > 1. Doh! After reducing the light transmission to something more sensible I stopped getting the error. Thanks Greg!

Date: Thu, 17 Mar 2016 14:49:01 -0700
From: Greg Ward <[email protected]>
To: Radiance general discussion <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error

Hi Galen,

I've never seen rcontrib run out of memory in this way before. What are your rendering options for rcontrib? Do any of your surfaces have greater than 100% reflectance or transmittance? This could cause the number of rays to increase rather than decrease on multiple bounces, which could lead to exponential memory growth during ray tree evaluation.

-Greg

From: Galen Burrell <[email protected]>
Subject: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:39:45 PM PDT

I?m running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed
(error code=12)
*** error: can?t allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system ? out of memory in samp_hemi: Cannot allocate memory
rcontrib: system ? read error from render process
Killed: 9

2)
rcontrib: system ? read error from render process
Killed: 9

I can?t establish a pattern for why/when these two errors occur.

Oddly, I don?t always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don?t understand why grid size would influence memory though??

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with
Yosemite installed Latest Radiance HEAD release; although the error
repeats on older versions from 6 months ago rcontrib settings are: ?n
12 ?ab 12 ?ad 100000 ?lw 1e-5 ?I+

Any clues on why I?m getting these errors?

Thank you!

Galen

Galen,

Here is something else that might improve the quality of your gridpoint calculations. I have seen "-lw 1e-5" to be inadequate in most daylit spaces. I usually drop this to around 1e-9 (and using high -ad settings) and get much better results when applying daylight coefficients to sky patches (the results are typically higher). When using a method where the direct sun is removed from a sky coefficient calculation that includes both sun and sky, to be added in via a third phase, this will generally also avoid the presence of negative numbers in the final results. You can easily test the -lw setting to see if it makes a difference.

Rick Mistrick

···

-----Original Message-----
From: Greg Ward [mailto:[email protected]]
Sent: Wednesday, March 23, 2016 11:21 AM
To: Radiance general discussion <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error

Good catch, Galen! I'm glad you found the explanation. Meanwhile, I changed the code so that (hopefully) we won't get runaway ray counts even if there are surfaces with >100% reflectance or transmittance.

Cheers!
-Greg

From: Galen Burrell <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error
Date: March 23, 2016 5:48:33 AM PDT

Just an update on this error: After searching through my materials definitions per Greg's suggestion, I found that I had a glass definition of 95% light transmission, which resulted in a transmissivity of > 1. Doh! After reducing the light transmission to something more sensible I stopped getting the error. Thanks Greg!

Date: Thu, 17 Mar 2016 14:49:01 -0700
From: Greg Ward <[email protected]>
To: Radiance general discussion <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error

Hi Galen,

I've never seen rcontrib run out of memory in this way before. What are your rendering options for rcontrib? Do any of your surfaces have greater than 100% reflectance or transmittance? This could cause the number of rays to increase rather than decrease on multiple bounces, which could lead to exponential memory growth during ray tree evaluation.

-Greg

From: Galen Burrell <[email protected]>
Subject: [Radiance-general] rcontrib malloc error
Date: March 17, 2016 1:39:45 PM PDT

I?m running view matrix calculations using rfluxmtx/rcontrib for the 3-phase method and keep getting one of two errors:

1)
rcontrib(2346) malloc: *** mmap(size=a really big number) failed
(error code=12)
*** error: can?t allocate region
*** set a breakpoint in malloc_error_break to debug
rcontrib: system ? out of memory in samp_hemi: Cannot allocate memory
rcontrib: system ? read error from render process
Killed: 9

2)
rcontrib: system ? read error from render process
Killed: 9

I can?t establish a pattern for why/when these two errors occur.

Oddly, I don?t always get the error with some gridpoint files (with 5900 points), but I consistently get it with another gridpoint file (with 7500 points). I don?t understand why grid size would influence memory though??

System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with
Yosemite installed Latest Radiance HEAD release; although the error
repeats on older versions from 6 months ago rcontrib settings are: ?n
12 ?ab 12 ?ad 100000 ?lw 1e-5 ?I+

Any clues on why I?m getting these errors?

Thank you!

Galen

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Galen,

I'd like to add that Rick is exactly right. An additional note: -lw is more
important for big spaces than small spaces.

--Tim

···

On Tue, Mar 29, 2016 at 8:27 AM, Richard Mistrick <[email protected]> wrote:

Galen,

Here is something else that might improve the quality of your gridpoint
calculations. I have seen "-lw 1e-5" to be inadequate in most daylit
spaces. I usually drop this to around 1e-9 (and using high -ad settings)
and get much better results when applying daylight coefficients to sky
patches (the results are typically higher). When using a method where the
direct sun is removed from a sky coefficient calculation that includes both
sun and sky, to be added in via a third phase, this will generally also
avoid the presence of negative numbers in the final results. You can
easily test the -lw setting to see if it makes a difference.

Rick Mistrick

-----Original Message-----
From: Greg Ward [mailto:[email protected]]
Sent: Wednesday, March 23, 2016 11:21 AM
To: Radiance general discussion <[email protected]>
Subject: Re: [Radiance-general] rcontrib malloc error

Good catch, Galen! I'm glad you found the explanation. Meanwhile, I
changed the code so that (hopefully) we won't get runaway ray counts even
if there are surfaces with >100% reflectance or transmittance.

Cheers!
-Greg

> From: Galen Burrell <[email protected]>
> Subject: Re: [Radiance-general] rcontrib malloc error
> Date: March 23, 2016 5:48:33 AM PDT
>
> Just an update on this error: After searching through my materials
definitions per Greg's suggestion, I found that I had a glass definition of
95% light transmission, which resulted in a transmissivity of > 1. Doh!
After reducing the light transmission to something more sensible I stopped
getting the error. Thanks Greg!
>
> Date: Thu, 17 Mar 2016 14:49:01 -0700
> From: Greg Ward <[email protected]>
> To: Radiance general discussion <[email protected]>
> Subject: Re: [Radiance-general] rcontrib malloc error
>
> Hi Galen,
>
> I've never seen rcontrib run out of memory in this way before. What are
your rendering options for rcontrib? Do any of your surfaces have greater
than 100% reflectance or transmittance? This could cause the number of
rays to increase rather than decrease on multiple bounces, which could lead
to exponential memory growth during ray tree evaluation.
>
> -Greg
>
>> From: Galen Burrell <[email protected]>
>> Subject: [Radiance-general] rcontrib malloc error
>> Date: March 17, 2016 1:39:45 PM PDT
>>
>> I?m running view matrix calculations using rfluxmtx/rcontrib for the
3-phase method and keep getting one of two errors:
>>
>> 1)
>> rcontrib(2346) malloc: *** mmap(size=a really big number) failed
>> (error code=12)
>> *** error: can?t allocate region
>> *** set a breakpoint in malloc_error_break to debug
>> rcontrib: system ? out of memory in samp_hemi: Cannot allocate memory
>> rcontrib: system ? read error from render process
>> Killed: 9
>>
>> 2)
>> rcontrib: system ? read error from render process
>> Killed: 9
>>
>> I can?t establish a pattern for why/when these two errors occur.
>>
>> Oddly, I don?t always get the error with some gridpoint files (with
5900 points), but I consistently get it with another gridpoint file (with
7500 points). I don?t understand why grid size would influence memory
though??
>>
>> System is Mac OS 10.7.5 (2 x 6-core); error repeats on my laptop with
>> Yosemite installed Latest Radiance HEAD release; although the error
>> repeats on older versions from 6 months ago rcontrib settings are: ?n
>> 12 ?ab 12 ?ad 100000 ?lw 1e-5 ?I+
>>
>> Any clues on why I?m getting these errors?
>>
>> Thank you!
>>
>> Galen

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general