Five-phase Method Tutorial: Errors in generating direct sun coefficient matrix

Hi Alireza,

Thanks for including your commands and the full errors reported -- this is helpful. I will offer my suggestions inline, below:

From: Alireza Hashemloo <[email protected]>
Subject: [Radiance-general] Five-phase Method Tutorial: Errors in generating direct sun coefficient matrix
Date: March 5, 2016 8:59:34 PM PST

Hello,

I am pursuing the Five-phase Method Tutorial (http://www.radiance-online.org/learning/tutorials/fivephasetutorialfiles/Tutorial-FivePhaseMethod_v2.pdf\) and I have encountered the following errors in my attempt to generate Direct sun coefficient matrix in Example 2:

Question/Error #1:

OS: Windows 7 (64-bit)
Radiance: 5.08
Command Line Environment: Git Bash

Command:

$ vwrays -c 4 -pj 1 -fa -vf views/view_A.vf -x 600 -y 600 | rtrace -h -opn -fa -ab 0 model_nosuns.oct | rcontrib `vwrays -vf views/view_A.vf -x 600 -y 600 -d` -fac -fo -o viewpics_ds/back_%04d.hdr -e MF:6 -f reinhart.cal -b rbin -bn Nrbins -m solar -c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 model_suns.oct

Error:
rtrace: warning - no light sources found
rtrace: system - write error: Invalid argument

The warning from the first rtrace command is nothing to worry about. You are using it to compute positions and normals, not values, so it doesn't matter. You can use the -w- on that command to avoid the warning. I also recommend using -opN rather than -opn for efficiency reasons.

The "Invalid argument" error is mysterious. I'm not sure why rtrace would report that unless the pipe to rcontrib was not being opened. Do the single back-quotes work in Git Bash? What happens if you leave off the `vwrays -vf views/view_A.vf -x 600 -y 600 -d` part? (It wouldn't be correct -- just wondering if that has something to do with the failure.)

Description: I have tested the same command on Windows shell (cmd.exe), and I receive an 'rcontrib' error in addition to the 'rtrace' errors that I encounter in Git Bash:

OS: Windows 7 (64-bit)
Radiance: 5.08
Command Line Environment: Windows shell (cmd.exe)

Command:

vwrays -c 4 -pj 1 -fa -vf views/view_A.vf -x 600 -y 600 | rtrace -h -opn -fa -ab 0 model_nosuns.oct | rcontrib `vwrays -vf views/view_A.vf -x 600 -y 600 -d` -fac -fo -o viewpics_ds/back_%04d.hdr -e MF:6 -f reinhart.cal -b rbin -bn Nrbins -m solar -c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 model_suns.oct

Error:
rcontrib: fatal - missing required modifier argument
rtrace: warning - no light sources found
rtrace: system - write error: Invalid argument

My guess is that your command-line is too long for cmd.exe, so the rcontrib command is getting truncated before it gets to its -m argument. Try running it as two commands, i.e.:

vwrays -c 4 -pj 1 -fa -vf views/view_A.vf -x 600 -y 600 | rtrace -h -w -opN -fa -ab 0 model_nosuns.oct > points.txt
rcontrib -x 600 -y 600 -fac -fo -o viewpics_ds/back_%04d.hdr -e MF:6 -f reinhart.cal -b rbin -bn Nrbins -m solar -c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 model_suns.oct < points.txt

(Replace the "-x 600 -y 600" with the output of "vwrays -vf views/view_A.vf -x 600 -y 600 -d" as I'm pretty sure back-quotes don't work with cmd.exe.)

Question #2:

The original command in the tutorial includes the "-n $procs" as part of the options passed to the 'rcontrib' call. However, I do not know where I can find the definition for "$procs" ?
According to the Radiance documentation available for rcontrib, the -n option is specified for multiple processes to accelerate computation on a shared memory machine. The Git Bash on Windows can handle 3200 open files as the maximum value passed to a "ulimit" call. But apparently, windows shell (cmd.exe) must be able to handle more open processes.

You will have to go from MF:6 to MF:4 using Git Bash, since MF:6 corresponds to over 5000 open files. I do not think you can have multiple processes under Windows using rcontrib, no matter which shell you use, so -n 1 (the default) is all you get.

Good luck!
-Greg

Alireza,

This behavior is consistent with running up against limitations on open files. I don't know how to change the limit on Windows, but found this:

Andy

···

On Mar 9, 2016, at 12:53 AM, Alireza Hashemloo <[email protected]> wrote:

Hi Greg and Rob,

Thank you for your responses:

- It does not appear that the single back quotes are the source of the problem as I tested the command following your recommendation without passing the `vwrays -vf views/view_A.vf -x 600 -y 600 -d` to "rcontrib" and the rtrace "invalid argument" error still appears.

- I have tested the approach to split the command into two parts as you had suggested:

part 1: vwrays -c 4 -pj 1 -fa -vf views/view_A.vf -x 600 -y 600 | rtrace -h -opN -fa -ab 0 model_nosuns.oct > points.txt
part 2: rcontrib -x 600 -y 600 -fac -fo -o viewpics_ds/back_%04d.hdr -e MF:4 -f reinhart.cal -b rbin -bn Nrbins -m solar -c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 model_suns.oct < points.txt

the part 1 appears to work properly considering that a "points.txt" file is generated. The part 2, however, does not appear to work: although I do not receive any error messages on my command line, the rcontrib call is completed immediately after it is run and 508 hdr images are generated in the given directory. However, each .hdr file is only 1 KB and the attempt to opening any of them is failed. It does not appear that the files have actually been written successfully.

In addition, could you elaborate about your suggestion to " Replace the "-x 600 -y 600" with the output of "vwrays -vf views/view_A.vf -x 600 -y 600 -d" " in part 2 of the command?
Finally, I was wondering where the definition for the "solar" modifier as the target light source passed to the 'rcontrib' call (-m solar) is made in the previous steps of the tutorial?

Thank you for your help.
Best Regards,

Alireza

On Mon, Mar 7, 2016 at 8:17 AM, Guglielmetti, Robert <[email protected]> wrote:
On 3/6/16, 9:53 AM, "Greg Ward" <[email protected]<mailto:[email protected]>> wrote:

I do not think you can have multiple processes under Windows using rcontrib, no matter which shell you use, so -n 1 (the default) is all you get.

That is true; rcontrib on Windows will silently ignore -n settings > 1. On OpenStudio, you will get a heads up about this.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

Hi Alireza,

Did you generate the XML files yourself? The ones I downloaded via the current version of the 5-phase bash script don't report this warning, which I would say indicates a problem. You can download the XML files using:

  curl -o bsdf/fullwindow_t45.xml http://www.radiance-online.org/learning/tutorials/fivephasetutorialfiles/fullwindow_t45.xml
  curl -o bsdf/fullwindow.xml http://www.radiance-online.org/learning/tutorials/fivephasetutorialfiles/fullwindow.xml

I can't speculate what is going wrong with your genBSDF command if you created these yourself, unless you send me the related input files.

Cheers,
-Greg

···

From: Alireza Hashemloo <[email protected]>
Subject: Re: [Radiance-general] Five-phase Method Tutorial: Errors in generating direct sun coefficient matrix
Date: March 20, 2016 1:54:42 PM PDT

Andy,

Thank you for your response. I have migrated to Mac in order to complete this step of the Five phase method tutorial.
I have pursued the tutorial with the exact scene descriptions and simulation settings.
I have encountered the following warning/error as follows:

after increasing the limit on the open files in Terminal via the "ulimit -n 9999" command, I have called the following command as shown in the tutorial:

Five-phase Method Tutoiral | Example 2 - Rendering | Direct Sun Coefficient Matrix (page 19) :

command:

$ vwrays -c 4 -pj 1 -fa -vf views/back.vf -x 500 -y 500 | \
rtrace -h- -opn -faa -ab 0 octs/model_nosuns.oct | \
rcontrib `vwrays -vf views/back.vf -x 500 -y 500 -d` -n $procs -fac -fo \
-o viewpics_ds/back_%04d.hdr -e MF:6 -f reinhart.cal -b rbin -bn Nrbins -m solar \
-c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 octs/model_suns.oct

Warnings:

rcontrib: warning - BSDF "bsdf/fullwindow_t45" has 169.4 front reflection rcontrib: warning \- BSDF &quot;bsdf/fullwindow\_t45&quot; has 218\.8 rear reflection

Error Description:

Although the given rcontrib warnings do not terminate the command call, it does not appear to me that any progress is made leaving the command running for over 24 hours.

Question:

1- What would be the reason for the tensor-tree BSDF file (fullwindow_t45.xml) generated in the previous steps of the tutorial to have over 100% reflection as shown in the rcontrib warnings?
2- How can I remedy this problem in this stage of the tutorial to generate the 5185 direct sun coefficient images?

Best Regards,
Alireza

On Wed, Mar 9, 2016 at 7:25 AM, Andy McNeil <[email protected]> wrote:
Alireza,

This behavior is consistent with running up against limitations on open files. I don't know how to change the limit on Windows, but found this:
c++ - Is there a limit on number of open files in Windows - Stack Overflow

Andy

On Mar 9, 2016, at 12:53 AM, Alireza Hashemloo <[email protected]> wrote:

Hi Greg and Rob,

Thank you for your responses:

- It does not appear that the single back quotes are the source of the problem as I tested the command following your recommendation without passing the `vwrays -vf views/view_A.vf -x 600 -y 600 -d` to "rcontrib" and the rtrace "invalid argument" error still appears.

- I have tested the approach to split the command into two parts as you had suggested:

part 1: vwrays -c 4 -pj 1 -fa -vf views/view_A.vf -x 600 -y 600 | rtrace -h -opN -fa -ab 0 model_nosuns.oct > points.txt
part 2: rcontrib -x 600 -y 600 -fac -fo -o viewpics_ds/back_%04d.hdr -e MF:4 -f reinhart.cal -b rbin -bn Nrbins -m solar -c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 model_suns.oct < points.txt

the part 1 appears to work properly considering that a "points.txt" file is generated. The part 2, however, does not appear to work: although I do not receive any error messages on my command line, the rcontrib call is completed immediately after it is run and 508 hdr images are generated in the given directory. However, each .hdr file is only 1 KB and the attempt to opening any of them is failed. It does not appear that the files have actually been written successfully.

In addition, could you elaborate about your suggestion to " Replace the "-x 600 -y 600" with the output of "vwrays -vf views/view_A.vf -x 600 -y 600 -d" " in part 2 of the command?
Finally, I was wondering where the definition for the "solar" modifier as the target light source passed to the 'rcontrib' call (-m solar) is made in the previous steps of the tutorial?

Thank you for your help.
Best Regards,

Alireza

On Mon, Mar 7, 2016 at 8:17 AM, Guglielmetti, Robert <[email protected]> wrote:
On 3/6/16, 9:53 AM, "Greg Ward" <[email protected]<mailto:[email protected]>> wrote:

I do not think you can have multiple processes under Windows using rcontrib, no matter which shell you use, so -n 1 (the default) is all you get.

That is true; rcontrib on Windows will silently ignore -n settings > 1. On OpenStudio, you will get a heads up about this.

Responses inline...

From: Alireza Hashemloo <[email protected]>
Subject: Re: [Radiance-general] Five-phase Method Tutorial: Errors in generating direct sun coefficient matrix
Date: March 20, 2016 3:45:01 PM PDT

Hi Greg,

Thank you for your response.

1- I was unsuccessful in my attempt to download the files via the "curl" call. I received the error : "No such file or directory" followed by "curl: (23) Failed writing body (0 != 1460)". May I ask if you could send me the two xml files as
an attachment to my email address?

The directory need to exist from where you run the curl command. I.e., make sure the "bsdf" directory already exists. Failing that, I will send you the files in an e-mail.

2- The tensor tree xml file (fullwindow_t45.xml) is generated via the following command:

$ genBSDF +f +b -geom meter -dim 0.5 3.5 1 2 -.3 0 -t4 5 bsdf/fullwindow.rad > bsdf/fullwindow_t45.xml

I can send you the related input files via a Google Drive link to your email address if that works for you in order to diagnose the cause for high reflection values.

Please e-mail me the files, which are not large.

3- Generating the direct sun coefficient images, how can I generate a BSDF file with Klems' subdivision scheme with higher resolution (each Klems' patch subdivided into four smaller patches) as a replacement to using a tensor-tree file?

There is no easy way to get more than 145 Klems patches with the current software. Sorry!

···

Thank you,
Alireza

On Sun, Mar 20, 2016 at 3:12 PM, Greg Ward <[email protected]> wrote:
Hi Alireza,

Did you generate the XML files yourself? The ones I downloaded via the current version of the 5-phase bash script don't report this warning, which I would say indicates a problem. You can download the XML files using:

  curl -o bsdf/fullwindow_t45.xml http://www.radiance-online.org/learning/tutorials/fivephasetutorialfiles/fullwindow_t45.xml
  curl -o bsdf/fullwindow.xml http://www.radiance-online.org/learning/tutorials/fivephasetutorialfiles/fullwindow.xml

I can't speculate what is going wrong with your genBSDF command if you created these yourself, unless you send me the related input files.

Cheers,
-Greg

From: Alireza Hashemloo <[email protected]>
Subject: Re: [Radiance-general] Five-phase Method Tutorial: Errors in generating direct sun coefficient matrix
Date: March 20, 2016 1:54:42 PM PDT

Andy,

Thank you for your response. I have migrated to Mac in order to complete this step of the Five phase method tutorial.
I have pursued the tutorial with the exact scene descriptions and simulation settings.
I have encountered the following warning/error as follows:

after increasing the limit on the open files in Terminal via the "ulimit -n 9999" command, I have called the following command as shown in the tutorial:

Five-phase Method Tutoiral | Example 2 - Rendering | Direct Sun Coefficient Matrix (page 19) :

command:

$ vwrays -c 4 -pj 1 -fa -vf views/back.vf -x 500 -y 500 | \
rtrace -h- -opn -faa -ab 0 octs/model_nosuns.oct | \
rcontrib `vwrays -vf views/back.vf -x 500 -y 500 -d` -n $procs -fac -fo \
-o viewpics_ds/back_%04d.hdr -e MF:6 -f reinhart.cal -b rbin -bn Nrbins -m solar \
-c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 octs/model_suns.oct

Warnings:

rcontrib: warning - BSDF "bsdf/fullwindow_t45" has 169.4 front reflection rcontrib: warning \- BSDF &quot;bsdf/fullwindow\_t45&quot; has 218\.8 rear reflection

Error Description:

Although the given rcontrib warnings do not terminate the command call, it does not appear to me that any progress is made leaving the command running for over 24 hours.

Question:

1- What would be the reason for the tensor-tree BSDF file (fullwindow_t45.xml) generated in the previous steps of the tutorial to have over 100% reflection as shown in the rcontrib warnings?
2- How can I remedy this problem in this stage of the tutorial to generate the 5185 direct sun coefficient images?

Best Regards,
Alireza

On Wed, Mar 9, 2016 at 7:25 AM, Andy McNeil <[email protected]> wrote:
Alireza,

This behavior is consistent with running up against limitations on open files. I don't know how to change the limit on Windows, but found this:
c++ - Is there a limit on number of open files in Windows - Stack Overflow

Andy

On Mar 9, 2016, at 12:53 AM, Alireza Hashemloo <[email protected]> wrote:

Hi Greg and Rob,

Thank you for your responses:

- It does not appear that the single back quotes are the source of the problem as I tested the command following your recommendation without passing the `vwrays -vf views/view_A.vf -x 600 -y 600 -d` to "rcontrib" and the rtrace "invalid argument" error still appears.

- I have tested the approach to split the command into two parts as you had suggested:

part 1: vwrays -c 4 -pj 1 -fa -vf views/view_A.vf -x 600 -y 600 | rtrace -h -opN -fa -ab 0 model_nosuns.oct > points.txt
part 2: rcontrib -x 600 -y 600 -fac -fo -o viewpics_ds/back_%04d.hdr -e MF:4 -f reinhart.cal -b rbin -bn Nrbins -m solar -c 4 -I -ab 1 -ad 100 -dt 0 -dc 1 -lw 1e-2 model_suns.oct < points.txt

the part 1 appears to work properly considering that a "points.txt" file is generated. The part 2, however, does not appear to work: although I do not receive any error messages on my command line, the rcontrib call is completed immediately after it is run and 508 hdr images are generated in the given directory. However, each .hdr file is only 1 KB and the attempt to opening any of them is failed. It does not appear that the files have actually been written successfully.

In addition, could you elaborate about your suggestion to " Replace the "-x 600 -y 600" with the output of "vwrays -vf views/view_A.vf -x 600 -y 600 -d" " in part 2 of the command?
Finally, I was wondering where the definition for the "solar" modifier as the target light source passed to the 'rcontrib' call (-m solar) is made in the previous steps of the tutorial?

Thank you for your help.
Best Regards,

Alireza

On Mon, Mar 7, 2016 at 8:17 AM, Guglielmetti, Robert <[email protected]> wrote:
On 3/6/16, 9:53 AM, "Greg Ward" <[email protected]<mailto:[email protected]>> wrote:

I do not think you can have multiple processes under Windows using rcontrib, no matter which shell you use, so -n 1 (the default) is all you get.

That is true; rcontrib on Windows will silently ignore -n settings > 1. On OpenStudio, you will get a heads up about this.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general