Error using BSDF material (ezxml.c)

Hi Greg, hi list,

I just wanted to use a BSDF material with an XML-file created by genBSDF,
but rvu throws the following error:

*rvu: fatal - File format error: BSDF "./test.xml" [error near line 1]:
root tag missing*

I could trace back the problem to line 499 in common/ezxml.c from where the
error
is returned:

* if (! *s) return ezxml_err(root, s, "root tag missing");*

The problematic call to the function *ezxml_parse_str* is the one in line
655. I checked
if using the else-branch there (i.e. "mmap failed, read file into memory")
solves the problem --
yes and no. The process throws other errors then:

*rvu: warning - Illegal argument: BSDF "test": unsupported
IncidentDataStructure for BSDF "tst"*
(for a Klems matrix BSDF)

*rvu: warning - Illegal argument for BSDF "test"*
(for a variable resolution BSDF isotropic and anisotropic)

The line numbers refer to the current head version...

Any clue what's going on here??

Cheers,
David

Hi David,

Thanks for sharing your files. The only problem I found was that your orientation vector was perpendicular to the plane of your BSDF. The error I got was a fairly useless "Illegal argument" warning. I just checked in a new version of src/rt/m_bsdf.c that offers a more accurate warning.

Meanwhile, try changing the orientation vector on your test BSDF to "0 1 0" or "1 0 0" so that it is in the plane of the surface and see if you still have problems.

Thanks for your help -- always appreciated!
-Greg

···

From: David Geisler-Moroder <[email protected]>
Date: April 24, 2012 12:30:47 AM PDT

Hi Greg,

I attached 4 BSDFs that I used yesterday:
klems_old.xml: a Klems-Matrix BSDF that I generated last year
klems.xml: a test Klems-Matrix BSDF I generated yesterday with the latest head distribution
tree_iso.xml: a test isotropic tensor tree BSDF I generated yesterday with the latest head distribution (-t3)
tree_aniso.xml: a test anisotropic tensor tree BSDF I generated yesterday with the latest head distribution (-t4)

Running objview check.rad crashed with the described errors...

I used binaries that I compiled yesterday on a 32bit Ubuntu 10.04 LTS running in a Virtual Box
[Linux version 2.6.32-31-generic (buildd@rothera) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) )].

Today in the morning I also tried running it on our compute server (64bit OpenSUSE):
Linux version 2.6.37.6-0.11-desktop (geeko@buildhost) (gcc version 4.5.1 20101208 [gcc-4_5-branch revision 167585] (SUSE Linux) )
The binaries there were compiled from the HEAD distribution of March 12.
There the "root tag missing" error does not occur, but the other two do.

Hope that's of any help...

Best,
David

Am 23. April 2012 18:05 schrieb Gregory J. Ward <[email protected]>:
Hi David,

Thanks for reporting this error. Can you send me the XML file that is giving you trouble?

What machine are you running on?

Cheers,
-Greg

Hi Greg,

thanks for spotting my stupid mistake!!!
A corrected orientation vector solves the "illegal argument" problems and
thus everything
works fine on the OpenSuse machine.

However, the error "root tag missing" is still thrown at the Ubuntu
machine. I did not have
enough time today, but I'll try to compile the newest head
- on the OpenSuse machine (and check if it still works)
- on the Ubuntu machine using a newer version of gcc
tomorrow.

I'll get back to you when I have the results.

Cheers,
David

···

Am 24. April 2012 17:39 schrieb Gregory J. Ward <[email protected]>:

Hi David,

Thanks for sharing your files. The only problem I found was that your
orientation vector was perpendicular to the plane of your BSDF. The error
I got was a fairly useless "Illegal argument" warning. I just checked in a
new version of src/rt/m_bsdf.c that offers a more accurate warning.

Meanwhile, try changing the orientation vector on your test BSDF to "0 1
0" or "1 0 0" so that it is in the plane of the surface and see if you
still have problems.

Thanks for your help -- always appreciated!
-Greg

*From: *David Geisler-Moroder <[email protected]>

*Date: *April 24, 2012 12:30:47 AM PDT

*
*

Hi Greg,

I attached 4 BSDFs that I used yesterday:
klems_old.xml: a Klems-Matrix BSDF that I generated last year
klems.xml: a test Klems-Matrix BSDF I generated yesterday with the latest
head distribution
tree_iso.xml: a test isotropic tensor tree BSDF I generated yesterday with
the latest head distribution (-t3)
tree_aniso.xml: a test anisotropic tensor tree BSDF I generated yesterday
with the latest head distribution (-t4)

Running objview check.rad crashed with the described errors...

I used binaries that I compiled yesterday on a 32bit Ubuntu 10.04 LTS
running in a Virtual Box
[Linux version 2.6.32-31-generic (buildd@rothera) (gcc version 4.4.3
(Ubuntu 4.4.3-4ubuntu5) )].

Today in the morning I also tried running it on our compute server (64bit
OpenSUSE):
Linux version 2.6.37.6-0.11-desktop (geeko@buildhost) (gcc version 4.5.1
20101208 [gcc-4_5-branch revision 167585] (SUSE Linux) )
The binaries there were compiled from the HEAD distribution of March 12.
There the "*root tag missing*" error does not occur, but the other two do.

Hope that's of any help...

Best,
David

Am 23. April 2012 18:05 schrieb Gregory J. Ward <[email protected]>:

Hi David,

Thanks for reporting this error. Can you send me the XML file that is
giving you trouble?

What machine are you running on?

Cheers,
-Greg

Hi Greg,

new/old head versions and different gcc versions do not matter.

The difference is still between 32bit and 64bit.
Printing out the pointer adress and first char before line 498 in ezxml.c

···

*
        fprintf(stderr, " s = %p\n", s);
        fprintf(stderr, " s[0] = %c\n", s[0]);
498 while (*s && *s != '<') s++; /* find first tag */
*
yields:
(on 32bit):*
s = 0xb76fe000
s[0] =
*(on 64bit):*
s = 0x7f1179377000
s[0] = <
*
It seems that the contents of the XML-File are not parsed correctly on the
32bit system.

Maybe you have some idea what/where the problem could be?

Best,
David

Am 24. April 2012 22:02 schrieb David Geisler-Moroder < [email protected]>:

Hi Greg,

thanks for spotting my stupid mistake!!!
A corrected orientation vector solves the "illegal argument" problems and
thus everything
works fine on the OpenSuse machine.

However, the error "root tag missing" is still thrown at the Ubuntu
machine. I did not have
enough time today, but I'll try to compile the newest head
- on the OpenSuse machine (and check if it still works)
- on the Ubuntu machine using a newer version of gcc
tomorrow.

I'll get back to you when I have the results.

Cheers,
David

Am 24. April 2012 17:39 schrieb Gregory J. Ward <[email protected]>:

Hi David,

Thanks for sharing your files. The only problem I found was that your
orientation vector was perpendicular to the plane of your BSDF. The error
I got was a fairly useless "Illegal argument" warning. I just checked in a
new version of src/rt/m_bsdf.c that offers a more accurate warning.

Meanwhile, try changing the orientation vector on your test BSDF to "0 1
0" or "1 0 0" so that it is in the plane of the surface and see if you
still have problems.

Thanks for your help -- always appreciated!
-Greg

*From: *David Geisler-Moroder <[email protected]>

*Date: *April 24, 2012 12:30:47 AM PDT

*
*

Hi Greg,

I attached 4 BSDFs that I used yesterday:
klems_old.xml: a Klems-Matrix BSDF that I generated last year
klems.xml: a test Klems-Matrix BSDF I generated yesterday with the latest
head distribution
tree_iso.xml: a test isotropic tensor tree BSDF I generated yesterday
with the latest head distribution (-t3)
tree_aniso.xml: a test anisotropic tensor tree BSDF I generated yesterday
with the latest head distribution (-t4)

Running objview check.rad crashed with the described errors...

I used binaries that I compiled yesterday on a 32bit Ubuntu 10.04 LTS
running in a Virtual Box
[Linux version 2.6.32-31-generic (buildd@rothera) (gcc version 4.4.3
(Ubuntu 4.4.3-4ubuntu5) )].

Today in the morning I also tried running it on our compute server (64bit
OpenSUSE):
Linux version 2.6.37.6-0.11-desktop (geeko@buildhost) (gcc version 4.5.1
20101208 [gcc-4_5-branch revision 167585] (SUSE Linux) )
The binaries there were compiled from the HEAD distribution of March 12.
There the "*root tag missing*" error does not occur, but the other two
do.

Hope that's of any help...

Best,
David

Am 23. April 2012 18:05 schrieb Gregory J. Ward <[email protected]>:

Hi David,

Thanks for reporting this error. Can you send me the XML file that is
giving you trouble?

What machine are you running on?

Cheers,
-Greg

--
DI Dr. David Geisler-Moroder
Hofwaldweg 14/20
6020 Innsbruck

Hi David,

I have no idea what's going wrong. You did a test before where you forced the program to take the "else" clause to avoid mmap(), and ended up with a different set of errors. Do you not get those errors when you pass the same BSDF through the 64-bit version? This is puzzling to me.

The fact that mmap() is not returning the file contents in 32-bit mode is very disturbing. This sounds like a header issue, but I have no way to debug it without downloading and installing ubuntu on a virtual machine. You could try commenting out the madvise() calls, not that I expect it to make a difference...

Avoiding the mmap() call should fix the problem, so it's worth trying that again.

Cheers,
-Greg

···

From: David Geisler-Moroder <[email protected]>
Date: April 25, 2012 2:19:29 AM PDT

Hi Greg,

new/old head versions and different gcc versions do not matter.

The difference is still between 32bit and 64bit.
Printing out the pointer adress and first char before line 498 in ezxml.c

        fprintf(stderr, " s = %p\n", s);
        fprintf(stderr, " s[0] = %c\n", s[0]);
498 while (*s && *s != '<') s++; /* find first tag */

yields:
(on 32bit):
s = 0xb76fe000
s[0] =
(on 64bit):
s = 0x7f1179377000
s[0] = <

It seems that the contents of the XML-File are not parsed correctly on the 32bit system.

Maybe you have some idea what/where the problem could be?

Best,
David

Hi Greg,

the other set of errors was only related to the wrong orientation vector
specified for the BSDF material.
So the only difference that is really there is the one related to mmap().

I also thought about avoiding mmap() on the 32bit machine. For Cygwin this
is done similarly in
common/ezxml.h by defining EZXML_NOMMAP. Maybe that's the best solution for
now.

Thanks for your help!

Cheers,
David

···

Am 25. April 2012 17:50 schrieb Gregory J. Ward <[email protected]>:

Hi David,

I have no idea what's going wrong. You did a test before where you forced
the program to take the "else" clause to avoid mmap(), and ended up with a
different set of errors. Do you not get those errors when you pass the
same BSDF through the 64-bit version? This is puzzling to me.

The fact that mmap() is not returning the file contents in 32-bit mode is
very disturbing. This sounds like a header issue, but I have no way to
debug it without downloading and installing ubuntu on a virtual machine.
You could try commenting out the madvise() calls, not that I expect it to
make a difference...

Avoiding the mmap() call should fix the problem, so it's worth trying that
again.

Cheers,
-Greg

*From: *David Geisler-Moroder <[email protected]>

*Date: *April 25, 2012 2:19:29 AM PDT

*
*

Hi Greg,

new/old head versions and different gcc versions do not matter.

The difference is still between 32bit and 64bit.
Printing out the pointer adress and first char before line 498 in ezxml.c
*
        fprintf(stderr, " s = %p\n", s);
        fprintf(stderr, " s[0] = %c\n", s[0]);
498 while (*s && *s != '<') s++; /* find first tag */
*
yields:
(on 32bit):*
s = 0xb76fe000
s[0] =
*(on 64bit):*
s = 0x7f1179377000
s[0] = <
*
It seems that the contents of the XML-File are not parsed correctly on the
32bit system.

Maybe you have some idea what/where the problem could be?

Best,
David

Hi David,

I tend to agree that the easiest fix is to take out the mmap() call for your system. We should log a bug to the Ubuntu developers, though.

You can put a -DEZXML_NOMMAP option in your rmake if you don't want to mess with the header.

Best,
-Greg

···

From: David Geisler-Moroder <[email protected]>
Date: April 25, 2012 9:30:32 AM PDT

Hi Greg,

the other set of errors was only related to the wrong orientation vector specified for the BSDF material.
So the only difference that is really there is the one related to mmap().

I also thought about avoiding mmap() on the 32bit machine. For Cygwin this is done similarly in
common/ezxml.h by defining EZXML_NOMMAP. Maybe that's the best solution for now.

Thanks for your help!

Cheers,
David

Hi Greg,

I recompiled the head with the NOMMAP define and everything works fine now.
Thanks again for all your help!!

By the way, I just ran the same rvu-command using the 32bit Windows
binaries provided
by NREL on
http://openstudio.nrel.gov/getting-started-developer/getting-started-radiance
(Rob G. - thanks a lot for offering those!!) -- no problems there as well.

Cheers,
David

···

Am 25. April 2012 19:29 schrieb Gregory J. Ward <[email protected]>:

Hi David,

I tend to agree that the easiest fix is to take out the mmap() call for
your system. We should log a bug to the Ubuntu developers, though.

You can put a -DEZXML_NOMMAP option in your rmake if you don't want to
mess with the header.

Best,
-Greg

Hi David,

I'm pretty sure the guys who developed the build system for NREL didn't use the NOMMAP #define, so must be their system doesn't have whatever header issues yours is running afoul of in 32-bit mode.

Thanks for the update -- glad it's working for you now!

Cheers,
-Greg

P.S. A side note on ezxml is that I found a problem after the 4.1 release that caused very long XML files with DOS eol (CR-LF) to be impossibly slow to load. This is fixed in the latest HEAD, which NREL is using.

···

From: David Geisler-Moroder <[email protected]>
Date: April 25, 2012 11:28:47 PM PDT

Hi Greg,

I recompiled the head with the NOMMAP define and everything works fine now.
Thanks again for all your help!!

By the way, I just ran the same rvu-command using the 32bit Windows binaries provided
by NREL on http://openstudio.nrel.gov/getting-started-developer/getting-started-radiance
(Rob G. - thanks a lot for offering those!!) -- no problems there as well.

Cheers,
David

Am 25. April 2012 19:29 schrieb Gregory J. Ward <[email protected]>:
Hi David,

I tend to agree that the easiest fix is to take out the mmap() call for your system. We should log a bug to the Ubuntu developers, though.

You can put a -DEZXML_NOMMAP option in your rmake if you don't want to mess with the header.

Best,
-Greg

By the way, I just ran the same rvu-command using the 32bit Windows binaries provided
by NREL on http://openstudio.nrel.gov/getting-started-developer/getting-started-radiance
(Rob G. - thanks a lot for offering those!!) -- no problems there as well.

Thank DOE and the Bonneville Power Authority for funding the work! Greg just merged all the changes we've made to get Radiance reliably building on all platforms with CMake, and I hope to post updated packages today. =)

Hi Greg, hi list,

a short follow-up on the NOMMAP-issue I had in April.

When re-compiling a HEAD version currently I forgot to add the NOMMAP
compile flag. Again, I ended up with the same error that I had in April,
but not always...
To cut a long story short, I could trace back the problem to something that
probably has nothing to do with RADIANCE and 32 or 64 bit. I'm running
RADIANCE on Ubuntu in a VirtualBox on a Win7 host system. Whenever the
XML-file to load is in a shared folder (via a VirtualBox guest addition)
and physically stored on the network directory, the error occurs. However,
copying the same file to a local directory in Ubuntu and using this one
everything works fine.

I have no idea why this EZXML MMAP doesn't work on shared folders or if
this has something to do with the formatting (NTFS vs. EXT4)??

Anyway, as a work-around specifying the NOMMAP compile flag or just working
on the local disk works...

Cheers,
David

···

2012/4/26 Gregory J. Ward <[email protected]>

Hi David,

I'm pretty sure the guys who developed the build system for NREL didn't
use the NOMMAP #define, so must be their system doesn't have whatever
header issues yours is running afoul of in 32-bit mode.

Thanks for the update -- glad it's working for you now!

Cheers,
-Greg

P.S. A side note on ezxml is that I found a problem after the 4.1 release
that caused very long XML files with DOS eol (CR-LF) to be impossibly slow
to load. This is fixed in the latest HEAD, which NREL is using.

*From: *David Geisler-Moroder <[email protected]>

*Date: *April 25, 2012 11:28:47 PM PDT

*
*

Hi Greg,

I recompiled the head with the NOMMAP define and everything works fine now.
Thanks again for all your help!!

By the way, I just ran the same rvu-command using the 32bit Windows
binaries provided
by NREL on
http://openstudio.nrel.gov/getting-started-developer/getting-started-radiance
(Rob G. - thanks a lot for offering those!!) -- no problems there as well.

Cheers,
David

Am 25. April 2012 19:29 schrieb Gregory J. Ward <[email protected]>:

Hi David,

I tend to agree that the easiest fix is to take out the mmap() call for
your system. We should log a bug to the Ubuntu developers, though.

You can put a -DEZXML_NOMMAP option in your rmake if you don't want to
mess with the header.

Best,
-Greg

Hi David,

That's an interesting find -- excellent sleuth work! It makes sense that a networked drive working through a Win7 virtual machine layer might run into troubles. I'm not sure what Unix does when mapping network drives in general, so it doesn't surprise me that having a Windows layer in there would mess things up.

Poking around with google leads me to articles saying that memory-mapping of networked files should be OK, even under Windows. So, it may be a problem with the implementation of VirtualBox. I didn't look for that issue, specifically.

Cheers,
-Greg

···

From: David Geisler-Moroder <[email protected]>
Date: November 7, 2012 5:20:10 AM PST

Hi Greg, hi list,

a short follow-up on the NOMMAP-issue I had in April.

When re-compiling a HEAD version currently I forgot to add the NOMMAP compile flag. Again, I ended up with the same error that I had in April, but not always...
To cut a long story short, I could trace back the problem to something that probably has nothing to do with RADIANCE and 32 or 64 bit. I'm running RADIANCE on Ubuntu in a VirtualBox on a Win7 host system. Whenever the XML-file to load is in a shared folder (via a VirtualBox guest addition) and physically stored on the network directory, the error occurs. However, copying the same file to a local directory in Ubuntu and using this one everything works fine.

I have no idea why this EZXML MMAP doesn't work on shared folders or if this has something to do with the formatting (NTFS vs. EXT4)??

Anyway, as a work-around specifying the NOMMAP compile flag or just working on the local disk works...

Cheers,
David

2012/4/26 Gregory J. Ward <[email protected]>
Hi David,

I'm pretty sure the guys who developed the build system for NREL didn't use the NOMMAP #define, so must be their system doesn't have whatever header issues yours is running afoul of in 32-bit mode.

Thanks for the update -- glad it's working for you now!

Cheers,
-Greg

P.S. A side note on ezxml is that I found a problem after the 4.1 release that caused very long XML files with DOS eol (CR-LF) to be impossibly slow to load. This is fixed in the latest HEAD, which NREL is using.

From: David Geisler-Moroder <[email protected]>
Date: April 25, 2012 11:28:47 PM PDT

Hi Greg,

I recompiled the head with the NOMMAP define and everything works fine now.
Thanks again for all your help!!

By the way, I just ran the same rvu-command using the 32bit Windows binaries provided
by NREL on http://openstudio.nrel.gov/getting-started-developer/getting-started-radiance
(Rob G. - thanks a lot for offering those!!) -- no problems there as well.

Cheers,
David

Am 25. April 2012 19:29 schrieb Gregory J. Ward <[email protected]>:
Hi David,

I tend to agree that the easiest fix is to take out the mmap() call for your system. We should log a bug to the Ubuntu developers, though.

You can put a -DEZXML_NOMMAP option in your rmake if you don't want to mess with the header.

Best,
-Greg