State of the SCons

[Georg_Mischler]

Hi all,

after all those years, I found that my CVS write access is
still active.

So I went and updated the SCons build system.

It now works nicely again with the last few SCons versions.
Unfortunately, SCons still reqires Python 2.7, I'm really
hoping for it to support Python 3 soon.

The compile results are now placed in a seperate "scbuild"
directory tree. Each target platform gets its own subbranch
in there.

Target configurations now differentiate between 32 and 46 bit
systems. I had to rename the platform config files for that.
Currently the build is autmatically set to the architecture
of the host platform.

It might be nice to be able to cross-compile at least for the
companion bit-architecture of the same platform, which should be
possible to do.
On the other hand, I'm not sure if its really worth the effort.
How many people are still running Radiance on 32 bit systems?
But at least the possibility should be kept open, so we won't
need another redesign when the first 128 bit systems come out in
a few years...

On linux, everything builds without a hitch. We get the same
executables as with makeall, and then some.
It's also much faster than ten years ago!

On Windows, everything without third-party dependencies builds
fine with the current VC 2015 "Community" compiler.
I haven't yet gotten around to fetching the Windows dev packages
of Tifflib and QT, so those parts need some more work.
X11 is probably not worth the hassle (though there might be ways).

The tifflib included with the NREL binaries didn't work, which
means they used a different compiler. In fact, they must be using
gcc on Windows, because I found (and fixed) some non-standard
allocation magic in src/cv/bsdftrans.cpp that only gcc accepts.

I haven't looked at qtrvu very closely. But even on a cursory
glance it looks straightforward to build. This should not be a
problem for SCons and MSCC, once all the dependencies are in place.
Ah... can we standardize on a file name extension for C++ files in
Radiance? Right now we have both *.cxx and *.cpp.

With the current MS compiler on 64 bit, there are a huge number
of coercion warnings all throughout the code. Most of those are
probably harmless, even if annoying.

There are two files tough, where the warning says that "the
result of a 32 bit shift operation was coerced to 64 bit. Was
there a 64 bit shift intended?"
   src/common/bmpfile.c
   src/util/rttree_reduce.c
I can't really tell, so I've just marked the lines with an
"XXX" comment for review.

If anyone wants to give the "new" SCons a try on any platform,
go ahead!

Please tell me about your results, particularly about any
changes to the config files you might find necessary.
Instructions are in ray/INSTALL.scons and ray/platform/README.
All changes are in CVS, the HEAD package should include those
changes in a day or so.

have fun!

-schorsch

···

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

[Greg_Ward]

Hi Schorsch,

Thanks for all your efforts! I guess we now have 3 working build systems for Radiance (the old rmake script, cmake, and SCons is back).

The 64-bit coercion warnings are as you say, mostly harmless. The ones isn bmpfile.c are going to be 8-bit values at most, so no worries there. However, the one you pointed out in rttree_reduce.c was a potential bug on large arrays, so I changed it to (size_t)1<<(expression). I hope your compiler accepts this without complaint. If it gives you another warning, please suggest a better strategy.

Cheers!
-Greg

···

From: Georg Mischler <[email protected]>
Subject: [Radiance-dev] State of the SCons
Date: March 9, 2016 6:27:00 PM PST

Hi all,

after all those years, I found that my CVS write access is
still active.

So I went and updated the SCons build system.

It now works nicely again with the last few SCons versions.
Unfortunately, SCons still reqires Python 2.7, I'm really
hoping for it to support Python 3 soon.

The compile results are now placed in a seperate "scbuild"
directory tree. Each target platform gets its own subbranch
in there.

Target configurations now differentiate between 32 and 46 bit
systems. I had to rename the platform config files for that.
Currently the build is autmatically set to the architecture
of the host platform.

It might be nice to be able to cross-compile at least for the
companion bit-architecture of the same platform, which should be
possible to do.
On the other hand, I'm not sure if its really worth the effort.
How many people are still running Radiance on 32 bit systems?
But at least the possibility should be kept open, so we won't
need another redesign when the first 128 bit systems come out in
a few years...

On linux, everything builds without a hitch. We get the same
executables as with makeall, and then some.
It's also much faster than ten years ago!

On Windows, everything without third-party dependencies builds
fine with the current VC 2015 "Community" compiler.
I haven't yet gotten around to fetching the Windows dev packages
of Tifflib and QT, so those parts need some more work.
X11 is probably not worth the hassle (though there might be ways).

The tifflib included with the NREL binaries didn't work, which
means they used a different compiler. In fact, they must be using
gcc on Windows, because I found (and fixed) some non-standard
allocation magic in src/cv/bsdftrans.cpp that only gcc accepts.

I haven't looked at qtrvu very closely. But even on a cursory
glance it looks straightforward to build. This should not be a
problem for SCons and MSCC, once all the dependencies are in place.
Ah... can we standardize on a file name extension for C++ files in
Radiance? Right now we have both *.cxx and *.cpp.

With the current MS compiler on 64 bit, there are a huge number
of coercion warnings all throughout the code. Most of those are
probably harmless, even if annoying.

There are two files tough, where the warning says that "the
result of a 32 bit shift operation was coerced to 64 bit. Was
there a 64 bit shift intended?"
src/common/bmpfile.c
src/util/rttree_reduce.c
I can't really tell, so I've just marked the lines with an
"XXX" comment for review.

If anyone wants to give the "new" SCons a try on any platform,
go ahead!

Please tell me about your results, particularly about any
changes to the config files you might find necessary.
Instructions are in ray/INSTALL.scons and ray/platform/README.
All changes are in CVS, the HEAD package should include those
changes in a day or so.

have fun!

-schorsch

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

[Guglielmetti_Robert]

Thanks Georg, and thanks Greg for hitting that one issue Schorsch raised
there. Other replies below...

It might be nice to be able to cross-compile at least for the
companion bit-architecture of the same platform, which should be
possible to do.
On the other hand, I'm not sure if its really worth the effort.
How many people are still running Radiance on 32 bit systems?
But at least the possibility should be kept open, so we won't
need another redesign when the first 128 bit systems come out in
a few years...

We gave up on providing 32-bit Mac and Linux packages here at NREL, but do
still make 32-bit Windows packages, only because we include Radiance with
OpenStudio and we are still supporting 32-bit Windows for OpenStudio users
(I forget why, but we just do). I was making universal Mac binaries, but
it was a PITA to maintain the tifflib and the packages are 2x as large, so
I dropped it.

On Windows, everything without third-party dependencies builds
fine with the current VC 2015 "Community" compiler.
I haven't yet gotten around to fetching the Windows dev packages
of Tifflib and QT, so those parts need some more work.
X11 is probably not worth the hassle (though there might be ways).

The tifflib included with the NREL binaries didn't work, which
means they used a different compiler. In fact, they must be using
gcc on Windows, because I found (and fixed) some non-standard
allocation magic in src/cv/bsdftrans.cpp that only gcc accepts.

I am "they". =) Not sure what didn't work for you, Georg. I use a 32-bit
windows tifflib from the interwebs
(http://gnuwin32.sourceforge.net/packages/tiff.htm), and rolled my own
64-bit when we started making 64-bit Windows packages. I do believe I used
gcc, but I can't remember and can't find my notes. Please let me know if
there's a better recommended way.

I haven't looked at qtrvu very closely. But even on a cursory
glance it looks straightforward to build. This should not be a
problem for SCons and MSCC, once all the dependencies are in place.
Ah... can we standardize on a file name extension for C++ files in
Radiance? Right now we have both *.cxx and *.cpp.

Maybe when you have time, you can look at qtrvu and help make it better.
The GUI was modeled after your Windows rvu, included with Rayfront. The
folks at Kitware wrote it, but only implemented the bits I felt were
absolutely necessary (limited budget). I haven't had access to Rayfront's
'rvu' in a while but I do recall there were some things we left out. As
far as the multiple filename extensions, I had no idea. =) I ASSumed .cxx
was for headers or some shit. Pretty sure you're the first person to look
at that code besides me since it was written! If it's a major deal, I
guess we can change it, but it doesn't seem to be hurting anyone except
for the anal retentive types here. =)

So good to see your name in my inbox again, Schorsch!

- Rob

···

On 3/9/16, 7:27 PM, "Georg Mischler" <[email protected]> wrote:

[Georg_Mischler]

Thanks Georg, and thanks Greg for hitting that one issue Schorsch raised
there. Other replies below...

This is one of the last mailing lists where people still seem to value
the old art of citation editing...

We gave up on providing 32-bit Mac and Linux packages here at NREL, but do
still make 32-bit Windows packages, only because we include Radiance with
OpenStudio and we are still supporting 32-bit Windows for OpenStudio users

I suspected as much. I'm usually way behind the curve in terms of
hardware, and even I have thrown out the last 32bit box last year.
I assume it will still build on 32bit anyway, but we don't need to go
the extra mile for that.

The tifflib included with the NREL binaries didn't work, which
means they used a different compiler. In fact, they must be using
gcc on Windows, because I found (and fixed) some non-standard
allocation magic in src/cv/bsdftrans.cpp that only gcc accepts.

I am "they". =) Not sure what didn't work for you, Georg. I use a 32-bit
windows tifflib from the interwebs
(http://gnuwin32.sourceforge.net/packages/tiff.htm), and rolled my own
64-bit when we started making 64-bit Windows packages. I do believe I used
gcc, but I can't remember and can't find my notes. Please let me know if
there's a better recommended way.

Lovely Windows...
I later noticed that I was compiling single-threaded, while the default
on Windows seems to be multi-threaded - whether you need it or not.
Which means you're actually linking to a different system library
(there's about a dozen of them). I've now built my own tifflib, also for
the headers, but had the same problem with that one at first.

Maybe when you have time, you can look at qtrvu and help make it better.
The GUI was modeled after your Windows rvu, included with Rayfront.

The old Winrview actually came from LBNL (for Desktop Radiance), and I
just fixed it to actually work as intended. I don't think that one has
ever been published in source. Would that fall under the Radiance licence?
Then maybe we could at least pick it for ideas. I remember I had to apply
some hacks to be able to stop a running simulation from the UI thread.

I also have something called "winimage" sitting on a disk, but no idea of
its state of maturity (Rayfront uses an image viewer in Python). It has the
same LBNL copyright, so we might be able to play with it as well.

What package do I need to fetch for the qt libraries?
Qtrvu looks nice, but playing around with it revealed a few glitches.
Getting the window size right is probably simple, why it always wants
you to hit "enter" twice remains to be seen.

I have no idea how much time I'll have for those things in the near or
not so near future, so don't hold your breath. I just saw an opportunity
in the past few weeks (starting with temporary access to Revit), so I
jumped on the topic again a bit.

Ah... can we standardize on a file name extension for C++ files in
Radiance? Right now we have both *.cxx and *.cpp.

As far as the multiple filename extensions, I had no idea. =) I ASSumed .cxx
was for headers or some shit. Pretty sure you're the first person to look
at that code besides me since it was written! If it's a major deal, I
guess we can change it, but it doesn't seem to be hurting anyone except
for the anal retentive types here. =)

Not a big issue, but confusing and distracting. The size and complexity of
the Radiance sources already makes them confusing enough...
The only counter example is src/cv/bsdftrans.cpp, and a (generated?) file
in CMakeFiles/.

Some people apparently do use .cxx, to distinguish the shortcut from "CPP"
for the preprocessor. Out in the wild though, .cpp seems to be about 5
times more popular. I suggest to decide on a standard now (no matter which),
while the "problem" is still not really one. We should expect the number
of C++ files to grow in the future.

-schorsch

···

Am 2016-03-10 18:05, schrieb Guglielmetti, Robert:

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

[Georg_Mischler]

Hi Greg,

I think then I'll "fix" the other instances as well anyway.
It may not technically matter, but any silenced compiler warning is
a distraction less.

-schorsch

···

Am 2016-03-10 17:28, schrieb Gregory J. Ward:

Hi Schorsch,

Thanks for all your efforts! I guess we now have 3 working build
systems for Radiance (the old rmake script, cmake, and SCons is back).

The 64-bit coercion warnings are as you say, mostly harmless. The
ones isn bmpfile.c are going to be 8-bit values at most, so no worries
there. However, the one you pointed out in rttree_reduce.c was a
potential bug on large arrays, so I changed it to
(size_t)1<<(expression). I hope your compiler accepts this without
complaint. If it gives you another warning, please suggest a better
strategy.

Cheers!
-Greg

From: Georg Mischler <[email protected]>
Subject: [Radiance-dev] State of the SCons
Date: March 9, 2016 6:27:00 PM PST

Hi all,

after all those years, I found that my CVS write access is
still active.

So I went and updated the SCons build system.

It now works nicely again with the last few SCons versions.
Unfortunately, SCons still reqires Python 2.7, I'm really
hoping for it to support Python 3 soon.

The compile results are now placed in a seperate "scbuild"
directory tree. Each target platform gets its own subbranch
in there.

Target configurations now differentiate between 32 and 46 bit
systems. I had to rename the platform config files for that.
Currently the build is autmatically set to the architecture
of the host platform.

It might be nice to be able to cross-compile at least for the
companion bit-architecture of the same platform, which should be
possible to do.
On the other hand, I'm not sure if its really worth the effort.
How many people are still running Radiance on 32 bit systems?
But at least the possibility should be kept open, so we won't
need another redesign when the first 128 bit systems come out in
a few years...

On linux, everything builds without a hitch. We get the same
executables as with makeall, and then some.
It's also much faster than ten years ago!

On Windows, everything without third-party dependencies builds
fine with the current VC 2015 "Community" compiler.
I haven't yet gotten around to fetching the Windows dev packages
of Tifflib and QT, so those parts need some more work.
X11 is probably not worth the hassle (though there might be ways).

The tifflib included with the NREL binaries didn't work, which
means they used a different compiler. In fact, they must be using
gcc on Windows, because I found (and fixed) some non-standard
allocation magic in src/cv/bsdftrans.cpp that only gcc accepts.

I haven't looked at qtrvu very closely. But even on a cursory
glance it looks straightforward to build. This should not be a
problem for SCons and MSCC, once all the dependencies are in place.
Ah... can we standardize on a file name extension for C++ files in
Radiance? Right now we have both *.cxx and *.cpp.

With the current MS compiler on 64 bit, there are a huge number
of coercion warnings all throughout the code. Most of those are
probably harmless, even if annoying.

There are two files tough, where the warning says that "the
result of a 32 bit shift operation was coerced to 64 bit. Was
there a 64 bit shift intended?"
src/common/bmpfile.c
src/util/rttree_reduce.c
I can't really tell, so I've just marked the lines with an
"XXX" comment for review.

If anyone wants to give the "new" SCons a try on any platform,
go ahead!

Please tell me about your results, particularly about any
changes to the config files you might find necessary.
Instructions are in ray/INSTALL.scons and ray/platform/README.
All changes are in CVS, the HEAD package should include those
changes in a day or so.

have fun!

-schorsch

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

[Guglielmetti_Robert]

Thanks Georg, and thanks Greg for hitting that one issue Schorsch
raised
there. Other replies below...

This is one of the last mailing lists where people still seem to value
the old art of citation editing...

Haha, I figured you'd appreciate it. I've all but abandoned the practice,
since MS Outlook makes it very difficult to do (and not get the formatting
screwed up), and even when it works, I've actually had people tell me they
received "blank" emails from me, because they didn't think to scroll down
past the first quoted bit =/

The old Winrview actually came from LBNL (for Desktop Radiance), and I
just fixed it to actually work as intended. I don't think that one has
ever been published in source. Would that fall under the Radiance
licence?
Then maybe we could at least pick it for ideas. I remember I had to
apply
some hacks to be able to stop a running simulation from the UI thread.

I also have something called "winimage" sitting on a disk, but no idea
of
its state of maturity (Rayfront uses an image viewer in Python). It has
the
same LBNL copyright, so we might be able to play with it as well.

Ah! I did not know that. I had ASSumed that yours was the only Windows
version, and so when we started the GitHub mirror we just rolled our own
rvu, thinking there were no open source options at the time. I have no
idea where the source for that Desktop Radiance rvu might be.

What package do I need to fetch for the qt libraries?
Qtrvu looks nice, but playing around with it revealed a few glitches.
Getting the window size right is probably simple, why it always wants
you to hit "enter" twice remains to be seen.

We updated qtrvu to use Qt5 a while back; specifically, we recommend Qt
5.3.2 for building OpenStudio, and that version seems to work fine for
qtrvu as well. Yeah it's definitely a little rough around the edges but
it's useable.

I have no idea how much time I'll have for those things in the near or
not so near future, so don't hold your breath. I just saw an opportunity
in the past few weeks (starting with temporary access to Revit), so I
jumped on the topic again a bit.

Understood. It's sure nice to hear from you again though!

- Rob

···

On 3/10/16, 4:09 PM, "Georg Mischler" <[email protected]> wrote:

Am 2016-03-10 18:05, schrieb Guglielmetti, Robert:

[Georg_Mischler]

Ah! I did not know that. I had ASSumed that yours was the only Windows
version, and so when we started the GitHub mirror we just rolled our own
rvu, thinking there were no open source options at the time. I have no
idea where the source for that Desktop Radiance rvu might be.

The sources to both winrview and winimage are sitting here on my
disk. I don't think the DR team changed anything relevant after
my fixes.

Unfortunately, my old contract with LBNL does not cover
redistribution in source form. To make this possible, we'd need
some statement from them that those programs can be considered a
part of the normal Radiance distribution and fall under the
"Radiance open source license".

@Greg, are you entitled to make such a statement?

We updated qtrvu to use Qt5 a while back; specifically, we recommend Qt
5.3.2 for building OpenStudio, and that version seems to work fine for
qtrvu as well. Yeah it's definitely a little rough around the edges but
it's useable.

I've played with the Qt4 tool for SCons on linux a bit.
But I didn't get far, probably because Suse installs Qt in
different places than the tool expects.

But now I'll fetch the Windows libraries and the Qt5 tool.
Has it actually ever been built on a non-Windows system?

-schorsch

···

Am 2016-03-14 16:51, schrieb Guglielmetti, Robert:

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

[Greg_Ward]

I think so. I'm generally the gatekeeper to the Radiance source tree, and can "welcome" new code that is offered. Since LBNL contracted you to develop it in the first place from what you say, this should not present a problem.

-Greg

···

Am 2016-03-14 16:51, schrieb Guglielmetti, Robert:

Ah! I did not know that. I had ASSumed that yours was the only Windows
version, and so when we started the GitHub mirror we just rolled our own
rvu, thinking there were no open source options at the time. I have no
idea where the source for that Desktop Radiance rvu might be.

The sources to both winrview and winimage are sitting here on my
disk. I don't think the DR team changed anything relevant after
my fixes.

Unfortunately, my old contract with LBNL does not cover
redistribution in source form. To make this possible, we'd need
some statement from them that those programs can be considered a
part of the normal Radiance distribution and fall under the
"Radiance open source license".

@Greg, are you entitled to make such a statement?

[Guglielmetti_Robert]

Yeah. Way it works on unix is that qtrvu is called when you specify qt as
the output device in a rvu call:

rvu -o qt foo.oct

That will launch qtrvu, even tho there's no actual executable called qtrvu
when you build on unix systems (with CMake). Works fine on Mac. Just
tested on a Ubuntu VM and it looks like it's not being packaged correctly,
so I'll have to look into that. =(

···

On 3/14/16, 8:26 PM, "Georg Mischler" <[email protected]> wrote:

But now I'll fetch the Windows libraries and the Qt5 tool.
Has it actually ever been built on a non-Windows system?