I've checked a refactored version of the Python test framework
into CVS today. As with the other proposed scripts, it now works
with both Python 2.7 and Python 3.x, and the only dependencies
are Python and Radiance. SCons simply invokes it, as can any
other tool, or the user directly.
The file ray/test/run_tests.py is now both a script and a module,
with identical functionality. The caller can pass in the Radiance
binary and library paths (if they're not already covered by PATH
and RAYPATH). If given, they are added to the local environment,
which then gets passed along to the test cases.
It will run all the test cases it discovers in the subdirectories
of ray/test/testcases/. Alternatively, one or several sections can
be tested seperately. More details are in README.txt and the
module docstrings.
The support modules have moved to lib/pyradlib/, with lib/ being
the standard Radiance library (similar to lib/tcl/ for trad).
Run_tests.py will find that directoy and add its parent to the
Python search path. A test case can import them as:
from pyradlib import lcompare
from pyradlib.pyrad_proc import Error, PIPE, ProcMixin
As can be seen from the latter, I've switched the existing test
cases to useing the same process management module as the scripts,
instead of os.system().
This same library can also be used by any other Python scripts,
although each of those will be on its own in actually finding its
location.
To avoid having to search all the relevant directories in each
test case again, the test runner temporarily injects an ad-hoc
module into sys.modules. Test cases can "import testsupport",
which will give them direct access to all the paths detected by
the runner. The information provided by that module can of course
be extended with whatever seems useful.
I think this is the easiest way to go forward with our testing
efforts.
I had a look at the CTest stuff, and it seems like it would only
run on Robs machine, because it contains hardcoded paths to some
local directories. That could be fixed, of course, but the
dependency to a much less common third-party tool would still
remain.
So what so you (or your build tool) need to do to use this?
1. Make sure the contents of ray/src/common/pyradlib/ are copied
to lib/pyradlib/
(either in the build tree or in the final installation).
2. Invoke ray/test/run_tests.py with suitable parameters.
3. Sit back and smile 
Of course, there's still a lot to improve, or more flexibility
to add. I'm open to suggestions!
Cheers
-schorsch
···
Am 2016-03-17 15:10, schrieb Georg Mischler:
The criteria I would apply are:
- Which framework is easiest to install for end users?
Or maybe better: Which is most likely to be already installed on the
system of the average Radiance user, even when not part of the
currently running build system?
* On unix
* on Mac (as far as different from other unixes)
* on Windows
- Which is easiest to maintain and to add test cases?
More specificly: Which has the shortest learning curve for the
average Radiance user before they can contribute tests?
- And last but not least, which has all the capabilities we need?
* Run a program with various arguments and input, and compare its
output against a prepared set of expected data.
* Eventually: Load a shared library and run similar tests on
individual functions (not easily possible with the current state
of librtrad, those pesky global variables once again).
-schorsch
Am 2016-03-16 22:28, schrieb Guglielmetti, Robert:
On 3/16/16, 3:02 PM, "Georg Mischler" <[email protected]> wrote:
That sounds like a lot of duplicate work...
Wouldn't it make more sense to maintain just one test suite, and
have either build system just invoke that?
Definitely a lot of duplicate work, you're right. I was thinking maybe a
little too specifically about a particular testing framework (CTest in my
case).
_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev
--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/