It occurs to me that what we are calling a "release," the rest of the world
has taken to calling a "release candidate" -- the version that is almost
complete, but still has a couple of annoying bugs hiding that users
identify as soon as they put it into service. Perhaps this would be a
convention we could adopt.
--
Randolph
Actually, the release candidate is the ‘b’ release in Greg’s parlance, which usually exists for a very brief time. If you look at the GitHub releases, we did make a 4.2.b.0, followed a week later by the official 4.2 release. It’s my understanding the HEAD is considered alpha code until right before an official release. Radiance major versions are released when they are released, generally following a brief period where it’s considered ‘beta’, and then the version goes back to ‘a’ in the HEAD and stays there until shortly before the next major release, again to ‘b’, and then, new version number. Twas ever thus.
At NREL we hang an additional identifier after the ‘a’ on each of our “releases” (which are really just snapshots, or tags), just to keep them in order. It’s not true semantic versioning, but it’s a way for us to keep track of which tag we’re using with which OpenStudio releases and whatnot. It can be confusing because our v5.0.a.5 predates the official release v5.0, which itself predates the latest set of packages we made: v5.0.a.8.
On Mar 12, 2016, at 7:24 PM, Randolph M. Fritz <[email protected]> wrote:
It occurs to me that what we are calling a "release," the rest of the world has taken to calling a "release candidate" -- the version that is almost complete, but still has a couple of annoying bugs hiding that users identify as soon as they put it into service. Perhaps this would be a convention we could adopt.
--
Randolph
_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev
We definitely have a more organic, non-standard release process for Radiance. I am open to suggestions for alternatives. One of our ongoing issues is the lack of a test suite to verify a release candidate and identify regressions. We haven't had the resources to create such a test suite, though Rob took a shot at it at some point.
-Greg
From: Rob Guglielmetti <[email protected]>
Subject: Re: [Radiance-dev] Release numbering
Date: March 12, 2016 6:44:54 PM PSTActually, the release candidate is the ‘b’ release in Greg’s parlance, which usually exists for a very brief time. If you look at the GitHub releases, we did make a 4.2.b.0, followed a week later by the official 4.2 release. It’s my understanding the HEAD is considered alpha code until right before an official release. Radiance major versions are released when they are released, generally following a brief period where it’s considered ‘beta’, and then the version goes back to ‘a’ in the HEAD and stays there until shortly before the next major release, again to ‘b’, and then, new version number. Twas ever thus.
At NREL we hang an additional identifier after the ‘a’ on each of our “releases” (which are really just snapshots, or tags), just to keep them in order. It’s not true semantic versioning, but it’s a way for us to keep track of which tag we’re using with which OpenStudio releases and whatnot. It can be confusing because our v5.0.a.5 predates the official release v5.0, which itself predates the latest set of packages we made: v5.0.a.8.
On Mar 12, 2016, at 7:24 PM, Randolph M. Fritz <[email protected]> wrote:
It occurs to me that what we are calling a "release," the rest of the world has taken to calling a "release candidate" -- the version that is almost complete, but still has a couple of annoying bugs hiding that users identify as soon as they put it into service. Perhaps this would be a convention we could adopt.
--
Randolph
As I recall, the way major releases have usually worked is that there's a
flurry of fixes afterwards as people adopt them, after which things settle
down, and the head goes back to being an alpha. So I was thinking in terms
of calling out the major releases as release candidates, and formalizing
the release after the flurry of fixes.
A number of us nibbled at the test suite at some point, but actually
designing it and developing it takes more time than any of us seem to have
available. 
Randolph
The SCons build system has had a testing infrastructure for many years,
unfortunately with only a minimal number of actual test cases.
As I have seen, the CMake system has one too, suffering the same problem.
It's just a matter of interested parties contributing test cases.
Covering all the existing code is one challenge. But it would be most helpful
if anyone adding features also provided test cases to demonstrate what they are
supposed to do, and even more importantly what they are NOT supposed to do.
Those could be just small(!) datasets with a description, and not necessarily
already integrated into a test suite. Someone who is familiar with one of the
suites can then integrate them. In the ideal case, no new feature wold go into
a "final release" without a comprehensive set of test cases covering it.
Besides that, given the "organic" development and limited resources, I'm not
sure if changing the release cycle would make much of a difference in when bugs
are found. There's only a small number of people checking out the HEAD version,
and those only try it on a very limited data set.
The majority will only download after a new final release has been announced,
no matter wether the release candidate has been around for a week or a month.
And only then will the new release be confronted with the "interesting" data
that actually triggers most bugs.
The important part here is to then create test cases specifically testing for
those bugs, to make sure they won't resurface later (regression testing).
QA is a dirty job, but someone's gotta do it...
-schorsch
Am 2016-03-14 02:20, schrieb Gregory J. Ward:
We definitely have a more organic, non-standard release process for
Radiance. I am open to suggestions for alternatives. One of our
ongoing issues is the lack of a test suite to verify a release
candidate and identify regressions. We haven't had the resources to
create such a test suite, though Rob took a shot at it at some point.-Greg
From: Rob Guglielmetti <[email protected]>
Subject: Re: [Radiance-dev] Release numbering
Date: March 12, 2016 6:44:54 PM PSTActually, the release candidate is the ‘b’ release in Greg’s parlance, which usually exists for a very brief time. If you look at the GitHub releases, we did make a 4.2.b.0, followed a week later by the official 4.2 release. It’s my understanding the HEAD is considered alpha code until right before an official release. Radiance major versions are released when they are released, generally following a brief period where it’s considered ‘beta’, and then the version goes back to ‘a’ in the HEAD and stays there until shortly before the next major release, again to ‘b’, and then, new version number. Twas ever thus.
At NREL we hang an additional identifier after the ‘a’ on each of our “releases” (which are really just snapshots, or tags), just to keep them in order. It’s not true semantic versioning, but it’s a way for us to keep track of which tag we’re using with which OpenStudio releases and whatnot. It can be confusing because our v5.0.a.5 predates the official release v5.0, which itself predates the latest set of packages we made: v5.0.a.8.
On Mar 12, 2016, at 7:24 PM, Randolph M. Fritz <[email protected]> >>> wrote:
It occurs to me that what we are calling a "release," the rest of the world has taken to calling a "release candidate" -- the version that is almost complete, but still has a couple of annoying bugs hiding that users identify as soon as they put it into service. Perhaps this would be a convention we could adopt.
--
Randolph_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev
--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/
I was gonna plug your tests too. I followed what you had in SCons when I
started the CTest stuff. I decided to start with tests on the core
elements we were using for OpenStudio's Radiance workflows, but never got
very far, nor very concrete (it's hard to ensure the correct binaries and
libs are being exercised, when you have multiple flavors of Radiance on
board).
We are about to build a rigorous testing and continuous integration
infrastructure around the OpenStudio and OS measures projects, which I
hope will pay dividends on the Radiance side of things. But I'm not
holding my breath on that last bit right now. We'll see.
- Rob
On 3/14/16, 5:03 AM, "Georg Mischler" <[email protected]> wrote:
The SCons build system has had a testing infrastructure for many years,
unfortunately with only a minimal number of actual test cases.
As I have seen, the CMake system has one too, suffering the same
problem.