Meta files and library

Hi,

our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".

I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.html

Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials, they
will inevitably get more attention in times to come:

A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).

I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.

Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.

Cheers
-schorsch

ยทยทยท

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

1 Like

I thought this got resolved with the NREL packages, but I guess not. I
don't see anything in the commit history where I had changed/fixed
anything in the CMake files so I think we still have an issue here.
Sarith, what version of the NREL installer are you currently using, where
objline does not work?

I agree with Schorsch, if we could use a standard vector format for this
it would be nice...

ยทยทยท

On 5/10/16, 2:02 AM, "Georg Mischler" <[email protected]> wrote:

Hi,

our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".

I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.
html

Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials,
they
will inevitably get more attention in times to come:

A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).

I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer
Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which
would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.

Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just
curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.

Cheers
-schorsch

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

Hi Schorsch,

Yes, "MLIB" was probably not the best choice of environment variable, and there are more standardized vector graphics formats around these days. The reason behind these choices is mostly historical. When I developed this format circa 1984, there wasn't much else around other than the plot package, which wasn't very capable and wasn't portable across architectures. At the time, I was relying on CP/M as well as Unix, and dot matrix printers for most of my output, so I rolled my own solution.

If you have a good, cross-platform alternative that wouldn't require much work or additional dependencies, we could certainly look into replacing this, but I've found it difficult to come up with something easier than just maintaining what we have.

Cheers,
-Greg

ยทยทยท

From: Georg Mischler <[email protected]>
Subject: [Radiance-dev] Meta files and library
Date: May 10, 2016 1:02:07 AM PDT

Hi,

our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".

I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.html

Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials, they
will inevitably get more attention in times to come:

A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).

I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.

Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.

Cheers
-schorsch

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Hi Rob,

I am using the win64 version of 5.0.a.10 that I found here:
https://github.com/NREL/Radiance/releases. I had the official 5.0 release
before that. Objline works fine on my Linux system. I was just testing see
if it is possible to implement it in a Windows environment by following the
workflow in the original csh script
<https://github.com/NREL/Radiance/blob/master/src/util/objline.csh> .

Sarith

(In case this unintentionally starts a new thread, this email was a reply to
http://www.radiance-online.org/pipermail/radiance-dev/2016-May/001600.html .
I have just subscribed to this mailing list.)

To add to the confusion, the distribution (via CVS) includes two of the
files with a *.met extension in ray/src/meta/.
Makeall installs those in parallel and redundantly to the *.mta files
coming with rad5R0supp.tar.gz. The executables are only looking for the
*.mta versions, though.

Cheers
-schorsch

ยทยทยท

Am 2016-05-10 16:44, schrieb Guglielmetti, Robert:

I thought this got resolved with the NREL packages, but I guess not. I
don't see anything in the commit history where I had changed/fixed
anything in the CMake files so I think we still have an issue here.
Sarith, what version of the NREL installer are you currently using, where
objline does not work?

I agree with Schorsch, if we could use a standard vector format for this
it would be nice...

On 5/10/16, 2:02 AM, "Georg Mischler" <[email protected]> wrote:

Hi,

our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".

I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.
html

Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials,
they
will inevitably get more attention in times to come:

A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).

I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer
Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which
would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.

Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just
curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.

Cheers
-schorsch

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Oh you know what, I don't think we ever followed this through on the Windows side since the default workflow is to pass the output from objline to X11meta. So far the only executable that depends upon X11 support we've attempted to support is rvu. Since the main benefit of you going through all the trouble to move csh scripts to Python is the Windows support, perhaps we should sort this out. =) I think this is one more reason to look at a standard vector format.

ยทยทยท

On 5/10/16, 1:18 PM, "Sarith Subramaniam" <[email protected]<mailto:[email protected]>> wrote:

Hi Rob,

I am using the win64 version of 5.0.a.10 that I found here: https://github.com/NREL/Radiance/releases. I had the official 5.0 release before that. Objline works fine on my Linux system. I was just testing see if it is possible to implement it in a Windows environment by following the workflow in the original csh script<https://github.com/NREL/Radiance/blob/master/src/util/objline.csh> .

Sarith

(In case this unintentionally starts a new thread, this email was a reply to http://www.radiance-online.org/pipermail/radiance-dev/2016-May/001600.html . I have just subscribed to this mailing list.)

But surely looking at diagrams with x11meta is only an intermediate step?
The real goal is to convert them to something that can be included in
whatever kind of documentation you're generating from your project.

Nonetheless, as long as the meta files are still there, having a way to
look at them on Windows would be nice. Maybe I can teach winimage to
convert them to an image on the fly...

Cheers
-schorsch

ยทยทยท

Am 2016-05-10 22:50, schrieb Guglielmetti, Robert:

Oh you know what, I don't think we ever followed this through on the
Windows side since the default workflow is to pass the output from
objline to X11meta. So far the only executable that depends upon X11
support we've attempted to support is rvu. Since the main benefit of
you going through all the trouble to move csh scripts to Python is the
Windows support, perhaps we should sort this out. =) I think this is
one more reason to look at a standard vector format.

On 5/10/16, 1:18 PM, "Sarith Subramaniam" > <[email protected]<mailto:[email protected]>> wrote:

Hi Rob,

I am using the win64 version of 5.0.a.10 that I found here:
https://github.com/NREL/Radiance/releases. I had the official 5.0
release before that. Objline works fine on my Linux system. I was just
testing see if it is possible to implement it in a Windows environment
by following the workflow in the original csh
script<https://github.com/NREL/Radiance/blob/master/src/util/objline.csh>
.

Sarith

(In case this unintentionally starts a new thread, this email was a
reply to
http://www.radiance-online.org/pipermail/radiance-dev/2016-May/001600.html
. I have just subscribed to this mailing list.)

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

The .met files are the hand-build ASCII originals. The .mta files are the converted binary files needed by the other programs. So, the .met files are sources in other words.

-Greg

ยทยทยท

From: Georg Mischler <[email protected]>
Subject: Re: [Radiance-dev] Meta files and library
Date: May 10, 2016 1:35:15 PM PDT

To add to the confusion, the distribution (via CVS) includes two of the
files with a *.met extension in ray/src/meta/.
Makeall installs those in parallel and redundantly to the *.mta files
coming with rad5R0supp.tar.gz. The executables are only looking for the
*.mta versions, though.

Cheers
-schorsch

Am 2016-05-10 16:44, schrieb Guglielmetti, Robert:

I thought this got resolved with the NREL packages, but I guess not. I
don't see anything in the commit history where I had changed/fixed
anything in the CMake files so I think we still have an issue here.
Sarith, what version of the NREL installer are you currently using, where
objline does not work?
I agree with Schorsch, if we could use a standard vector format for this
it would be nice...
On 5/10/16, 2:02 AM, "Georg Mischler" <[email protected]> wrote:

Hi,
our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".
I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.
html
Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials,
they
will inevitably get more attention in times to come:
A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).
I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer
Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which
would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.
Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just
curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.
Cheers
-schorsch
--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/
_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

True, but there are converters from metafile to encapsulated PostScript, Targa, and BMP files that can be used and viewed on Windows.

All the same, I'm not opposed to converting to an existing cross-platform vector format if there's one that comes preinstalled everywhere. Is there one?

-Greg

ยทยทยท

From: "Guglielmetti, Robert" <[email protected]>
Subject: Re: [Radiance-dev] Meta files and library
Date: May 10, 2016 1:50:26 PM PDT

Oh you know what, I don't think we ever followed this through on the Windows side since the default workflow is to pass the output from objline to X11meta. So far the only executable that depends upon X11 support we've attempted to support is rvu. Since the main benefit of you going through all the trouble to move csh scripts to Python is the Windows support, perhaps we should sort this out. =) I think this is one more reason to look at a standard vector format.

On 5/10/16, 1:18 PM, "Sarith Subramaniam" <[email protected]<mailto:[email protected]>> wrote:

Hi Rob,

I am using the win64 version of 5.0.a.10 that I found here: https://github.com/NREL/Radiance/releases. I had the official 5.0 release before that. Objline works fine on my Linux system. I was just testing see if it is possible to implement it in a Windows environment by following the workflow in the original csh script<https://github.com/NREL/Radiance/blob/master/src/util/objline.csh> .

Sarith

(In case this unintentionally starts a new thread, this email was a reply to http://www.radiance-online.org/pipermail/radiance-dev/2016-May/001600.html . I have just subscribed to this mailing list.)

Ah, I didn't remember that the history of meta files goes that far back.

If I understand it right, the primary purpose of the package is to
create diagrams from tabular numerical data. Its functionality is almost
completely independent from the rest of Radiance, with mgf2meta as the
lonely exception that actually processes Radiance specific data.

Compared to more "friendly" ways to create diagrams (spreadsheets,
statistical software), the current approach has the following
advantages:

- simple implementation
- included in the distribution
- scriptable within pipelines
- no thirdparty dependencies

The disadvantages:

- limited output connectivity (conversion is currently only possible
ย ย ย into Windows bitmaps, targa, and postscript).
- limited graphics resolution (2^14 units), possibly not accurate
ย ย ย enough for large format high resolution output
- limited graphics capabilities (no curves, only 5 colors)

So what we're looking for would be a diagramming toolkit with more
flexibility on the output side, and not too many other disadvantages.

The least invasive solution would be to simply add a few converters,
eg. meta2svg or meta2pdf. This would at least widen the path to
downstream processing.

On the other hand, if we ignore mgf2meta, we could use pretty much any
open source diagramming tool out there that we like. There will be
nothing that is "preinstalled everywhere" though, the topic is simply
too diverse for that.

In between, using eg. SVG as an internal format might be an option.
Writing is easy, since we have very simple content. Reading is slightly
more involved, even when we're restricting it to our self-generated files.
But then we already have ezxml in the distribution, so it might actually
be manageable.

Cheers
-schorsch

ยทยทยท

Am 2016-05-10 16:48, schrieb Gregory J. Ward:

Hi Schorsch,

Yes, "MLIB" was probably not the best choice of environment variable,
and there are more standardized vector graphics formats around these
days. The reason behind these choices is mostly historical. When I
developed this format circa 1984, there wasn't much else around other
than the plot package, which wasn't very capable and wasn't portable
across architectures. At the time, I was relying on CP/M as well as
Unix, and dot matrix printers for most of my output, so I rolled my
own solution.

If you have a good, cross-platform alternative that wouldn't require
much work or additional dependencies, we could certainly look into
replacing this, but I've found it difficult to come up with something
easier than just maintaining what we have.

Cheers,
-Greg

From: Georg Mischler <[email protected]>
Subject: [Radiance-dev] Meta files and library
Date: May 10, 2016 1:02:07 AM PDT

Hi,

our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".

I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.html

Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials, they
will inevitably get more attention in times to come:

A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).

I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.

Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.

Cheers
-schorsch

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Indeed, the metafile tools were dragged into Radiance at some point I wanted to be able to script things like plots and diagrams when doing data analysis, and it was just more convenient. They have very little in common with the rest of Radiance, though they were the first appearance of the .cal language in tools like bgraph.

We can pipe the output of psmeta to ps2pdf or similar.

How is SVG supported on different systems? I hadn't heard of it before. Can you just send output to a web browser, or ??

As I said, these tools have been no effort for me to maintain for many decades, so unless we really need better vector output for some reason, it's not that big a deal to write a converter to SVG or PDF if that's all we're talking about. On the other hand, it wouldn't be that difficult to replace it as far as Radiance is concerned -- I would still use it for automating plot generation, since I've no way to pipe data to Excel.

Cheers,
-Greg

ยทยทยท

From: Georg Mischler <[email protected]>
Subject: Re: [Radiance-dev] Meta files and library
Date: May 10, 2016 3:41:21 PM PDT

Ah, I didn't remember that the history of meta files goes that far back.

If I understand it right, the primary purpose of the package is to
create diagrams from tabular numerical data. Its functionality is almost
completely independent from the rest of Radiance, with mgf2meta as the
lonely exception that actually processes Radiance specific data.

Compared to more "friendly" ways to create diagrams (spreadsheets,
statistical software), the current approach has the following
advantages:

- simple implementation
- included in the distribution
- scriptable within pipelines
- no thirdparty dependencies

The disadvantages:

- limited output connectivity (conversion is currently only possible
into Windows bitmaps, targa, and postscript).
- limited graphics resolution (2^14 units), possibly not accurate
enough for large format high resolution output
- limited graphics capabilities (no curves, only 5 colors)

So what we're looking for would be a diagramming toolkit with more
flexibility on the output side, and not too many other disadvantages.

The least invasive solution would be to simply add a few converters,
eg. meta2svg or meta2pdf. This would at least widen the path to
downstream processing.

On the other hand, if we ignore mgf2meta, we could use pretty much any
open source diagramming tool out there that we like. There will be
nothing that is "preinstalled everywhere" though, the topic is simply
too diverse for that.

In between, using eg. SVG as an internal format might be an option.
Writing is easy, since we have very simple content. Reading is slightly
more involved, even when we're restricting it to our self-generated files.
But then we already have ezxml in the distribution, so it might actually
be manageable.

Cheers
-schorsch

Am 2016-05-10 16:48, schrieb Gregory J. Ward:

Hi Schorsch,
Yes, "MLIB" was probably not the best choice of environment variable,
and there are more standardized vector graphics formats around these
days. The reason behind these choices is mostly historical. When I
developed this format circa 1984, there wasn't much else around other
than the plot package, which wasn't very capable and wasn't portable
across architectures. At the time, I was relying on CP/M as well as
Unix, and dot matrix printers for most of my output, so I rolled my
own solution.
If you have a good, cross-platform alternative that wouldn't require
much work or additional dependencies, we could certainly look into
replacing this, but I've found it difficult to come up with something
easier than just maintaining what we have.
Cheers,
-Greg

From: Georg Mischler <[email protected]>
Subject: [Radiance-dev] Meta files and library
Date: May 10, 2016 1:02:07 AM PDT
Hi,
our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".
I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.html
Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials, they
will inevitably get more attention in times to come:
A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).
I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.
Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.
Cheers
-schorsch

So the plot files are really cal files?
Or just a very similar, but subtly incompatible variation thereof?
The documentation sometimes talks about "plot files", and sometimes
about "graph files". Are those the same?

SVG has pretty much become the standard for vector graphics on the web
(and elsewhere, such as scalable icons in Linux GUIs). All modern
browsers can render it. As an XML format it may even be embedded directly
in the HTML of a web page. Pretty much all other relevant and up-to-date
vector graphics software understands the format. The most popular free
tool for editing seems to be Inkscape.

The format has several advanced capabilities that most people rarely
use, if at all, and would probably be irrelevant to us (eg. styling with
external css, animation with Javascript). A file containing a diagram
with some lines, fills, and text can actually be very straightforward
and simple. There are also defined subsets of the format (eg. "SVG Tiny"),
primarily intended for mobile use.

As long as nobody complains eg. about output resolution, it's clearly
easier to keep working with meta files internally for the moment.
But then, it would still be nice if we could simplify things a bit.
Is there really still a need for pexpand? And I'm not quite sure
what psort is good for to begin with.

Since those tools are not used anywhere else, wouldn't it make sense to
ditch $MLIB, and simply use $RAYPATH/meta instead?

The documentation also mentions (usually under "see also") a number of
manpages that either never existed or have been removed: graph(1G),
plot(1), plot(5), plotout(1), primout(3), t4014(1), mx80(1), impress(1).

And last (for the moment): Looking at the manpage, it appears that
plotin(1) has a very similar if not identical purpose as bgraph(1).
What am I missing there?

Cheers
-schorsch

ยทยทยท

Am 2016-05-11 01:00, schrieb Gregory J. Ward:

Indeed, the metafile tools were dragged into Radiance at some point I
wanted to be able to script things like plots and diagrams when doing
data analysis, and it was just more convenient. They have very little
in common with the rest of Radiance, though they were the first
appearance of the .cal language in tools like bgraph.

We can pipe the output of psmeta to ps2pdf or similar.

How is SVG supported on different systems? I hadn't heard of it
before. Can you just send output to a web browser, or ??

As I said, these tools have been no effort for me to maintain for many
decades, so unless we really need better vector output for some
reason, it's not that big a deal to write a converter to SVG or PDF if
that's all we're talking about. On the other hand, it wouldn't be
that difficult to replace it as far as Radiance is concerned -- I
would still use it for automating plot generation, since I've no way
to pipe data to Excel.

Cheers,
-Greg

From: Georg Mischler <[email protected]>
Subject: Re: [Radiance-dev] Meta files and library
Date: May 10, 2016 3:41:21 PM PDT

Ah, I didn't remember that the history of meta files goes that far back.

If I understand it right, the primary purpose of the package is to
create diagrams from tabular numerical data. Its functionality is almost
completely independent from the rest of Radiance, with mgf2meta as the
lonely exception that actually processes Radiance specific data.

Compared to more "friendly" ways to create diagrams (spreadsheets,
statistical software), the current approach has the following
advantages:

- simple implementation
- included in the distribution
- scriptable within pipelines
- no thirdparty dependencies

The disadvantages:

- limited output connectivity (conversion is currently only possible
into Windows bitmaps, targa, and postscript).
- limited graphics resolution (2^14 units), possibly not accurate
enough for large format high resolution output
- limited graphics capabilities (no curves, only 5 colors)

So what we're looking for would be a diagramming toolkit with more
flexibility on the output side, and not too many other disadvantages.

The least invasive solution would be to simply add a few converters,
eg. meta2svg or meta2pdf. This would at least widen the path to
downstream processing.

On the other hand, if we ignore mgf2meta, we could use pretty much any
open source diagramming tool out there that we like. There will be
nothing that is "preinstalled everywhere" though, the topic is simply
too diverse for that.

In between, using eg. SVG as an internal format might be an option.
Writing is easy, since we have very simple content. Reading is slightly
more involved, even when we're restricting it to our self-generated files.
But then we already have ezxml in the distribution, so it might actually
be manageable.

Cheers
-schorsch

Am 2016-05-10 16:48, schrieb Gregory J. Ward:

Hi Schorsch,
Yes, "MLIB" was probably not the best choice of environment variable,
and there are more standardized vector graphics formats around these
days. The reason behind these choices is mostly historical. When I
developed this format circa 1984, there wasn't much else around other
than the plot package, which wasn't very capable and wasn't portable
across architectures. At the time, I was relying on CP/M as well as
Unix, and dot matrix printers for most of my output, so I rolled my
own solution.
If you have a good, cross-platform alternative that wouldn't require
much work or additional dependencies, we could certainly look into
replacing this, but I've found it difficult to come up with something
easier than just maintaining what we have.
Cheers,
-Greg

From: Georg Mischler <[email protected]>
Subject: [Radiance-dev] Meta files and library
Date: May 10, 2016 1:02:07 AM PDT
Hi,
our volunteer Sarith Subramaniam is currently working on objline.py.
When trying to test it, he noticed that the NREL releases do not include
the supporting meta files in lib/meta. Since they do include the
executables that require those files, that seems slightly suboptimal...
But it shouldn't be hard to fix. Just copy over "ray/lib/meta/*.mta".
I've found a previous mention of the topic here:
http://radiance-online.org/pipermail/radiance-general/2015-October/011273.html
Some unsorted thoughts on the topic of meta files in general, given that
due to the availability of objline.csh/py, and some newer tutorials, they
will inevitably get more attention in times to come:
A few messages down the above thread, Greg mentions the undocumented(!)
envvar "MLIB". The first thing running through my mind seeing this was:
"This must be the worst possible name for an environment variable, ever"
(way to generic).
I also never quite understood the point of inventing yet another
proprietary vector format just for Radiance. Or are such meta files used
anywhere else that I'm not aware of?
They do not seem to be the same as the ISO standardized "Computer Graphics
Meta Files" (http://www.fileformat.info/format/cgm/egff.htm), which would
be a possible candidate for something like that.
Another possible approach would be SVG (or a suitable subset thereof),
which has excellent support libaries available nowadays.
Of course, I haven't looked into how much work it would be to use either
of those (or yet something else), so this basically just amounts to me
ranting at the moment without any real purpose. But I'm still just curious
what the motivation behind this format was (or which format it actually
is), and what other people are thinking about possible alternatives.
Cheers
-schorsch

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Hi Schorsch,

I only have a response to a few of these, so I'll pull them out...

From: Georg Mischler <[email protected]>
Date: May 11, 2016 1:31:34 AM PDT

So the plot files are really cal files?
Or just a very similar, but subtly incompatible variation thereof?
The documentation sometimes talks about "plot files", and sometimes
about "graph files". Are those the same?

The "graph files" are related to .cal files in an odd way. Specifically, functions in graph files use the .cal language, but you only get one line per definition, so end-of-line must be escaped with a backslash ('\') if you want to keep a function neat and semicolons at the end are optional. You can specify as many variables and related functions as you need, subject to the same restrictions. Other lines in the graph file define strings rather than expressions, and are parsed differently. So, it's a mix, but pretty easy to work with. I use it all the time when I want to make a quick plot. Try:

include=function.plt
include=polar.plt
DEG:PI/180
xmin=0
xmax=360
abs(x):if(x,x,-x)
A(x)=abs(sin(x*DEG)+cos(x*DEG))

Then, give the file to bgraph and pipe the output to psmeta or meta2bmp to see what comes out.

...
As long as nobody complains eg. about output resolution, it's clearly
easier to keep working with meta files internally for the moment.
But then, it would still be nice if we could simplify things a bit.
Is there really still a need for pexpand? And I'm not quite sure
what psort is good for to begin with.

The pexpand command is called internally, as is psort by image converters that want their vector primitives sorted.

Since those tools are not used anywhere else, wouldn't it make sense to
ditch $MLIB, and simply use $RAYPATH/meta instead?

Except that RAYPATH is a list of directories, where MDIR is a single location. All the same, a function that finds the first meta directory in $RAYPATH could be used instead.

The documentation also mentions (usually under "see also") a number of
manpages that either never existed or have been removed: graph(1G),
plot(1), plot(5), plotout(1), primout(3), t4014(1), mx80(1), impress(1).

Yeah, RIP many disused graphics output devices. We could remove the references if anyone cares.

And last (for the moment): Looking at the manpage, it appears that
plotin(1) has a very similar if not identical purpose as bgraph(1).
What am I missing there?

The plotin tool converts from old-school plot primitives to metafile equivalents. I don't know if anyone uses the original graph or plot routines anymore, so this could probably be retired as well. The functionality is better in bgraph, anyway.

-Greg

The "graph files" are related to .cal files in an odd way.

"Subtly incompatible" it is then...
So a parser needs to use different patterns/escapes for seperating
statements from each other. Apart from not seeing any of the simulation
specific variables, is that the only difference?

The pexpand command is called internally, as is psort by image
converters that want their vector primitives sorted.

That would seem to make more sense as library routines then.
Not sure if refactoring as such would be worth the effort now, though.

Since those tools are not used anywhere else, wouldn't it make sense to
ditch $MLIB, and simply use $RAYPATH/meta instead?

Except that RAYPATH is a list of directories, where MDIR is a single
location. All the same, a function that finds the first meta
directory in $RAYPATH could be used instead.

The new Python scripts do exactly that to find their "pyradlib" support
library. Granted, that one has a more unique name (deliberately so),
which is less likely to collide with something a user might have within
their project.

Yeah, RIP many disused graphics output devices. We could remove the
references if anyone cares.

If you have any interest in avoiding confusion among new users,
you should care.

The plotin tool converts from old-school plot primitives to metafile
equivalents. I don't know if anyone uses the original graph or plot
routines anymore, so this could probably be retired as well. The
functionality is better in bgraph, anyway.

Ah, so "plot files" are indeed something different from "graph files".
Since the plot(5) man page presumably documenting that format is gone,
keeping a program around that depends on it doesn't make much sense.

Cheers
-schorsch

ยทยทยท

Am 2016-05-12 02:27, schrieb Gregory J. Ward:

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Responses inline...

From: Georg Mischler <[email protected]>
Subject: Re: [Radiance-dev] Meta files and library
Date: May 12, 2016 12:58:14 AM PDT

The "graph files" are related to .cal files in an odd way.

"Subtly incompatible" it is then...
So a parser needs to use different patterns/escapes for seperating
statements from each other. Apart from not seeing any of the simulation
specific variables, is that the only difference?

Well, the .cal routines have their own set of runtime flags that determine which features are supported. The bgraph program uses the default flags, which are (E_VARIABLE | E_FUNCTION), so it doesn't support input channels as in Radiance (although most users never see or use these).

Also, the syntax is similar but the interpretation is very different when the variables match any of the standard ones ("keywords") such as "include", which loads another file, and the many string variables like "title", "Blabel", etc. The variables "?data" are also special with their own syntax, and load an array of floating-point data that ends with a semicolon or EOF. So, quite a bit different.

The pexpand command is called internally, as is psort by image
converters that want their vector primitives sorted.

That would seem to make more sense as library routines then.
Not sure if refactoring as such would be worth the effort now, though.

Yes, I suppose, although rholo also calls separate driver programs (in a subdirectory of the executables), and rvu is capable of something similar.

Since those tools are not used anywhere else, wouldn't it make sense to
ditch $MLIB, and simply use $RAYPATH/meta instead?

Except that RAYPATH is a list of directories, where MDIR is a single
location. All the same, a function that finds the first meta
directory in $RAYPATH could be used instead.

The new Python scripts do exactly that to find their "pyradlib" support
library. Granted, that one has a more unique name (deliberately so),
which is less likely to collide with something a user might have within
their project.

Yeah. We should decide if it's worth the risk to change it.

Yeah, RIP many disused graphics output devices. We could remove the
references if anyone cares.

If you have any interest in avoiding confusion among new users,
you should care.

Of course I care -- I just wonder if anyone else looks at these man pages, anymore. Easy enough to change.

The plotin tool converts from old-school plot primitives to metafile
equivalents. I don't know if anyone uses the original graph or plot
routines anymore, so this could probably be retired as well. The
functionality is better in bgraph, anyway.

Ah, so "plot files" are indeed something different from "graph files".
Since the plot(5) man page presumably documenting that format is gone,
keeping a program around that depends on it doesn't make much sense.

Well, it was never my package, so I've no idea if it's still "out there," somewhere. Used to be standard on BSD Unix, anyway. Anyone else know what we're talking about?

-Greg

ยทยทยท

Am 2016-05-12 02:27, schrieb Gregory J. Ward:

> Ah, so "plot files" are indeed something different from "graph files".
> Since the plot(5) man page presumably documenting that format is gone,
> keeping a program around that depends on it doesn't make much sense.

Well, it was never my package, so I've no idea if it's still "out there,"
somewhere. Used to be standard on BSD Unix, anyway. Anyone else know what
we're talking about?

Plot goes all the way back to Unix v7. It appears that the GNU plotting

utilities (https://www.gnu.org/software/plotutils/), not to be confused
with gnuplot will still read the format.

So I suppose it is still "out there," but I am not sure that it matters
much any more.

Randolph

Just checked the SuSe installation DVD:
Besides gnuplot there's also a gnu plotutils package containing
"libplot" and commands like "plot", "graph", etc.
Not installed by default, but with a deeply nested package
productivity/graphics/visualization/graph.

So it evidently still exists in some form or other.
Which probably also means we should keep "plotin" and the related
manpage references around for another while. There's even a
"plotout" still sitting in the attic...

Cheers
-schorsch

ยทยทยท

Am 2016-05-12 19:37, schrieb Randolph M. Fritz:

Ah, so "plot files" are indeed something different from "graph

files".

Since the plot(5) man page presumably documenting that format is

gone,

keeping a program around that depends on it doesn't make much

sense.

Well, it was never my package, so I've no idea if it's still "out
there," somewhere. Used to be standard on BSD Unix, anyway. Anyone
else know what we're talking about?

Plot goes all the way back to Unix v7. It appears that the GNU
plotting utilities (https://www.gnu.org/software/plotutils/), not to
be confused with gnuplot will still read the format.

So I suppose it is still "out there," but I am not sure that it
matters much any more.

Randolph
_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

--
Georg Mischler -- simulations developer -- schorsch at schorsch com
+schorsch.com+ -- lighting design tools -- http://www.schorsch.com/

Just to be confusing "gnuplot" and "GNU plotutils" are entirely different
programs. The gnuplot developers had the name first and are keeping it.

In any event, the GNU plotutils will, as far as i know, work with plotin
and (if it is resurrected) plotout.

Randolph

Just to revive this old thread.

Iโ€™m having issues with the lib/meta folder missing. I did install latest from NREL.

Where can I find the files here in 2022? :slight_smile:

Also any modern alternative to the meta files since 2016?

Mathias

EDIT: Found the meta files here
https://github.com/LBNL-ETA/Radiance/releases/download/05eb231c/Radiance_Auxiliary_05eb231c.zip

Hi Mathias,

There is a meta/ directory in the auxiliary zip file, which you may need to install someplace and assign the $MDIR environment variable to. I donโ€™t know if the packages take care of this or not. I thought the essential files would get copied in the building of the src/meta folder, but perhaps there are some files missing, or your environment variable isnโ€™t set. (Remember that it can be only one directory and should end in a โ€˜/โ€™ character, unlike Radiance search paths.)

Sorry for the hassle โ€“ I donโ€™t use the packages myself, which is a problem for answering these sorts of questions.

-Greg