Radiane MIME types

I've now reviewed RFC 4288, *Media Type Specifications and Registration Procedures* (current best practices) with an eye to registering Radiance .pic/.hdr files as an internet type. I've avoided looking at other file types, at least for the moment. Here's some of my notes on this:

1. Since 2005, only a standards body can register an unqualified type; image/hdr, say. So we'd probably best come in as a "vendor" (broadly construed) type; image/vnd.radiance, perhaps. (I can make a case for vnd.radiance.hdr, but it seems like overkill).

2. We're encouraged (but not required) to submit information about the format of our files. I think it would be a good idea to submit the Radiance picture file format, possibly lightly edited and expanded. I'll sign up for the [email protected] mailing list and see what we might submit.

3. We need to conduct a security analysis, though I don't think we have major issues here. The main one is that .pic format does use compression, which might be used in a resource-denial attack.

...to be continued...

Randolph

···

On Oct 25, 2008, at 2:51 PM, R Fritz wrote:

It looks pretty straightforward; I'll have to review the submission requirements in more detail, but I don't see serious problems. We'll probably have to come in as an organization-specific type ("image/vnd.radiance"), since an unqualified type now has to come from a recognized standards body. I'll get back with more details later this week.

Randolph

On Oct 25, 2008, at 11:33 AM, Axel Jacobs wrote:

Randolph,

The official mime types are assined by the IANA:
http://www.iana.org/assignments/media-types/

Getting .hdr accepted by IANA would mean that eventually all other
mime assignment, e.g.
for the various LINUX desktops:
http://standards.freedesktop.org/shared-mime-info-spec/shared-mime-info-spec-latest.html
will pick it up

I started a thread here:
http://luminance.londonmet.ac.uk/radiance_mailinglists/dev/2006-November/000761.html
but it didn't lead anywhere.

When I said I was not sufficiently confident about the file format,
then this is what I meant:
(see http://radsite.lbl.gov/radiance/refer/filefmts.pdf)

HDR comes in two flavours:
- RGBE
- XYZE

Each of those two flavours has three sub-flavours:
- non-rle
- rle
- mixed

Greg (http://luminance.londonmet.ac.uk/radiance_mailinglists/dev/2006-November/000766.html):
-----------8<--------------

Both, RGBE and XYZE come in flat and in RLE flavours. However, the
official file specs only mention two FORMAT= string, i.e. 32-
bit_rle_rgbe
and 32-bit_rle_xyze. How does this work?

I decided not to sub-type based on the presence or absence of run-
length encoding. Since the reader routines identify RLE on a per
scanline basis (and in fact there can be a mix of RLE and
uncompressed scanlines), there seemed no need for a separate format
specifier.

The MIME specs have the notion of subclasses. So would RGBE and
XYZE be
subclasses of RADIANCE HDR?

Again, I wouldn't bother to distinguish these within MIME. It would
be like distinguishing between different classes of TIFF. Any
software that opens an RGBE file will also open an XYZE file, even if
it won't display the colors correctly.
-----------8<--------------

So there you go...

I've started working on a mime XML file following the freedesktop.org
style. This is to make my yearly chore of rolling LEARNIX somewhat
easier. It's not well tested yet, but here is the first draft. The bit
with the extensions seems to work, but I haven't tried yet what
happens when the extension is removed (mime magick).

Ignore all non-HDR stuff. I just thought I might as well leave it in,
in the hope of getting some comments.

<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">

<mime-type type="image/x-hdr">
   <comment xml:lang="en">HDR image</comment>
  <alias type="image/x-radiance"/>
  <alias type="image/x-rgbe"/>
   <magic priority="50">
     <match type="string" value="#?RADIANCE" offset="0"/>
     <match type="string" value="FORMAT=32-bit_rle_rgbe" offset="0:500"/>
   </magic>
   <glob pattern="*.hdr"/>
   <glob pattern="*.unf"/>
   <glob pattern="*.xyze"/>
   <glob pattern="*.rgbe"/>
</mime-type>

<mime-type type="model/x-radiance-control">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance input</comment>
   <glob pattern="*.rif"/>
</mime-type>

<mime-type type="application/x-radiance-octree">
   <comment xml:lang="en">Radiance octree</comment>
   <magic priority="50">
     <match type="string" value="#?RADIANCE" offset="0"/>
     <match type="string" value="FORMAT=Radiance_octree" offset="0:500"/>
   </magic>
   <glob pattern="*.oct"/>
</mime-type>

<mime-type type="application/x-radiance-ambient-cache">
   <comment xml:lang="en">Radiance ambient cache</comment>
   <glob pattern="*.amb"/>
</mime-type>

<mime-type type="model/x-radiance-holodeck-control">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance holodeck input</comment>
   <glob pattern="*.hif"/>
</mime-type>

<mime-type type="application/x-radiance-holodeck">
   <comment xml:lang="en">Radiance holodeck</comment>
   <glob pattern="*.hdk"/>
</mime-type>

<mime-type type="model/x-radiance-geometry">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance geometry</comment>
   <glob pattern="*.rad"/>
   <glob pattern="*.norm"/>
</mime-type>

<mime-type type="model/x-radiance-triangle-mesh">
   <comment xml:lang="en">Radiance triangle mesh</comment>
   <glob pattern="*.rtm"/>
</mime-type>

<mime-type type="model/x-radiance-material">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance material</comment>
   <glob pattern="*.mat"/>
</mime-type>

<mime-type type="model/x-radiance-view">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance view</comment>
   <glob pattern="*.vf"/>
</mime-type>

<mime-type type="model/x-radiance-cal">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance cal</comment>
   <glob pattern="*.cal"/>
</mime-type>

<mime-type type="model/x-radiance-bgraph">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance bgraph</comment>
   <glob pattern="*.bgraph"/>
</mime-type>

<mime-type type="model/x-radiance-options">
  <sub-class-of type="text/plain"/>
   <comment xml:lang="en">Radiance options</comment>
   <glob pattern="*.opt"/>
</mime-type>

</mime-info>

Axel

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev

Hi Randolph,

thanks for chasing this up. My comments are below...

Encoding considerations: Binary preferred. This is a binary image
  type with a text header; "binary" or "base64" are the only
  reasonable choices.

Applications that use this media type: Radiance, HDR Workshop,
    Photoshop, TBD

+++ pfstools
more here: http://luxal.dachary.org/webhdr/software.shtml
.hdr is listed as RGBE

Additional information:
     Magic number(s): Text "#?RADIANCE\n" (octet sequence 23 3f 52 41
       44 49 41 4e 43 45 0a) at the beginning of the file
     File extension(s): pic, hdr
     Macintosh file type code(s): TBD

--- The .pic extension is officially dead and buried. Stick to .hdr.
It might be worthwhile also aiming for .rgbe and .xyze

+++ I don't think #?RADIANCE\n is quite good enough. It also matches
octrees (FORMAT=Radiance_octree). I would suggest that
FORMAT=32-bit_rle_rgbe or FORMAT=32-bit_rle_xyze needs to be in the
specs as well. The problem is that they don't have a fix position,
since this depends on the commands which were used to generate the
image.
I believe this is what the offset thingy is there for (although this
particular line does not work somehow):
<match type="string" value="FORMAT=32-bit_rle_rgbe" offset="0:500"/>

Here is a full .hdr header:

----------8<--------------
#?RADIANCE
oconv skies/sky.mat materials/barnsley2.mat skies/sky.rad
objects/barnsley-1.8m_7.5deep.rad objects/lightshelf_horizontal2.rad
rpict -vu 0 0 1 -vf views/floor2.vf -x 1600 -y 1600 -ps 4 -pt .08 -dp
1024 -ar 80 -ms 0.08 -ds .3 -dt .1 -dc .5 -dr 1 -sj .7 -st .1 -ab 4
-af tmp/b2.amb -aa .1 -ad 1536 -as 392 -av 0.01 0.01 0.01 -lr 8 -lw
.002
SOFTWARE= RADIANCE 3.8 lastmod Mon Dec 3 11:50:14 GMT 2007 by root on
axel-desktop
VIEW= -vtl -vp 3.75 3.75 1.5 -vd 0 0 -1 -vu -1 0 0 -vh 7.5 -vv 7.5 -vo
0 -va 0 -vs 0 -vl 0
CAPDATE= 2008:03:13 12:04:37
FORMAT=32-bit_rle_rgbe
pfilt -r .6 -x /2 -y /2
EXPOSURE=3.078936e+00
<blank line>
-Y 800 +X 800

... and an octree header:

#?RADIANCE
oconv materials/barnsley.mat skies/sky.mat objects/gnd_floor.rad
objects/top_floor.rad skies/sky.rad objects/lightshelf_sloped.rad
FORMAT=Radiance_octree
<blank line>
----------8<--------------

Would you mind if I forwarded your email to the pfstools mailing list?
I'd be happy to act as a gatekeeper and feed you only sensible
comments back. Can I leave the username/password in there?

Regards

Axel

I apologize that I don't have time at the moment to look at this thoroughly.

I just checked in the switch from ".pic" to ".hdr" throughout the HEAD release of Radiance 4.0a.

The "#?RADIANCE" line at the beginning of the file is, as Axel notes, not a unique indicator of a RGBE or XYZE file. For that, you really need to find the format string, and I think the notion of "magic numbers" fails in most specifications of a header that could be any size, with the important string buried in it somewhere. I only added the "#?RADIANCE" line as an afterthought in fact to provide something basic for the Unix "file" command to work with.

-Greg

···

From: "Axel Jacobs" <[email protected]>
Date: November 7, 2008 3:25:42 PM PST

Hi Randolph,

thanks for chasing this up. My comments are below...

Encoding considerations: Binary preferred. This is a binary image
  type with a text header; "binary" or "base64" are the only
  reasonable choices.

Applications that use this media type: Radiance, HDR Workshop,
    Photoshop, TBD

+++ pfstools
more here: http://luxal.dachary.org/webhdr/software.shtml
.hdr is listed as RGBE

Additional information:
     Magic number(s): Text "#?RADIANCE\n" (octet sequence 23 3f 52 41
       44 49 41 4e 43 45 0a) at the beginning of the file
     File extension(s): pic, hdr
     Macintosh file type code(s): TBD

I am turning back to this project, having finally found a bit of time. I've updated my draft registration form. It can be downloaded from <http://students.washington.edu/rfritz/radreg/Radiance%20Registration%20Form.txt > user/password rad/rad.

Two questions only:

   1. Is the file header different on Windows because of the difference in newline convention?
   2. Is *Rendering with Radiance* still available from Booksurge?

Randolph

Hi Randolph,

Thanks for working on this!

  1. Is the file header different on Windows because of the difference in newline convention?

No, a single newline ('\n') character should terminate each header line no matter what system you're on. A carriage return ('\r') will be ignored, and not produced by any of the Radiance tools.

  2. Is *Rendering with Radiance* still available from Booksurge?

You can buy it from Amazon:

  http://www.amazon.com/Rendering-Radiance-Science-Lighting-Visualization/dp/0974538108/ref=pd_bbs_sr_1/002-5711964-3059265?ie=UTF8&s=books&qid=1186177315&sr=8-1

The link perpetually says it's out of stock, but since it's print-on-demand, it doesn't matter. I think it comes from BookSurge, but I don't think they have a link for it anymore.

Cheers,
-Greg

Mark Baker on the IETF list thinks we're good to go. I'm ready to submit the attached to the IANA on behalf of Greg Ward and "The Radiance Lighting Simulation Group" (which means us) & will do so tomorrow.

Randolph

Radiance Registration Form.txt (2.21 KB)

Well done. Thanks, Randolph!

-Greg

···

From: R Fritz <[email protected]>
Date: January 6, 2009 11:22:09 AM PST

Mark Baker on the IETF list thinks we're good to go. I'm ready to submit the attached to the IANA on behalf of Greg Ward and "The Radiance Lighting Simulation Group" (which means us) & will do so tomorrow.

Randolph

I just contacted the IANA to request a status update on this. They're apparently swamped with requests, so it's still pending after a month. I hope they're not *too* swamped.

Randolph

···

On Jan 8, 2009, at 9:52 AM, Gregory J. Ward wrote:

Well done. Thanks, Randolph!

-Greg

From: R Fritz <[email protected]>
Date: January 6, 2009 11:22:09 AM PST

Mark Baker on the IETF list thinks we're good to go. I'm ready to submit the attached to the IANA on behalf of Greg Ward and "The Radiance Lighting Simulation Group" (which means us) & will do so tomorrow.

Randolph

_______________________________________________
Radiance-dev mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-dev