Mac OS X and nfs

radiance-general
1

Hi all,

I'm at my wits end trying to get my machines to play nicely when parallel processing. I have a 4 computer setup which I am trying to farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports to give read write access and sync to all machines in my private network (192.168.0.0/255.255.255.0). The server and the other linux box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up the mount for the exported folder. In the Finder, I am able to access, read, and copy files from the Macintosh to the server, and to erase files that are on the server. I can also use X11 to run single processor renderings on files that are located on the nfs server. However, when I try to run the parallel renderings from the Macintosh, the process just freezes. Using the top command, I see that rpict is there, but 0% of the computers resources is devoted to the process, and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure that they all match.

Just to test what was going on, I tried setting up one of my Macintoshes as the nfs server. In this instance, the linux box can mount the file system, but the process freezes when it starts the rendering, whether it be a single rpict command or an rpiece command. The other Macintosh is capable of running and completing the parallel rendering process, but only about 10% of the computers resources is devoted to the process (this is something that I'll just have to look into).

Could somebody please help me figure out what is going on?

Thanks.

2

Hi John,

I think that one thing to look into in greater depth is what kind of control you can exercise on the configuration/setup of NFS on the Panther based machines. In particular you need to have both nfsd running as well as something called statd (I am assuming these are the names used in the apple based systems). The latter is responsible for reporting file status for locking requests among other things.

It sounds like you can get a linux to linux nfs setup running ok. I think that you need to debug and get a stable apple to apple nfs setup running (ie full processor performance). Then you can work on figuring out how to get the two to work together. I suspect that there are some mount and/or export related switches that may need to be set to make nfs work in a heterogenous environment like this. It is difficult enough in a homegenous one.

-Jack

John S. An wrote:

···

Hi all,

I'm at my wits end trying to get my machines to play nicely when parallel processing. I have a 4 computer setup which I am trying to farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports to give read write access and sync to all machines in my private network (192.168.0.0/255.255.255.0). The server and the other linux box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up the mount for the exported folder. In the Finder, I am able to access, read, and copy files from the Macintosh to the server, and to erase files that are on the server. I can also use X11 to run single processor renderings on files that are located on the nfs server. However, when I try to run the parallel renderings from the Macintosh, the process just freezes. Using the top command, I see that rpict is there, but 0% of the computers resources is devoted to the process, and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure that they all match.

Just to test what was going on, I tried setting up one of my Macintoshes as the nfs server. In this instance, the linux box can mount the file system, but the process freezes when it starts the rendering, whether it be a single rpict command or an rpiece command. The other Macintosh is capable of running and completing the parallel rendering process, but only about 10% of the computers resources is devoted to the process (this is something that I'll just have to look into).

Could somebody please help me figure out what is going on?

Thanks.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

3

Hi,
about homogeneous environment:
some time ago (macosX 10.0) I did some rpiece experiments....
It seems to me that I simply employed the Appletalk protocol to share the
ambient file....
may be....
just macosx to macosx.....
cheers,
giulio

PS however at the end I bought a dual athlon Linux machine.... but now it is
different: G5...

···

-----Original Message-----
From: Jack de Valpine [mailto:[email protected]]
Sent: 22 December 2003 21:05
To: [email protected]
Subject: Re: [Radiance-general] Mac OS X and nfs

Hi John,

I think that one thing to look into in greater depth is what kind of
control you can exercise on the configuration/setup of NFS on the
Panther based machines. In particular you need to have both nfsd running
as well as something called statd (I am assuming these are the names
used in the apple based systems). The latter is responsible for
reporting file status for locking requests among other things.

It sounds like you can get a linux to linux nfs setup running ok. I
think that you need to debug and get a stable apple to apple nfs setup
running (ie full processor performance). Then you can work on figuring
out how to get the two to work together. I suspect that there are some
mount and/or export related switches that may need to be set to make nfs
work in a heterogenous environment like this. It is difficult enough in
a homegenous one.

-Jack

John S. An wrote:

Hi all,

I'm at my wits end trying to get my machines to play nicely when
parallel processing. I have a 4 computer setup which I am trying to
farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports
to give read write access and sync to all machines in my private
network (192.168.0.0/255.255.255.0). The server and the other linux
box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up
the mount for the exported folder. In the Finder, I am able to
access, read, and copy files from the Macintosh to the server, and to
erase files that are on the server. I can also use X11 to run single
processor renderings on files that are located on the nfs server.
However, when I try to run the parallel renderings from the Macintosh,
the process just freezes. Using the top command, I see that rpict is
there, but 0% of the computers resources is devoted to the process,
and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure
that they all match.

Just to test what was going on, I tried setting up one of my
Macintoshes as the nfs server. In this instance, the linux box can
mount the file system, but the process freezes when it starts the
rendering, whether it be a single rpict command or an rpiece command.
The other Macintosh is capable of running and completing the parallel
rendering process, but only about 10% of the computers resources is
devoted to the process (this is something that I'll just have to look
into).

Could somebody please help me figure out what is going on?

Thanks.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

___________________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for acceptability of content and viruses.

4

Thanks for your advice Jack. I don't know how many people on the list are interested in the intricacies of making nfs work on Mac OS X, but with Apple hardware one of the many hardware options for running Radiance, I'll post updates on my experience. So here goes...

In trying to get a stable Mac OS X only parallel rendering going, I switched the server/client relationship. In other words, the machine that was previously the client became the server, and the server became the client. All other settings remained the same, but this time, the vast majority (~98%) of the processes were allocated to rpict. I'm not sure why this set up is more stable than the other, but it is.

So then I tried mounting the nfs volume from the Linux machines, and tried starting an rpict process with an -af switch, and it also worked. So I thought my problem was solved, but (there's always a but)...

File locking seems to work on the Linux machines, but not on the Macintosh client. When I begin an rpict process on the Macintosh client, and then attach another rpict from the Linux client, then the output file always shows that the ambient values are corrupt.

So, once I figure out how to make the Macintoshes abide by the file-locking protocols, I think all will be well.

One note, for anyone trying make nfs adjustments on the Macintosh, the shareware NFS Manager (http://www.bresink.de/osx/NFSManager.html) by Marcel Bresink is extremely helpful. Unlike most UNIX/Linux distributions, Mac OS X reserves files such as /etc/fstab and /etc/exports for system functions, and thus the settings need to be adjusted in the application NetInfo Manager. NFS Manager puts a friendlier GUI interface to dealing with NetInfo Manager.

In response to Guilio's comment about trying to use Appletalk to share the ambient file, the problem arises in the file locking. I don't think Radiance knows how to deal with file locking in the Appletalk protocol. I too tried initially to see if I could share files over Appletalk, and Radiance always gave me an error message and then quit. I did find some discussion in the developers forum on allowing Radiance to share files over Samba. I haven't had a chance to explore this option any further, but has the Samba file sharing implementation been incorporated into the HEAD version of Radiance?

I hope I'm not boring everyone with this discussion. If so, just let me know, and I'll stop posting any updates.

···

Message: 5
Date: Mon, 22 Dec 2003 16:05:09 -0500
From: Jack de Valpine <[email protected]>
To: [email protected]
Subject: Re: [Radiance-general] Mac OS X and nfs
Reply-To: [email protected]

Hi John,

I think that one thing to look into in greater depth is what kind of
control you can exercise on the configuration/setup of NFS on the
Panther based machines. In particular you need to have both nfsd running
as well as something called statd (I am assuming these are the names
used in the apple based systems). The latter is responsible for
reporting file status for locking requests among other things.

It sounds like you can get a linux to linux nfs setup running ok. I
think that you need to debug and get a stable apple to apple nfs setup
running (ie full processor performance). Then you can work on figuring
out how to get the two to work together. I suspect that there are some
mount and/or export related switches that may need to be set to make nfs
work in a heterogenous environment like this. It is difficult enough in
a homegenous one.

-Jack

John S. An wrote:

Hi all,

I'm at my wits end trying to get my machines to play nicely when
parallel processing. I have a 4 computer setup which I am trying to
farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports
to give read write access and sync to all machines in my private
network (192.168.0.0/255.255.255.0). The server and the other linux
box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up
the mount for the exported folder. In the Finder, I am able to
access, read, and copy files from the Macintosh to the server, and to
erase files that are on the server. I can also use X11 to run single
processor renderings on files that are located on the nfs server.
However, when I try to run the parallel renderings from the Macintosh,
the process just freezes. Using the top command, I see that rpict is
there, but 0% of the computers resources is devoted to the process,
and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure
that they all match.

Just to test what was going on, I tried setting up one of my
Macintoshes as the nfs server. In this instance, the linux box can
mount the file system, but the process freezes when it starts the
rendering, whether it be a single rpict command or an rpiece command.
The other Macintosh is capable of running and completing the parallel
rendering process, but only about 10% of the computers resources is
devoted to the process (this is something that I'll just have to look
into).

Could somebody please help me figure out what is going on?

Thanks.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

--__--__--

Message: 6
From: Giulio Antonutto <[email protected]>
To: "'[email protected]'" <[email protected]>
Subject: RE: [Radiance-general] Mac OS X and nfs
Date: Tue, 23 Dec 2003 09:36:20 -0000
Reply-To: [email protected]

Hi,
about homogeneous environment:
some time ago (macosX 10.0) I did some rpiece experiments....
It seems to me that I simply employed the Appletalk protocol to share the
ambient file....
may be....
just macosx to macosx.....
cheers,
giulio

PS however at the end I bought a dual athlon Linux machine.... but now it is
different: G5...

-----Original Message-----
From: Jack de Valpine [mailto:[email protected]]
Sent: 22 December 2003 21:05
To: [email protected]
Subject: Re: [Radiance-general] Mac OS X and nfs

Hi John,

I think that one thing to look into in greater depth is what kind of
control you can exercise on the configuration/setup of NFS on the
Panther based machines. In particular you need to have both nfsd running
as well as something called statd (I am assuming these are the names
used in the apple based systems). The latter is responsible for
reporting file status for locking requests among other things.

It sounds like you can get a linux to linux nfs setup running ok. I
think that you need to debug and get a stable apple to apple nfs setup
running (ie full processor performance). Then you can work on figuring
out how to get the two to work together. I suspect that there are some
mount and/or export related switches that may need to be set to make nfs
work in a heterogenous environment like this. It is difficult enough in
a homegenous one.

-Jack

John S. An wrote:

Hi all,

I'm at my wits end trying to get my machines to play nicely when
parallel processing. I have a 4 computer setup which I am trying to
farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports
to give read write access and sync to all machines in my private
network (192.168.0.0/255.255.255.0). The server and the other linux
box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up
the mount for the exported folder. In the Finder, I am able to
access, read, and copy files from the Macintosh to the server, and to
erase files that are on the server. I can also use X11 to run single
processor renderings on files that are located on the nfs server.
However, when I try to run the parallel renderings from the Macintosh,
the process just freezes. Using the top command, I see that rpict is
there, but 0% of the computers resources is devoted to the process,
and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure
that they all match.

Just to test what was going on, I tried setting up one of my
Macintoshes as the nfs server. In this instance, the linux box can
mount the file system, but the process freezes when it starts the
rendering, whether it be a single rpict command or an rpiece command.
The other Macintosh is capable of running and completing the parallel
rendering process, but only about 10% of the computers resources is
devoted to the process (this is something that I'll just have to look
into).

Could somebody please help me figure out what is going on?

Thanks.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

5

Hey John,

Yes, NFS is just a big nasty mess to deal with.

It looks like there was some funny configuration going on on the OS X client/server only setup. It would be worth taking another pass at trying to identify what is going on in this configuration.

For OS X to linux NFS server, I think that you will need to determine how to set which nfs protocol version is used (2 or 3) for each system. Additionally you have to have statd running on both the client and server for locking to work. Further you will have to determine what switches need to be set on the OS X to enforce synchronous read/writes.

-Jack

John S. An wrote:

···

Thanks for your advice Jack. I don't know how many people on the list are interested in the intricacies of making nfs work on Mac OS X, but with Apple hardware one of the many hardware options for running Radiance, I'll post updates on my experience. So here goes...

In trying to get a stable Mac OS X only parallel rendering going, I switched the server/client relationship. In other words, the machine that was previously the client became the server, and the server became the client. All other settings remained the same, but this time, the vast majority (~98%) of the processes were allocated to rpict. I'm not sure why this set up is more stable than the other, but it is.

So then I tried mounting the nfs volume from the Linux machines, and tried starting an rpict process with an -af switch, and it also worked. So I thought my problem was solved, but (there's always a but)...

File locking seems to work on the Linux machines, but not on the Macintosh client. When I begin an rpict process on the Macintosh client, and then attach another rpict from the Linux client, then the output file always shows that the ambient values are corrupt.

So, once I figure out how to make the Macintoshes abide by the file-locking protocols, I think all will be well.

One note, for anyone trying make nfs adjustments on the Macintosh, the shareware NFS Manager (http://www.bresink.de/osx/NFSManager.html) by Marcel Bresink is extremely helpful. Unlike most UNIX/Linux distributions, Mac OS X reserves files such as /etc/fstab and /etc/exports for system functions, and thus the settings need to be adjusted in the application NetInfo Manager. NFS Manager puts a friendlier GUI interface to dealing with NetInfo Manager.

In response to Guilio's comment about trying to use Appletalk to share the ambient file, the problem arises in the file locking. I don't think Radiance knows how to deal with file locking in the Appletalk protocol. I too tried initially to see if I could share files over Appletalk, and Radiance always gave me an error message and then quit. I did find some discussion in the developers forum on allowing Radiance to share files over Samba. I haven't had a chance to explore this option any further, but has the Samba file sharing implementation been incorporated into the HEAD version of Radiance?

I hope I'm not boring everyone with this discussion. If so, just let me know, and I'll stop posting any updates.

Message: 5
Date: Mon, 22 Dec 2003 16:05:09 -0500
From: Jack de Valpine <[email protected]>
To: [email protected]
Subject: Re: [Radiance-general] Mac OS X and nfs
Reply-To: [email protected]

Hi John,

I think that one thing to look into in greater depth is what kind of
control you can exercise on the configuration/setup of NFS on the
Panther based machines. In particular you need to have both nfsd running
as well as something called statd (I am assuming these are the names
used in the apple based systems). The latter is responsible for
reporting file status for locking requests among other things.

It sounds like you can get a linux to linux nfs setup running ok. I
think that you need to debug and get a stable apple to apple nfs setup
running (ie full processor performance). Then you can work on figuring
out how to get the two to work together. I suspect that there are some
mount and/or export related switches that may need to be set to make nfs
work in a heterogenous environment like this. It is difficult enough in
a homegenous one.

-Jack

John S. An wrote:

Hi all,

I'm at my wits end trying to get my machines to play nicely when
parallel processing. I have a 4 computer setup which I am trying to
farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports
to give read write access and sync to all machines in my private
network (192.168.0.0/255.255.255.0). The server and the other linux
box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up
the mount for the exported folder. In the Finder, I am able to
access, read, and copy files from the Macintosh to the server, and to
erase files that are on the server. I can also use X11 to run single
processor renderings on files that are located on the nfs server.
However, when I try to run the parallel renderings from the Macintosh,
the process just freezes. Using the top command, I see that rpict is
there, but 0% of the computers resources is devoted to the process,
and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure
that they all match.

Just to test what was going on, I tried setting up one of my
Macintoshes as the nfs server. In this instance, the linux box can
mount the file system, but the process freezes when it starts the
rendering, whether it be a single rpict command or an rpiece command.
The other Macintosh is capable of running and completing the parallel
rendering process, but only about 10% of the computers resources is
devoted to the process (this is something that I'll just have to look
into).

Could somebody please help me figure out what is going on?

Thanks.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

--__--__--

Message: 6
From: Giulio Antonutto <[email protected]>
To: "'[email protected]'" <[email protected]>
Subject: RE: [Radiance-general] Mac OS X and nfs
Date: Tue, 23 Dec 2003 09:36:20 -0000
Reply-To: [email protected]

Hi,
about homogeneous environment:
some time ago (macosX 10.0) I did some rpiece experiments....
It seems to me that I simply employed the Appletalk protocol to share the
ambient file....
may be....
just macosx to macosx.....
cheers,
giulio

PS however at the end I bought a dual athlon Linux machine.... but now it is
different: G5...

-----Original Message-----
From: Jack de Valpine [mailto:[email protected]]
Sent: 22 December 2003 21:05
To: [email protected]
Subject: Re: [Radiance-general] Mac OS X and nfs

Hi John,

I think that one thing to look into in greater depth is what kind of
control you can exercise on the configuration/setup of NFS on the
Panther based machines. In particular you need to have both nfsd running
as well as something called statd (I am assuming these are the names
used in the apple based systems). The latter is responsible for
reporting file status for locking requests among other things.

It sounds like you can get a linux to linux nfs setup running ok. I
think that you need to debug and get a stable apple to apple nfs setup
running (ie full processor performance). Then you can work on figuring
out how to get the two to work together. I suspect that there are some
mount and/or export related switches that may need to be set to make nfs
work in a heterogenous environment like this. It is difficult enough in
a homegenous one.

-Jack

John S. An wrote:

Hi all,

I'm at my wits end trying to get my machines to play nicely when
parallel processing. I have a 4 computer setup which I am trying to
farm together via nfs for some renderings, and I've come to an impass.

Set-up:
2 linux boxes running Mandrake 9.1
2 Macintoshes running Panther (10.3.2)

One of the linux box is set as the nfs server. I set the /etc/exports
to give read write access and sync to all machines in my private
network (192.168.0.0/255.255.255.0). The server and the other linux
box have no problems communicating and parallel processing.

On the Macintoshes, I used the shareware program NFSManager to set up
the mount for the exported folder. In the Finder, I am able to
access, read, and copy files from the Macintosh to the server, and to
erase files that are on the server. I can also use X11 to run single
processor renderings on files that are located on the nfs server.
However, when I try to run the parallel renderings from the Macintosh,
the process just freezes. Using the top command, I see that rpict is
there, but 0% of the computers resources is devoted to the process,
and it remains in that state indefinitely (5 days).

I doubled checked the UID and GID on all for machines to make sure
that they all match.

Just to test what was going on, I tried setting up one of my
Macintoshes as the nfs server. In this instance, the linux box can
mount the file system, but the process freezes when it starts the
rendering, whether it be a single rpict command or an rpiece command.
The other Macintosh is capable of running and completing the parallel
rendering process, but only about 10% of the computers resources is
devoted to the process (this is something that I'll just have to look
into).

Could somebody please help me figure out what is going on?

Thanks.

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

_______________________________________________
Radiance-general mailing list
[email protected]
http://www.radiance-online.org/mailman/listinfo/radiance-general

--
# John E. de Valpine
# president
#
# visarc incorporated
# http://www.visarc.com
#
# channeling technology for superior design and construction

6

Hi,

do you use nfs/udp or nfs/tcp? Try tcp, I know from some friends
that they solved some problems with this. OS X amd Linux play
nicely together in general.

CU Lars.