starting rpict -S 1 -P pfile

Hi!

I am currently changing my Makefile that is doing some rendering. So far, the views had been in the Makefile itself, each as one target. Now I wanted to change this and use a viewfile together with the -P option of rpict for multi-processing.

I am wondering now how to understand the manual. I used a line like that:

rpict -S 1 -P pics.pfile -o pic%02d.pic scene.oct < scene.vf &

Because I thought that the rpict process must go to the background, so that I can start the other processes with

rpict -P pics.pfile &

However, I did not get any output. So was the & after the first rpict the problem? Will it read the scene and send itself to the background, waiting for more processes? Or do I simply have to wait before starting the second rpict, until the first one has read the entire scene?

TIA + CU, this multi-processor radiance stuff is really not easy... Lars.

pics: scene.oct scene.vf
  rpict -S 1 -P pics.pfile -o pic%02d.pic scene.oct < scene.vf & \
  rpict -P pics.pfile & \
  rpict -P pics.pfile &

···

from my Makeile:

Hi Lars,

It may just be that you need to use the -PP option rather than -P, which doesn't do parallel processing. (The -P "persist mode" is a relatively little-used option that permits serial execution without the associated startup costs -- it's more valuable for rtrace than rpict.)

Also, each of your rpict -PP commands should be handed the same set of views on the standard input. Otherwise, nothing will happen.

-Greg

···

From: "Lars O. Grobe" <[email protected]>
Date: October 15, 2005 9:59:06 AM PDT

Hi!

I am currently changing my Makefile that is doing some rendering. So far, the views had been in the Makefile itself, each as one target. Now I wanted to change this and use a viewfile together with the -P option of rpict for multi-processing.

I am wondering now how to understand the manual. I used a line like that:

rpict -S 1 -P pics.pfile -o pic%02d.pic scene.oct < scene.vf &

Because I thought that the rpict process must go to the background, so that I can start the other processes with

rpict -P pics.pfile &

However, I did not get any output. So was the & after the first rpict the problem? Will it read the scene and send itself to the background, waiting for more processes? Or do I simply have to wait before starting the second rpict, until the first one has read the entire scene?

TIA + CU, this multi-processor radiance stuff is really not easy... Lars.

from my Makeile:

pics: scene.oct scene.vf
    rpict -S 1 -P pics.pfile -o pic%02d.pic scene.oct < scene.vf & \
    rpict -P pics.pfile & \
    rpict -P pics.pfile &

It may just be that you need to use the -PP option rather than -P, which doesn't do parallel processing. (The -P "persist mode" is a relatively little-used option that permits serial execution without the associated startup costs -- it's more valuable for rtrace than rpict.)

Also, each of your rpict -PP commands should be handed the same set of views on the standard input. Otherwise, nothing will happen.

I cannot use -PP because my openmosix doesn't support shared memory. I forgot to pass the view file, as I read that command line parameters don't have to be passed. Of course, I don't pass views as command line parameters, but from stdin, so I have to feed the view file for each process, right? :wink:

I think that if I start more than one -P process, they WILL run parallel (even if they use much more memory than the shared -PP processes), right?

Thank You, CU Lars.

There's no sense in using the -P option if you're after parallel runs, because -P insists on serial runs -- it's a mechanism to connect sequential invocations to the same process. It's exactly the opposite of what you need. Just use multiple rpicts if OpenMosix doesn't support shared memory. It will be the same as using -PP in that case, since there's nothing about the -PP option that *requires* shared memory. It just ends up using it if it's available.

-Greg

···

From: "Lars O. Grobe" <[email protected]>
Date: October 15, 2005 11:50:29 AM PDT

It may just be that you need to use the -PP option rather than -P, which doesn't do parallel processing. (The -P "persist mode" is a relatively little-used option that permits serial execution without the associated startup costs -- it's more valuable for rtrace than rpict.)

Also, each of your rpict -PP commands should be handed the same set of views on the standard input. Otherwise, nothing will happen.

I cannot use -PP because my openmosix doesn't support shared memory. I forgot to pass the view file, as I read that command line parameters don't have to be passed. Of course, I don't pass views as command line parameters, but from stdin, so I have to feed the view file for each process, right? :wink:

I think that if I start more than one -P process, they WILL run parallel (even if they use much more memory than the shared -PP processes), right?

Thank You, CU Lars.

Hi Greg!

There's no sense in using the -P option if you're after parallel runs, because -P insists on serial runs -- it's a mechanism to connect sequential invocations to the same process. It's exactly the opposite of what you need. Just use multiple rpicts if OpenMosix doesn't support shared memory. It will be the same as using -PP in that case, since there's nothing about the -PP option that *requires* shared memory. It just ends up using it if it's available.

Well, the reason why I wanted to use -P is that I need some way to make the processes render just the missing frames. So they need to be synced. If I use just several rpict's without -PP, I will have to split my view files, and that means that I have to regroup the views in the view files each time that the cluster configuration changes.

The problem with using rpict with -PP is that rpict will share memory, and than openmosix will not allow the processes to migrata any more. So if the -PP option is the only way to make multiple rpict processes sync, I will have to go back and use a target for each view in my Makefile. Than I can use make -j # with # being the number of machines, the most generic way to distribute the processes. I am not too happy with that because I will have the problem with the viewfiles sooner or later when starting animations - I cannot have one target per frame, so I will have to find a way to render ONE viewfile's views in parallel without that "shared memory".

In fact I am seriously considering using ranimate for my stills now, as this allows me to distribute processes from ONE viewfile. I could simply add HOST-lines pointing to localhost, which is a bit silly, but should work fine.

Thank you for all the help, and have a nice week-end, we have a nice sunny autumn day here and it is a pity to stay in the lab :wink:

CU Lars.

Hi Lars,

You don't need the -P or -PP option to synchronize rpict using -o. If you read the man page carefully, you'll see that the coordination happens simply because rpict -S skips -o outputs that are already started. This is the same way -PP works, so the two functions are orthogonal.

-Greg

P.S. You could use ranimate for this purpose, but there's no real advantage, and you would have to turn frame interpolation off.

···

From: "Lars O. Grobe" <[email protected]>
Date: October 16, 2005 3:44:58 AM PDT

Hi Greg!

There's no sense in using the -P option if you're after parallel runs, because -P insists on serial runs -- it's a mechanism to connect sequential invocations to the same process. It's exactly the opposite of what you need. Just use multiple rpicts if OpenMosix doesn't support shared memory. It will be the same as using -PP in that case, since there's nothing about the -PP option that *requires* shared memory. It just ends up using it if it's available.

Well, the reason why I wanted to use -P is that I need some way to make the processes render just the missing frames. So they need to be synced. If I use just several rpict's without -PP, I will have to split my view files, and that means that I have to regroup the views in the view files each time that the cluster configuration changes.

The problem with using rpict with -PP is that rpict will share memory, and than openmosix will not allow the processes to migrata any more. So if the -PP option is the only way to make multiple rpict processes sync, I will have to go back and use a target for each view in my Makefile. Than I can use make -j # with # being the number of machines, the most generic way to distribute the processes. I am not too happy with that because I will have the problem with the viewfiles sooner or later when starting animations - I cannot have one target per frame, so I will have to find a way to render ONE viewfile's views in parallel without that "shared memory".

In fact I am seriously considering using ranimate for my stills now, as this allows me to distribute processes from ONE viewfile. I could simply add HOST-lines pointing to localhost, which is a bit silly, but should work fine.

Thank you for all the help, and have a nice week-end, we have a nice sunny autumn day here and it is a pity to stay in the lab :wink:

CU Lars.