I am attempting to run the below rtpict command on a google cloud VM instance with 192 CPUs. The command successfully runs. The problem I’m running into is that the max number of parallel process I can observe is 64.
I believe it may relate to the MAX_NPROCS set at raypcalls.c:157
I’m hoping to avoid modifying the source code myself and recompiling to prevent adding any further complexity to the rendering workflows I’ve created, and distribution to other team members.
What would you suggest is the best way around this limit? or is there another source of the limit on the number of processes that could be occurring when using rtpict?
There are usually diminishing returns as you increase the number of ray-tracing processes. Are you seeing near-linear speed-ups out to 64 processes, or are you just trying to push things to see how far you can go? At some point, the overhead of talking to all those proceses becomes the major bottleneck, if you don’t hit a problem earlier with ambient cache sharing or the like.
In any case, you are welcome to add -DMAX_NPROCS=100 or whatever you want to try. (Be sure to get the latest HEAD source though, because I just noticed a typo in the macros surrounding that setting.) There is a system-imposed limit as you can see based on the number of file descriptors handled by the select() call, which we have no control over.
Cheers,
-Greg
P.S. I should add that for a single output image, rxpiece might be the better choice as it does not rely on raypcalls nor does it have the same controlling-process bottleneck.