parallel dctimestep

radiance-general
1

Dear all,

This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?

Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?

Thanks,
Santiago

···

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for viruses and acceptability of content

2

Hi Santiago,

I pretty much did the exact same thing that you mentioned to run multiple instances of dctimestep with Python. The syntax is something like:

/import multiprocessing as mp/

/import os//
/

/#Run dctimesteps, 16 at at time./

/#runCommand is just a call to os.system
/

/#commands is a list of dctimestep commands like ["dctimestep dc.mtx sky0001.smx > res0001.smx", ///"dctimestep dc.mtx sky0002.smx > res0002.smx" ...]
//

/pools = mp.Pool(processes=16)/

/pools.map(runCommand, commands)/

For the above, you'd have to do some additional scripting to run rmtxop and compile all the results into a single file. There is also a method that was discussed by Wangda Zuo and Andy Mcneil in this paper: http://www.ibpsa.org/proceedings/BS2011/P_1155.pdf

Sarith

//

···

On 9/22/2017 9:55 AM, Santiago Torres wrote:

Dear all,

This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?

Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?

Thanks,
Santiago

____________________________________________________________
Electronic mail messages entering and leaving Arup business
systems are scanned for viruses and acceptability of content

_______________________________________________
Radiance-general mailing list
[email protected]
https://www.radiance-online.org/mailman/listinfo/radiance-general

1 Like
3

Hi Santiago,

A research assistant at LBNL spent some time implementing a GPU version of dctimestep, and found that the process was i/o-bound, meaning that the matrix multiplications, etc. took much less time to do that just loading the input and writing out the result. His process was still a bit faster than the original dctimestep, but since then, I implemented some optimizations to avoid multiplying zero rows and vectors (mostly nighttime points from gendaymtx), which meant the GPU implementation was really no faster.

If you have an SSD or RAMdisk for i/o, then you may yet eek some benefit from running the calculation in parallel, but you'd have to interleave the matrices in the end or handle subsequent calcs by month. Interleaving would probably require transposing each matrix, concatenating, then transposing the result. It's all doable with rcollate, but that will take a bit of time as well.

Cheers,
-Greg

···

From: Santiago Torres <[email protected]>
Date: September 22, 2017 7:55:53 AM PDT

Dear all,

This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?

Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?

Thanks,
Santiago

4

Dear Sarith, Greg,

Thank you for your replies. In this case I am running the annual calculation of an image, and each time-step (daytime) is taking 3-4 minutes to run, so the total calculation takes more than 9 days. And I would like to increase the resolution to reduce noise…

So I thought if I run 10 dctimestep processes in parallel, it should take less than a day. I hope this makes sense. I will try it out and let you know if I find problems.

Thank you,
Santiago

···

From: Sarith Subramaniam [mailto:[email protected]]
Sent: 22 September 2017 16:53
To: [email protected]
Subject: Re: [Radiance-general] parallel dctimestep

Hi Santiago,

I pretty much did the exact same thing that you mentioned to run multiple instances of dctimestep with Python. The syntax is something like:

import multiprocessing as mp

import os

#Run dctimesteps, 16 at at time.

#runCommand is just a call to os.system

#commands is a list of dctimestep commands like ["dctimestep dc.mtx sky0001.smx > res0001.smx", "dctimestep dc.mtx sky0002.smx > res0002.smx" ...]

pools = mp.Pool(processes=16)

pools.map(runCommand, commands)

For the above, you'd have to do some additional scripting to run rmtxop and compile all the results into a single file. There is also a method that was discussed by Wangda Zuo and Andy Mcneil in this paper: http://www.ibpsa.org/proceedings/BS2011/P_1155.pdf<http://secure-web.cisco.com/1Dg4sbIwWzfdNHSDFjoErkam7XpFZkZl_YajPwbdNSfN3qC1GSwtzOsoHjzUTCbiw63iGUUuE2gD4bllTAnDKmlmn_oe9aU9yRt8wzVUSrmsrE1PKj6ErqSMsFikzAe_P4HEM_ZQk3m52HImTpdUxnGXUzrzYTkz-kgXQZX-U9wtAWex8uSdvjmJXLQZIBdOdjRn7Um1GekbwQI--4karqReEQedYXrhJsex6RkYtNIiYFMCYiwet0zel4nY98Gol/http%3A%2F%2Fwww.ibpsa.org%2Fproceedings%2FBS2011%2FP_1155.pdf>

Sarith

On 9/22/2017 9:55 AM, Santiago Torres wrote:

Dear all,

This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?

Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?

Thanks,

Santiago

____________________________________________________________

Electronic mail messages entering and leaving Arup business

systems are scanned for viruses and acceptability of content

_______________________________________________

Radiance-general mailing list

[email protected]<mailto:[email protected]>

https://www.radiance-online.org/mailman/listinfo/radiance-general<https://secure-web.cisco.com/1dNm1tL5ZkC_5oSXGas8FrDHQjSEoA-AXQFIbvI-laeD2_dA8ZZxclAD1SI0QMbRn3l7yrnbDa-7x9KSVqSDSTJ3bk0nLMZT75aOcULZ2TMPQzn3kYPcm_Vpu7o6w5IfTvXvVEOjQE9m0GLvQr7ApX14n8GJGbpG4u9HQmjOqTWn9t7lgXoUehVcv90pQYsw4Lx80fWfdCqcYRrWNqbGCOHyq05iKumfv2LomyE18zxqH84Dccnn9kXeQJ4djRZyB/https%3A%2F%2Fwww.radiance-online.org%2Fmailman%2Flistinfo%2Fradiance-general>

5

Ah, that's a different problem!

How much memory is on your machine? If there is less RAM than the combined size of the component images, then you won't see the benefit of disk caching. You can try decreasing the memory required for caching by compacting your pictures (e.g., running "ra_rgbe -r" on each of them).

If you don't have as much free RAM as you have images and you're running off a hard disk rather than an SSD, that is the worst case for dctimestep and image processing. Running in parallel may help you a bit, but you still need to think about the i/o requirements. Having a bunch of processes fighting to access the same disk information might not do that much.

Cheers,
-Greg

···

From: Santiago Torres <[email protected]>
Date: September 22, 2017 9:13:50 AM PDT

Dear Sarith, Greg,

Thank you for your replies. In this case I am running the annual calculation of an image, and each time-step (daytime) is taking 3-4 minutes to run, so the total calculation takes more than 9 days. And I would like to increase the resolution to reduce noise…

So I thought if I run 10 dctimestep processes in parallel, it should take less than a day. I hope this makes sense. I will try it out and let you know if I find problems.

Thank you,
Santiago

From: Sarith Subramaniam [mailto:[email protected]]
Sent: 22 September 2017 16:53

Hi Santiago,

I pretty much did the exact same thing that you mentioned to run multiple instances of dctimestep with Python. The syntax is something like:
import multiprocessing as mp

import os

#Run dctimesteps, 16 at at time.

#runCommand is just a call to os.system

#commands is a list of dctimestep commands like ["dctimestep dc.mtx sky0001.smx > res0001.smx", "dctimestep dc.mtx sky0002.smx > res0002.smx" ...]

pools = mp.Pool(processes=16)

pools.map(runCommand, commands)

For the above, you'd have to do some additional scripting to run rmtxop and compile all the results into a single file. There is also a method that was discussed by Wangda Zuo and Andy Mcneil in this paper: http://www.ibpsa.org/proceedings/BS2011/P_1155.pdf
Sarith

On 9/22/2017 9:55 AM, Santiago Torres wrote:
Dear all,

This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?

Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?

Thanks,
Santiago

6

Hi Santiago and Greg,

To add to my original reply, I ran my simulations using the standard version of dctimestep. I use a fairly souped-up Microsoft Azure cloud-based VM which has� 250GB of RAM. If you are parallelizing image-based simulations, you'd want to explicitly specify the month-name, or number, in the string formatting option( something like -o results/Jan%03d.hdr). If not, there is a chance that the results from one instance of dctimestep might get overwritten by the other (I learnt this the hard way).

Sarith

···

On 9/22/2017 11:38 AM, Greg Ward wrote:

Ah, that's a different problem!

How much memory is on your machine? �If there is less RAM than the combined size of the component images, then you won't see the benefit of disk caching. �You can try decreasing the memory required for caching by compacting your pictures (e.g., running "ra_rgbe -r" on each of them).

If you don't have as much free RAM as you have images and you're running off a hard disk rather than an SSD, that is the worst case for dctimestep and image processing. �Running in parallel may help you a bit, but you still need to think about the i/o requirements. �Having a bunch of processes fighting to access the same disk information might not do that much.

Cheers,
-Greg

*From: *Santiago Torres <[email protected] <mailto:[email protected]>>

*Date: *September 22, 2017 9:13:50 AM PDT

*

Dear Sarith, Greg,

Thank you for your replies. In this case I am running the annual calculation of an image, and each time-step (daytime) is taking 3-4 minutes to run, so the total calculation takes more than 9 days. And I would like to increase the resolution to reduce noise�

So I thought if I run 10 dctimestep processes in parallel, it should take less than a day. I hope this makes sense. I will try it out and let you know if I find problems.

Thank you,

Santiago

*From:*Sarith Subramaniam [mailto:[email protected]]
*Sent:* 22 September 2017 16:53*
*

Hi Santiago,

I pretty much did the exact same thing that you mentioned to run multiple instances of dctimestep with Python. The syntax is something like:

/import multiprocessing as mp/

/import os/

/#Run dctimesteps, 16 at at time./

/#runCommand is just a call to os.system/

/#commands is a list of dctimestep commands like ["dctimestep dc.mtx sky0001.smx > res0001.smx",� "dctimestep dc.mtx sky0002.smx > res0002.smx" ...]/

/pools = mp.Pool(processes=16)/

/pools.map(runCommand, commands)/

For the above, you'd have to do some additional scripting to run rmtxop and compile all the results into a single file. There is also a method that was discussed by Wangda Zuo and Andy Mcneil in this paper: http://www.ibpsa.org/proceedings/BS2011/P_1155.pdf <http://secure-web.cisco.com/1Dg4sbIwWzfdNHSDFjoErkam7XpFZkZl_YajPwbdNSfN3qC1GSwtzOsoHjzUTCbiw63iGUUuE2gD4bllTAnDKmlmn_oe9aU9yRt8wzVUSrmsrE1PKj6ErqSMsFikzAe_P4HEM_ZQk3m52HImTpdUxnGXUzrzYTkz-kgXQZX-U9wtAWex8uSdvjmJXLQZIBdOdjRn7Um1GekbwQI--4karqReEQedYXrhJsex6RkYtNIiYFMCYiwet0zel4nY98Gol/http%3A%2F%2Fwww.ibpsa.org%2Fproceedings%2FBS2011%2FP_1155.pdf>

Sarith

On 9/22/2017 9:55 AM, Santiago Torres wrote:

    Dear all,

    This might be a silly question, but I couldn't find any reference. Is there a way to run dctimestep in parallel?

    Or, would it be possible to create a weather tape for each month, create a matrix from each, and run each month separately?

    Thanks,

    Santiago

_______________________________________________
Radiance-general mailing list
[email protected]
https://www.radiance-online.org/mailman/listinfo/radiance-general

7

Thank you Sarith. I just wanted to share my successful implementation of multiprocessing in Py 3.7.

import os
import multiprocessing as mp

def oconve (string):
cmd_oconv = "oconv -r 2048 -f "+ string
os.system(cmd_oconv)

oconv_lst = [’\tmp\sky0.sky \scene\G1\opaque\Name…opq.mat \scene\G1\opaque\Name…opq.rad > \tmp\Spectral_0.oct’,
‘\tmp\sky1.sky \scene\G1\opaque\Name…opq.mat \scene\G1\opaque\Name…opq.rad > \tmp\Spectral_1.oct’,
‘\tmp\sky2.sky \scene\G1\opaque\Name…opq.mat \scene\G1\opaque\Name…opq.rad > \tmp\Spectral_2.oct’]

n_cpu = mp.cpu_count()
with mp.Pool(processes=n_cpu) as pools:

pools.map(oconve, oconv_lst)