Replacing rmxtop with rcollate, rcalc for speedup & reduced file size

radiance-general
1

Hello,

Based on the following two discussions, I am now of the understanding that replacing rmtxop with combination(s) of rcollate & rcalc can help speedup simulation runtime, while also reducing the memory requirements:

rcalc for dctimestep

dctimestep out-of-core

I am running radiance 5.4a on WSL, and trying to test a workflow using Sarith’s 5 phase tutorial.

Based on my understanding of the the discussions above, I replaced:

dctimestep  matrices/vmtx/v.mtx matrices/tmtx/blinds.xml matrices/dmtx/daylight.dmx skyVectors/NYC.smx | rmtxop -fa -t -c 47.4 119.9 11.6 - > results/5ph/3ph/3phAnnual.ill

with:

dctimestep -od matrices/vmtx/v.mtx matrices/tmtx/blinds.xml matrices/dmtx/daylight.dmx skyVectors/NYC.smx > matrices/temp3P.mtx

rcollate -fd3 -t -ho matrices/temp3P.mtx | rcalc -id3 -od -e '$1=47.4*$1 + 119.9*$2 + 11.6*$3' | rcollate -hi -fd -or 40 -oc 100 > results/5ph/3ph/3phAnnual.txt

this seems to be working as expected (also switching from ASCII in original command to “Double” in the new one really keeps the file size in check).

However, I am struggling to replace the final step of combining all the matrices properly.

Original:

rmtxop results/5ph/3ph/3phAnnual.ill + -s -1 results/5ph/3phdir/3phAnnualDir.ill + results/5ph/cds/annualRSun.ill > results/5ph/5Phannual.ill

My current (failed) implementation of this step to replace rmtxop, inspired by Andy’s original 5-phase tutorial:

rlam -id results/5ph/3ph/3phAnnual.txt results/5ph/3phdir/3phAnnualDir.txt results/5ph/cds/annualRSun.txt | getinfo - | rcalc -id3 -e '$1=($1-$2)+$3' | rcollate -hi -fa -t -or 40 -oc 100 > results/5ph/5Phannual.ill

Something seems messed up with this step however, as the generated .ill does looks absolutely wrong, even containing some negative values!

I would be really grateful if someone could guide me with the proper setup for the last step. I also ask myself - is this even beneficial - or should I just keep things simple by continuing to use rmtxop for this final step ?

Many thanks!
Hamza

2

Hi Hamza,

You should spend some time getting acquainted with the new rcomb command and what it can do. You can add or subtract matrices, convert them from RGB to Y, multiply two matrices, and quite a few other things. It is part of the latest 6.0a release found here.

The man page is included in the release, and I am linking a PDF of it, below.

Cheers,
-Greg

rcomb(1) man page PDF

3

Thanks a lot for the swift reply Greg & for rcomb :slight_smile: .

I am trying to learn the ropes still, however I now successfully managed to replace the original rmtxop command with rcomb as follows :slight_smile: :

original:

rmtxop -fa results/dcDDS/dc/annualR.txt + -s -1 results/dcDDS/dcd/annualRDir.txt + results/dcDDS/cds/annualRSun.txt > results/dcDDS/annualR.ill

new:

rcomb -n 10 -h -fa -e "co=(ci(1)-ci(2))+ci(3)" results/dcDDS/dc/annualR.txt results/dcDDS/dcd/annualRDir.txt results/dcDDS/cds/annualRSun.txt > results/dcDDS/annualR.ill

So far, so good.

The addition of multiprocessing in rcomb, and it’s ability to also do multiplication, now got me really excited and I wanted to replace dctimestep with rcomb to see if it helps speed things up a bit.

Original:

dctimestep matrices/dc/illum.mtx skyVectors/NYC.smx > matrices/temp2P.mtx

Rcomb:

rcomb -n 10 -h -e "co=ci(1)*ci(2)" matrices/dc/illum.mtx skyVectors/NYC.smx > matrices/temp2P_rcomb.mtx

However, I am now getting the following error:

skyVectors/NYC.smx: mismatch in size or #components

Any tips on what I am doing wrong, and what should I fix in the command, would be greatly appreciated. I am attaching the matrix files (currently in ASCII for readability & cross-checking results, but eventually intended to be ‘double’) in the link below. Apologies for the rookie mistakes.

test_matrices.zip

Best,
Hamza

4

Hi Hamza,

For the first replacement, you did nothing wrong, but it will be faster if you mimic the syntax you used for rmtxop in rcomb, but without the ‘+’ signs, since addition is the default operation for rcomb. Like so:

rcomb -fa results/dcDDS/dc/annualR.txt -s -1 results/dcDDS/dcd/annualRDir.txt results/dcDDS/cds/annualRSun.txt > results/dcDDS/annualR.ill

Also, I don’t think you’ll gain much with the -n option, since each operation is relatively fast. I reserve that for big matrix multiplies and things where processing each row takes significant time. You can try with and without to see what you get. Maybe with ASCII i/o, it makes a difference?

In the dctimestep example, you are actually concatenating two matrices, and it’s hard to beat dctimestep for that unless the first matrix is large enough to run into memory problems. In any case, your expression for rcomb would do per-element multipication, which is not the same thing. You want to do:

rcomb matrices/dc/illum.mtx -m skyVectors/NYC.smx > matrices/temp2P.mtx

instead.

Hope this helps!
-Greg

1 Like
5

Thank you so much for the explanation Greg - this works now as expected!

Many thanks and wish you a great day ahead. :slight_smile:

EDIT: I will try to test the benefit of -n on larger matrices and let you know if there is a big difference or not.