I have a skill code with about 5000 lines. Inside the top level procedure, everything is very simple: call two lower level functions (let's call them A and B). There is no global variables shared between function A and B except a cell view they will work on. So, I basically can run them step by step on a cell view if I like to do so.Originally posted in cdnusers.org by email@example.com
I can't explain the following facts:
(1). if I run the top level procedure, which means that A and B will be run in sequence, then, it will take forever to finish the second function (B). (>24 hours). VERY SLOW!! The function A takes about 1 minute cputime to run, which is ok.
(2). Run A and B step by step: run function A, which still takes about 1 minute. then, save the cell view and quite opus. restart opus and open the saved cell view. then, run function B on it, which takes about 5 minutes to finish, VERY FAST!!
(3). Run A and B separately. After A finished, immediately start function B, both phenomena of item 1&2 may happen. Some times, I got B takes looong time (>24 hrs) and some times, I got B takes less than 5 minutes. If I interrupt the B function in the middle using "ctrl-C", and then, restart B, the chance is high that B will finish fast (<5 minutes).
My goal is to run the top level procedure directly like in (1) in a reasonable short time.
I have used profile('memory) to look into the memory usage. function A will take ~300mB and B will take ~1GB. I added needNCells to increase the initial memory allocations in my code so that the allocated numbers are large than necessary values reported by profileSummary and gcsummary. By the way, the computer I used has 6GB physical memory, much larger than the needed 1.5GB.
Also, I tried to insert "gc" in several locations of the code to force it collect garbage and tried to make sure all the used variables are local so that their memory can be recycled when a function is not used anymore.
But none of the above methods could help me reduce function B cputime if I run A&B together by calling the top level function.
What have I missed? Please help!