HOUDINI ONLY USING 2-10% CPU?

[chrisdunham95]

Having a real issue recently on my main pc - its simming out a pyro sim for a student project - and its using all 32gb of ram - but only 2-10% cpu on average. Sometimes it will go up for a small time AND at the beggining of the simulation it seems to use 90% CPU atleast, - its then also using one of my hard drives to compute - assuming this is due to ram being used - The specific scene im working on now is loading of my second 3tb drive. (I do realise simming using files on my SSD would be quicker but dont have enough space right now to do that)  - all three hard drives have 100gb + free space also. 

Is this a houdini issue?

Or my hardware?

Or simply the simulation is high res and is just taking a long while? 

Specs;

 

Intel Core i7-5820K - 3.30ghz - 32gb DDR4 - Nvidia GeForce GTX 770 - 

ONE 500GB SSD - TWO 3TB standard seagate hard drives 

 

Any advice or help would be great! 

 

[Atom]

If you have enabled OpenCL most of the computation will happen on your graphics card, not the CPU. Are you monitoring your GPU as well? Try disabling OpenCL (it may crash on large sims anyway) and see if your CPU usage goes up.

Also, split the difference on the simulation resolution. Move it down a notch.

[Guest tar]

2% to 10% sounds weird - smoke solver uses ~95% of all threads, including hyper threading on OsX.

[chrisdunham95]

On 01/05/2016 at 9:29 PM, Atom said:

If you have enabled OpenCL most of the computation will happen on your graphics card, not the CPU. Are you monitoring your GPU as well? Try disabling OpenCL (it may crash on large sims anyway) and see if your CPU usage goes up.

Also, split the difference on the simulation resolution. Move it down a notch.

 

On 02/05/2016 at 5:52 AM, marty said:

2% to 10% sounds weird - smoke solver uses ~95% of all threads, including hyper threading on OsX.

Awww i never thought about open CL ill check my files thanks atom! - i have been dropping res but its for projects for a student film day so some sims literally just need more res 

 

Thanks guys! 

[symek]

So called 'simulation' is a series of steps performed one after another mostly  - and only one of them is highly multi-threaded smoke simulation. How the whole job is performing depends on every step. From my experience the easiest way to harm your simulation (that is slow it down) is to have collision object which is hard to voxelize, like high-res mesh of deforming character. If you do heavy sim, lots' of version etc, bake your collision objects to disk. 

[lukeiamyourfather]

On 5/1/2016 at 2:03 PM, chrisdunham95 said:

its using all 32gb of ram - but only 2-10% cpu on average

That's because the memory is full and it starts using swap so the bottleneck becomes the storage device being used for swap rather than the processor being the bottleneck. Get more memory and the processor will be better utilized because it won't be limited by swap anymore.

Edited May 11, 2016 by lukeiamyourfather

[hkspowers]

Can confirm Luke's assessment in conjunction with Symek's.  I'm running dual 12 core  E5-2680 v3 @ 2.50GHz and I get a similar problem to yours but I know exactly why.

When it does the actual pyro calculation in the sim, procs get pegged at 100%, but when it goes to write out the huge geo file for that frame the cpu tanks to 2%, but that's only because it's waiting on the file write over the network.  So you will see in my case a very predictable pattern of peaks and valleys in the task manager as the sim progresses.   If I was writing locally to a pci based ssd, the whole thing would be sped up as it's not bottlenecked by my storage solution. 

 

 

 

[Guest tar]

Sounds like SideFx sound do the writing on a background thread.

 

Edit: There is a 'Save to Disk in Background' option. Are people using it?

Edited May 14, 2016 by tar

[hkspowers]

I've used that quite a bit and while it's great for home use, it does seem to increase the sim time considerably if you are doing a heavy sim.  For smaller sims it's pretty transparent and hardly noticeable which makes it great for pops and quick pyro sims.

[Guest tar]

Interesting - the popup help does say it needs more Ram to run in the background, so it must be competing for resources on large ram-usage sims.  An SSD drive should be super helpful for paging/swap.

[malexander]

16 hours ago, marty said:

Interesting - the popup help does say it needs more Ram to run in the background, so it must be competing for resources on large ram-usage sims.  An SSD drive should be super helpful for paging/swap.

Yes, any sort of background saving requires that the data is kept around for saving while a new copy of the data is created to start simulating the next frame. Once the save is complete the data from the previous frame will be freed. This causes peak memory usage to be higher.

[anim]

a few things you can try to reduce your RAM/Disk usage:

- disable caching on Dopnet, as you are saving your sim to disk anyways

- make sure you optimize your geo before saving to disk, so just keep fields you really need, don't forget to Volume Compress them, and many times you can reduce the grid resolution of vel field to half and remove all values from voxels with no density

 

[chrisdunham95]

Thanks for the replys guys i think your correct regarding th bottlenecking and simply needing more ram! - thanks for the tips appreciated all!