cudarsjanis Posted February 3, 2016 Share Posted February 3, 2016 Is there any way how I can limit Houdini RAM and CPU usage while writing out simulations with ROP outputdriver? My system crashed few times already while writing out whitewater particles. and I had to start all over again. In Adobe software there's simple setting in preferences - how much RAM to leave for other software to use. In houdini what I found is that I can bump down cache size on DOP, and that is basically it. I'm new to houdini. Also I'm using save to disk in background. so it should use less ram Now simulation is 85% done. so gonna render everything I hope. What I did is I unchecked write cache al all on DOP net and whitewater DOP, don't know if this is the most efficient way to render. Any advice would be appreciated. Thank you My system is not so bad, so I should be able to write out large simulations. Intel® Core™i7 Quad Core Processor i7-4790k (4.0GHz) 8MB Cache 32GB KINGSTON HYPER-X FURY DUAL-DDR3 1600MHz (4 x 8GB) 4GB NVIDIA GEFORCE GTX 970 Quote Link to comment Share on other sites More sharing options...
Skybar Posted February 3, 2016 Share Posted February 3, 2016 Reduce the resolution of your simulation. It pretty much needs as much RAM as it needs. Quote Link to comment Share on other sites More sharing options...
Atom Posted February 3, 2016 Share Posted February 3, 2016 You can also try the resume cache feature that way you can pick up where the crash happened. I just learned about this. It is located on the Cache TAB of the AutoDop Network. Try activating Save Checkpoints and read the help card. Quote Link to comment Share on other sites More sharing options...
fathom Posted February 3, 2016 Share Posted February 3, 2016 what operating system? windows really sucks with memory management in houdini. linux is boatloads better. save in background actually INCREASES ram, so if you're hitting a problem with ram, turn it off. save in bg will spawn off copies of your sim data to a write cache after each frame is simmed so it can continue the sim while a separate thread does the writing from that cache. if you have a really massive data set or a slow drive, you might end up with two or three or more whole sims in ram at the same time. Quote Link to comment Share on other sites More sharing options...
Guest tar Posted February 3, 2016 Share Posted February 3, 2016 32 GB is not so much for simulations - more than enough for most programs and rendering etc, but simulations require a lot. Think 64GB as the median. Quote Link to comment Share on other sites More sharing options...
cudarsjanis Posted February 3, 2016 Author Share Posted February 3, 2016 (edited) what operating system? windows really sucks with memory management in houdini. linux is boatloads better. save in background actually INCREASES ram, so if you're hitting a problem with ram, turn it off. save in bg will spawn off copies of your sim data to a write cache after each frame is simmed so it can continue the sim while a separate thread does the writing from that cache. if you have a really massive data set or a slow drive, you might end up with two or three or more whole sims in ram at the same time. I'm on Win 7 drive isn't slow as I'm running my system on SSD Writing data on SATA hdd, though. I rendered out the with lower particle count. around 3 Gb per frame on peak now. not so much detail in foam now as the most of the particles are bubbles. So save in background really uses more RAM at the end? ok, so not gonna use that for large sims. Edited February 3, 2016 by cudarsjanis Quote Link to comment Share on other sites More sharing options...
fathom Posted February 3, 2016 Share Posted February 3, 2016 yeah, i figured it was windows.... writing directly to sata.... that should be reasonably fast. turning off save in bg means your sim will park while writing to disk, but you don't have the ram to spare. are you doing much post processing on your geo prior to writing? you might need to unload any sops you're adding into the mix to help ease the ram load. Quote Link to comment Share on other sites More sharing options...
LukeLetellier Posted February 6, 2016 Share Posted February 6, 2016 what operating system? windows really sucks with memory management in houdini. linux is boatloads better. There really needs to be a fix for this. There are those of us who are _relatively_ technical minded, but not to a Linux degree - and I'd much rather spend my time trying to master Houdini then trying to figure out how to simply install a driver or a program. I know for many of you that's laughable, but I do have priorities in my life - and I'd rather focus my efforts elsewhere. Quote Link to comment Share on other sites More sharing options...
Syrena Posted February 6, 2016 Share Posted February 6, 2016 (edited) Is there any way how I can limit Houdini RAM and CPU usage while writing out simulations with ROP outputdriver? My system crashed few times already while writing out whitewater particles. and I had to start all over again. In Adobe software there's simple setting in preferences - how much RAM to leave for other software to use. In houdini what I found is that I can bump down cache size on DOP, and that is basically it. I'm new to houdini. Also I'm using save to disk in background. so it should use less ram Now simulation is 85% done. so gonna render everything I hope. What I did is I unchecked write cache al all on DOP net and whitewater DOP, don't know if this is the most efficient way to render. Any advice would be appreciated. Thank you My system is not so bad, so I should be able to write out large simulations. Intel® Core™i7 Quad Core Processor i7-4790k (4.0GHz) 8MB Cache 32GB KINGSTON HYPER-X FURY DUAL-DDR3 1600MHz (4 x 8GB) 4GB NVIDIA GEFORCE GTX 970 What you want to do is make sure you disable caching on the Autodop node (which you already had), and inside on the object. Disable pre-conditionar maybe? Clean any residual caches in the cache manager. When you write out your sim files add dot sc e.g. bgeo.sc (use the BLOSC compression) which should be the default if you're using compressed fluids workflow. Now you want to get rid of attributes you don't need before the ROP with an attribute delete node. You might want to delete primitive groups you don't need as well since it was packed data (depending on what type of sim it is). I just started to learn Houdini, and these things helped to reduce file size, and speed things up reading them back in. Not sure if this is what you're trying to do. Your system is enough to do a lot of sims. I have a similar system, with 6700K 4.6ghz and Titan X SC with 32 Gigs of DDR4 memory. I don't see any improvement using OPENCL for fluid sims. I understand you have to make the whole workflow opencl friendly, but I haven't don't that yet, and I haven't got to learning PYRO either, except for making little torches for lighting. Edited February 6, 2016 by Syrena Quote Link to comment Share on other sites More sharing options...
Guest tar Posted February 6, 2016 Share Posted February 6, 2016 There really needs to be a fix for this. There are those of us who are _relatively_ technical minded, but not to a Linux degree - and I'd much rather spend my time trying to master Houdini then trying to figure out how to simply install a driver or a program. I know for many of you that's laughable, but I do have priorities in my life - and I'd rather focus my efforts elsewhere. This sounds fine in theory but memory issues may not be fixed for a while, so the realpolitik is you have to use Linux or OsX as it has superior mallocs. Think of learning Linux, which has tones of help online with Ubuntu or Mint, as a long term investment in the best architecture for the future. Quote Link to comment Share on other sites More sharing options...
LukeLetellier Posted February 6, 2016 Share Posted February 6, 2016 (edited) This sounds fine in theory but memory issues may not be fixed for a while, so the realpolitik is you have to use Linux or OsX as it has superior mallocs. Think of learning Linux, which has tones of help online with Ubuntu or Mint, as a long term investment in the best architecture for the future. Yes. You're probably right. . There's just so much to learn in our industry and so little time! Edited February 6, 2016 by LukeLetellier Quote Link to comment Share on other sites More sharing options...
goldleaf Posted February 7, 2016 Share Posted February 7, 2016 Man, I was worried at first about diving into Linux. These days, Ubuntu or Linux Mint have so many drivers ready to go out of the box, it's very, very straightforward to get going, and as intimated earlier, there's lots of help online. And if you can't completely shed Windows, either Dual Boot, or get a second machine + display and use http://synergy-project.org/ to share your keyboard/mouse. I find Mint is a wonderful blend of stability (based on Ubuntu Long Term Support), recent features/fixes, and easy to install drivers. But back to the OP: yeah 32GB will get eaten up pretty fast, especially with SIMs volumes. I've worked on 96GB and 128GB workstations, and still find myself hitting the limit. Another win for having gobs of RAM, is it gives you more time to catch a mistake that's eating all of your memory. Accidentally hitting 0.0001 on Voxel Size, or scattering billions of points, can be annoying (working in manual cook mode is a good workaround). Quote Link to comment Share on other sites More sharing options...
Guest tar Posted February 8, 2016 Share Posted February 8, 2016 Ya - a good tip for fiddling with Linux is to have a second computer on the Google, then you just search your issues when the main one drops to grub! Quote Link to comment Share on other sites More sharing options...
cudarsjanis Posted February 8, 2016 Author Share Posted February 8, 2016 What you want to do is make sure you disable caching on the Autodop node (which you already had), and inside on the object. Disable pre-conditionar maybe? Clean any residual caches in the cache manager. When you write out your sim files add dot sc e.g. bgeo.sc (use the BLOSC compression) which should be the default if you're using compressed fluids workflow. Now you want to get rid of attributes you don't need before the ROP with an attribute delete node. You might want to delete primitive groups you don't need as well since it was packed data (depending on what type of sim it is). I just started to learn Houdini, and these things helped to reduce file size, and speed things up reading them back in. Not sure if this is what you're trying to do. Your system is enough to do a lot of sims. I have a similar system, with 6700K 4.6ghz and Titan X SC with 32 Gigs of DDR4 memory. I don't see any improvement using OPENCL for fluid sims. I understand you have to make the whole workflow opencl friendly, but I haven't don't that yet, and I haven't got to learning PYRO either, except for making little torches for lighting. Getting rip of attributes that I don`t need is a good idea. Thanx for the tip! Not sure If I want to set up Linux and try to learn that as well. Already struggling with Houdini Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.