simulating lots of pieces

[jim c]

I've got a glass shatter simulation that I'm trying to run and I'm having problems. When I run it with around 2-3K objects, it runs OK. When I crank it up to 20K objects the simulation, or rather Houdini crashes on the 2nd frame .

I'm using the ODE solver, and lots of small rectangular cubes. Any idea what to look for? Task Manager shows that I've still got at least 1 gig of physical RAM left.

Also I'm using H10, Apprentice HD. Is it possible to save out the sim with the Apprentice HD version? I thought I read somewhere that this was not possible?

Edited January 27, 2012 by jim c

[LaidlawFX]

You can save a geometry sequence out of Apprentice HD.

Simple optimization would be: To upgrade to h11.1 if you are using Apprentice since it's free. There has been a lot of updates since h10 came out. Also check how much your virtual memory is pinging, sometimes that will max out prior to physical memory. If going from 2k to 20k kills its try some inbetweens, 4k, 8k, 12k, 16k, see if you can get away with less and when your scene becomes useless to render. Split up your sim if you find you need that many and can't do it in one go. Calculate one half first, then sim the second half on top of it, or other cheating tricks. Also simplify your geometry as much as possible and swap out the low poly stuff for higher resolution afterwards. There are more optimizing tricks, but I would try the above first.

[jim c]

OK I'll try that. I do need to reduce the size no matter what since the geo files that it's saving out are huge (over 1 gig per frame). I'll try H11.1 this weekend and report back how that goes.

[jim c]

I've been trying it under H11. So far it crashes at 10K pieces. I'm going to try and post the file here and see if anyone can get it to work, maybe it's something about my machine. I have a hard time understanding how H can't handle this.

[Erik_JE]

Another tip is to make sure you are deleting unnecessary groups and attributes before writing out geo. It will save you a lot of diskspace.

[jim c]

OK I got a bit further with this. Anything over 10K pieces (where each piece is just a box - 6 polygons) causes Houdini to crash. But I fiddled around and found some smaller numbers that I guess will work. I'm currently rendering in a smaller area, at about 8900 objects.

So I went to run a simulation of 300 frames, after 7 hours it had rendered 100 frames, and had filled up the C drive (the drive where Houdini was installed). The temp folder in my user account was completely filled up with some sort of temp sim files, each around 400Mb. Is this normal? Is there a way to have these stored elsewhere? Is there a reason it keeps filling up?

More importantly is there a way to render the next 200 frames? What I'm concerned about is that if I put the frame number at 100 and hit play, then it will have to re-sim the previous 100 frames and blow up or take forever just to get back where it left off.

Edited February 1, 2012 by jim c

[jim c]

A couple of interesting notes:

Trying Houdini 11.1.67 made no performance difference, at least none that I could tell. So I went back to H10.

I got frustrated with H after a while, so I thought I would try Maya, since I had heard that it had a fast solver. Following a tutorial (http://area.autodesk.com/tutorials/destroying_a_wall_using_rigid_bodies_and_particles) at the Autodesk site I was able to make a simple test work. So I exported the base shattered glass objects (around 3K objects) to an obj file. Loaded up in Maya just fine. Added the Active Rigid Body and Field. First off Maya took FOREVER just trying to set up the rigid body objects. But that finally finished. Then I tried to run the simulation. Ha! Absolutely unworkable. Houdini RBD objects are extremely slow and for my case completely unsuitable to use. But if you set things up correctly and use the ODE solver then you can get some work done. In comparison I gave up waiting for Maya to ever get to the 3rd frame! Simulating 8900 objects, Houdini would solve around 1 frame every 2-3 minutes. Maya wasn't even close to this. So I'm wondering is Maya really that horrible or did I do something wrong? I then tried the Bullet Plugin for Maya (http://bulletphysics.org/mediawiki-1.5.8/index.php/Maya_Dynamica_Plugin). I loaded up my 3K object file and tried assigning active objects to it! It was taking 1-2 second per OBJECT just to add the nodes to the solver or whatever the script was actually doing. I gave up on that.

Edited February 1, 2012 by jim c

[Erik_JE]

Maya with over ~5000 objects in the scene becomes super slow. The Dynamica/Bullet solver is pretty good tho and I think you can get it to work if you just let it chug through the initial node setup.

On the other hand you could also try the bullet solver for Houdini someone at the SESI forums wrote until H12 arrives where it should be built in.

[jim c]

Well I think I'll stick with what I have for the moment. My only problem now is getting the remaining frames done. I have frame 1 - 100 simulated and stored to disk as per frame bgeo objects. But I'm not sure how to get this thing to do the next 200 frames.

This is what it looks like so far (somewhere around frame 5 or so)

Edited February 2, 2012 by jim c

[edward]

Is there a way to have these stored elsewhere? Is there a reason it keeps filling up?

Yes, set the HOUDINI_TEMP_DIR environment variable prior to starting Houdini.

[lukeiamyourfather]

Yes, set the HOUDINI_TEMP_DIR environment variable prior to starting Houdini.

Or disable caching on the DOP network all together and it won't put temporary files anywhere. If you're still having problems with crashing, are you using 32-bit or 64-bit Houdini?

[LaidlawFX]

Another thing to check with your RBD is how you are importing your objects, there are RBD Point, RBD Fracture, RBD Glue, RBD Static, etc... so you might want to try different methods, especially the point loader. There are some example files on that one I need to play with myself.

Also for RBD make sure the implicit volume used for calculation is as simple as possible. By default if it is by volume it does a ray trace of a 30x30x30 box, play with these settings to get the lowest threshold to preserve the shape. You can even create proxy shapes to make the solve quicker.

Lower the number of refinement calculations in your solver as possible. The less it has to calculate while maintaining the general sim, the better.

If you can wait for bullet in h12 this will help you out a lot, even RBD in h12 is a lot better. Bullet is designed for a sim like you need. Simple objects and massive amounts of them.

Also can't think of a single case why an h10 dop sim would be better than a h11 version. There seems like there may be another factor in play.

You can try 3Ds Max too, if you need to just get the job done. I'm willing to bet there are some better plugin for dynamics than there are for Maya, just based on the user base.

Also have you tried to just do a particle sim and instance boxes onto them? That would certainly get you your high threshold if you can control the particles right.

[jim c]

Thanks!RBD just didn't cut it for what I needed. The ODE solver is what did the trick, plus the RBD Point object. RBD Fracture won't work since it can't use ODE. The other issue could be my machine. I on;ly have 4 Gb of ram at the moment (though hopefully that's going to change soon with a jump to 12-16Gb more ram).

I did try the particle sim solution. Easier to set up but the problem was the starting frames - when I instanced the boxes to them they would spin in place along the Y axis for 8 or 9 frames before straightening out and then flowing along the lines of the explosion. The DOPs simulation simply looks better in this case. I can live with what I have for now (at this point I just want to get the damn scene done and rendered so I can have a friggin video to show for it!!!).

[LaidlawFX]

Hope it comes out good enough for the job.

An upvector probably would have solved the spinning boxes for the particle sim.

Also ram was def your killer. Dynamics in general are just ram heavy, unless you a gpu based engine, which Houdini does not have. 12 gigs would be the lowest to consider using dynamics, production level at many facility is 24gigs, or higher for dynamic specific people.

[Owl]

....I got frustrated with H after a while, so I thought I would try Maya, ...

a bit late but - while i would've chose houdini, in maya you don't make 5k separate objects you should import obj as single object and made ncloth/ bulet object out of it.

for maya 1 shape = 1 houdini sop node

so 5k objects = 5k geo/sop nodes