fractured pieces bullet sim memory issues

[Dispel]

I have a pre-fractured bullet sim that is pretty simple in every aspect other than having a lot of pieces. No constraints, only gravity, just some static collider geometry and the fractured pieces I'm trying to sim breaking apart.

I'm using bullet because RBD is too slow for my needs on this.

When my sim gets to the frame that my object activates, the dop network spends 5-10 minutes on the "Add Data" step while houdini starts to use up more and more RAM.. For about 6k pieces, it will consume 32+ GB of RAM which is the limit for my system. I am trying to achieve more than that, ideally I would wish for 100k but I'm not familiar with my limitations in this area.

Any way I can optimize a sim like this so that it doesn't use up as much RAM? I have caching turned off by the way, I'm just outputting a bgeo sequence.

[goldleaf]

Something is definitely sounds fishy. I've run 50,000 constrained fracture pieces, and the maximum memory useage I recorded was ~10GB Ram during the ~7.5 hour sim (there were 65,898 constraints generated).

A couple of thoughts, since I don't have the file in front of me:

- Make sure in the Collisions tab, under volumes, that you uncheck the creat volume option (it shouldn't generate it, but just in case...)

- A workflow I tried, since RBD Point Objects are so flexible, is to use that rather than RBD Fracture. During an internship, I did this, and documented it here: http://cartojellybea...ure-points.html. That also makes it much, much lighter to store the animated points on disk, and either instance the pieces at rendertime, or copy them onto their appropriate piece.

- The RBD Fracture Object is a digital asset, so if you use the Profile Monitor, you might be able to find which DOP inside of it is the culprit, and then figure out if it's something you can delete

My experience was on a Linux 64 machine, I'm not sure what your specs are, but perhaps that's the issue?

Edited January 15, 2013 by goldleaf

[lukeiamyourfather]

How many points and primitives are there when the memory usage is high?

[Dispel]

Cool suggestions goldleaf.

I will uncheck that for all objects and see if it helps...

I haven't looked at point objects, I will try to look at it shortly, but my understanding was that this basically just makes them behave for collisions as if they are spheres with a certain radius? Is that not the case? (I will read up on your info shortly)

Good idea with the RBD Fracture Object, though I don't understand how to use the profile monitor correctly yet. I will investigate this soon also. (After I try the first thing)

The highest I managed to achieve so far, with 32 GB of RAM and 32GB of virtual memory was 10k objects with 370k polys.

[Dispel]

Seems like all the memory gets taken up at my dopimport node from my pre-fractured objects sop network. Basically it is just a fractured object, the only extra stuff needed for my sim that is included on the objects is name attribute, and the pieces are in groups.

The dopimport settings are:

Use Single Object (Off)

Import style: Tranform Input Geometry

Import by Name (Off)

Inverse Transform (Off)

Transform Geometry with Position Data (Off)

Transform Geometry with Geometry Data (Off)

Preserve World Space Positions (On)

Add Dop Object Name Attribute (On)

Add Dop Object Id Attribute (Off)

Add to Existing Velocity Values (Off)

Delete Abandoned Primitives (On)

Point Velocities: Instantaneous Point Velocities

Vector Attrs to Transform: N

Any ideas? I think I can set velocities to No Point Velocities, not sure what else will make a difference.

Edited January 15, 2013 by Dispel

[Dispel]

I have tried toggling as many different settings as I can on the dopimport node to only very minor effect on memory usage. The object coming into the dopimport says, as I am testing on it, a 16GB memory usage on dopimport, with the following stats:

92473 points

123,231 prims

369693 verts

123231 polys

3623 prim groups (indicating I have 3623 pieces)

2 point attrs: P, N

1 vert attr: uv

1 prim attr: name

1 detail attr: varmap

[Dispel]

I can't seem to delete or manipulate any of the node connections within the RBD Fractured Object DOP, I'm just getting permission denied errors. Why is this?

[Dispel]

Nevermind, I figured it out. "Allow Editing of Contents" right click menu. Ha.

Okay, so I optimized the Fracture Object a little bit (having it ignore RBD stuff, and not applying unneeded collision data) but so far it has only affected the sim speed, not the memory footprint.

[goldleaf]

I just made a sphere, fractured it to 6000 pieces, and created an rbd fracture object out of it, and my ram is very low, even while simming. It might be something related to whatever you're fracturing? Or getting unneeded geometry from the fracture?

Also, something else I thought of, turn off record impacts (that too might just help with sim times, but if they're intersecting at creation/activation, that's an impact).

Can you post the file? I can take a look at it as soon as I can...

Edited January 16, 2013 by goldleaf

[Dispel]

Is that "Impact Data"?

I already had that off, unless you mean something else.

I could post the file but it's about 80MB because of required-to-be-included collider cached geometry. I will see about it this evening. I'd really like to figure this out.

[Dispel]

Sorry, I have been slow to post this because of a different job that came up, while the job involving this scene already passed deadline, so I had to make do with what I had. But I would still like to know how to do better. I have uploaded it. 63MB. (Sorry for the size, but again, requires some animated collider geo, and has cached version of the objects.)

One thing to note with it, is that in my "GROUND" geo (the important piece of geo being simulated, everything else is some form of passive collider geometry) for this upload I have set all my scatter sops to 200 points for lightness so other people can sim it, but the highest i have managed to sim with it is setting them all to 1500 points. Use the downstream ROP to update it in case you want to try with more points. Because voronoi takes up some RAM, I update the geometry, then I save and quit and reopen so that I am starting the sim with a clean slate memory wise.

Edited December 1, 2013 by Dispel

[goldleaf]

I had a few minutes to look at it, and I had some success. The attached file (just a replacement for the hip scene you sent) has the ground object as a simple object, rather than a fracture object. The fracture object as a lot of overhead since it makes a new object for each piece, and since the pieces aren't moving anywhere, it's much more efficient to represent it as a single object (in this file, I made it an rbd object with a mass of 0, which I explain below).

This may or may not help, but before I did anything to the scene it took 30 and 4.25GB Ram to cook the scene from frame 129-130, and this file now took 25 seconds, but only used 2.9GB memory. I realize now that your ground was created on frame 140, and my rbd object version came in on fr. 1, so maybe this was all bogus But unless the static stuff separates and has holes all over, you might be able to get away with a single rbd object like the attached file has.

Here are some other observations:

• The Merge DOP for the static/dynamics objects was set to mutual, which may or may not have a peformance hit, but since the static aren't at all affected by the dynamic sim, might as well keep it at Left Affects Right
  
• When I used the performance monitor I was seeing almost all of the time being Unaccounted, which might mean that Bullet is the memory culprit (maybe - and I think it might be the tree)
  
• The Bullet Library doesn't like objects which are really small (smaller than ~0.2 units can get unstable), so since the tree is scaling from nothing up to full size, that could be using up memory and/or causing slowdowns. Maybe start it with some size, then start the translate/scale?
  
• Since the tree point count doesn't seem change, perhaps using render out an MDD cache, and use that to deform a static model (may or may not be of help, since the DOP object is still deforming, but it's definitely more efficient/faster than reading a per-frame bgeo sequence)
  
• Bullet sees objects with a Mass of 0 as static, so you can leverage that to get static objects
  
• If you can get away with it, the static stuff could be represented by cubes, as those are far faster/lighter for bullet than convex shapes, and meld it together into a compound object, or use rbd point object with mass of 0
  

Hope that helps!

Glad you were at least able to get the job done for the client!

groundBreak_ideas.hipnc