redshift Redshift Instancing Issues (RBD-Sim Piece Detailing)

[m-egger]

Hello,

I recently went through Steven Knipping's tutorial on Rigid Bodies (Volume 2). And now I have a nice scene with a building blowing up. The simulation runs off low-res geometry and then Knipping uses instancing to replace these parts with high-res geometry that has detailing in the break-areas. The low-res geometry has about 500K points as packed disk primitives so it runs pretty fast in the editor. Also, it gets ported to Redshift in a reasonable amount of time (~10 seconds). However, when I switch to the high-res geometry, as you might guess, the 10 seconds turn into around 4 minutes, with a crash rate of about 30% due to memory. When I unpacked it I think it was 40M points, which I can understand are slow to write/read in the Houdini-Redshift bridge, but is there no way to make Redshift use the instancing and packed disk primitives?

My theory kind of is that RS unpacks all of that and that's why it takes forever, because when I unpack it beforehand, it works somewhat faster - at least for the low-res. The high-res just crashes.

I probably don't understand how Redshift works, and have wrong expectations. It would be nice if someone could give me an explanation. Attached you'll find an archive with the scene with one frame (Frame 18) of the simulation included as well as the folder for the saved instances of low- and high-res geometry. 

Thanks a lot for your help,

Martin

(Here's the pic of the low-res geometry, the high-res is basically every piece bricked down into ~80 times more polygons.)

Martin_RBDdestr_RSproblem_archive.zip

Edited December 27, 2017 by tortoise
Rename Topic to fit.

[Atom]

The unpacking is probably the problem, that is why you may want to re-configure your scene to use Redshift proxy files instead. Then you can leverage the @instancefile attribute. When all your geometry is on the disk, and not in the scene, Redshift does not have to go through the Houdini single-core API to fetch and extract things. This gives better performance on export.

I put together an overview on how to setup this kind of system in my latest video here.

[Atom]

After looking over your file I think you can just bypass your instance nodes. Instead of setting your string attribute s@instance_hi set s@instancefile and point it to the .rs proxy files you generate with the technique I mention in the video.

[m-egger]

Oh wow, that's the perfect video for my issue! Thanks so much. 

I haven't gotten to it yet, but it looks like it has to work.

I raise my hat to you, Atom!

[m-egger]

Alright, so I tried out your approach and it seems to almost work. The thing that's missing now is the transformation of the original pieces. It looks like this now: 

Now I wonder if redshift stores the transformation values/matrices for the instances in a different manner as houdini? 

That's the code I used from Steven Knipping's original setup to transform the Pivot and Transform from the original points to the instances.

Any more ideas? 

 

Edited December 23, 2017 by tortoise

[Atom]

When you created all the .rs files, were the source pieces at world origin? I think they need to be at world origin so when they are copied to points at a later time they will be in the correct location.

I think the .rs export writes the current location of the vertices in world space. I don't think it embeds a transform.

 

Edited December 23, 2017 by Atom

[m-egger]

Baby steps. I tinkered with the instances a lot and fingured out that the instancing if applied to the packed sim data projects an instance to every unpacked point. I fixed that by copying a point to every instance (with keeping the orient on the copy-to-node) and transferring the needed attributes. 

My next issue that you identified very precisely Atom was that the saved instances are saved at the center. So I had to transform the instances' center to the origin (when saving out the proxies) for the instances to appear where they were intended.

And my last and not yet solved issue is that the rotation is not applied entirely to the instances. It's weird, but I feel like I'm getting there.

I can post some pics and the hip tomorrow.

At this point I think I'm just looking for a way to get the intrinsic rotation of the sim's pieces to the points that I clone the instances onto. Any suggestions I'd be very happy about.

[Heileif]

How did the instancing go?

[m-egger]

To be honest, I gave up on it as I didn't feel it actually paid off to spend this time setting up. Probably because porting to the live-view still took about a minute, even with the instances.

I was pretty close though. I think all that was missing was a deeper understanding of prim intrinsics. And the way the sim worked originally and how Redshift handles its instances (where the center is when saving and where it is projected to when re-loaded), because that is hardly documented, as far as I could find out.

(Also, I unfortunately lost the saves, so that cut my motivation to go on as well.)

[Heileif]

Aha, looks like i have to take up the torch when there is time then!

Would be nice to get this working, will upload i HIP file if have some success. With the information in this thread i will get a good start at least 

[wode]

I found the solution.

1) Create a number of points equal to the number of total packed prims you have in your sim (not the number of unique pieces. for my building it was around 18,000.) I used the points generate sop for this. Each point corresponds to a packed prim. Drop down a wrangle and plug your points into the first input. Attach your simulated packed prims to the second input. For each point, you will create 3 new attributes: v@pivot, 3@transform, and s@name. The pivot and the transform are primintrinsics and you can copy them from the current packed prim attached to the second input (current meaning the one indexed at @ptnum). Also copy the packed prims positions and piece string attribute (@name_orig in the tutorial) onto your point. You can use the @name to create a new attribute called s@instancefile that points to wherever that particular piece's .rs proxy file is located on disk (this of course could have been done in one step but I like to break it up). Now you have all the attributes on your points that Redshift needs to find and instance your proxies.

2) Make sure to add the RS obj parameters to whichever object contains your new instancing points, and BE SURE to untick the box under instancing that says 'Ignore Pivot Point Attribute'. Anddd your done!

 

Edited June 3, 2018 by wode

[m-egger]

Wow. You are a genious for figuring this out - thank you very much! I definitely have to try this out soon.

[wode]

I reworked my solution upon realizing that the ignore pivot attribute tickbox was still there. Much easier now, but let me know if for whatever reason it doesn't work for you. Kind of came at the problem in a very nonlinear fashion, so I may have missed some steps in trying to draft up a linear solution.

Edited June 3, 2018 by wode

[Activate]

This works great. Thank you! Starting with "fresh" points really did the trick :-)

[Activate]

BTW, does anyone know how to texture a setup such as this one where the same pieces (from the wall) are re-used to create the whole building? Perhaps that is not a very common set up.

[tricecold]

One thing I am not sure here, this is not true instancing as every piece is a unique object and can be loaded only once, so there isn't any memory advantage here

[karan2898]

I tried generating new points and then instancing to that points but still its not working properly does anyone got success if so then please upload screenshot