Retiming Particle Sim

[Timm Dapper]

Hi there!

I have a few scenes here to which I need to add particle based smoke effects. The tricky part about these shots is that their timing is different for pretty much every shot from real time to pretty strong slow motion. What helped with shots that have constant timing was piping the particle sim through DOPs and using the Scale Time parameter. Works quite well. The problem are shots that have time ramps (sometimes several of them) already embedded in the object animation.

I tried animating the Scale Time paramater, but it seems I can't set keyframes. I can apply expressions, but they don't seem to be evaluated every time.

I also thought I could go through CHOPS and use the Warp Node. My first step was to set up the Warp so that I get a smooth Motion Curve instead of the bumpy curve that has the time warp in it. I was planning to make the simulation with a constant Time Scale in that scene, then apply the inverse of the Warp to the particle simulation that I get out to transform it back into the timing that I started with. Any there goes the problem: How would I go about getting the exact inverted warp? Or alternatively. is there another way to do this that I haven't considered?

I know one could argue about having time warps in the original motion curves, but for now this is a situation I have to deal with for now :-(

Thanks in advance!

Timm

[dbukovec]

hi!

write out the particle points with ROP output driver, read them back with a file SOP, then use the TimeWarp SOP to retime it, also before that use a Time Blend SOP to produce geomtry in between frames.

[sanostol]

another way would be to load them into chops and do it there.

[Allegro]

you'll likely need to find a way to do it with chops. If you have changing point counts, the time warp/blend sops will not work.

Edited July 28, 2009 by Allegro

[Andy Nicholas]

Funny you should ask this, I've literally just done a project that had a huge amount of retiming particles on 7 or 8 shots each with different time ramps. Personally, I'd rather retime the input geometry to work at a consistent frame rate, i.e. the highest one. On this project, all the cameras, characters, particles and FX were done at 100fps and retimed using a curve that we used across the entire production (not just Houdini, it was used in XSI, Maya, and even Massive) before rendering.

If you try and sim with a changing time step; a ) I'm not sure if it works properly - buggy at best, and b ) the behaviour of your simulation won't be consistent (to be fair, it's probably not that noticable, but in certain situations it might make a big difference, especially collisions).

As far as the retiming goes, I avoided CHOPs (as I could never get that method to work properly) and ended up making my own time blend that uses particle IDs to identify which particles to blend between (hint: ask a programmer how they'd do it, and recreate that solution in geometry using an attribute transfer SOP). It's pretty fast and handles particles being created and destroyed. I've used it on a couple of jobs now.

Andy

Edited July 29, 2009 by Andy Nicholas

[Allegro]

ended up making my own time blend that uses particle IDs to identify which particles to blend between.

curse you, now I want to do this and can't seem to figure it out.

edit: Oh, hey! Look at that, didn't even notice that they added the Match by Attribute option on the point sop!

edit again: Wow... it's not even new... how did I never notice this?!

Edited July 29, 2009 by Allegro

[Allegro]

So I unlocked the timeblend and replaced the blendshape node with a point node

Match By Attribute: check

Attribute to Match: id

Position:

$TX + (($TX2-$TX)*($FF - floor($FF)))

$TY + (($TY2-$TY)*($FF - floor($FF)))

$TZ + (($TZ2-$TZ)*($FF - floor($FF)))

Is there a better way to do this? vops perhaps? Since the import attribute vop only runs on ptnum, I'm not sure how to iterate through based on id.

[mrice]

here's a stab at doing this with python...

I'd be really interested to know how the speed compares to Andy's attribute transfer solution

id_retime.hipnc

Edited July 31, 2009 by mrice

[Andy Nicholas]

here's a stab at doing this with python...

I'd be really interested to know how the speed compares to Andy's attribute transfer solution

Thanks a lot for sharing that, I just downloaded it to take a look.

It's great. What you've done is a very elegant way of doing it. Frankly I was surprised that it would work, just because I've always avoided doing anything like trying to pull inputs at different times. I was especially surprised to see it working with live particle system. Nice!

I did have a couple of odd things happening when I tried File SOPs to load the data instead of doing it live. The current frame slider kept jumping around and I didn't quite get the result I expected. It might be a bug on the OSX version though, I'm not sure.

The method I'm using is implemented in a very different way. Your method has the advantage of being able to do it live, whereas I usually have to rely on pulling data from files on adjoining frames. I've also tried using my version with the Time Shift SOP, but that'll only work on a live particle system if I use a Cache SOP (which effectively doesn't make it live anymore!).

[...a few minutes later...]

So I've just run a test. To keep it fair for both systems and to ensure we're comparing like with like, I piped the particle system into a Cache SOP so that the interrogation time for the input geometry to our retime systems is exactly the same. I had to use two TimeShift SOPs for my system so that I can send it the correct data (instead of using my usual method of two File SOPs to pull the data).

With around 11500 particles I get a frame time of 0.35 seconds with my system, and with yours it takes 2.32 seconds. I must admit, I was quite surprised as I'd been considering rewriting mine in Python to see if I could speed it up. One reason i can think of for the performance difference is that I'm using VEX for some of the processing. From what I understand, that's going to be more thread efficient than Python.

[Andy Nicholas]

Is there a better way to do this? vops perhaps? Since the import attribute vop only runs on ptnum, I'm not sure how to iterate through based on id.

Yes, plus you also need to handle the situation where particles are born or killed. It's a bit ambiguous with the Point SOP as to what it'll do.

You've also hit the nail on the head with respect to the fact that VOPs only works with importing attributes based on ID. The key is to generate an attribute to tell VOPs which point corresponds to which ID attribute. That way it's just a simple double lookup in VOPs to get the position to interpolate with. It also lets you trap for the case where there isn't a corresponding point, in that situation you can add it to a group for it to be deleted, as generally you don't want it hanging around in a fixed position (which is the alternative way to handle it).

Generating the lookup attribute is what I do in the Attribute Transfer SOP.

There... I've given it away now :-)

Let me know how you get on.

Cheers

Andy

[mrice]

With around 11500 particles I get a frame time of 0.35 seconds with my system, and with yours it takes 2.32 seconds. I must admit, I was quite surprised as I'd been considering rewriting mine in Python to see if I could speed it up. One reason i can think of for the performance difference is that I'm using VEX for some of the processing. From what I understand, that's going to be more thread efficient than Python.

Thanks for the test!

Yeah I'm not surprised that python is way slower. Reading from disk might speed it up some, I'll give that a shot.

[Allegro]

I'm still doing it with the point sop, but I'm also doing some attribute transfer followed by a delete sop which seems to correctly handle the creation/death of particles throughout the sim. It doesn't seem like it would be especially accurate though if some particle were moving slow and others fast as attribtransfer relies on a distance threshold...

[ranxerox]

thanks a lot for this ! I have been using houdini to retime + massage particles from realflow and this sop is extremely useful. I would be interested to know how the other solutions mentioned here were done (I have struggling to implement them). In any case, I made a slight modification to the sop, so that I could animate the time parameter. I've included my setup and changes. I'm not sure the setup is entirely correct, but the results look good with my shot.

thanks again !

-ranxx

here's a stab at doing this with python...

I'd be really interested to know how the speed compares to Andy's attribute transfer solution

od_particle_retime.hip

[mrice]

thanks a lot for this ! I have been using houdini to retime + massage particles from realflow and this sop is extremely useful. I would be interested to know how the other solutions mentioned here were done (I have struggling to implement them). In any case, I made a slight modification to the sop, so that I could animate the time parameter. I've included my setup and changes. I'm not sure the setup is entirely correct, but the results look good with my shot.

thanks again !

-ranxx

Hey ranxx,

Glad to hear its working for you!

I tried looking at your hip but I dont think the asset is embedded in the scene. As is, you can animate time by keying the 'outrangey' parameter. Its much more intuitive (for me at least) to retime by rate though, like how the warp chop works. I'll try to take this asset further in the next couple of days.

[ranxerox]

oops sorry ! I've embedded the otl and attached the new file.

-ranxx

Hey ranxx,

Glad to hear its working for you!

I tried looking at your hip but I dont think the asset is embedded in the scene. As is, you can animate time by keying the 'outrangey' parameter. Its much more intuitive (for me at least) to retime by rate though, like how the warp chop works. I'll try to take this asset further in the next couple of days.

od_particle_retime.hip

[mrice]

ranxx, I think its the python sop which isnt embedded. But this version should be friendlier to animate.

Another straightforward and stable way to do this without python is by loading the particles as a pointcloud, then you can search for a matching id. In a simple case this was about 10x faster than the python sop for me, but when I got up to 10,000+ particles the overhead of reading in larger files and traversing a larger tree began to slow it down substantially and this python sop was slightly faster.

id_retime_v2.hipnc

[bhaveshpandey]

hey Michael!

just wanted to say thanks..your asset helped me a lot in my simulation..

I hope to be able to do this sometime

[inoue]

this attach file is easiest way I think.

birth time id is always new number, so Its only concerned about kill time.

then I replaced floor with ceil .

It seems to be working.

timeblend.hip

[riviera]

I know I'm spreading the shameless advertising of our asset library qLib all over the place, but it happens that we worked at the issue of particle retiming to quite some extent.

I'm at work right now where we have limited facebook access, so I can only post the github address: http://qlab.github.io/qLib/ -- but if you check our our page on facebook (especially the "photos" section), you'll find screenshots with related explanations about how to retime particles using qLib tools (there are various example files in the distribution, too).

(For instance, check this video:

. This is an animated boolean, 20 frames long, slowed down with a 10x factor or so. It's an actual example scene that covers various ways of proper sub-frame emission, death, subframe-accurate age attributes and attribute mapping.)

cheers )

[riviera]

I can even provide some explanations on the various aspects of particle retiming if anyone's interested. (Although hopefully everything's explained in the qLib example scene -- look for the "Timeblend qL" one)