best way to reduce number of points in a FLIP post simulation?

[Memo]

like the title says, whats the best way to randomly reduce the number of points in a FLIP system POST simulation? (losing the same particles every frame obviously, not different ones each frame).

I.e. I have about 10M points simulating, afterwards I want to reduce this down, not sure how much to, probably around 1K-1M, need to have a play.

 

I tried creating an attribute called rndval = rand($ID), and then delete where $RNDVAL<ch("delete_amount") (HSCRIPT). It works alright, but is super slow. I found doing a group and blasting by group is slighty faster, but still painfully slow.

 

I was doing this after the simulation (i.e. import the DOPFields into a geo and do it down the chain there).

 

In my sim I've turned off reseeding, so I've now tried doing the grouping before the sim (my input points are static, just a standard box flip field). There is no ID defined at that point, so I'm grouping on rand($PT) < ch("delete_amount"). THis is now done once at the start of the sim, which is quite slow, and then while the sim is running it's not evaluated so all is good. The cost of blasting alone is quite negligible. This works ok now, but the downside is I can't tweak my theshold post-sim! I.e. the cached data includes the thresholded group! WHat I'd really like to be able to do is cache the sim at full res, and then reduce it afterwards. Is there anyway of doing this that isn't super slow?

[eetu]

With that setup you could add a rand($PT) attribute to the points pre-sim, and then do the thresholding and deleting afterwards.

 

It would probably be fastest if you do thresholding and adding to a group inside a VOP SOP (Add Point to Group VOP) and then blast the group. Adding a random absed on ID into that wouldn't probably slow it down measurably, so no need to for the pre-sim modifications in that case.

Vex/VOPs are crazy fast compared to expressions in these sorts of things.

[Memo]

 

Wow that's amazing! I had no idea VOP SOP were THAT Much faster! Below are my results for 3 methods for reading a 367,161 point particle system from sequence of BGEOs for 50 frames.

 

1.

file read - 4.574s

attribcreate rand($ID) - 11.636s

delete by expression $RNDVAL<ch("delete_amount") - 34.245s

Total (excluding file io) - 45.881s

 

2.

file read - 4.613s

attribcreate rand($ID) - 11.473s

group by expression $RNDVAL<ch("delete_amount") - 21.887s

blast group - 1.003

Total (excluding file io) - 34.363

 

3.

file read - 4.564

VOPSOP: compare rand(ID) < promoted parameter ch("delete_amount"); if so add to group - 0.106

blast group - 0.941

Total (excluding file io) - 1.047

 

Madness!!

 

interestingly, the VOP random documentation says the return value is 0...1, but I'm pretty sure it's signed. WHen I tried 'if rand(id)<0.5' I got 82K points (out of 367K, exactly a quarter!). However when I mapped it from -1...1 to 0...1 I got 182,849 points (182,849 / 367,161 = 0.498, which is pretty good. Also identical point count to what delete & group return).

http://www.sidefx.com/docs/houdini13.0/nodes/vop/random

 

amazing thanks. I'll try and never use expressions again!

Edited May 26, 2014 by Memo

[mawi]

Try a wrangle node with...

if(rand(i@id)<threashold)

removepoint(0,@ptnum);

[ranxerox]

interestingly, the VOP random documentation says the return value is 0...1, but I'm pretty sure it's signed. WHen I tried 'if rand(id)<0.5' I got 82K points (out of 367K, exactly a quarter!). However when I mapped it from -1...1 to 0...1 I got 182,849 points (182,849 / 367,161 = 0.498, which is pretty good. Also identical point count to what delete & group return).

http://www.sidefx.com/docs/houdini13.0/nodes/vop/random

 

amazing thanks. I'll try and never use expressions again!

 

 

 if you do a quick check on the output of the random vop you can see that it's not signed (just wire it into an add attribute vop).  There must be a problem somewhere else in your vop sop.

 

 -G

[Memo]

@mawi,

I tried that but doesn't work. Just doesn't remove anything. I even tried just removepoint(0, 'ptnum) or removepoint(0, 5) and nothing dissappears. 

These docs say it should work in any context. I tried a point wrangle in a normal SOP network. 

 

@ranxerox,

yes you're right. I think I was comparing against a rndval attribute which already existed on my data.

[JuriBryan]

removepoint and all other actual geometry manipulations work only in some vops.
use them in a attribute vop and it will work with out a problem, and they are very very fast.
(check the docs, it says which vops work with those nodes)

[mawi]

@mawi,

I tried that but doesn't work. Just doesn't remove anything. I even tried just removepoint(0, 'ptnum) or removepoint(0, 5) and nothing dissappears. 

These docs say it should work in any context. I tried a point wrangle in a normal SOP network. 

 

Nope.

It runs in CVEX, you will need a attribute wrangle SOP.

[JuriBryan]

in the latest builds it works in basically vop context besides the old vopsop.
 

[Memo]

Thanks guys, yea that worked. Sorry I am trying to read as much documentation as I can, but since it's such a goliath piece of software sometimes I don't know what to look for. 

 

Interestingly I found the VEX (attribwrangle) to be a lot slower than the VOP. I wondered if it's the pointremoving from within the VEX that's slow, so I tried grouping first in VEX and then Blasting, and that is

indeed why it is. Add/removegroup from a pointwrangle followed by a blast is a lot (3x) faster than a removepoint from an attribwrangle. Interestingly wrangleattrib/setgroup is an order of magnitude slower!

Also I wondered if passing in external parameters via ch() was slow, turns out that's negligible.

 

My results are below. I ran them a few times and always within a few % of this.

 

P.S. this is for a 1M point grid, running over 100 frames, modulating the threshold sinusoidally and running 5 simultaneous branches so they all get the same input. 

 

 

EDIT:

I initially uploaded an incorrect version which didn't have the pointwrangle add/removegroup. Correct version attached. 

point_delete_tests.00008.hipnc

Edited May 28, 2014 by Memo

[Skybar]

Seems like CVEX isn't quite as well multithreaded? Also it is quite pointless to compare VOPSOP and Point Wrangle, since they're essentially the same and will have very similar results.

[ikarus]

I would think it's a threading issue, you can't have multiple threads access the geometry data set and delete elements simultaneously.  I imagine removepoint calls enter a queue to be executed to avoid memory errors.