Fluid Sim, Higher Resolution?

[Atom]

Ryan: Thanks for the detailed explanation. I still don't even know where to click to view how many particles are in my simulation. Are particles just FLIP points? What node do you middle click on to get that information?

The tip for the main body of water being a non-emitter flip object and turning on ReSeeding has helped a lot. At the start of this process I basically turned off all features of FLIP because each feature incrementally slows down the simulation process. I did not realize reseeding would actually maintain an approximate particle count, that is good to know. I also altered the Activation on my fluid feed tubes. For the first 48 frames they dump fluid into the system continuously, then they pulse emit every 36 frames. if($F < 48,1,$F %36). This helps reduce the new particles per-frame count.

I am trying the compressed cache approach, it does use less memory. At a particle separation of 0.03 my system only reports using 10GB of system RAM. This seems deceptive to me, I thought well if I have more ram lets lower the particle separation. I dropped it to 0.02 and tried exporting the compressed cache and Houdini crashed after 8 frames even though the system reported only using 12GB. So the "real" physical memory limit is there, but somehow masked by the compression technique.

Returning to particle separation 0.03 I managed to kick out 320 frames over night, which is the best result yet and it is still running. I may reach my 1,518 count quicker than 15 days.

One thing of note is that the ParticleFluidSurface node can not detect what resolution the compressed cache was rendered at. So it is important to manually match the particle separation on that node to the value of the actual particle separation from the original sim.

Edited January 27, 2017 by Atom

[Diego A Grimaldi]

hey Atom, to see how many particles you have you can either middle click your DOP IO and see the count or I always have a display on my viewport that tells me points and prims count and some more info depending on the selected node
You can display that by clicking D on the viewport, go to Guides, and under Additional Information where it says Geometry Information switch that from Off to Always On.

[rbowden]

I'm gonna assume you are talking to me (Ryan) when you say Rob 

Reseeding is supposed to maintain particle count but you should always be double checking your particle counts to be sure. Reducing your particle separation from .03 to .02 could be a pretty drastic change. Again, double check your particle counts to see how big of a change it is. Sometimes it could be something as big as a 10mil point change.

I'm not sure what you mean about the particle fluid surface not detecting the resolution? If you used to shelf to set this up (which you did in the file you posted earlier in this thread), the fluid compress particle separation and the particle fluid surface are both channel referenced to use whatever particle separation you use on flip object. If that isn't the case, then you probably broke that link when testing out some things.

As for viewing the amount of particles, @Diego A Grimaldi is correct that you can just middle click on your DOP IO node to do this. If you want to do it in the flip simulation, you can just middle mouse click on any of the nodes and it will tell you how many points you have in the flip object (I usually just do it on the flipfluidobject itself though). Be careful with Diegos suggestion of the viewport particle count display. If you have that up in your viewport, anytime you change frame, it is going to cook your simulation because it is going to try and read that frames particle count. So only keep it on if you are doing a flipbook or going frame by frame manually. 
 

[Atom]

Oops, sorry, I meant Ryan.

Quote

I'm not sure what you mean about the particle fluid surface not detecting the resolution? 

I moved my HIP file from one machine to another. My main machine is still cranking away with a particle separation of 0.03. However, the HIP file I copied to the other machine still has a particle separation of 0.08. I also copied the current compressed cache from one machine (while it is still calculating) to the other machine so I could preview the resolution so far.

It is true that the Particle Separation for the ParticleFluidSurface is connected via an expression to the particle separation of the FlipObject. But when I activate Load From Disk on the compressed cache it is effectively pulling in a 0.3 resolution for the ParticleFluidSurface. If you don't actually change this value you will see that your 0.3 cache will be surfaced with a 0.8 down-res resolution. The inverse also seems to be true I can surface my 0.3 cache at 0.2 for instance. I do realize this does not give any more resolution but the resulting surface does exhibit some up-res characteristics.

[schwungsau]

32GB is more then enough for this kind of sim's..... i had 200million particle with 32GB without problems. 
its about how to plan your simulation.  first things, i plan the  efficiency of the flip container. most setup i see, they huge fliptank where 80-90% is emty and not used  --> it still used full ram and calculation power --> 90% waste of Ram.

slicing up container and transfers the flip particles is faster, efficient and you pump in more resolution, that's one way to optimize simulations.

[Atom]

Good to know it is possible to squeeze more out of my aging system. My current setup does have a lot of empty space due to the waterfall distance to ground.

Do you or anyone know of a HIP file setup floating around that demonstrates this chopping up of FLIP and seamlessly recombining it on a single machine? I know if you have HQue system setup you can divvy up sections to other machines but I only have one indie license at this time and it is installed on my fastest machine with the most memory.

[Guest tar]

9 minutes ago, schwungsau said:

 most setup i see, they huge fliptank where 80-90% is emty and not used  --> it still used full ram and calculation power --> 90% waste of Ram.

 

Where is the wasted Ram happening? The particles individually contain all the info needed in FLIP.  

[schwungsau]

for flip particles you need a voxel grid for calculation, otherwise you just could regular particles.

 

[Guest tar]

31 minutes ago, schwungsau said:

for flip particles you need a voxel grid for calculation, otherwise you just could regular particles.

 

Has anyone tried to optimise this field. Setting the values to 16bit seems to increase the amount of Ram needed!

[nigelgardiner]

Hi Atom

Schwungsau is corrrect in saying you may have a needlessly large flip container. If you click onto the flip solver and go over to the scene view the bounds should be displayed and you can drag then to the needed size.

I haven't had chance to look at your hip file so you may have already done this.

Also scale is really important with flip in determining the particle separation you actually need, put a camera on it and work from there, you'll be surprised at how coarse a particle separation you can get away with.

Interesting movement and splashes etc can all be created by driving the flip with a noisy velocity field and what not which really helps with a natural look. Flip won't do this by itself.

Also reseeding may or may not be your friend.

Hope I'm not stating the obvious, apologies if so.

 

cheers

Nigel.

 

[Atom]

The decision to convert the main body from an emitter to just an initial volume of fluid results in the waterfall thinning out over time. I could increase the frequency of the pule on the feed tubes but I am wondering if there anyway to force reseeding to keep up the particle amount to a specific value/volume? Like a fluidic rest state perhaps..?

Edited January 28, 2017 by Atom

[stickman]

rbowden... thanks for this reply... very helpful!
I have an additional question (hope it's okay to add to this thread)... I like to manually correct sims in other software (specifically LW / modo). I see some sculpting options in houdini, but haven't discovered a workflow-friendly method (or tool such as chrono-sculpt) with which to sculpt particle sims, mesh sequences, etc.
I suspect I'm just searching under the wrong words... is there such a tool / workflow? Pointers appreciated.

[schwungsau]

3 hours ago, stickman said:

rbowden... thanks for this reply... very helpful!
I have an additional question (hope it's okay to add to this thread)... I like to manually correct sims in other software (specifically LW / modo). I see some sculpting options in houdini, but haven't discovered a workflow-friendly method (or tool such as chrono-sculpt) with which to sculpt particle sims, mesh sequences, etc.
I suspect I'm just searching under the wrong words... is there such a tool / workflow? Pointers appreciated.

There much better options then chrono-sculpt in Houdini. (basically, you could build a rig which works like chrono-sculpt). its my favorite workflow --> shoot particle in some direction and make cool in post processing --> artist friendly and fast.
but there is no one node-do-it all-tool.  Typically you start with sculpting / upresing particles before you start to mesh --> this make meshing fast and efficient and looks better then just drop down a fluid surface node. maybe, i do to tutorial on this.
you should look in pointclouds. pointclouds can calculate stuff are very fast, you can get normals and/or density on particles --> you will have attributes like a mesh --> now you can sculpt, displace or deform your particles like mesh in zbrush or build custom deform tools etc.

[Atom]

Here is the render test with the pulse on the feed tubes increased from every 36 frames to every 24 frames. The fluid seems to thin out a little slower. So I guess it is a balancing act to keep particle count as low as possible while not letting that count drop below a certain threshold.

I need to maintain the volume amount for 1,518 frames.

Edited January 28, 2017 by Atom

[Atom]

Here is the result after exporting the cache first, then surfacing as a second step. This does allow me to fit a higher resolution simulation on my 32GB machine. Thanks everyone for explaining the process to me.

Now if I can just get the dancing right I think the interaction with the fluid can work.

 

And a few more cache frames finish up...

Edited January 29, 2017 by Atom

[galagast]

Hey Atom,

What was your process when you converted the Main Body to an initial volume? Did you run a simulation just to fill up the area for the Main body?

1. If so, did you make it settle before picking a frame to use as an initial volume?
2. And, were you using the same Particle Separation value for the the initial volume and your current flowing sim?
3. Lastly, just throwing it here out of curiousity.. is there a pscale attribute on the Main Body? Is it the same as in you current sim's pscale?

I'm asking because looking at the volume loss.. sometimes it is caused by mismatching Particle Separation values.
Or probably.. the fluid simply is not not fully settled?

[haggi]

On 1/27/2017 at 7:21 PM, schwungsau said:

slicing up container and transfers the flip particles is faster, efficient and you pump in more resolution, that's one way to optimize simulations.

Could you explain this a bit further what you mean by transferring the flip particles? Do you mean to transfer them from one container to another if they hit the boundaries similiar to a distributed sim?

[Atom]

@Galagast:

I create the initial volume using a box cookied with the landscape. This produces a nice initial fit with little settling. I lowered edge length to get a better fit, less blobby. I left the division size of the emitter to the default of 0.1, my sim is at 0.03. thanks for that tip. I have linked these values in the past. I am not adjusting pscale at all although I thought about trying out a previous suggestion of lowering pscale as particles fall to help reduce spreading.

I think my loss of volume is the result of fluid falling over the side and being removed from the simulation once it reaches the border of the bounding area. For now, increasing Particle Scale Radius has helped out quite a bit. It keeps the perceived volume up even if particles are being removed from the system over time.

[schwungsau]

On 1/30/2017 at 3:46 AM, haggi said:

Could you explain this a bit further what you mean by transferring the flip particles? Do you mean to transfer them from one container to another if they hit the boundaries similiar to a distributed sim?

there many ways to transfer flip particles to another constainer. you simulation waterfall there is no need to sim the falling water and lake, where the waterfall hits it in the container. sim first the waterfall and the sim the lake. in his current example i split the sim in 5 different containers to maximize resolution.

[Atom]

So when you mention container are you referring to FlipSolvers? I should have a solver for each "container" or component (i.e. falls, main body, feed tube etc...)? Would they still mix together visually..?

Would these all reside in the same AutoDopNetwork or should each solver exist, solitary, in it's own network?

Edited January 31, 2017 by Atom