What's the best way to cache simulation data to disk?

[magneto]

Hi,

Everyone seems to use a different method. These are some of the operators I have seen people use to cache their simulation to disk:

DOP I/O

File

File Cache

ROP Output Driver

I am leaning towards DOP I/O SOP because it has both the capability to import DOP data which you always need to do I think and also the ability to read/write to disk.

What do you use and recommend using?

Thanks

[anim]

I think DOP I/O and File Cache are made for ease of use, they still contain ROP Output Driver SOP and File SOP inside

I would use File Cache SOP for generic caching of any geometry, but if you directly want to cache data from DOP sim, then DOP I/O may be handy, but you can still use Dop Import Fields SOP + File Cache SOP combo for that

[magneto]

I want ease of use

I see that pretty much all of their parameters are the same. So that's why I thought there must be some history behind these.

[Skybar]

I generally use DOP I/O with DOP-stuff, and File Cache for anything else - mainly because it's easy to just hit "Render", and then tick "Load from disk". If I'm not going to use the written files at the same place though, I tend to swap the File Cache for a ROP Output and read that in somewhere else with File.

[magneto]

What about loading from disk using DOP I/O or File Cache and then loading them somewhere else in another subnet using Object Merge? Is that nonsensical?

[Skybar]

That seems like it would just be confusing, unless you need them at both places. You could just do a File SOP with a relative path to pick the same files up.

[Solitude]

What about loading from disk using DOP I/O or File Cache and then loading them somewhere else in another subnet using Object Merge? Is that nonsensical?

I do that all the time, usually putting a null under the cache and labeling it in such a way that I know it's the cache, untouched. For a pyro sim, for instance, I might use the dop i/o to cache it, put a null under it called "OUT_pyroCache" or something useful and then call that into another object to emit particles from that sim, or use the velocity field, etc. After the null I might do things like delete the extra fields I won't use for rendering, and maybe use a volume ramp or volume vops to prep the sim for rendering, then put a null under that called "OUT_render". This way you have one place to save and fetch the sim from, and it can also be the same node you are rendering. If you use a file node in other objects, you have no 'good' way of knowing where you're getting this data that's feeding your secondary sim. I think it makes more sense than having many file sops (and then possibly re-reading another copy of the geo from disk again).

I use dop i/o and file cache (makes directories for you, and takes care of saving and loading in one place) for home use (each studio tends to have their own version of file cache that they use to put things in the right places) -- then I'll use fetch nodes pointing to the rops inside of those otls in /out if I need to chain anything together.

Edited November 20, 2013 by Solitude

[magneto]

Thanks Solitude, that's awesome. So you fetch the cached sim from say OUT_pyroCache using Object Merge? Now that you said it, having a single place to save/load the cache for something make sense.

I also have a hard time finding good names for the objects. For example the cache subnet, do you name it fluid_cache, or cache_fluid? I haven't seen many examples. What if you have a cached sim, like FLIP particles, that you need to use in another dopnet. So you have null nodes like OUT_surface, OUT_vel, OUT_particles. What would you call that subnet? fluid_source? It would be nice to see other people's conventions regarding these.

Thanks

[Solitude]

Yes, I object merge it into other objects, or subnets. I usually have something like /obj/particle_fluid (whatever the shelf tool naming convention is, or whatever I've renamed it to match my scene better) ... then stuff like /obj/particle_fluid/OUT_vel /obj/particle_fluid/OUT_cache (all fields, straight out of the cache basically), then, /obj/particle_fluid/OUT_particles /obj/particle_fluid/OUT_mesh (surfaced particles). The naming you have makes sense. Do what works for you, but as long as it will make sense to others as well.

[magneto]

Thanks Solitude, that makes a lot of sense. I was getting confused because some files I have seen had fluid_source, fluid_dops, source_fluid, another fluid_dops for spray, etc so it was hard to see what was coming from where