Distributed SOP cooks?

[Daniel_Daniel]

I'd watched the video showcasing distributed particle sims, but only recently it occurred to me that this technique only applies to DOPs?

Am I correct in saying that there is no general solution to distribute SOP cooks? For example, slicing and aggregating (map-reducing) VDB clouds or any VEX snippets on large meshes?  

Edited April 3, 2021 by Daniel_Daniel
mention VEX

[underscoreus]

As far as I know no, there is no built-in solution for general distributed sop cooks. I guess the same can be said for DOPS. No general way there either, always specialized tools. One for FLIP, one for Pyro etc.
But for SOPS I think you'd have to make your own little setup if you wanted to distribute it. However, most cooks in SOPS should not be that long anyway so distributing the task might create more work than it removes. Whenever I've encountered these kinds of issues, and it's always been because I ran out of system resources, I've always just split my geometry/volume in two, three, four, or five pieces and ran the task on each part and merged/fused it back together.

[Daniel_Daniel]

EDIT: Hmm... one could partition geo [101 of wedge ROP] with id attributes for each slice and distribute them then. As opposed to DOPs, SOP networks are "stateless", which should obviate any need for communication between slices. A bit of overlap between them should do the trick? 

Thanks, I see. Yes, distributed cooks are an edge case. Hmm... what about buffered reads then? The problem is mostly limited memory (yes, sometimes even 64GB isn't enough ) 

I hoped maybe for a distributed solution from the new PDG workflows, but it seemed to only distribute variations of separate cooks.

Edited April 3, 2021 by Daniel_Daniel