Is it beneficial to build custom solvers?

[magneto]

Hi,

From what I can tell, other than using the default Pyro, Flip, RBD solvers AS IS, it seems like no one even touches microsolvers, or make their own solvers. I mean even after years of experience in film, they just rely on the default tools.

Is this normal? I always thought the best Houdini guys would be surfing on cutting edge custom solvers that they built that put them ahead of everyone in ways not possible with the default solvers.

If you don't need these, then why even go beyond the default solvers?

Is this just a hidden skill for bragging rights that doesn't have much use in real production?

Thanks

[fathom]

i can't speak for other folks, but i've cracked open different solvers to add/change things.  and the way they're made, they typically allow for modification without having to unlock by wiring in microsolvers as part of the "inputs" to the main solver.

[Annon]

I've built our studios pyro wrapper which includes support for micro solvers and cvex "plugins".

These are used when more complicated things need to be achieved.

Also I know a few people who've added more complexity to flip and bullet stuff.

I mean all the default pyro tools are made out of microsolvers, so you can see what's going on easy enough...

Plenty of people extend default setups, either in sop solvers or microsolvers.

[JonathanGranskg]

I've only done some minor changes to the FLIP solver when I played around with airfields and that was mostly some changes to droplet detection system to make it work properly.

 

I don't think many people build full-blown solvers from scratch. Like Christian and Miles said it's more about adding extra stuff.

[Skybar]

Define "custom". Sometimes vanilla pyro is enough, sometimes you need extra fields or whatever. Houdinis strongpoint is the ability to modify stuff according to your needs. If you dont have those needs then you dont need to.

[Jason]

I find that mostly commonly strategic surgery is performed on Solver depending on very specific shot needs. If we find that the solver addition we've done is a good general purpose idea, we sometimes lobby Side Effects for a native implementation to be used for upcoming projects... some things are just not worth trying to maintain yourselves. Just having the ability to jump into a solver like we do is still very awesome, though.

 

Trying to maintain a specialized/branched version of the Solvers is probably a lot more work than you think. It's great for say 8 months, but then Side Effects moves forward and it becomes a game of catch-up where you might not be fully aware of where the subtleties of the new changes lie. Wrapping entire solvers is puzzlingly difficult to maintain, even.

[CiaranM]

I can't remember the last time I didn't use a microsolver on a job, but that usually just involves plugging them into the inputs of the smoke solver, not building something 100% custom from the ground up. Why re-invent the wheel (and who has the time anyway)?

Clients are always wanting smoke to do some magical bullshit, so microsolvers are a necessity.

[Guest tar]

Is this just a hidden skill for bragging rights that doesn't have much use in real production?

 

 

I see this differently. All knowledge is useful for problem solving. i.e. knowing that 0.999 = 1, and, 1+1 !=2 in abstract algebra is useful no? It's part of the foundation to understanding that the world is not the common-sense world that immediately see around you.  You only can construct new ways of doing things if you can use the proper building blocks, so, building solvers is not about 'bragging', but is of the essence of being a true artist.

[NSugleris]

I see this differently. All knowledge is useful for problem solving. i.e. knowing that 0.999 = 1, and, 1+1 !=2 in abstract algebra is useful no? It's part of the foundation to understanding that the world is not the common-sense world that immediately see around you.  You only can construct new ways of doing things if you can use the proper building blocks, so, building solvers is not about 'bragging', but is of the essence of being a true artist.

 

I agree.  I think of it as having just more specific tools in your garage.   You probably won't need it everytime but knowing there's more options to problem solve with is always a plus in my book.

[magneto]

I see this differently. All knowledge is useful for problem solving. i.e. knowing that 0.999 = 1, and, 1+1 !=2 in abstract algebra is useful no? It's part of the foundation to understanding that the world is not the common-sense world that immediately see around you. You only can construct new ways of doing things if you can use the proper building blocks, so, building solvers is not about 'bragging', but is of the essence of being a true artist.

I agree with you but I was told by other tds that I met that if you are building custom solvers etc, you are screwing around on company's dime. That's why I wanted to ask here.

 

They basically said the default solvers work and they do everything you need. No need to complicate things.

[Guest tar]

I agree with you but I was told by other tds that I met that if you are building custom solvers etc, you are screwing around on company's dime. That's why I wanted to ask here.

 

They basically said the default solvers work and they do everything you need. No need to complicate things.

 

That's superficially true but you also want to work with great producers whom can sell your skill-set, then you get to do r & d as your job.

 

The top producers want to work with the top artists, so become excellent at what you do and you can do what you enjoy.

[Skybar]

Its kinda arbitrary to say that the default solvers are good for them, and should be good for everyone. Everyone got different needs, if the default isnt enough for a job should you then complain that it is bad or work around it? With this mindset, Pixar/Disney etc. should never have to innovate to create new stuff. Which of course they do all the time.

[xs2222]

my 2 pennies worth , correct me if i am wrong ,

aren't most of the solvers from the shelf , built from multiple solvers , 

it depends on the type of task you are involved with , if it needs tweaking then by all means , 

for instance if the existing solvers are slow for the task involved , then you can strip it down to basics and for speed and efficiency , or build no top of them

 

personally would go for the simple approach and start adding nodes as and when needed  

[Annon]

As mentioned before, you don't have to build everything from the ground up. Adding into the inputs of the pyro solver is mostly enough.

So I doubt there's any artist properly building a solver from scratch. Maybe they unlock it and add something in there to the main solver, but mostly I bet people just bolt things ontop of it...

[anim]

...

From what I can tell, other than using the default Pyro, Flip, RBD solvers AS IS, it seems like no one even touches microsolvers, or make their own solvers. I mean even after years of experience in film, they just rely on the default tools.

...

not sure what you are basing this on, but you can't generalize like this, it's simply not true in general

you can see forum being full of custom setups and solvers, so even from that you can imagine that real production tools would be even more sophisticated, and it is true in many cases

but it's also true that there is so many Houdini artists who will just use default solvers and few tricks they have learned from tutorials/forums and will never want to build their own as it seems scary to them

but overall, if you need some functionality that is not out of the box, or you want to customize/extend existing solvers, then microsolvers to the rescue

but there is very blurry line between what you consider microsolvers and what not as it's just a abstract name, but any Wrangle (Geometry, POP, Field) in DOPs is a microsolver

Sop Solver is a microsolver, any POP DOP node is a microsolver or isn't it? Would you not call it microsolver, just because it's not in MicroSolvers submenu? Doesn't matter, they are all DOP nodes that modify data and you can use them for whatever you consider them useful, I can't imagine to actually not be able to use them

but to make it short I would say it is almost impossible to avoid using microsolvers and it's pretty common to even build custom solvers from scratch, for specific purpose.

[outtacontex]

I think it's beneficial to do in your spare/personal time, as it helps understand aspects of DOPs. It'll just speed up shot dev significantly, because you'll have a better understanding of the inner workings of things. The majority of the time (also what several others have mentioned), I've seen modified solvers, where tds dive in and add to or manipulate things as they see fit. The more knowledgeable you become with it, the quicker you can set things up....so you very well may be able to build a solver from scratch depending on production/shot demands. 

[magneto]

not sure what you are basing this on, but you can't generalize like this, it's simply not true in general

you can see forum being full of custom setups and solvers, so even from that you can imagine that real production tools would be even more sophisticated, and it is true in many cases

but it's also true that there is so many Houdini artists who will just use default solvers and few tricks they have learned from tutorials/forums and will never want to build their own as it seems scary to them

but overall, if you need some functionality that is not out of the box, or you want to customize/extend existing solvers, then microsolvers to the rescue

but there is very blurry line between what you consider microsolvers and what not as it's just a abstract name, but any Wrangle (Geometry, POP, Field) in DOPs is a microsolver

Sop Solver is a microsolver, any POP DOP node is a microsolver or isn't it? Would you not call it microsolver, just because it's not in MicroSolvers submenu? Doesn't matter, they are all DOP nodes that modify data and you can use them for whatever you consider them useful, I can't imagine to actually not be able to use them

but to make it short I would say it is almost impossible to avoid using microsolvers and it's pretty common to even build custom solvers from scratch, for specific purpose.

 

Sorry I meant the TDs I talked to in person who work at big studios. I don't think they are lying but they are also not necessarily on odforce like active posters here. I mainly asked if they used the actual Gas microsolvers which they said no, let alone brand new solvers built from scratch.

 

I am just wondering if one could make a better FLIP/pyro/smoke/grain solver than SESI with all the papers around and even a better UI could make a huge difference if everything is designed with great detail.

[anim]

I am just wondering if one could make a better FLIP/pyro/smoke/grain solver than SESI with all the papers around and even a better UI could make a huge difference if everything is designed with great detail.

 

yes and no

as the main functionality of FLIP/Pyro/Smoke is in blackboxed microsolvers, so while you can use it for your own solver, you will still use the same microsolvers

you can of course do your own version of the solver, take slightly different approach, but the crucial steps like Project Non Divergent, etc. will most likely be the microsolvers SESI provided, so even "from scratch" is a vague term as really the heavy lifting is done already, to implement many papers you would need to make your own "microsolvers", but for simple ones Houdini is quite flexible as you can see on forums

 

grain is a different beast, Jeff made it completely using VEX, proving that you could indeed do it before on your own  and that you can possibly improve it much more (similarly to what Ripple Solver is as well, just VOPs).

And if you look at the code there is more than grains, there are some SPH kernels already implemented and similar, so if you want to extend grains to handle liquids like NVIDIA's Flex, it may be already half way there

 

On the other hand TDs are usually not physicists/matematicians so unless they really understand the papers and find it enjoyable to implement complex equations they don't want to go that deep

But within certain range of complexity it's quite fun, and I find Gas Microsolvers white nice ballance, not too low level, but very flexible/powerful

And it makes me very happy to see some cool papers implemented very cleverly into existing microsolvers like preserving angular momentum from APIC paper into Gas Field To Particle and Gas Particle To Field DOPs in H15

then it would be a shame not to use such goodies in your own tools

 

Personally I didn't have a need to build any of the existing solvers from scratch. If you have an idea how to improve existing solver, then you can just edit it. But understanding microsolvers at their individual functions it's very valuable as many of them are so powerful

Edited October 29, 2015 by anim

[Guest tar]

You kind of need at least undergraduate and perhaps graduate level mathematics to implement proper simulation tools.  At that level you're usually a developer, not a TD.