Point Instancing Render Issue with Smoke elements

[bifrost]

Hey guys! I ran into this problem a couple of weeks ago. Whenever I am rendering a smoke element, sometimes my render have some flickers in them. It just randomly appears on a random smoke element in a random shot. So it's not shot specific.

I am using the point instance method to render where essentially I have a set of points linked to the file path on disk. Has anyone experienced this problem before? Any help on this is greatly appreciated.

Posting this in FX because I have only had this issue with smoke elements. Not even fire. Just the density.

-bf

[TheDude]

Hi Bifrost -- A few months ago I was also running into this issue. Mine was a bit different as I was using only a few wedges and using packed disk prim to pull in. Same visual though -- it almost looked like entire cube-positions in random places didn't register any volume data.

The only workaround I found for this was to actually create delay load procedurals for the wedges. Obviously that isn't an option in your situation, since I'm guessing you have a ton of instances... What's got me wondering is if anyone has seen this before H15... I never heard anyone mention this before that release.

Sorry to say I never had time to chase down the issue further once I had a workaround working.

Instead of having the points link the instancefile, you could try directly creating geometry as raw delay load procedurals(again this could take a lot of setup time depending on how many you have), and then have each instance link to that, instead of pulling directly file location on disk. That is at least somewhat more similar to the workaround. No idea if it'll work tho :/

Edited October 6, 2016 by TheDude

[bifrost]

20 hours ago, TheDude said:

Hi Bifrost -- A few months ago I was also running into this issue. Mine was a bit different as I was using only a few wedges and using packed disk prim to pull in. Same visual though -- it almost looked like entire cube-positions in random places didn't register any volume data.

The only workaround I found for this was to actually create delay load procedurals for the wedges. Obviously that isn't an option in your situation, since I'm guessing you have a ton of instances... What's got me wondering is if anyone has seen this before H15... I never heard anyone mention this before that release.

Sorry to say I never had time to chase down the issue further once I had a workaround working.

Instead of having the points link the instancefile, you could try directly creating geometry as raw delay load procedurals(again this could take a lot of setup time depending on how many you have), and then have each instance link to that, instead of pulling directly file location on disk. That is at least somewhat more similar to the workaround. No idea if it'll work tho :/

Hey Jamie! Yes, this exactly sounds like a problem that we are experiencing at the minute. I am glad that I am not the only one here. As far as points go, we have over 100 clusters of smoke elements but we are trying to fix this through delay load geometry. And yes, I have looked in the forums, both odForce and SideFX, but in vain. Seems like not many people are running into this issue. Probably because they don't use point instancing method. And I am wondering the same if this issue exists before H15. May be it's a bug in this particular version of Houdini where it doesn't take voxel data into consideration properly. Either way, I am thinking of writing an email to SideFX about this.

Thanks for taking time to reply.

-bf

Edited October 7, 2016 by bifrost

[TheDude]

Hi Bifrost, just threw together a hip file.

 

It will be a lot of work, but this is the workaround I used -- delay load procedural worked every time on the farm, whereas packed disk would break. If you have the time to get up a geometry node for every instance sequence, I would recommend taking the time to do that just because it is consistent.

If you render Volume_One explicitly it will work (if you have 3 volume sequences, you will just need to create those three geometries... or 100 in your case    ). I then tested to see if you can instance explicit delay load procedurals fast point instancing, in order to give you the ability to keep your current setup, but if you render instance1 you will see it error -- It can't do that, which I was hoping would be the answer to your issue.

One idea I had was that since hard drives either read or write, but not both, that if the farm is in heavy load, then as it builds the geometry from disk, it could be potentially interrupted as it has to write something, and then resumes reading the rest of the geometry and so "skips" areas. I don't know how volumes are read, perhaps in buckets until 100%? If that is the case though, it doesn't make sense that it would read it in one go every time for delay load procedurals, but not for packed disk... I dunno super confused as to what is happening.

Sorry for the bad news, super annoying problem to find at the end of a pipeline right at render time. 

JG_Instance_Example.hip

Edited October 7, 2016 by TheDude

[bifrost]

On 10/07/2016 at 11:57 AM, TheDude said:

Hi Bifrost, just threw together a hip file.

 

It will be a lot of work, but this is the workaround I used -- delay load procedural worked every time on the farm, whereas packed disk would break. If you have the time to get up a geometry node for every instance sequence, I would recommend taking the time to do that just because it is consistent.

If you render Volume_One explicitly it will work (if you have 3 volume sequences, you will just need to create those three geometries... or 100 in your case    ). I then tested to see if you can instance explicit delay load procedurals fast point instancing, in order to give you the ability to keep your current setup, but if you render instance1 you will see it error -- It can't do that, which I was hoping would be the answer to your issue.

One idea I had was that since hard drives either read or write, but not both, that if the farm is in heavy load, then as it builds the geometry from disk, it could be potentially interrupted as it has to write something, and then resumes reading the rest of the geometry and so "skips" areas. I don't know how volumes are read, perhaps in buckets until 100%? If that is the case though, it doesn't make sense that it would read it in one go every time for delay load procedurals, but not for packed disk... I dunno super confused as to what is happening.

Sorry for the bad news, super annoying problem to find at the end of a pipeline right at render time. 

JG_Instance_Example.hip

Alright! So because we have over 100 clusters/points, this is too slow of a method. But we started looking into Packed Disk Primitives recently. Funny thing, there were some box like flickers with that method too but it's insane fast. So we opened a ticket with SideFX support to see if we can get their attention on this. They asked for an example file and voila, they were able to reproduce the same problem at their end.

So just waiting to hear back from them. I think Jeff or someone else is probably looking at it right now. I am hoping for a good news. Let's see. Just thought I will reply here too just in case if anyone else ran into this issue. No, you ain't the only one.

 

-bf

[TheDude]

No way! That's awesome to hear, thanks for actually following through with this, much appreciated and hope to hear more soon on it!

[bifrost]

1 hour ago, TheDude said:

No way! That's awesome to hear, thanks for actually following through with this, much appreciated and hope to hear more soon on it!

I will keep you updated regarding this.

[bifrost]

So SideFX replied confirming it's a bug and it's being looked upon. So I am guessing they will be fixing the same in near future builds for H. So, for now, we are using some manual methods to do this. But it's great to know that it's a bug and not us.

[TheDude]

Good news!

I reported an RFE again with an attached hip file showing it breaking. They looked into it, and the next week i got an email.

 

"This problem should be fixed in Houdini 16.5.304."

 

Finally!! haha, pretty excited to not have to have hundreds of individual geo nodes in the future