Debugging with Houdini Engine Debugger + UE4

[ron2812]

Hi everyone!

I'm kinda new to Houdini and use it with UE4.
My Digital Asset crashes all the time though so I thought I'll debug it with the Houdini Engine Debugger.
This works well and actually is pretty awesome.

When I reproduce my crash in UE4, the connection gets closed and the Houdini Console shows the following errors:
Thrift: Mon Oct 29 18:09:38 2018 TPipe ::GetOverlappedResult errored GLE=errno = 109
Thrift: Mon Oct 29 18:09:38 2018 TConnectedClient died: TPipe: GetOverlappedResult failed

But where to go from here?
Is there a way to find out what node crashed? Or how should i proceed to narrow down the problem?
Is there any kind of log file I could check?

If you need more information, please let me know!
Thanks for you help in advance!
best,
Ron

[LaidlawFX]

Hello Ron,

Welcome to the forum!

If you got the Houdini Engine Debugger working that's a good first step.

If you can share your HDA and or hip with the forum that be easiest to answer questions. It saves from the guessing game. Often reproducing your asset so it can share publicly will clear out the issue on your end too.

I'm going to guess your HDA takes inputs and more of a modifier than a generator. If it's a generator you should be able to repro the issue in Houdini easy enough unless you are creating bad geometry. For a modifier like asset, create the same number input on a dummy HDA, then use the Houdini Debugger to visualize the geometry in Houdini. If it crashes then you have bad geo, and should probably submit an RFE with your problem geo, or find out why that geo is problematic. For instance does it have bad ngons? With the geo connected to your dummy HDA save the hip file out so you can shut down the debugger. Then you can debug the scene with your engine geometry only in Houdini. This should make debugging your HDA alot easier. If it keeps on crashing with your asset now, just disable, or remove parts of yoru HDA until you can narrow down what region is crashing the HDA. If it's just locking up but then finishing in Houdini you can run the performance monitor too, but with a straight crash it will only help so much. If you don't have any issue with the HDA exclusively in Houdini with your input geometry. It's possible houdini is producing a piece of geometry it does not see as an issue, but the connection is choking on. This is a bit to varied an answer as we don't know what your HDA is actually doing. Houdini does spit out a couple different logs, but they won't help you as much. The Error code is not common, so you would have to ask support what the code is specificly, but they would need the sterlized example file the same as most people on the forum would to help you debug your issue deeper.

Hope that helps.

-Ben

[ron2812]

Hi Ben!

Thank you for your detailled answer!
It's kind of a mix - I'm working on a wall generator with flagstones on top. The wall is generated procedually based on a curve input. The flagstones come in from UE4 as it is an already modelled mesh.
All i do is placing instances of the input object along the wall with random rotation and random scale.

It almost looks like there's a problem when working with meshes coming in from UE4. It works pretty ok when I bypass that mesh and use a proxy box.
 

Tha HDA + HIP file is attached (i hope everything is included as I'm a beginner in Houdini).
Thank you for your help! You're saving me hours of pain :-)

If you need anything else, please let me know!

best,
Ron

RuinwallGenerator.zip

[LaidlawFX]

If it does not crash when you use a box, then it certainly does sound like your custom asset has some geometry issues. You could try and just run that mesh through a clean sop, polydoctor sop,  or such and get rid of any n-gons or other trash geometry that it may inadvertently have with out your knowing. 

If crashes occasionally with the box then you may have two issues. A box theoretically should not cause geometry issues, so it would probably be something else funky happening in the system. 

The hda is not included in the .zip just the embedded version in the .hip. Also the vertex color randomizer is not included. You can generally strip the secondary .hda when sharing. 

To make your .hda a bit easier to use you can use the inputs in the sop node as opposed to the object merge. 

Also save out the geometry from unreal as seen through Houdini Engine Debugger. With your hda to keep it from crashing you can disconnect all the nodes after your inputs "../INPUT" so you can see what that geometry gets imported with the Houdini Debugger. Save that scene and then test what's wrong with that geo or hda.

[ron2812]

Hi Ben!

Thanks! I'll give that a try! Maybe something is wrong with the input geo since I did some zRemeshing over a high-poly mesh. This sometimes causes errors.
Also I found one problem where I used the input-geometry bbox for some calculations and a switch where I can turn off the input mesh. Maybe this causes some of the problems as well.

Could you elaborate the last statement a little bit more?
Do you mean exporting the mesh (the flagstone object) as FBX from Unreal?

Thanks for taking the time to check out my problem! Really appreciate it!

best, Ron
 

[LaidlawFX]

Hey Ron, 

So I'm not sure you are actually using the Houdini Engine Debugger to test your tool. So with Houdini open go to Windows > Houdini Engine Debugger. Set it to named pipe and turn it on. Then in Unreal go to the settings for Houdini Engine, turn off the connection, set the option to named pipe and turn it on. No instead of Houdini running in a background process called HARS it will function through your open Houdini session This way you can do run time debugging of your Houdini functions in Unreal. So the geometry you have imported and the HDA you are using will run in that session. Be warned this is not your typical houdini session, there is no history for example, but you can see what houdini is doing with the imported geometry. If you saved your HDA prior to this test with out your inputs connected to the rest of the HDA, then when you connect the geometry that is causing it to crash you will see what it looks like on Houdini. If your geo is possibly messed up into ngon shit this is a good way to find out. Also you can slowly lead test your hda and see what functions are actually working in the HDA. To help debug once you have this active session hip file you can save this hip. It will presserve the input geometry than came from Unreal as it will be locked. Since the nodes are locked you can share the geometry with out needing to have the Unreal session and all those associated needs. This makes it easy to debug and share. Also the .hda is a seperate file from the .hip. So when you share a .hip without the .hda it will embed the file in the .hip but for practical purposes you can't edit or manipulate this embedded file to share back with you. There are some exceptions for this, but it's easier to just have the .hda in the first place.

-Ben

[ron2812]

Hey Ben!

This is really valueable information! Thank you!
Thanks to you I was able to debug the file further and from what it looks like, the input geo is OK when it comes into Houdini. No issues there.

I also use a Subdivide node to split up my generated wall (to do vertex painting where I need more vertices) and generated a UV with a UV Scale node (to fix Texel density)
What I found out is that the problems mostly occur when I assign a Material in UE4 to the generated wall mesh and later change subdivision or modify the uv.

In my example:
--> I connected the Engine Debugger, got everything from UE4 visible in Houdini and started to modify my parameters and curve - all good
--> I added my flagstone asset in UE and was able to see it in Houdini (geo looks fine, no crash)
--> Changed the divisions and UV scale on the generated wall (still everything fine)
--> Assigned a UE4 Material to my generated wall asset (simple material with brick textures) - still works
--> Changed subdivision parameter again - BAM - UE4 crashes
--> the last subdivision change is visible in Houdini - so the last command was processed by Houdini but by that time UE4 was gone.
--> Saved HIP file, unlocked nodes, fooled around in Houdini with parameters - no problems there.

From that perspective it seems like it's more a UE4 problem than a Houdini problem.
As if UE4 Materials don't like it if you change the underlying geometry or UV channels on the fly.

I wonder if this is a known thing (or maybe I'm totally mislead there) but to me it doesn't seem uncommon to assign materials before and then change geometry and generate UVs in realtime.
So strange nobody seems to have that issues? Or maybe I'm on the wrong path here and this should work as expected.

-Ron

[ron2812]

I need to add that this doesn't happen ALL time.
Sometimes it works a while before it crashes when I cange UV and division parameters and a material is assigned.

On my last try, I was able to change UV and subdivisions while a Material was assigned.
But when I changed a parameter for the random rotation it crashed immediatelly.

Generally speaking it works fine when I just use generic Houdini geometry. Once I add a Material or a custom mesh in UE4 things get a bit whacky.

 

[LaidlawFX]

Yeah you definitely have a bug with the Houdini Engine plugin. Well technically it is not a bug with Houdini per say and that of UE4 not being able to handle the diverse array of geometry that Houdini can. They won't fix that anytime soon in UE4 so the Houdini Engine team makes safe catches around the types of "bad" geometry that will cause it to crash. You should send in the example to support, especially if you can get a close to 100% repo case, even if it's slide the slider until it crashes.

On the other side I have never had much luck with the subdivide SOP and Houdini Enging to other applications in general. We used to have this issue a lot when i was doing Voronoi Fracture with the Max plugin in for Maya when it first came out. My biggest recommendation would be just not to use the subdivide sop as convenient as it is in Houdini. The subdivide process really depends on clean geometry coming in, and unless you know 100% how they are being created the algorithm that is subdivide can do some interesting things when faced with ngons you can't see. Many programs can not handle the diverse geometry collection that Houdini can so Houdini keeps on trucking. Especially game engine only like very clean geometry for performance sake, and are not designed for funny shapes. For a substitute you can use the divide sop on faces, you can add edge loops( a bit trickier in houdini), bevels, etc. The triangulate options on divide sop are nice option for cleanup too. The Clean Sop and polydoctor can help too if you go down the route of cleanup. I would stay away from cleanup processes to early in your development if you can, triangulation being different than cleaning up overlapping faces and such as a last note.

[ron2812]

Hey Ben!

Sorry to bump this thread again, but I still got problems with geo coming from UE4...
But I made something really simple to reproduce the crash.
I made an HDA with just a curve input, a resample node and a copy-to-points from an Object Merge.

So now when I import the HDA into UE4 I do the following:
- set Curve as Input and draw a simple curve
- Assign a mesh (from FBX) to the object merge -> things look ok
- Change the input mesh SLOWLY a couple of times (box1->box2, box2->box1, box1->box2 etc.)

UE4 crashes here everytime when switching out the meshes. Mostly after the 3rd switch.
As for the meshes i just made 2 simple boxes in Modo. Nothing fancy (to be sure there's no broken geo).

It can't get simpler than that and I still got crashes all the time. I attached the hda and the 2 boxes as FBX.
Any idea what could go wrong here?
 

Thanks in advance!
Best,
ron

 

box1.fbx

box2.fbx

curve_spawner.hdalc

[LaidlawFX]

Hey man certainly send this into support at sidefx.com, or use the submit page on the website. If it's a crash like you say it is then I can only tell you how to avoid it. They can fix it. In the email subject say BUG: Unreal Houdini Engine Crash. And for the body explain the repro step by step and attach the assets and Geo like you did. If it's a geo issue like I think they should be able to fix it in one of the daily builds quite quickly depending on how busy they are at the moment. I send in about a hundred request a year. They are pretty good at it especially on crash related issues.

[ron2812]

Hey Ben!

I submitted it as bug - let's see what happens :-)

One thought came to mind... maybe this is a thing already and I'm just not aware of it.
In my case all i wanna do is basically spawn UE4 assets on given positions with rotation/scale etc.
So there's really no need to have the geo going into Houdini (which is the cause of my troubles). All I basically need is to get a list of spawn-points with the required attributes back into Unreal and
place instances of my mesh according to the attributes.

I wonder if there's a pre-made blueprint or something for that task.
This sounds like a logical thing to do if you use premade meshes from UE4 that don't change but only get instanced in a certain way/position.

talk soon,

best,
Ron

[LaidlawFX]

I'm not a Unreal expert, too much time with proprietary engines unfortunately. I highly suspect you can do that though. Plenty of games build world's in Unreal as such, it might just be a bit of google Foo.

You may hate me for saying this though but since you said it was geometry issue. Houdini definitely needs a fix in this case I completely believe. HOWEVER, the system has been used for years in production up into this point. So you may just have an asset management/approval process that is more the culprit. Game engine's really care a lot about the mesh's provided for performance, more so than render pipelines. So they don't handle outside the norm meshes well. I've had issues like this before in proprietary engine where this smells more like an edge case. Building up good wrappers around exporters takes a lot of work, so submitting this edge case is certainly good. So you may fix your troubles with a standardized pipe. Apologies if that is a sore topic, it generally is in production.