Open world roads

[Pancho]

Hi there,

trying to bend my head around this for a few days now.

Since I'm creating my terrain and roads (using pathfinding and manual approaches) in Houdini, I wonder how to create/import these roads inside UE4.

I've seen a lot the videos on road creation inside UE4 using splines. The basic seems to be to pathinstance and pathdeform small chunks of road. Very memory friendly. But in all cases the splines were created inside UE4. With an open world which might change during the developments every day, this seems to be a very destructive workflow.

Since I already created my road inside of Houdini (as a nurbs curve, primitive curve or as geometry), what do I do with it to finally create it inside of UE4?

Importing the geo seems like a no go, since in an open world this mesh might be really heavy, although it would be easy to create the geo inside Houdini giving me full control over most parameters. Would UE4 slice this mesh in a memory friendly way, so that it can handle it?

I couldn't find anything yet on importing spline components via an assett, to create the streets with a blueprint. I imaginge since UE4 uses instances for these roads which are bend along the spline, that this might be a good approach, but how would it work practically?

Most of the videos covering this subject show the creation of a small racetracks or something similar. A no brainer in terms of performance, since the geo can be easily imported. But in an openworld with 20km of road this seems to be not the right option, or?

If you have any experience on this subject, I'd be glad if you could share it.

Cheers
Tom

[sant0s81]

Maybe that gives you an idea:

 

[Pancho]

Ha,ha! THanks for the tip!!!

: ) Watched it already. But I was a bit disappointed when he just instanced the rails tracks with visible gaps. : /

Hoped that I would find the secret in this video. Unfortunately I can't find anything on the web so far covering this topic in regards to what splits my brain.

I just can't tell which way I should bring in the geo so that it suits the needs of a game engine. Probably I could just generate the road mesh in Houdini which would be a no brainer. But since the street is long, how would I prepare the geo in order to just load parts visible to the camera or reduce it's LOD in the distance or do something else to it that it is UE4 "safe"?

Bringing in a heightfield is one thing, bringing in the instances another, but the roads... ???

 

[sant0s81]

@Pancho

I dont know if "Distance Culling" or similar ways in UE4 work on a single static mesh since I guess, you probably PolyExtrude a grid along the spline and so you have that massive geometry.
Maybe Tesselation is the way to go?
That tutorial here shows a nice way with lots of controls:

https://www.youtube.com/watch?v=W1MRDXUWID4&list=LL78sjhUe0B31hAz2XdaouuA&index=42&t=0s

Another way probably is to kinda split the street in many parts and use Distance Culling or LoD.

edit: and I dont know, if you have to change the HDA in UE4 to a static mesh to get all these LoD and Culling options.

Edited August 10, 2020 by sant0s81

[Pancho]

Thanks for the reply.

Funnily I just started to look at culling after it was menitonend in the "From ant to space" (or something) video from Unreal. THe guy presented it in combination with world space displacement for flowers. The further away from the camera, the lower the instances would be placed. If they are beneath ground some form of culling lets them disappear.

I guess cutting the street into pieces would be the way to go. Probably the heightfield could still carry the color information of the street even if the geo isn't there anymore.

In combination with LODs this could work nicely. Will give it a try asap!

Are you familiar with UE4?

 

[sant0s81]

@Pancho

https://www.sidefx.com/forum/topic/58506/

and

https://www.youtube.com/watch?v=kJyWSzfwEk8

could be helpfull.

Since a curve from an HDA can be used as a Spline Actor in UE4, that should work.
Never tried building a street, but I will also test around a bit, since I need something like that soon.
Its also interesting for rivers since it has the same problem when you have a large world.

And check "World Compositing" in UE4 - there are some interesting tutorials and posts in the web and maybe that could also give you some freedom regarding the mesh problems.

I am more into landscapes and using UE4 for that since some weeks now - so if you have any questions about that, let me know.
But I am also not using UE4 for real time rendering and I dont really try to optimize for frame rate

 

[Pancho]

I just used UE4 as a render engine for some animations. All this VAT to export animations is a pain. Houdini engine should take care of this. The videos about the LABS tool pretend that it is simple, but it's like "Pitfall", the C64 game. I guess I don't need to get into details. The nurse was pretty busy whiping away the foam from my mouth for hours, days, weeks? : )

The pain with every new package is not doing some stuff, but to get the philosophy of how things are done. In H I don't care about a few 1.000k, but in Unreal? So just the get info whether it's bessed to export spline, geo or whatever is difficult to get. That's almost as terrible as converting a receiving high res DEM models! : )

Cheers!

 

[sant0s81]

1 hour ago, Pancho said:

That's almost as terrible as converting a receiving high res DEM models!

@Pancho

What do you mean?
I got MASSIVE LiDAR data from Houdini into UE4 with crazy amount of details.

Check my LiDAR Post under WIP.

But yea, I think working in UE4 but coming from a normal 3D app is difficult first - but UE5 is coming and it looks awesome.

Right now I am playing with the Dev-Rendering version of UE4 4.26 where you can use volumetric clouds.

Its not as easy as it looks in the video here, but it works and gives crazy lighting effects:
https://www.twitch.tv/videos/702287886

Also check that video, great for beautiful ground fog etc:
https://www.youtube.com/watch?v=Xd7-rTzfmCo

Grüße,
sant0s

 

Edited August 10, 2020 by sant0s81

[Pancho]

Wow! I'll check the wip!

Did you capture the lidar yourself? Need to capture something like a long road. Thought to mount a lidar like the one from DIJ on a bike, but not sure whether this will work. Tried photogrammetry. While Nuke is able to retrieve a pass the point clouds are a mistery. Agisoft on the other hand comes up with great points clouds, but looses the spatial sequence, so instead of one long passage, you get a wired mix of small chunks. Still looking for a solution. Something which works as Nuke could/should work.

[sant0s81]

@Pancho

No, the data I am using in that example are scans from Thueringen Geo-Department (working on a project for the Planetariumn in Jena).

https://www.geoportal-th.de/de-de/Downloadbereiche/Download-Offene-Geodaten-Thüringen/Download-Höhendaten

Maybe you can find a long road in that data - there is quite some detail. But of course, depends what you need.
I use to check first on google maps to find an area that I could use, than I hope there is LiDAR data available and in Houdini I modify the point cloud with the Heightfield tools if more details are needed.

But thx for the Info that DJI has a LiDAR scanner... I was actually thinking to build one (there are some awesome DIY LiDAR scanners on hackaday.com).

Photogrammetry could be a solution if you only need the road - and if you have a drone and fly slowly over the ground and than use that data with Heightfields...

 

 

[Pancho]

Ah, don'T build one. THe ones I've seen are more a gadget, than a lidar scanner. The two from DIJ are between 800-1500€ I think. ONe seems to be a single scanner, the other a triple one. Both are not rotating, but scanning in one direction. Wonder whether the speed of a car at 30-50km/h would still deliver nice results. Scanning distance is something like 40m max as far as I can remember.

 

[sant0s81]

@Pancho

sounds interesting.
Would love to get my hands on a LiDAR scanner.

We are working on a Project where we have to scan different little spaces (gardens mostly) and get them into UE4.
If you have some sources or experience, let me know.

[Pancho]

Sounds like a good project for getting into lidar hardware. For this purpose a rotating one might be nice, but one which could be carried around is probably even better.

 

[LaidlawFX]

If you know how your terrain data is sliced for the 20kx20k world, you can just align the slices of the road geometry in Houdini. 

A roundtrip-able spline data is not common with off the shelf tooling. It's generally in the domain of studio pipelines. Even then mostly it's a one way process. It starts in the creation software saved to an external file then backed down in the engine. It generally does not come back in from the engine. However with Houdini Engine, you can take the splines you've created and write them out as your own data fromat. .svg should be able to handle most work by splines. However, Houdini does not have a native spline export tool. So you would be making one in python, or C++. You could write the data out as .csv if you really want, but then everything truely is a custom pipeline.

One of the more efficient ways of doing open world roads is not to use mesh prefabs for the road, but to bake the main road texture directly into the heightmap. Then you instance decals onto the road to make it unique. This saves you from having multiple meshes representing the surface of terrain. You can then add street lamps, road signs etc via this method. The prefab for meshes works generally best in simple contained game environments like a city you teleport in and out of. Especially a city that does not use a heightmap. With a 20K world you will most certainly be using a heightmap nowadays.

[sant0s81]

14 hours ago, LaidlawFX said:

One of the more efficient ways of doing open world roads is not to use mesh prefabs for the road, but to bake the main road texture directly into the heightmap

@LaidlawFX

Thank you for the information.
Is there something out in the net, that explains that?

I found that example of building a road - and in the comments the author says, that the street is a flat plane 9x9 polygons:
https://www.youtube.com/watch?v=bdYB_VVapNY

14 hours ago, LaidlawFX said:

However, Houdini does not have a native spline export tool

I thought, that you can export splines from Houdini to UE4 with an HDA and in UE4 the splines will be recognized as UE4 Splines and you can modify them the same way, you can with UE4 native splines.

I am confused

[LaidlawFX]

I've only worked on a few AAA franchises, with a few GDC talks you may have watched. Maybe you should wait on someone else to answer 

[sant0s81]

@LaidlawFX

Ehehhe - sry, didnt want to question your skills.

Like I said, just confused

edit: actually I saw that video here yesterday and I guess, thats quite similar about what you are talking?
https://www.youtube.com/watch?v=UjwT1qw7r08

And I am just watching you talking

Edited August 18, 2020 by sant0s81

[rates]

I came across this commercially available plugin which looks like it aims to address some of these issues. It will create Unreal Landscape splines from GIS map data (e.g roads) https://terraformpro.com/terraform/

As far as I can tell there doesn't seem to be a native way to turn Houdini curves (or other data) into Unreal Spline components or Landscape Splines. Via an HDA you can instance stuff onto points controlled by Houdini curves, but this wouldn't help in the case of roads where the instanced components need to be continuous and deforming (which Spline components or Landscape Splines can handle). You can sample from existing Unreal spline components into an HDA and then do stuff, but this wouldn't help you if you already have your road curves in Houdini. Apparently outputting Houdini curves to Spline components will be in Houdini Engine 2, but is not in the alpha yet: https://www.sidefx.com/forum/topic/75103/

Landscape textures work for country roads that blend into the surrounding terrain at relatively low terrain resolution like in Ghost Recon Wildlands I guess, but how would you handle stuff that needs sharp transitions with better resolution than 1 vertex per meter, like curbs and footpaths over 20km of suburbia? Surely that would put you back in the same position that raised the OP's question. i.e trying to instance deforming geometry along curves.

[sant0s81]

5 hours ago, rates said:

Surely that would put you back in the same position that raised the OP's question. i.e trying to instance deforming geometry along curves

But wouldnt something like that work?
https://www.youtube.com/watch?v=W1MRDXUWID4

 

Actually, adding tesselation on the street that is generated with a HDA to have a super low poly mesh by distance to camera.

 

[rates]

2 hours ago, sant0s81 said:

But wouldnt something like that work?
https://www.youtube.com/watch?v=W1MRDXUWID4

I don't think so, that link describes tessellating and displacing ue4 height-map landscapes, how would you place curbs and sidewalks to follow a winding road using tiling textures? The landscape mask resolution is still limited to 1 vertex per meter. Also tessellation won't generate collision so feet would penetrate.