Jump to content
Sign in to follow this  
sant0s81

[Solved] Custom LiDAR importer with python

Recommended Posts

Nothing fancy, just to show how easy it is to go super simple from LiDAR to UE4 with that cool tool now :D

lidar_to_hf_to_ue4.thumb.jpg.6e49b370940e21b1ca1d1b86f06172b6.jpg

  • Like 1

Share this post


Link to post
Share on other sites

Hi, no, this is just default comment from Python SOP.

3 hours ago, sant0s81 said:

# Add code to modify contents of geo. # Use drop down menu to select examples.

 

In the latest version, https://forums.odforce.net/applications/core/interface/file/attachment.php?id=56259 classification is just an integer attribute on the points.

 

 

 

Share this post


Link to post
Share on other sites
13 hours ago, sant0s81 said:

What do you mean loading under a second to load?

 

Julien means the same thing I already mentioned. Loading and building attributes in Python is much slower than doing same thing in C++ most of the time. Lidar Import SOP is a fast C++ node compared to pylas. Loading your *.las example takes 20ms with Lidar Import SOP on my machine but same file takes almost 6 seconds with Python SOP with pylas  (3 seconds just for transforming numpy nd array to serialized form and another 2.5 seconds for setPointFloatAttribValues method).

If you do have enough memory and you do end up reading most of the file anyway it's faster just to use Lidar Import SOP to load all points fast and then use Python SOP to add only additional data that Lidar Import SOP cannot read (like classification) and blast what you don't need. Like this:

laspy_lidar_faster_import.thumb.png.933c90d966b9028ce7c04befee05a755.png

 

 

 

  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

Okay, wow....thats cool :)

But anyway, you must have crazy machines... I load at leas 20 till 25 seconds for the big .las.

@pezetko

One thing I just realised, I cannot load .laz - only .las.
Is that a restriction?
I can convert them, no problem - but somewhere I saw it may work witz .laz, too.

Edited by sant0s81

Share this post


Link to post
Share on other sites
Posted (edited)

Another quick test - so the HF is on maximum for UE4, means 8129 x 8129.
Of course there is only one texture now and it looks shitty - but was only to test with the resolution.

The LiDAR file out of the box, combined with super little erosion to keep the main shape, alreadygives some crazy details on the ground.
The trees are also on the "correct" positions filtered by the classification.

Tomorrow I will try to make it with more layers of the HF and the different classifications from the LiDAR.

Have a good evening everyone.

 

Edited by sant0s81
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

Nice,

I think fast ssd is much more important nowadays.

I didn't try *.laz with laspy, may worth a try. I know I submitted a few RFEs for Lidar Import SOP, one for *.laz support as well as one for supporting a newer version of *.las format. If you find some of those features important, submit yours, more "votes" (RFEs) doesn't hurt ;)

 

Btw: if you replace

np.concatenate(array)

with

np.ravel(array)

the second one is 10x faster.

 

And this is even faster than the second one (but only few ms).

array.reshape(-1)

 

Edited by pezetko
better code example
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

Just a few more optimizations to native python types, and testing on bigger las file.

Python laspy SOP is 1.2 - 1.6x slower than Import Lidar SOP (1.6x if I apply scale, offset and add classification attribute what Import Lidar SOP does not perform), not bad.

This is the code:

from laspy.file import File
import numpy as np

node = hou.pwd()
geo = node.geometry()

file_path = geo.attribValue("file_path")

def load_color(inFile):
    missing_color = ["red", "green", "blue"]
    for spec in inFile.point_format:
        if spec.name in missing_color:
            missing_color.remove(spec.name)

    if missing_color:
        return None
    color = np.vstack((inFile.red, inFile.green, inFile.blue)).transpose()
    return (color / 255.0).reshape(-1) # transform from 1-255 to 0.0-1.0 range)


with File(file_path, mode='r') as inFile:
    # --- load point position
    coords = np.vstack((inFile.X, inFile.Z, inFile.Y)).transpose()  # 632 ms
    scale = inFile.header.scale	# should be already np.array
    offset = inFile.header.offset # there is no offset in simple.las example from laspy library

    # --- compute scaled and offseted positions and transform in 1d array
    pos = (coords*scale+offset).reshape(-1)     # 300 ms
    geo.setPointFloatAttribValues("P", pos.tolist())     # 2203 ms

    # --- add classification attribute
    geo.addAttrib(hou.attribType.Point, "classification", 0, False, False)
    geo.setPointFloatAttribValues("classification", inFile.Classification.tolist())     # 450 ms

    # --- load color
    colors = load_color(inFile)
    if colors is not None:
        geo.addAttrib(hou.attribType.Point, "Cd", (1.0,1.0,1.0), False, False)  # add color atttribute
        geo.setPointFloatAttribValues("Cd", colors.tolist())
        
    # --- load intensity
    geo.addAttrib(hou.attribType.Point, "intensity", 0.0, False, False)  # add intensity atttribute
    geo.setPointFloatAttribValues("intensity", (inFile.intensity / 512.0).tolist()) # transform from 1-255 to 0.0-1.0 range)

 

laspy_lidar_comparsion.png.e9a8e754e2cb048a26a83f7bdefe2995.png

Edited by pezetko
fixed inFile.x to inFile.X
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)
16 hours ago, pezetko said:

I didn't try *.laz with laspy, may worth a try. I know I submitted a few RFEs for Lidar Import SOP, one for *.laz support as well as one for supporting a newer version of *.las format. If you find some of those features important, submit yours, more "votes" (RFEs) doesn't hurt ;)

@pezetko

Awesome upadtes, thanks!

What do you mean with REFs?
References?

I can upload some more *.*las and also *.*laz to try.
I guess the wetransfer link will expire soon - shall I gonna upload on google drive or something?

And you can download very detailed LiDAR of Thueringen/Germany here: https://www.geoportal-th.de/de-de/Downloadbereiche/Download-Offene-Geodaten-Thüringen/Download-Höhendaten
Choose the orange LiDAR date, they are soooo crazy detailed... :D

And I have a 12,xGB LiDAR file here if you are interested to test with crazy amount of data.
Here is a screenshot - just a little cutout, combined with real world texture and some HF textures - and of course some quick and dirty scattered trees ^^
steiger_wip.thumb.jpg.5e9324c45b9b1bacf25deeb087020d6e.jpg

Or do you mean with votes, sending that feature to SideFX?
Already did. :)

Edited by sant0s81

Share this post


Link to post
Share on other sites
7 hours ago, sant0s81 said:

What do you mean with REFs?

Requests for Enhancement: https://www.sidefx.com/bugs/submit/

I was able to download the file from google drive but my old K4000 can't display it in Houdini.

 

  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

Just realized that there is a difference between inFile.x (x position with scale and offset applied) and inFile.X (X position without any scale or offset). So Lidar Import SOP produces scaled and offset position from las.

 

And for the laz there is laszip executable from https://rapidlasso.com/laszip/ that you have to have somewhere on your PATH. There is also c++ based laz-perf https://pypi.org/project/lazperf/

Edited by pezetko
laz
  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×