Data-driven visualization: How to match a folder full of images to a bunch of points with attributes

[Triplett]

Hello. 

I only started using Houdini a few months ago, so this may be a little too complex of a question for this forum, but I have been spinning my wheels for too long not to try. 

Here is my goal in as simple terms as I can come up with:

I am trying to reassemble a complex image from a folder of cropped and masked pieces of that image. I also have a Json file that offers pixel coordinates for the upper left corner of each piece, the Object ID (filename for each piece) and a bunch of attributes that explain what each piece is. (Parts of a house, a tree, etc.) In the short term, I would like to use the trace node on each piece to create a simple flat piece of geometry for each piece, and copy that piece to its corresponding point. The "corresponding point" is the point that matches an "ID" attribute to the filename of each cropped image (ex: "melgaco2_1521041_etc.png"). I also have a folder of "white and alpha" mask versions of each cut out image that have the same ID numbers in the file name to use with the trace node. I did not share those here, but I can if needed. 

Below is a screenshot of the cut up image files,  and a composite of the imported Json file with cubes at the origin point of each piece over top of the original image:

 

Here are my problems:

1. I don't know how to import all of these images into Houdini. I tried a File Pattern via the TOP network, and I created 160 "work items"but I am a little lost trying to figure out how to do anything with those items since they are not geometry yet. 

2. Once I could get a detail? for every image piece via the trace node, I would need to create an attribute wrangle that would match it to a point that has the same "ID" attribute as the image file's filename. (I come from a GIS background, so I keep thinking this would be easy, but this kind of "join" is hard to find int eh documentation)

3. I am trying to make this procedural - I would like to be able to create an HDA that requires two inputs: a path to the file folder of images, and a link to the corresponding Json. I am getting lost about what "Level" everything needs to be on, especially when I started messing around with the Top Network File Pattern Node and "work items." Should each image file piece be its own object, or can I keep them as primitives or details of one object? 

4. I have barely started learning about the img environment, so I imagine I might need to be working more in the COP network, but I am struggling to understand how to "move" between environments. If anyone out there knows a tutorial that clarifies how to connect nodes between the Geo and img environments please share

Thanks for any help,

SuperviselyJson_Python_ET_v008.hipnc

melgaco2_jpg.zip

Edited August 7, 2020 by Triplett
accidentally added an extra image.

[Gorrod]

Hi, I'm not 100% certain I understand you correctly and this might be too obvious.

You basicly just want to trace a bunch of images, copy those to point positions and apply attributes given by your json file.

Since I don't have your json file and your hipfile crashes for me, here is a dummy scene with your pictures traced and assembled.

Maybe this helps?

assemblePic.zip

Edited August 8, 2020 by Gorrod

[Triplett]

Thank you Gorrod! This is a huge step forward. I completely missed the file directory parameter. I think that alone will open up a lot of opportunities for this project in the future.  

I was able to duplicate your python code for my Json file and I pointed the img directory to my mask files (rather than the corresponding images you see above). The results of the Trace are heavy, but manageable with some optimization and I can at least see the way forward. The one part of your network I duplicated, but definitely didn't understand was the HScript you added to the "image input" bar for the trace node: `points(-1, 0, "imgFile")`  - Until I looked it up here: https://www.sidefx.com/docs/houdini/expressions/points.html - (I have struggled to understand when I am supposed to use HScript, when VEX etc.) I also need to understand dependencies better. 

My next task is to pull out the scale of each image as an attribute on the primitives (maybe?) so they don't all come out extremely tiny. I also need to move the pivot to the upper left corner for each. Right now the scale is set to 1m per pixel of the original image for the points, but the carved up images come in at a single universal scale. I am going to try to create an attribute wrangle in the for each loop that uses this function:  https://www.sidefx.com/docs/houdini/vex/functions/teximport.html  ... although I wonder if I would be better off tying this in Python with this function: https://www.sidefx.com/docs/houdini/hom/hou/imageResolution.html (all advice is welcome)

You can see how wonky the scale and p ivot positions are here: 

 

I am hopeful that this .zip will not cause the same errors in the .hipnc file as before. I also made sure that the correct json was inside.

 

 

Melgaco2_Houdini_Import_Sharing.zip

[Gorrod]

Well you already know what to do. Just move/offset the pivot or point position of the point you're copying to and set the scale attribute. You can do that before copying your traced images, take a look at the instance attributes which will get picked up by the copytopoints sop.

If you choose to use teximport vex function, you don't need to put it inside the for loop since it runs over all points anyway. Both python and vex have the same result, teximport, in my case, is just way faster.

[Triplett]

Thanks Ben. I think I follow you for the most part. Transforming the pivot for each image in the forEach loop will be easy once I figure out how to use teximport to add the width and height as attributes on the point. I originally thought I would add imageResolution to the python import script at the top.

I am a total beginner with VEX, so forgive me if this looks bonkers. I am getting a little lost on the syntax when they refer to the "map" in the documentation. s@imgFile doesn't appear to be correct:

Also, since I am still trying to learn what language to use in which context, why did you need to / choose to use HScript in the trace node above?

[Gorrod]

Declaring your function as an int doesn't work. Either write:

int hasRes = teximport(s@imgFile, "texture:resolution", v@scale);

or leave out the return value variable:

teximport(s@imgFile, "texture:resolution", v@scale);

As for Hscript vs Python vs Vex, there are numerous explanations already online. This is just how I see it:

The short version is "HScript is Houdini's legacy scripting language."

Which you can mostly use like you would with python for importing, renaming files and so on. That's pretty oldschool though, so as given in the documentation, you should rather use python for that now. 

However Hscript is the go to to use as expression functions for changing parameter values on nodes, like I do with the trace SOPs "file" parameter here. You could also do that in python, but it's a bit more cumbersome.

Vex works a bit different and is mostly only used for editing geometry and geometry attributes as well as in shaders, cops and chops. For the geometry editing you could also use python, it would just be way slower since it is python and by default only single threaded. You can not use vex to change any parameters of nodes.

 

 

[Triplett]

Thanks Ben. Your advice has been extremely helpful. I made your short edit to the VEX in the attribute wrangle and I was able to get the points to recognize the scale of each image via "teximport" but unfortunately this project keeps introducing new strange  problems. The result you can see below shows that the scale is correct for some objects, but incorrect for others. Sometimes the scale is off in the X-axis, and sometimes on the Y-axis, but never both!

1. I checked one of the more obvious objects - the sky - and noticed that if I set the Y-scale to 6x with an edit node, it lined up very well. Other objects are 1/3, 1/5 of their correct scale in either the X or the Y axis.
2. Looking at the geometry spreadsheet for the point that the traced sky gets copied to, I saw that the scale was correct. (5404 x 860) - This matches what I see in the image's exif data outside of Houdini also. 
3. The Z scale (scale[2]) is set to 4 for all of the objects. I don't know where it got this number since each cropped mask image only has an x and a y resolution. 
4. I created a box object that was 5404 [x] 860 [y] and 1 [z] and it looked to be the exact correct size I needed for the sky, but as per point #1 the traced sky is 1/6 of what it should be in the Y axis. 

It is as though there was a random expression on the scale vector, but I never wrote one, there are no edit nodes in the network, and that also wouldn't explain why a handful of objects still seem like they are scaled and placed correctly.

I am attaching a zipped project here along with a few images of the spreadsheet and the VEX etc. I have a feeling it has something to do with instancing, but I can't figure it out. As always, I would be super grateful for any help you or anyone else on the board can offer.
 

 

 

Melgaco2_Houdini_Import_Sharing.zip

[Triplett]

Is it possible that this is a bug? I am not sure how to move forward if the scale in the geometry spreadsheet doesn't match the scale in the viewer.

[Gorrod]

Have you maybe tried setting every piece to the same size before copying them onto the points and scaling them. (e.g. setting the trace nodes "Scale To Size" parameter) as well as centering every piece to the origin?

As I said before I can't open your file because of some gamedev tool that isn't loading or something, so I can't be of much help here.

[Triplett]

I haven't used any of the SideFx Lab / Game Dev tools yet in this file but I do have it installed. I am not sure why that will cause an error. 

I Just tried to save out as a .hip file in the hopes that this .zip folder will work for you. I searched for how to uninstall the SideFX labs tools (game dev) without luck. It seems like the issue is with the environment file on my setup not matching yours. It is sort of annoying that this one installation makes it so hard to share things. 

Melgaco2_Houdini_Import_Sharing.zip

[Triplett]

That was it! I can't believe it was a simple checkbox. I tried it earlier before the scaling was corrected but it didn't come together so I didn't think of that box again. Thanks so much Ben.