Jump to content

Image sequence projection for baking onto geometry

Recommended Posts

Hi everyone,

I'm trying to solve a tricky problem in Houdini, and I could use some help! I need to project my live action plate onto some 3D dummy geometry and essentially bake down every individual frame. Essentially what I'm trying to produce is a 3D representation of my scene using the live action images as a diff map, because each projection should either overwrite or paint in new areas of the scene.

My geo matches the scene and the camera is tracked and solved from the plate and matches well. The shot doesn't include any moving objects so we don't have to worry about that.

I have experimented with the sop solver, the bake texture rop, and material layers, but I haven't quite made a breakthrough just yet - as these are concepts that I don't have very much experience with.

Any guidance or suggestions will be very welcome!

Many thanks :)

Edited by markingleukc
Link to comment
Share on other sites

4 minutes ago, Skybar said:

You could just UV Texture SOP and use the perspective from camera. Then just apply your sequence as a texture and render?

I'll try to explain better, I think it's a tricky concept to visualise the first time around.

If you imagine: for every frame of my sequence as my shot camera moves through the scene, Houdini makes a new camera which is frozen in the main camera's position in 3D space. That new camera projects its corresponding frozen frame onto the geometry, as a new layer on top of the camera projections came before it. What you would end up with (in theory) is a 3D scene with colour information applied onto the matched 3D geo.

I'm sure there must be a solution using either the SOP solver or even using hscript, as you can easily iterate actions over each frame with these.

Link to comment
Share on other sites

Ah right I see, yes you could do that in a SOP solver. Like I described, but instead apply the texture to your point color and just keep adding for every frame. It would require your geometry to be fairly highres though to not be a blobby color mess.

Link to comment
Share on other sites

what did you use to track your shot? Syntheyes does have the feature that does this already so it may be easier than setting it up in Houdini from scratch



but other than that yes, you can create point on the geometry for each pixel on the desired resolution in it's uvspace, then project color from camera on every frame and combine in Solver using your favorite math. Then save each point's value to corresponding pixel in COPs

Edited by anim
Link to comment
Share on other sites

1 hour ago, markingleukc said:

I'll try to explain better, I think it's a tricky concept to visualise the first time around

It's a compositing 101 technique for removing camera motion, and, as Anim explained SynthEyes has the feature or we do it in Nuke every day.


Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...