Welcome to od|forum

Register now to gain access to all of our features. Once registered and logged in, you will be able to contribute to this site by submitting your own content or replying to existing content. You'll be able to customize your profile, receive reputation points as a reward for submitting content, while also communicating with other members via your own private inbox, plus much more! This message will be removed once you have signed in.

mattjakob

Velocity from animated sequence / Optical Flow

Hello,

I was wondering if it's possible to create velocity vectors from an animated sequence (texture). Ideally something similar to Optical Flow.

I can extract movement from geometry (eg. use trail to generate v vector of animated VOP) and I can do basic image analysis (eg. convolution kernel operations on coloured points using VOP) so I feel it could be kind of possible to do it but can't get my head around it.

In terms of results I'd like to create a particle system that reacts to an animated texture (eg. overlay particles on a animated texture and have them follow the movements in the video)

 

thanks!

Edited by mattjakob

Share this post


Link to post
Share on other sites

AFAIK people generate vectors in external apps (usually After Effects or Nuke with plugins like Twixor or go fancy with openCV libs or hacked ffmpeg builds), and then use that image sequence to make a velocity field. I assume that's how the amazing video below was done.

(edit)

Ah, he says in another video he uses deep flow: https://thoth.inrialpes.fr/src/deepflow/

(/edit)

Had a quick look around last night and this morning, found this javascript image processing library.  They have an implementation of 'Lucas-Kanade optical flow', I have a suspicion it'd be good starting point to port to vex:

https://github.com/inspirit/jsfeat

https://github.com/inspirit/jsfeat/blob/master/src/jsfeat_optical_flow_lk.js

 

-matt

 

 

 

 

3 people like this

Share this post


Link to post
Share on other sites
On 16.3.2017 at 1:10 AM, mestela said:

Had a quick look around last night and this morning, found this javascript image processing library.  They have an implementation of 'Lucas-Kanade optical flow', I have a suspicion it'd be good starting point to port to vex:

i don´t think this would be possible to implement as long as vex doesn´t support arbitrary sized matrices. on the other hand, it wouldn´t be much of a problem in python/numpy for which several implementations already exist. performance might be an issue, though ...

Share this post


Link to post
Share on other sites

Will love to know more about this stuff, been reading a lot and its driving me crazy.

So far I created a motion vector with Twixtor Pro and imported to houdini, used the Cd to drive the velocity in pop, got some particles following the movement but not nearly close to that awesome video

 

yes.jpg

Share this post


Link to post
Share on other sites
17 hours ago, Matsidefx said:

Hello,

Can you explain your work ?? Thanks to share

Hey Mat,

I created a motion vector with twixtor pro, here's a tutorial (

)

Wich get me something like the top image here

video_0.jpg

Then I modified to get sharper colors like the bottom one, and imported that sequence to a grid with attrib from map. Used that to drive the vel of a pop network, is not a good result but I think may be a starting point

 

Cheers

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now