Houdini and Machine Learning/Computer Vison

[rayalez]

Hi!! I am interested in learning ML and computer vision. Have you heard about houdini being applied in these areas?

Or something related.

Or maybe you can imagine some interesting project idea where it can be useful?

I just want to come up with some ideas ob how can I use it for ML projects.

Any pointers/suggestions/ideas are welcome!!

[pclaes]

I don't know of much that has been done in that field directly in relation to Houdini, but it is a very interesting subject.

 

There are definitely plenty of areas that could benefit from this, but I don't know if houdini is the best platform for it. That said, it tends to be good for prototyping things quickly and making use of the existing tools and functionality before writing your own custom software if you require more speed or specific libraries.

You may also want to learn a bit of Massive to see how data is handled there. That said, Houdini 14 looks to have some of the funtionality Massive provides and then some.

 

 

A recent brief discussion that might interest you is here:

forums.odforce.net/topic/21428-procedural-intelligent-dynamic-character-animation/

 

The actual machine learning part is something you would have to figure out yourself, but everything else in regards to inputs, sensors, data processing, data structures, multi-threading and visualization, Houdini could provide for you.

 

Things to look at within houdini:

- vops (multithreading), pointclouds, cvex, volume sampling and manipulation of volume data (volume vop), this can provide information about the enviromnent or you could run your learning in a voxel environment to make use of some of the data structures like vdb for speed gains.

- sensor vops (a bit slow, but could provide you with the vision inputs as it renders a thin strip of pixels from the agents point of view) (there are help files showing examples how to use them)

- pathfinding (find shortest path sop)

- sopsolver - most likely you will need a dynamic context to loop over your data. A sopsolver allows you to loop over any kind of data.

- chops - for capturing inputs from external devices. For instance you can hook up the gimbal of an iphone so it outputs the gimbal sensor information from the phone and then capture that data in chops to drive some parameter/behavior.

- python - for scripting your own custom nodes or accessing some external data through python modules.

- HDK - for accessing all of houdini's raw functionality and programming your own custom node. (This is definitely the most complex approach as this requires an already deep understanding of houdini first. )

 

What would you like to use/apply machine learning for?

Having character learn to walk is a very practical example. Another example, I suppose, is ants learning to find the shortest path to their food. Or self-stabilizing systems (like how steam/pressure is managed in factories or cooling systems work in reactors. I have seen machine learning be applied to learning how to play super mario and make the character get to the end of the level. Or you have AI type solutions playing games like Starcraft 1 (The Berkely Overmind - this is actually a really interesting thing as under the hood it computes a high-risk map and manages resources to perform attacks - kind of like guerilla strikes attacking the enemy where they are weakest, continuously probing for new weaknesses).

 

Good luck!

[lukeiamyourfather]

The focus for Houdini is elsewhere but does have a few neat tools like Peter has pointed out. These are useful in the context of CGI where things can sit and process for a while but they are not useful for real-time applications like robotics, surveillance, autonomous vehicles, augmented reality, etc.

 

What kind of computer vision projects are you thinking about doing? Or is it rather, you want to use Houdini no matter what so what kind of projects could you do with Houdini that involve computer vision?

 

If you haven't already check out OpenCV which is intended for real-time applications and is very accessible (C++, Python, Android, lots of others).

 

http://opencv.org/about.html

 

Some fun projects from Hack A Day that use OpenCV.

 

http://hackaday.com/2014/06/20/autonomous-balloon-popping/

http://hackaday.com/2014/04/10/never-lose-your-pencil-with-oskar-on-patrol/

http://hackaday.com/2014/03/08/automated-phone-crackerapp-tester-steps-it-up-a-notch/

http://hackaday.com/2012/12/02/pixar-style-lamp-project-is-a-huge-animatronics-win/

[pclaes]

On a little side note, this Ted talk is just released and is very interesting and goes into computer learning:
http://video.ted.com/talk/podcast/2014X/None/JeremyHoward_2014X-480p.mp4

[lisux]

I always find these topics, like machine learning or fuzzy logic, quite interesting but useless in practice for vfx.

Is like all the fuzzy logic stuff in Massive, cool but a pain in production.

At the end and imperative system is the most practical solution.

Cheers

[edward]

I should note that ML can also be viewed as a (multidimensional) interpolation problem. You're given some "training examples" as points on a graph that you are then supposed to interpolate. So any of the "example-based" methods in CG can also be viewed as ML. For example, the BlendPose CHOP is given "training examples" which then allow it to "predict" data that it has never seen before. In fact, it's pretty close to a "radial basis function network" as it's known in the AI community. But in CG, it's known as "Pose-Based Deformation" because of the way it is applied (amongst other things in CG). Other examples of AI techniques applied in CG include tracking and facial retargeting.