Generating Volume-data From Audio

[hkrol]

Hi!

I'm a student who just discovered the power of Houdini in search of the easiest way to:

- create Volumetric 3D-Textures from sound

- Render them using Ray Marching

Houdini seems to be very suitable for this since it has audio input built in (CHOPS) and very advanced ways to render volumetric data.

What I want to do is basically taking a specific number of audio samples per frame and generate tree dimensional abstract volumes from them (in ways and transformations I still have to find out).

I think there would be many ways to do this, e.g. mapping CHOP-data along axes in object space (taken from P in a VEX network) and piping it into the opacity-output. In this case: how would I get my CHOP-data into the VEX-network?

Is this possible at all or do I have to find ways to manually generate i3d-files (for every frame of my audio data) and render them with Houdini in the end?

by the way: what I don't want in my project is to manipulate polygon-objects' vertices

Another possibility I found before discovering Houdini was to program a 3D-texture-node in maya and feeding that data into a fluid-voxel-container.

But working with nodes feels much more intuitive and faster than writing c++/python code and reloading/rewiring plugins all the time.

I would be very glad if someone could give me a hint on how to approach my 3d-audio-volumes.

- Henning

[Jason]

Yeah, there seem to be a couple of limitations making dumping of CHOP data into Volumes just a little awkward. It would seem that the Channel SOP would still need to be extended by SESI to possibly allow CHOP data to be yanked into Volumes directly.

However, I'd think that the Next Best Thing would be to take advantage of the standalone program i3dconvert, which can convert a 3d grid of points (with point attributes) to i3d files. So, in other words, you can transfer CHOP data to the point attributes a grid of points you've set up, save it out, run i3dconvert, now render the i3d using the Mantra: Image3d Volume Procedural.

If you go via this bank-of-points method, all the rest of Houdini's attribute handling tools can be used (like AttribTransfer). You may be able to get some joy from the VolumeMix SOP in "User" mode - where you could look up into a CHOP for every voxel in the Volume but you're bound to traverse every voxel -where sometimes you'd rather draw into a voxel grid.

Good luck - hope to see some results!

Jason

PS. To render with the standard Volume shaders ("basic smoke") remember to name your attribute "density".

[hkrol]

Thank you for your reply, Jason.

In the last days I was testing and experimenting a lot.

At the moment I'm having trouble converting a regular point cloud to i3d.

The point cloud is a regular grid and the points have one attribute: "density" (showing a soft sphere inside the grid)

I exported that geometry as a .pc-file which I can successfully read in again.

But after converting that file with i3dconvert (using "-t -d 64 64 64" as options) and trying to render the texture the Houdini Console pops up and says: "Error: Invalid header."

Is there an obvious error in my described process?

another question: It should be possible to sample any arbitrary deformed point cloud using a regular point cloud (via your mentioned AttribTransfer-node), shouldn't it?

- Henning

edit: it may be helpful to know that I'm using version 9.0.794 64bit on windows x64

Edited December 14, 2007 by hkrol

[andrewlowell]

Cool Project!

Are you doing ok with the point transformations / audio integration? ... since sound can be parsed into 3D space by quite a few different aespects of sound, what type of "spread" did you have in mind to form this volumetric grid? Volume / Panning / Frequency ? Or what type of volumetric puff?

[hkrol]

hi andrew.

What I plan to to is to find forms which are, from my eyes, the best representation for specific tones or instruments. They could be soft, coarse, pulsating, heavily distorted, ...

I'm still at the stage to find the best technique for rendering these clouds with varying density (maybe also color). The most promising would be a VEX-shader I found here on this forum ( http://forums.odforce.net/index.php?showto...ost&p=21435 ). I would like to simplyfy the code so there would only be values and densities adding up. But being no coder this is not that easy for me.

I haven't yet started playing around with CHOPS very much, but I'd like to get into that after this step. Isolating frequencies and such will be very specific for the music (that hasn't been chosen yet). What I _have_ done is a simple wobbling tube of the current frame's waveform but I'd very much like it to become more complex than that

I will post some results if I create anything worth watchable.

Edited December 17, 2007 by hkrol

[andrewlowell]

VERY COOL!

It's not a very charted path but don't get discouraged, Houdini can do it very well in fact. I'll post a simple example similar to what you might be talking about in a few days for you to dissect.

I'm finishing up a book on Houdini audio techniques right now, so watch for that. You're definitely looking in the right place with CHOPs and Houdini.

[andrewlowell]

Here's what I could do in a short time, hopefully it will get you started. I'd really like to see some volumetric audio effects you could come up with!

This is an array based on pan vs frequency, then copied onto particles. This is just with stereo but you might try a surround recording in order to fully visualize audio with a volumetric puff.

link to file

http://www.andrew-lowell-productions.com/a...panfreqAnim.zip

preview

http://www.andrew-lowell-productions.com/a...anfreqarray.mov

[Jason]

I exported that geometry as a .pc-file which I can successfully read in again.

But after converting that file with i3dconvert (using "-t -d 64 64 64" as options) and trying to render the texture the Houdini Console pops up and says: "Error: Invalid header."

Is there an obvious error in my described process?

Hm, I wonder. I know they changed the format of the i3d file and there was a switch to throw to convert old format to new format. Perhaps this tool is writing out the old format i3d's, is what I'm thinking - which would be a bug we should tell SESI about, if thats the case.

[hkrol]

thanks for your replies.

I'm happy to be able to work without i3d-files at the moment. I just write out my point clouds.

I have also thrown out most of the stuff in the VEX-shader I posted the link to. It suits my needs at the moment and renders quite fine.

Thank you for the example, Andrew. Your CHOP-network looks very advanced to me. I have to have a deeper look at that after the holidays.

For my project I'm just at the stage of finding the right music and developing a dramaturgic concept. I hope this doesn't take too much time as it's not very easy.

I've attached a very first test-rendering. I'm not going into such things as surround sound. In fact, mono may be enough for me. As long as the forms are visually adequate to the music they don't have to have any strict logic on how they are built.

... well I just figured out that I'm not allowed to attach an mp4-video so here it is:

http://h-krol.de/studium/test_v02.mp4

I wish you all a few relaxed days

- Henning

[andrewlowell]

Cool video!

Are you kidding, I'm still working on that part too, it will probably take another few years if I'm lucky!

For my project I'm just at the stage of finding the right music and developing a dramaturgic concept. I hope this doesn't take too much time as it's not very easy.

- Henning

Edited December 22, 2007 by andrewlowell