Save out field data to text file, and more!

[jopojelly]

I am doing an experiment (research) where I want to save some simulation data to disk so I can run it through a machine learning algorithm, but I am hoping I could get some guidance or suggestions as to how to approach this.

In the end, I want to save out training examples to disk which include:

• Value at a cell and 6 neighboring cells from a scalar field at time step t  -- (pressure)
• Value of the gradients/divergence between the center cell and the 6 neighboring cells from a vector field at time step t  -- (velocity)
• Value of a cell from a scalar field at time step t + 1  -- (pressure)

In a previous project, I wrote a new SOP Operator in the hdk and used ofstream to write to disk, but it was painful and the lack of an IDE made it a slow to code. I am left wondering if there are functionalities of houdini that would make this task easier that I just don't know about. 

Basically, I want to do some simple math, save out scalar and vector field data as lines in a text file ("[a,b,c,d,e,f,g,h...]\n"), and also save cell data from the next timestep, all presumably in a SIM node attached to my Multisolver after the "gas project non divergent" node. (The data will then be loaded into a separate python program to train the Machine learner, but that's not important.)

Any, and I mean any, guidance or suggestions on how to best utilize houdini to get this data out would be much appreciated. Thanks

[symek]

Take a look at HOM interface to Houdini's geometry. Saving / loading volumes (or other geometry) into/from array/numpy array is a matter of 3 lines of code:

https://www.sidefx.com/docs/houdini/hom/hou/Volume#allVoxelsAsString

 

hth.

[petz]

i´m with szymon. using numpy might be a very good option especially sice you are using python anyways. put all values into an array and save it to disk as binary data. should be fairly fast and easy to read back into python for further processing.

 

6 hours ago, jopojelly said:

I am doing an experiment (research) where I want to save some simulation data to disk so I can run it through a machine learning algorithm

that sounds interesting! what are you using machine learning for? training samples for faster solve, building meta-models, setting up a rsm model, ...?

[millag]

You should take a look at this:
http://archive.sidefx.com/docs/hdk15.5/_h_d_k__geometry__g_e_oio.html
I think this would be the cleanest way to get your data in/out of Houdini.