Using External Python Modules with Compiled C++ Dependencies (Windows)

[Wooshum]

Hi All,

When using external python modules from a python operator I usually use:

import sys
pyModPath = node.parm('pyModPath').eval()
sys.path.append(pyModPath)

This works perfectly for pure python modules but I am having issues with modules that require compiled external dependencies, in particular i am needing to use the pyproj module: https://pypi.python.org/pypi/pyproj? 

I have tried using a precompiled python wheel when installing through PIP but it still throws an error that the DLL could not be found in HOU (have double checked that I am using and installing for 64bit). I am typically installing the pip modules in the root Python27 folder and linking to the 'Lib\site-packages' folder in that directory (for pyre python I can link to modules anywhere). Unfortunately I am stuck using Windows at the moment.

As a note: when using the pyproj module outside of Houdini (through the command line) it works fine.

Does anyone have any tips / suggestions how to get these compiled modules functioning within HOU?

[Stalkerx777]

Doesn't matter compiled or not, just make sure it's in your PYTHONPATH or sys.path.append it. Most important here is compatibility of compiled module with Houdini's python interpreter. Your system Python version should be the same as Houdini's.

[Wooshum]

Thanks for that Alex. Looks like Houdini 15.5 is bundled with Python 2.7.5 (October 2015) - will install as the system version and give it a go. 

[james_grieve]

Sorry for digging up an old post, but did you ever find a resolution to your issue here? I am seeing a similar problem with Houdini 16.5 being unable to load a compiled python package on Windows. I have verified that the package works with the system python installation, however when importing into Houdini it does not work.

Using WinDBG I have tracked this issue to a problem loading the MSVCR90.dll runtime library - this resolves correctly on the system python install but not within Houdini.

Any thoughts?

[james_grieve]

Just following up here with the solution to this issue.

The problem is that the current version of Houdini (16.5) is built on Windows using Visual Studio 2015. The Houdini python build is also built using VS 2015. The problem arose as the default python 2.7.5 on Windows is built using VS 2008, so any precompiled binary modules, or modules built locally with the system Python 2.7.5, will be built using VS 2008. This then leads to incompatibilities which manifest in a generic DLL not found error.

The solution is to build the module locally using Houdini's python install. You will need to have VS 2015 installed, and then it simply a matter of executing setup.py on the module to be built while running in Houdini's python environment. After that everything should work correctly. If it is doing the right thing, you will see references to "Microsoft Visual Studio 14.0" in the build messages that indicate the correct compiler version is being used.

Hopefully this is helpful to others as it was very non-obvious to me at the time.

[oddo]

Hey James,

Sorry to bring up this old topic but I am trying to achieve what you did there. Building pyproj for Houdini (preferably 17 but 16.5 would work too). Using my local python 2.7 I had to install Visual Studio 2008 build tools for python 2.7 but I was able to build pyproj 1.9.5.1 as well as 1.9.6.

Using  

C:\Program Files\Side Effects Software\Houdini 16.5.439\python27\python.exe setup.py build

I can see the references to Visual Studio 14.0 but in the end I get an error.

 fatal error LNK1169: one or more multiply defined symbols found

How did you manage to build it?

 

//EDIT:

I managed to build it and use it within Houdini. So in Case anyone stumbles upon this here is what I did:

https://github.com/jswhit/pyproj/issues/149#issuecomment-436305206 this issue on github got me on the right track. The duplicate now comes as a standard with newer versions of C++. So deleting it as describes seems to be fine.

I got the source tar for 1.9.6 build it like described above with houdini 17s own python version. This creates a build folder with the compiled library inside (pyproj-1.9.6\build\lib.win-amd64-2.7\pyproj). I coped that pyproj folder to a "lib" folder next to my hipfile and use it within a python like this:

import sys
import os
libPath = os.path.dirname(hou.hipFile.path())+"/lib"

if libPath not in sys.path:
    sys.path.append(libPath)

import pyproj

 

Hope this helps somebody.

Edited January 24, 2019 by oddo
Solved it!

[Cynwulf]

Hi there,

Just a quick update for this one, does it require another compile in the 18.5 or is there a newer way to import/use pyproj inside Houdini?

Thanks,

Chris

[oddo]

Hey Chris,

I haven't tried this but there is a chance it could work without the recompile. If I remember correctly I had to recompile it to make it work with the python 2 version that comes with Houdini.

Houdini 18.5 now apparently comes with python 3.7? (https://www.sidefx.com/docs/houdini/news/18_5/viewport.html#python-scripting)

The way how I appended the lib path is still a valid way of doing it if you don't want to install the pyproj library into your Houdini python (I wouldn't do that)