Jump to content

Using External Python Modules with Compiled C++ Dependencies (Windows)


Wooshum

Recommended Posts

Hi All,

When using external python modules from a python operator I usually use:

import sys
pyModPath = node.parm('pyModPath').eval()
sys.path.append(pyModPath)

This works perfectly for pure python modules but I am having issues with modules that require compiled external dependencies, in particular i am needing to use the pyproj module: https://pypi.python.org/pypi/pyproj? 

I have tried using a precompiled python wheel when installing through PIP but it still throws an error that the DLL could not be found in HOU (have double checked that I am using and installing for 64bit). I am typically installing the pip modules in the root Python27 folder and linking to the 'Lib\site-packages' folder in that directory (for pyre python I can link to modules anywhere). Unfortunately I am stuck using Windows at the moment.

As a note: when using the pyproj module outside of Houdini (through the command line) it works fine.

Does anyone have any tips / suggestions how to get these compiled modules functioning within HOU?

Link to comment
Share on other sites

  • 1 year later...

Sorry for digging up an old post, but did you ever find a resolution to your issue here? I am seeing a similar problem with Houdini 16.5 being unable to load a compiled python package on Windows. I have verified that the package works with the system python installation, however when importing into Houdini it does not work.

Using WinDBG I have tracked this issue to a problem loading the MSVCR90.dll runtime library - this resolves correctly on the system python install but not within Houdini.

Any thoughts?

Link to comment
Share on other sites

Just following up here with the solution to this issue.

The problem is that the current version of Houdini (16.5) is built on Windows using Visual Studio 2015. The Houdini python build is also built using VS 2015. The problem arose as the default python 2.7.5 on Windows is built using VS 2008, so any precompiled binary modules, or modules built locally with the system Python 2.7.5, will be built using VS 2008. This then leads to incompatibilities which manifest in a generic DLL not found error.

The solution is to build the module locally using Houdini's python install. You will need to have VS 2015 installed, and then it simply a matter of executing setup.py on the module to be built while running in Houdini's python environment. After that everything should work correctly. If it is doing the right thing, you will see references to "Microsoft Visual Studio 14.0" in the build messages that indicate the correct compiler version is being used.

Hopefully this is helpful to others as it was very non-obvious to me at the time.

Link to comment
Share on other sites

  • 7 months later...

Hey James,

Sorry to bring up this old topic but I am trying to achieve what you did there. Building pyproj for Houdini (preferably 17 but 16.5 would work too). Using my local python 2.7 I had to install Visual Studio 2008 build tools for python 2.7 but I was able to build pyproj 1.9.5.1 as well as 1.9.6.

Using  

C:\Program Files\Side Effects Software\Houdini 16.5.439\python27\python.exe setup.py build

I can see the references to Visual Studio 14.0 but in the end I get an error.

 fatal error LNK1169: one or more multiply defined symbols found

How did you manage to build it?

 

//EDIT:

I managed to build it and use it within Houdini. So in Case anyone stumbles upon this here is what I did:

https://github.com/jswhit/pyproj/issues/149#issuecomment-436305206 this issue on github got me on the right track. The duplicate now comes as a standard with newer versions of C++. So deleting it as describes seems to be fine.

I got the source tar for 1.9.6 build it like described above with houdini 17s own python version. This creates a build folder with the compiled library inside (pyproj-1.9.6\build\lib.win-amd64-2.7\pyproj). I coped that pyproj folder to a "lib" folder next to my hipfile and use it within a python like this:

import sys
import os
libPath = os.path.dirname(hou.hipFile.path())+"/lib"

if libPath not in sys.path:
    sys.path.append(libPath)

import pyproj

 

Hope this helps somebody.

Edited by oddo
Solved it!
Link to comment
Share on other sites

  • 2 years later...

Hey Chris,

I haven't tried this but there is a chance it could work without the recompile. If I remember correctly I had to recompile it to make it work with the python 2 version that comes with Houdini.

Houdini 18.5 now apparently comes with python 3.7? (https://www.sidefx.com/docs/houdini/news/18_5/viewport.html#python-scripting)

The way how I appended the lib path is still a valid way of doing it if you don't want to install the pyproj library into your Houdini python (I wouldn't do that)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...