RESOLVED: Python COM Access Restricted In Houdini

[geneome]

From my last posts (1, 2) one could discern that I had some sort of goal in mind. Well, I did, but after about a week of trying to figure it out I keep hitting a wall so it looks like I'll see if yet another post might lead someone to answer my cries.

On a Vista 64 machine I have a device whose SDK is a COM dll. The dll comes in both a 32 and 64 bit version, so both are on my machine. I have a python script that can poll the data from the device and running it in IDLE with a 32 bit Python 2.5.4 and comtypes 0.6.0 I can poll data from the device - so it works! I can see everything I do on the device printed nicely in IDLE. But in Houdini (9.5.379 32 bit) with comtypes added to it's python directory, when I run the same script using either the python shell, making a python operator type, or even using the script sop, I get an error. Specifically: "AttributeError: 'module' object has no attribute [COM method]". The fact that it's throwing an error regarding an attribute in the dll makes me think that it's at least seeing the dll. This is reinforced when I turn off the device, both IDLE and Houdini post the error which says it can't find the device.

So maybe this just will not work in Houdini. Maybe throwing in Python COM modules into Houdini just pushes the intended use of Python in Houdini too far. But what I would like to understand is why everything works beautifully in IDLE but when I have all the right ingredients in Houdini it doesn't work. Could it possibly be some issue with the permissions Python has in Houdini to access other parts of my machine? Or could it be an issue like 32 bit Houdini is looking at the 64-bit dll (which doesn't make sense to me if the 32-bit Python can see the right dll and work. It's a shame I can't get it working since it always has seemed that I'm really close to getting this device working in Houdini. It seems that Houdini can feasibly get device information (1, 2) without going a beyond-my-power HDK route.

Edited April 12, 2009 by geneome

[symek]

From my last posts (1, 2) one could discern that I had some sort of goal in mind. Well, I did, but after about a week of trying to figure it out I keep hitting a wall so it looks like I'll see if yet another post might lead someone to answer my cries.

On a Vista 64 machine I have a device whose SDK is a COM dll. The dll comes in both a 32 and 64 bit version, so both are on my machine. I have a python script that can poll the data from the device and running it in IDLE with a 32 bit Python 2.5.4 and comtypes 0.6.0 I can poll data from the device - so it works! I can see everything I do on the device printed nicely in IDLE. But in Houdini (9.5.379 32 bit) with comtypes added to it's python directory, when I run the same script using either the python shell, making a python operator type, or even using the script sop, I get an error. Specifically: "AttributeError: 'module' object has no attribute [COM method]". The fact that it's throwing an error regarding an attribute in the dll makes me think that it's at least seeing the dll. This is reinforced when I turn off the device, both IDLE and Houdini post the error which says it can't find the device.

So maybe this just will not work in Houdini. Maybe throwing in Python COM modules into Houdini just pushes the intended use of Python in Houdini too far. But what I would like to understand is why everything works beautifully in IDLE but when I have all the right ingredients in Houdini it doesn't work. Could it possibly be some issue with the permissions Python has in Houdini to access other parts of my machine? Or could it be an issue like 32 bit Houdini is looking at the 64-bit dll (which doesn't make sense to me if the 32-bit Python can see the right dll and work. It's a shame I can't get it working since it always has seemed that I'm really close to getting this device working in Houdini. It seems that Houdini can feasibly get device information (1, 2) without going a beyond-my-power HDK route.

Sorry, it's me again with some lazy comments Isn't the IDLE working with different version of Python than Houdini? It is actually. Houdini's Python is compiled by SESI, while comtypes relying on ctypes you're using come for standard C distribution, right? Did you manage to run ctypes from within Houdini (possibly compiled on vc8)? I think this is a crucial issue here. If you can't do a single call with ctypes in Houdini, the problem with comtypes clearly comes from that. I can't do it now, but I suppose the easiest test with ctypes could clarify situation.

[geneome]

Sorry, it's me again with some lazy comments

Nonsense! I'm always glad to hear from you

Isn't the IDLE working with different version of Python than Houdini? It is actually. Houdini's Python is compiled by SESI, while comtypes relying on ctypes you're using come for standard C distribution, right?

The IDLE I've been using is Python 2.5.4 and Houdini uses 2.5.2. So yes, it's a different version. Houdini's python is compiled by SESI, and their python distribution comes with ctypes as a module which the comtypes module I added relies on. The one oddity is that though Houdini has a ctypes module it was was missing _ctypes.pyd so I took that from my python 2.5.4 version and put it in Houdini.x\python\lib (where a bunch of other pyd files are). Without _ctypes.pyd the script was complaining it couldn't find it (since the ctypes module _init_.py (in both Python 2.5.4 and Houdini) imports some things from it. Once I added it Houdini seemed to see _ctypes.pyd and import from it.

Did you manage to run ctypes from within Houdini (possibly compiled on vc8)? I think this is a crucial issue here. If you can't do a single call with ctypes in Houdini, the problem with comtypes clearly comes from that. I can't do it now, but I suppose the easiest test with ctypes could clarify situation.

Actually that never even crossed my mind, I'll run a ctypes test and post back. Everything I've been using has been compiled on VC8, so that shouldn't be an issue. I have run into things compiled on VC9 returning an error in Houdini, and the error usually notes that's the issue (e.g. MSVCR90.dll not found).

Edited April 10, 2009 by geneome

[geneome]

In Houdini I ran the ctypes test (Houdini 9.5.379\python\lib\ctypes\test\runtests.py) and got the following:

		find_library('c') ->  None
		find_library('m') ->  None
----------------------------------------------------------------------
Ran 349 tests in 0.141s (0 modules skipped)
Unavailable resources: SEH, printing, refcount

OK

EDIT: BTW, I needed to add _ctypes_test.pyd from my python 2.5.4 install to Houdini.x\python\lib to get the tests to work.

EDIT EDIT: I get the same result when I run the ctype test in Python 2.5.4.

Edited April 10, 2009 by geneome

[geneome]

Small update: SYmek got me thinking about Houdini's Python version 2.5.2 and the one I have this working on (2.5.4). I took a look at the release notes from the Python 2.5 releases and it seems that there were a few fixes to the ctypes module after 2.5.2 was released. This might be the issue, so when I get a chance I'm going to do some module replacement to Houdini's ctypes with a 2.5.4 version to see if it's that easy to resolve my issue. Fixed since 2.5.2 (the first one is a big red flag to me):

- Assigning methods to ctypes.Structure and ctypes.Union subclasses

after creation of the class does now work correctly. See Issue #1700288.

- Issue #3547: Fixed ctypes structures bitfields of varying integer

sizes.

- Issue #3554: ctypes.string_at and ctypes.wstring_at did call Python

api functions without holding the GIL, which could lead to a fatal

error when they failed.

- Issue #2682: ctypes callback functions no longer contain a cyclic

reference to themselves.

Edited April 11, 2009 by geneome

[geneome]

Figured it out - but this will be embarrassing on my part. And it actually makes me feel kind of silly wasting so much time trying to find work-arounds for what I thought was a bigger issue. But since I posted about it I am obligated to post the resolution:

The comtypes module in Houdini wasn't compiled! (SYmek: You may begin laughing) I had no .pyc or .pyo files anywhere. And that was it, after I compiled everything using the Python compileall module in Houdini itself (along with adding my own _ctypes.pyd) it all worked out. Now I can get Houdini's command window to print the data from the device just fine. Now I need to see if it's feasible to get this data into Houdini beyond printing to a window, but that is for another day. And if I get anythin usable out of it I'll be sure to post about it.

[symek]

Figured it out - but this will be embarrassing on my part. And it actually makes me feel kind of silly wasting so much time trying to find work-arounds for what I thought was a bigger issue. But since I posted about it I am obligated to post the resolution:

The comtypes module in Houdini wasn't compiled! (SYmek: You may begin laughing) I had no .pyc or .pyo files anywhere. And that was it, after I compiled everything using the Python compileall module in Houdini itself (along with adding my own _ctypes.pyd) it all worked out. Now I can get Houdini's command window to print the data from the device just fine. Now I need to see if it's feasible to get this data into Houdini beyond printing to a window, but that is for another day. And if I get anythin usable out of it I'll be sure to post about it.

Gosh, I wouldn't guess! It's strange since theoretically Python doesn't need compilation of pure python modules to run it. Anyways, great you solve it out.

cheers!

[geneome]

Gosh, I wouldn't guess! It's strange since theoretically Python doesn't need compilation of pure python modules to run it.

My thoughts exactly, that's why I wasn't paying any attention to what was actually in the comtypes folder. Big lesson learned on my part in this case.