Running python scripts in a houdini command line

[sibarrick]

Has anyone else encountered this situation and found a solution.

We've only just started using H9 (9.5 actually) in production properly and I hit a problem with python. Our render script is a python script that we run in a houdini command line window (on windows) this has been in use for years. Unfortunately now that Houdini ships with its own python the 2 seem to conflict and my external installed one tries to recompile the houdini python source and then python crashes. The versions are the same 2.5.2 but that doesn't help when the houdini .py files get reconverted in .pyc files bad stuff happens and it doesn't like it.

Any one know how to run external python files in a houdini shell on windows without this happening?

[poltergeist]

What kind of crash? Was there some message or pythonic exception traceback?

And this is not related to .pyc-files, I believe. Can you explain how you running script and what this script imports? It's not clearly for me (sory, but my English is very bad).

Anyway, try to...

set PYTHONHOME=c:\Python25

...before running render script. (c:\Python25 - is where your local python installed)

Edited August 21, 2008 by poltergeist

[kubabuk]

Hi Simon,

I'm not sure if this's exactly it, but which compiled version of houdini are you using VC7, VC8? I spent ages before in the end I figured it out that to run external python scripts I need to stick with VC7. On VC8 and 64 since SESI uses VB2005 to compile it - python installed in your system wont run and gives you nasty errors while importing hou module. Is that what you had problems with?

Best,

kuba

PS. As far as I remember sidefx started using VC8 with H9.1 so in early versions of H9 this problem didn't exist.

[sibarrick]

Well I've uninstalled python2.5 now as there were other issues with trying to use that, so this is from python2.4 but the error is much the same....

'import site' failed; use -v for traceback

Traceback (most recent call last):

File "c:\ubin\run_renders.py", line 434, in ?

main()

File "c:\ubin\run_renders.py", line 369, in main

import string, socket

File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.186\python\lib\socket.py", line 45, in ?

import _socket

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

Also see attached image

I have to use VC8 builds as I need fbx export and mixing VC7 and VC8 causes issue with compile hdk code.

Basically I just open a houdini command line window and type the name of my render script which is "run_renders" and bang that error pops up.

The reason I thought it was related to .pyc files is because each time i run the script the .pyc files are touched. And when we did a verbose output on it they were complaining about bad magic. We force the render script to run with houdini's python install and the error went away. Trouble with that is there is no easy way to force our scripts to run with Houdini's internal python. Or at least none that I could think of.

I tried setting an enviroment var for PYTHONHOME but that didn't help.

I should also add none of my scripts do anything with hou module as all of this predates H9.

Edited August 21, 2008 by sibarrick

[poltergeist]

Look. $HFS/bin/hcmd.exe - is your houdini command line. If you enter there: "echo %PYTHONHOME%", you will get smth like this:

"C:\PROGRA~1\SIDEEF~1\HOUDIN~1.186\python"

hcmd is setting your PYTHONHOME environment variable to it's python version. Then, if you start interactive python console and type following code...

C:\Documents and Settings\mag>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print '\n'.join(__import__('sys').path)
C:\WINDOWS\system32\python25.zip
C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python\DLLs
C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python\lib
C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python\lib\plat-win
C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python\lib\lib-tk
C:\Python25
C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python
C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python\lib\site-packages
>>>

... you will see, that you run your local python version, but it uses modules from houdini's python lib. To correct this, I suggest...

C:\Documents and Settings\mag>echo %PYTHONHOME%
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.170/python

C:\Documents and Settings\mag>set PYTHONHOME=c:\Python25

C:\Documents and Settings\mag>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print '\n'.join(__import__('sys').path)
c:\Python25\lib\site-packages\pygments-0.9-py2.5.egg
c:\Python25\lib\site-packages\genshi-0.4.4-py2.5.egg
c:\Python25\lib\site-packages\pyopengl-3.0.0b1-py2.5.egg
C:\WINDOWS\system32\python25.zip
c:\Python25\DLLs
c:\Python25\lib
c:\Python25\lib\plat-win
c:\Python25\lib\lib-tk
C:\Python25
c:\Python25\lib\site-packages
c:\Python25\lib\site-packages\PIL
c:\Python25\lib\site-packages\win32
c:\Python25\lib\site-packages\win32\lib
c:\Python25\lib\site-packages\Pythonwin
c:\Python25\lib\site-packages\wx-2.8-msw-unicode
>>>

I tested your case...

C:\Documents and Settings\mag>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.170\python\lib\socket.py", line 45, in <module>
	import _socket
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
>>>

... and get your runtime error and successfully fixed it:)

C:\Documents and Settings\mag>set PYTHONHOME=c:\Python25

C:\Documents and Settings\mag>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> help(socket)
Help on module socket:

NAME
	socket

FILE
	c:\python25\lib\socket.py

DESCRIPTION
	This module provides socket operations and some related functions.
	On Unix, it supports IP (Internet Protocol) and Unix domain sockets.
	On other systems, it only supports IP. Functions specific for a
	socket are available as methods of the socket object.
...

Enjoy:)

Edited August 21, 2008 by poltergeist

[poltergeist]

And to clear about .pyc files: http://www.network-theory.co.uk/docs/pytut...ythonfiles.html

[edward]

We force the render script to run with houdini's python install and the error went away. Trouble with that is there is no easy way to force our scripts to run with Houdini's internal python. Or at least none that I could think of.

Just curious, why is that?

[sibarrick]

Just curious, why is that?

Because to do it I had to specify the entire path to the python exe in the houdini install path - I could add it to my path but then I'd have to update it for every install of Houdini and on every machine. I figured there has to be an easier way...

Oh, yeah, also my scripts currently only work in 2.4 so although I got the errors to go away the scripts still don't run because 2.5 doesn't include all the libraries I use. That's the other issue I alluded to earlier when I said I'd removed 2.5

Edited August 22, 2008 by sibarrick

[sibarrick]

Enjoy:)

I do, I do!

Thankyou.

[sibarrick]

As an extension to this I have set autorun in the registery to use my install of python - set PYTHONHOME=c:\Python25

HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor\AutoRun

and/or

HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun

This now means my scripts work for any shell - not sure if it will cause any other issues further down the line... I'll keep you posted.

[kubabuk]

Thanks guys that is very useful.