Jump to content
gramx

PYRO on GPU OpenCL crash

Recommended Posts

Hi

 

I am doing some Pyro sims and have been using openCL mode which is way way faster!

 

I currently have a Quadro K4000 and we are looking at buying a TitanX to test.

 

I am having an issue where my graphics card keeps crashing whilst I am doing Pyro sims, small sims work fine. I am using H14.0.258 I also have the lastest NVidia drivers.

 

Is anyone else having issues with openCL, its so fast I want to get it working! I also used GPU-Z to check video RAM it was using just over 1 gig when it crashed, Ieaveing 2 gig available.

 

Thanks

 

Graham

Share this post


Link to post
Share on other sites

Are you monitoring your GPU temperature? When GPUs overheat they crash.

 

You may want to use a custom fan/temperature graph to make sure the GPU fan turns on before it gets too hot. I generally set my fan to be "full-on" just before it reaches 60 degrees celsius. Most graphics card tend to have conservative fan curves in their bios by default. This allows the vendor to produce a product that runs quiet and will wear out quicker.

 

MSI Afterburner is a free program on Windows that allows you to create a custom fan curve for your GPU.

https://gaming.msi.com/features/afterburner

Edited by Atom

Share this post


Link to post
Share on other sites

No, but I just did another test with the same Pyro sim. Temperature got to 58 degrees, fan speed was at 35% and RAM 1.6gig The same simulation works fine under CPU, but a lot slower. Should I try a faster fan speed?

Share this post


Link to post
Share on other sites

The same simulation works fine under CPU, but a lot slower. 

Have you tried OpenCl on CPU not GPU? I get almost the same speed increase (1.5x-3x) wether I run OpenCL on CPU or GPU

Edited by radurock

Share this post


Link to post
Share on other sites

So how do you run OpenCL on the CPU exactly?

 

I have the same Quadro4000 card at work on a 16 core machine. Can OpenCL use all 16 cores? The Quadro is really a lame card, it only has 256 cores compared to over a thousand even for a modest gaming card.

Share this post


Link to post
Share on other sites

So how do you run OpenCL on the CPU exactly?

 

I have the same Quadro4000 card at work on a 16 core machine. Can OpenCL use all 16 cores? The Quadro is really a lame card, it only has 256 cores compared to over a thousand even for a modest gaming card.

You edit the houdini environment :

 

HOUDINI_OCL_DEVICETYPE=CPU
 
HOUDINI_OCL_VENDOR=Intel® Corporation (or AMD)
 
Install OpenCL from : https://software.intel.com/en-us/articles/opencl-drivers (OpenCL™ Runtime 14.2 for Intel® CPU and Intel® Xeon Phi™ coprocessors for Windows* (64-bit)  , or Linux)
 
Then you check Houdini -> Help -> About page -> Show Details  and look for OpenCL Platform to be set to your CPU.
 
All these are in the link above posted by Graham, although it's a bit confusing finding the OpenCL runtime-only download link. 
  • Like 2

Share this post


Link to post
Share on other sites

It would be worth confirming that GPU-Z does measure OpenCL ram effectively too. Not all tools do measure hardware utilisation correctly. i.e. CPU monitors 

 

There is an env var to set that might be more accurate for this purpose. HOUDINI_OCL_REPORT_MEMORY_USE=1

Share this post


Link to post
Share on other sites

Was running some tests in Windows with Gtx980 and OpenCL smoke is super killer fast. 4GB is enough to produce some nice Fx work!

 

 

 

 

I have the same Quadro4000 card at work on a 16 core machine. Can OpenCL use all 16 cores? The Quadro is really a lame card, it only has 256 cores compared to over a thousand even for a modest gaming card.

 

Cores on the GPU aren't comparable to cores on the CPU. Compare the Compute Units. i.e the GTX 980 has 2048 cores in 16 Compute Units, therefore it has 16 'cores' for comparison purposes.

 

EDIT: ./deviceQuery shows this as

(16) Multiprocessors, (128) CUDA Cores/MP:     2048 CUDA Cores

 

 

I currently have a Quadro K4000 and we are looking at buying a TitanX to test.

 

 

 

We are trying to find out the OpenCL Nvidia driver uses 32bit or 64bit address bits. If it's limited to 32bit, as it appears by all indications, then you'll be limited to 4GB GPU ram currently. Meanwhile AMD has a 64 bit driver and can access all available GPU Ram for OpenCL. 

Edited by marty

Share this post


Link to post
Share on other sites

Will,  I have test the K4000 for pyro Sim  .

That's quite fast in bigger value of division.  But when the division value is such small.   I will crash. Also dont konw the reason. 

Edited by Rio.Dominia

Share this post


Link to post
Share on other sites

Dang, OSX left out of the loop again...?

 

 

OsX in H14 has OpenCL; AMD OpenCL on OsX is very fast, but Nvidia OpenCL on OsX is lacklustre. i.e. the smoke OpenCL, at default settings, is 22fps in Windows, and 11fps on OsX. Same machine.

Share this post


Link to post
Share on other sites

Is anyone else having issues with openCL, its so fast I want to get it working! I also used GPU-Z to check video RAM it was using just over 1 gig when it crashed, Ieaveing 2 gig available.

 

I'm not convinced GPU-Z is accurate for measuring, I've had the same issue you just described.  I've used openCL a bit for smoke sims, and have generally found that you can get about 10 million voxels per gig on the video card.  So my 660 ti with 3 gigs, can do about a 30 mil voxel sim before it starts erroring.  Also I've found that there are some weird memory leaks or something that happen sometimes when it errors. So when I get an openCL error I'll usually restart Houdini and open the scene again, and start fresh.

  • Like 1

Share this post


Link to post
Share on other sites

 Thanks for advice! When I am back at work I will test the GPU RAM again with:-   HOUDINI_OCL_REPORT_MEMORY_USE=1

Share this post


Link to post
Share on other sites

 

You edit the houdini environment :

 

HOUDINI_OCL_DEVICETYPE=CPU
 
HOUDINI_OCL_VENDOR=Intel® Corporation (or AMD)
 
Install OpenCL from : https://software.intel.com/en-us/articles/opencl-drivers (OpenCL™ Runtime 14.2 for Intel® CPU and Intel® Xeon Phi™ coprocessors for Windows* (64-bit)  , or Linux)
 
Then you check Houdini -> Help -> About page -> Show Details  and look for OpenCL Platform to be set to your CPU.

 

Thanks for the tip, this worked fine for me  :)  I also got crashes using the GPU (gtx 970 with 4GB and latest drivers) With the CPU (i7-3930K) I get even a little bit faster results. But I also get this warning message when I dive into the dop net

WARNING: Linking two modules of different data layouts!

WARNING: Linking two modules of different target triples: C:\Program Files (x86)\Common Files\Intel\OpenCL\bin\common\clbltfnshared.rtl: 'i686-pc-win32-elf' and 'x86_64-pc-win32-elf'

 

But besides that everything seems to work fine.

Share this post


Link to post
Share on other sites

Awesome news! So OpenCL in Houdini can use more than 4Gb on the GPU now?

 

It always could, i.e. AMD and Intel CPU OpenCL drivers allowed it. Nvidia is just late to the party. 

 

EDIT: Nvidia will most likely always drag it's feet as OpenCL competes with CUDA. OpenCL 1.2 is a 2011 spec!  AMD already has OpenCL 2.0 implemented and available with their latest cards, it should allow sharing of CPU memory with the GPU. i.e. out of core. once the devs code for it IIRC

Edited by marty

Share this post


Link to post
Share on other sites

 

Good eye, Marty.  If anyone has a chance to try these out successfully, please let us know.  I've tried 349.16 under Linux, which also reports being 64-bit, and so far still can't successfully allocate above 4GB in any process, even on a K6000.  We've got a bug report in with Nvidia to hopefully figure this out.  (I've also got a simple non-Houdini test app I could post that shows the same problem, if anyone has experience compiling and running such things.)

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×