Heightfields causing OpenCL issues on Farm

[Whatsinaname]

In H16.0.573 I created a terrain using Heightfields without caching the end result, as everything computed fairly quick. Unfortunately, that scene didn't seem to work on the farm due to an OpenCL issue, clCreateProgramWithSource() failed.

The machines on the farm do not have graphics cards, so OpenCL won't work in this environment. Therefore I debugged my scene and it seems like some Heightfield SOPs are using OpenCL. I was too lazy to inspect the individual nodes, but it was great if it was easier to see if a certain node will try using OpenCL, as this won't work on many farms.

 

 

 

Edited April 19, 2017 by Whatsinaname
typo

[Atom]

Can you try a heightfield convert at the end and then lock that node?

[lukeiamyourfather]

What hardware is used on the farm? What operating system?

[Solitude]

There's supposed to be an intel opencl fallback for machines that don't have gpus. I thought it would be automatic, but try  HOUDINI_OCL_DEVICETYPE=CPU  as an environment variable on the farm machines and see if that fixes it.

Side note though, you should probably have your stuff baked to disk if it's going to the farm, unless you're simming out the erosion and such which use opencl.

[Guest tar]

Take a look at jlait's answers from this thread:

https://www.sidefx.com/forum/topic/49073/?page=1#post-223001

 

Quote

CPU with GPU counts as a GPU device as far as OpenCL is concerned.

Ideally, your GPU-less servers will just automatically fall back to the Intel OpenCL CPU drivers. These should work on all i-series processors and the post-Phenom AMD processors

Quote

To repeat, you do *NOT* need a CPU with a GPU core.

One of the awesome things about OpenCL is that it has a CPU runtime that can run on the CPU without touching the GPU. This gets confusing when you do have a CPU with a GPU core, as you'll have two different OpenCL drivers available for the same chip. But in your case, it should be straightforward. If you are running Houdini 16.0, we include the Intel OpenCL CPU drivers with the install. So when a job that needs OpenCl runs on your servers, it should just fallback to the CPU. If it isn't, there is something else going wrong. (Look for a “Failed to load HFS OpenCL driver” on startup, or similar.)

 

[Whatsinaname]

I presume the machines on the farms are mostly Xeons, running on Linux.

[lukeiamyourfather]

42 minutes ago, Whatsinaname said:

I presume the machines on the farms are mostly Xeons, running on Linux.

There's a driver from Intel for using OpenCL on Xeon processors. Most Linux distributions don't have this in their repositories nor is it part of the kernel at this point.

https://software.intel.com/en-us/articles/opencl-drivers#latest_linux_driver

[Solitude]

Houdini 16 comes with the Intel opencl driver now.

[lukeiamyourfather]

17 minutes ago, Solitude said:

Houdini 16 comes with the Intel opencl driver now.

Is that documented somewhere? Does that include Linux and not just Windows? Asking because the evidence would seem to indicate otherwise. I'm curious is all, not flinging poo.

[Solitude]

Jeff mentions it in that thread posted above, and he talked about it on the beta forums. Otherwise this is the only real mention I know of:

http://www.sidefx.com/docs/houdini16.0/news/16/dynamics    "Built-in Intel OpenCL driver with CPU fallback for simulation on the farm."

[johner]

Yes, Houdini 16 ships with a built-in Intel OpenCL CPU driver on Linux and Windows, and should fall back to in on any machine where a GPU is not present (e.g. a render farm machine).  We have seen a few cases where library conflicts prevent it from loading, but you should see "Unable to load HFS OpenCL driver" if that's the case.

Usually you can do:

$ hgpuinfo -l
[*HFS OpenCL Platform*]   Intel(R) OpenCL
Platform Vendor           Intel(R) Corporation
Platform Version          OpenCL 1.2 LINUX
OpenCL Device             Intel(R) Xeon(R) CPU           X5650  @ 2.67GHz
OpenCL Type               CPU
Device Version            OpenCL 1.2 (Build 57)
Frequency                 2670 MHz
Compute Units             24
Device Address Bits       64
Global Memory             24102 MB
Max Allocation            6025 MB
Global Cache              256 KB
Max Constant Args         480
Max Constant Size         128 KB
Local Mem Size            32 KB
2D Image Support          16384x16384
3D Image Support          2048x2048x2048
.....

That HFS driver should always be first, so that if you do HOUDINI_OCL_DEVICETYPE=CPU it will be the device chosen.  Also:

$ hconfig -h HOUDINI_USE_HFS_OCL
HOUDINI_USE_HFS_OCL
    Set to its default value of 1, this variable tells Houdini to load
    the built-in CPU OpenCL driver that is shipped in $HFS (64-bit
    Windows and Linux only). This built-in CPU device can be selected
    using the regular OpenCL device specifications, e.g.
    HOUDINI_OCL_DEVICETYPE=CPU.
    Houdini will also fall back to using this driver if the usual OpenCL
    device selection process fails, making it safer to submit OpenCL
    jobs to a renderfarm that has no GPUs. Set this variable to 2 to
    disable this fallback mechanism, or 0 to disable the built-in device
    completely.
    On OSX this variable has no effect.

 

Edited April 19, 2017 by johner

[Whatsinaname]

This is very good to know, Thanks a lot.

 

It also seems like OpenCL issues are affecting Flipbooks (= Flipbook crashing) on some constellations (speaking about Xeons running Linux).