Instances and other

[sanostol]

I'm doing a lot with instances right now and I have some hard times rendering them, to be honest.

Let's say I have 5000 instances, no material applied and start the render they "instantly" start to render, great.

That's the way I like it. But as soon as I have a shader on the instances I have very long times

staring at the evaluating python message.

I started to compile my shaders, and assign the compiled ones, it does help but not very much.

on the other side I get errors like "Warning:Invalid channel vm_diplacementboud .....",

when opening scenes with the compiled shaders.

I tried to use delayed load, but it does not help to speed up the ifd generation.

So I thought maybe the material archive of the delayed load procedural does what I want.

But I do not get it to work. how do I have to use the Mantra Archive to get a valid material archive,

that I can use with the delayed load and most important does it speed up the ifd generation.

Right now I have the feeling I'm doing something wrong. I have to wait sometime 4 to 8 minutes for one ifd.

Writing the ifds for 300 frames is quite timeconsuming and there is no render yet.

another thing I'm not sure about is, is it better to compile shaders always?

I have a scene with a lot of assets, that have a lot of different shaders( most are type of the new shading preset), and when I open my scene

and want to start rendering it takes quite some time to compile all shaders. What is the best way to handle this?

when I compile them by hand I have to drop in a new one and apply all values again, quite timeconsuming.

many questions, hope You can help

best regards

Martin

[LaidlawFX]

I'm doing a lot with instances right now and I have some hard times rendering them, to be honest.

Let's say I have 5000 instances, no material applied and start the render they "instantly" start to render, great.

That's the way I like it. But as soon as I have a shader on the instances I have very long times

staring at the evaluating python message.

I started to compile my shaders, and assign the compiled ones, it does help but not very much.

on the other side I get errors like "Warning:Invalid channel vm_diplacementboud .....",

when opening scenes with the compiled shaders.

I tried to use delayed load, but it does not help to speed up the ifd generation.

So I thought maybe the material archive of the delayed load procedural does what I want.

But I do not get it to work. how do I have to use the Mantra Archive to get a valid material archive,

that I can use with the delayed load and most important does it speed up the ifd generation.

Right now I have the feeling I'm doing something wrong. I have to wait sometime 4 to 8 minutes for one ifd.

Writing the ifds for 300 frames is quite timeconsuming and there is no render yet.

Sadly it doesn't sound like your times aren't too bad. The time to call all those python calls is the trade off for being able to render your scene(Instance doesn't mean cure all it just manages the files differently often faster, but there are always trade offs). Alternatively depending on the density of the mesh you may be better off saving a thousand small "rocks" per say or "leaves" as one delayed load and it will be quicker to load at the cost of little bit more ram. You know your system so if you watching your task manager, activity monitor, etc... and it's not getting close to your ability try this.

Also just because it is delayed load and an instance doesn't necessarily means it will be faster it just means your computer will be more apt to handle it. There are two big process going on under the hood of the render engine, the geometry process which you can skip a lot of when you use delayed load and instances(as long as there is no displacement since then it has to make a copy of every instance into ram defeating the purpose of instances). This is the dicing and computation of the geometry in your scene, and the second is the shading process this runs the shader(third is the filtering and quantization of the shader, and prior to all this you can say the the compile of the scene to ifd, so four to two depending on how you want to break it down). For the geometry process make sure you do not have high poly objects in the far distance, a ten million poly object jammed into the size of pixel will call a million times depending on some shading process. Try the different render engines to see what goes faster, they are a combination of the different geometry and shading process that mantra provides. Most of my instincts say go with PBR nowadays, but if you want to cache out your occlusion as a point cloud with the irradiance function on the mantra rop with the micropolygon or raytrace engine and save the time processing you would of filtering the noise of soft area lights in PBR, it may be a good trade off, but it also takes some time to make these point clouds.

Since you are talking about making this many ifds, are you sending it to a farm? Once an ifd is done you can just pass that off you do not need to wait for them all to be done for mantra to start rendering, talk to your Pipeline person and you can make a work around for that. Depending on the how computationally intensive your ifd generation is going to be split it off into a few chunks to get them done faster.

If your more at a school level as opposed to production if you don't need to render it don't render it whether it be extra frames or un-parallaxed geometry(it looks the same since the camera doesn't change). Comp tricks do wonders.

another thing I'm not sure about is, is it better to compile shaders always?

I have a scene with a lot of assets, that have a lot of different shaders( most are type of the new shading preset), and when I open my scene

and want to start rendering it takes quite some time to compile all shaders. What is the best way to handle this?

when I compile them by hand I have to drop in a new one and apply all values again, quite timeconsuming.

If you do have the delayed load make sure you saved out your shader via the mantra archive. You will no longer be able to manipulate your shader afterwards once it is archived, but this will help be quicker in the load up into your ifd. If you do want to manipulate your shader afterwards on the object(or instance) geometry if you have one object you can just put a regular shader onto the object, just make sure that on your procedural delayed load material that you do not have it connected to your material ifd. (general statement)

Make sure you cheat your shaders alot too, do not have everything cast rays(preferably if you can't see it's use), cheating is the key, and especially do not have your instances be displaced. Make sure you uncheck true displacements on your shader and if you do not have this on your shader add it. If you say, but I want displacement only displace on the nearest objects and separate off that group. Use the environment map option on your shader to avoid actual raytracing if you can. Unchecking true displacements will make everything into a normal map.

Also make smaller shaders. The big mantra surface is cool for testing and doing smaller projects, but when being used so often as you do in a scene means it will be called millions of times for shading... that's a lot of vex. (Hopefully by the end of the week I'll post a document on this). Steal some of the old shaders from 10 if you can. There was a good methods to their madness, before the mantra surface. Those shader presets weren't intended to be used for "5000" instances I has a rough time with 20,000 learning the hard way.

Also when you say you recompile the shader by hand what are you doing? It sounds tedious. Automate your process the best you can Houdini will let you build interfaces and .otl when you are getting into big complex scenes like your talking the less button pressing the better. Each extremely large scene like this are different so the assets will change. Lately I like working in one big geometry network and force feeding everything through object merges into other geometry nodes and instances with the names relative referenced to the number of the object(_1, _2, as my instance names). So I can just duplicate the instance a hundred times with out actually having to change the names. I loose a little are direct ability, but it is a lot easier of a work flow a lot less button pushing.

Warning:Invalid channel vm_diplacementboud - as for this depending on what version of houdini 11 you are using the max shader displacement is not linked to the shader bounds correctly so re-connect it, or pull it up to the surface and manually adjust it like you need to do in renderman. currently it is set to some float default that does not change.

...hope that is enough, a little chaotic, but hopefully it becomes organized more in a few days...

[sanostol]

thanks a lot for taking the time to reply that detailed.

one question remains, how do I create a material archive, the archive ROP let me only select a geo object, what is the workflow for this?

Martin

[LaidlawFX]

thanks a lot for taking the time to reply that detailed.

one question remains, how do I create a material archive, the archive ROP let me only select a geo object, what is the workflow for this?

Martin

Whatever material that is assigned at the object level or sop level of the geo object will get archived in the .ifd. If you don't bring back in the ifd with the delayed load, then the relative reference will be saved of that material if one was assigned. This lets you say store a car with twelve materials be archived.

Welcome

[sanostol]

thanks again, mate

last night i tried a new way, and it seems to work fine. too

i removed all shaders from the instanced geometries and instead added a parameter shop_surfacepath to the instancegeo( pointcloud) pointing to the surfaceshader and now the ifd generation is blazing fast again.

in this case it works as the instances all have the same shader. I have to check if I can still override parameters by point attributes, but I guess it works, too

unfortunately I can not find or add a shop_materialpath parameter, so I can not assign a material. when i try to add one manualy it is renamed to shop_materialpath1 and i could not find the shop_materialpath, but it was late last night

on the other side the material just combines displacementshader and surfaceshader, and I really don't want to displace the instances for ram reasons. so it works fine for now

Martin

[pclaes]

If you are curious as to how it works under the hood you could also have a look at this custom instancer (you'll need to compile it):

http://www.peterclaes.be/blog/?p=151

In simple terms it will call a single procedural (thereby making the ifd extremely light). It does require some setup work. It basically steals the materials from "null objects" in the scene instead of working with ifd archives.

I'm currently quite busy moving house, but should be continuing my development half December.