Duotone self shadow

[Rvb]

I'm working on a cartoon shader for a project for my semester. When I create a new shader and add a duotone, the shadow on it looks terrible (see attachment). How can I fix this?

Rogier

[Mario Marengo]

I'm working on a cartoon shader for a project for my semester. When I create a new shader and add a duotone, the shadow on it looks terrible (see attachment). How can I fix this?

You rarely see the terminator line as harshly as you do with this "duotone" shader because typical reflectance uses irradiance, whereas the duotone shader ignores it (which is something you actually do want in this case because it's what gives you the "toon" look).

Additionally, there's a difference between cast shadows and self-shadowing. The latter is much more dependent on the accuracy of the surface representation than the former. Self shadowing is also where you'll see the effects of the Shadow Bias parameter (in the Shadows tab of a light) more strongly.

So, to illustrate, here are a couple of coarsely defined polygonal surfaces -- and I mean "coarsely" from the point of view of being able to resolve such a high frequency feature as the terminator line.

And if you just assign your duotone shader to that and press render (MP engine in these images), you'll get a mess at the terminator line. Notice however, that the cast shadow (from the cube onto the torus) is pretty smooth despite the coarse tessellation.

But you don't usually see such a bad artifact when rendering "normal" materials, like plastic, so what's going on?

The reason is that typical reflectance is based on irradiance, which increasingly attenuates the incoming light intensity as it approaches the tangent plane -- this is in contrast to the duotone shader which switches between "shaded" and "black" exactly when the incident illumination crosses the tangent plane (an instantaneous on-off switch... very hard to catch).

Here's the exact same image as above except using the usual "diffuse" illumination (in the attached hipfile, switch the Algorithm parameter between "Duotone" and "Duotone Irradiance" to compare the two approaches).

But you don't want that because it stops looking "toony". I just mentioned it to try to explain what it is you're seeing (so that you may have a hope of fixing it).

So, since the "rattiness" of the terminator is a result of insufficient geometric detail, the first thing you can try is to render all polygonal surfaces as subD's (object: Render>Geometry>Polygons As Subdivision). Implicit, primitive, and parametric surfaces will be left as they are, which is fine.

Still a little blocky (see the terminator line on the inside of the torus, just below the cube), but much better.

The little bit of the artifact that remains results from a mix of two separate issues: Raytracing Bias, and Shading Quality (or "dicing").

In this case, given the scale of the geometry relative to the camera, the raytracing bias default of 0.01 seems OK -- if it were too low you'd start seeing artifacts that look like this (rendered with a bias of 0.0001):

So what remains is to increase the shading quality (object: Render>Dicing>Shading Quality and Ray Shading Quality). When computing the subdiv surface, Mantra refines the surface a lot (a heck of a lot more than our wireframe), to the point that it decides it's sufficient for "most cases" (the usual illumination, materials, etc). But in this special case it may not be quite enough (we're seeing a much finer version of the chunky poly render above, but it's not quite fine enough yet). To refine it further, increase both the "Shading Quality" and "Ray Shading Quality" parameters (one or the other will be used depending on the render engine) until you're happy. Here it is with a quality of 2, which I think is enough to hide the remaining bits of nastiness in this case.

One last thing to notice is that we didn't touch the ROP's sampling parameters (Pixel Samples, Variance AA, etc) as this issue has nothing to do with "noise removal" at the pixel level -- it is entirely a geometry refinement problem.

HTH,

Cheers.

[Mario Marengo]

Ooops. Forgot the hip file. Here it is:

duotone.002.hip

[Rvb]

Wow thank you Mario for your detailed explanation! I will look in to it and I'm sure it will be enough!

Will show the finished project when done

[amazingjay]

Will show the finished project when done

So...? where's the final project? I'm really curious and interested.

[Rvb]

I totally forgot! I will upload the final result somewhere today. Here is a quick image .

[amazingjay]

I totally forgot! I will upload the final result somewhere today. Here is a quick image .

...we're still waiting for the final result as you promised! =)

Edited January 14, 2011 by amazingjay

[amazingjay]

...we're still waiting for the final result as you promised! =)

...and waiting.