Pbr Memory Usage, Overall Flaky Mantra [solved]

[anamous]

Anyone having stability problems with Mantra in general and specifically PBR?

- More often than not, with MPBR and PBR scenes (and micropoly with GI Light set to path tracing), mantra dies while rendering, giving me a command exit code 11 (core dumped). This happens when rendering IFDs too. Sometimes re-rendering helps, sometimes it doesn't. It looks completely random to me.

- MPBR is very slow, and while PBR is much quicker, it eats memory like crazy, specially if I drive up the samples (the only way I know of that enhances the otherwise very noisy renders). The same scene would use 2GB in MPBR, and about 14GB (!) in PBR, eventually dying. I noticed that reducing the thread count helps, but only to a certain extent. Even setting the tile size to something ridiculous like 2 doesn't help - and the whole scene contains about 10.000 polys.

- Often, while rendering a sequence, Mantra will die during one of the frames. When I restart it manually, it has no problems finishing the frame and a few more, but the problem comes up later in the sequence.

All of this occurs on an 8-core MacPro running Ubuntu 64 7.10. Funnily, the same scene manages to render on WinXP 64 on the same machine, but only at a 30% load.

Obviously I don't expect any software to be 100% bug-free, but this major flakiness is currently the only thing standing between my company and a few houdini licenses. The frustrating thing being that Mantra has some really great features, and when finetuned is one the quickest renderers around (specially the GI light can make mentalray users go "huh?").

cheers

[symek]

Could you prepare some example scene and attach it here? Your problem sounds interesting. As you probably know PBR is not bullet-proof engine yet. Still lots of work has to be put there.

One thing that amaze me (although I'm far away to debate it) is that you make your decision about Houdini dependant on its PBR engine :shocking:

[anamous]

No, I'm sorry if this was too vague: the overall flakiness has nothing to do with PBR, sometimes simple scenes with the micropoly renderer stop in mid-render too. I will upload a scene by tomorrow (as soon as I'm back in office).

And in advertising as you know, Mantra has to compete with MR, Vray, Brazil etc., and some sort of GI solution (final gathering or light caches etc.) is almost always involved. Seeing that there are basically two ways of achieving this kind of effect in Mantra (GI light and PBR), PBR suddenly becomes one of the selling points, even if right now it's more of a perspective. We are actually looking at Houdini for two things: effects (DOPs, CHOPs, etc.) and rendering (bringing in animated scenes from other packages and lighting and shading in H.

I would of course love to hear opinions

[symek]

No, I'm sorry if this was too vague

No, it wasn't. All is fine . Paradoxically this surprise comes from my scepticism about Mantra GI possibilities, not yours since I hardly consider Mantra as GI capable renderer at all. All GI services except AO (which is crazy fast and stable) simply doesn't work correctly now. Photons pipe line is apparently broken due to the PBR entrance. Irradiance computes only one bounce and its rather slow. The point is... I don't care. Micro-polygon engine satisfies me more than that. I don't mean Mantra shouldn't be GI engine, I hope some one will look after proper photons support in non-PBR mode in next realize. Plus I surely understand your expectations. The point is that I really like to work with passes and standard lighting models after all so I can easily wait while things will settle down. PBR is simply not ready for me.

As to crashing Mantra in general, the only reason I can tell is threading. Generally you should avoid using more than 4 cores. Even that can be problem. Personally I tend to work on 2 cores per frame (if RAM allows). Threading is always stability/performance issue. It simply helps overcome RAM limitation but almost every engine I know deals better with fewer threads.

And last clarification: I could be wrong here, but I have an impression that you identify GI support in Mantra with VEX Global Illumination Light Shader. It's reasonable as this is the only exposed solution for GI (except PBR) which is easily reachable for a new user. But, in contrast, VEX GI itself is unreasonable! This shader hasn't changed for years now and simply doesn't perform the real power of many Mantra solutions!

The problem is that SESI doesn't supply such crucial tool in standard distribution

Edited March 16, 2008 by SYmek

[anamous]

Thanks alot for your insight.

I've been playing around with Mantra on and off for a few years now, always trying to duplicate what our MR and Brazil TDs manage to set up, so my experience is limited. If you split renderers into two camps (exreme generalization), the micropolygon renderers and the pure raytracers, I think that Mantra is the micropolygon-capable renderer with the best (theoretical) support for all the features of modern raytracers (indirect illumination, glossy stuff, quick IBL etc.) - IMO Mantra has the potential to fit right between both camps.

The GI Light is perfect for softening the lighting of a scene, it is quick, very controllable (great AO as you mentioned), and I haven't had the need for photon maps yet. In theory, if the path tracing options would finally work as advertised (supporting the options in the PBR tab of the Mantra ROP such as diffuse bounce count) this would be a great argument in favor of Mantra.

I can only speak for my small shop, and after having a long hard stare at many options, Mantra is the one that keeps wowing us. I didn't expect it to, but in our internal benchmarks (and being the renderer we know the least of) it has made hardcore Maya users ask for parts of the pipeline moved over to Houdini. If only SESI would work on its stability, IMO currently a sour point since the major revamp for 9.

Maybe my opinion isn't very popular and I'm sure I'm wrong about alot of things

[Jason]

Hi there,

I'm assuming you are using the latest 9.1 release (9.1.179) and not an aging 9.0 build; it's crucial to keep up to date with Mantra releases since there are continual bug-fixes going in there. And please be vocal about you issues on the SESI forums so that they can have the opportunity to fix your problems, which they will if you report them.

SYmek is probably right that there is some sort of threading issue which is causing Mantra to die on your end; multithreading is definitely the prime culprit for crashes in Mantra. I haven't been doing much PBR work lately but it has been rock-solid with some heavy hard surface and volume rendering we've been doing lately. Does it crash in MP mode?

My opinion of Mantra wrt PBR and GI in general is this: the architecture of Mantra seems to me to be very good. The shader set and training materials are nowhere close to adequate, however. There are so many avenues and possibilities in Mantra which are sadly undermarketed and virtually undocumented. For instance:

SYmek is concerned about multiple-bounce irradiance. Why does this not seem to exist in Mantra? The reasons are that if you don't have a your illumination samples managed by the renderer (ie, if you force your shaders to send a bunch of rays out) then if enable multiple bounces you are going to get an exponential explosion of number of rays being sent out. Fine, you might say, I'll take the hit - but then unknowing people will have dreadfully slow scenes and complain about that. PBR is the answer in Mantra to managing those illumination samples. So you might think you're trapped with MP or PBR, but this is not true either since there are ways to cross-pollinate the two contexts - see the pathtrace() function. Photon mapping with the PBR engine is far nicer than the old photon mapping engine and here you can using the PBR engine to generate the photons but access them in the MP engine.

Now take all this along with point-clouds (with pcwrite() too), volume rendering, irradiance caching, light exports, etc and I think all the possibilities can be quite flabbergasting, especially since there are not demos, documentation, tutorials or honed shaders to make this very easily accessible to average user. Even for large facilities there is barely enough talent with time to dig around and discover these great things that Mantra might be capable of. I wish that I had had the time or opportunity to look into all of these groovy things.

[anamous]

Hey Jason

SYmek is concerned about multiple-bounce irradiance. Why does this not seem to exist in Mantra? The reasons are that if you don't have a your illumination samples managed by the renderer (ie, if you force your shaders to send a bunch of rays out) then if enable multiple bounces you are going to get an exponential explosion of number of rays being sent out. Fine, you might say, I'll take the hit - but then unknowing people will have dreadfully slow scenes and complain about that. PBR is the answer in Mantra to managing those illumination samples. So you might think you're trapped with MP or PBR, but this is not true either since there are ways to cross-pollinate the two contexts - see the pathtrace() function. Photon mapping with the PBR engine is far nicer than the old photon mapping engine and here you can using the PBR engine to generate the photons but access them in the MP engine.

Does this actually work as opposed to 9.0.x? In 9.0 the PBR parameters (diffuse bounces, etc.) didn't have an effect on the pathtrace() function, which for some reason was still limited to 1 bounce.

SESI should pay one or more of the Mantra gurus (you, Mario, Jens, others) to write shaders and docs (assuming that they themselves are busy enough enhancing it), the market potential for a well-rounded easy-to-use Mantra pipeline is amazing IMHO. I agree that currently it's an unused opportunity.

[Jason]

Hey Jason

Does this actually work as opposed to 9.0.x? In 9.0 the PBR parameters (diffuse bounces, etc.) didn't have an effect on the pathtrace() function, which for some reason was still limited to 1 bounce.

Yeah, you know - in another thread here on odforce it seemed as if the pathtrace() function (used by way of the VEX Global Illumination SHOP) wasn't actually providing multiple bounces although Andrew Clinton (SESI) says it should. I got too busy to follow up on it.

SESI should pay one or more of the Mantra gurus (you, Mario, Jens, others) to write shaders and docs (assuming that they themselves are busy enough enhancing it), the market potential for a well-rounded easy-to-use Mantra pipeline is amazing IMHO. I agree that currently it's an unused opportunity.

Ah, if we only had the time:)

[symek]

SYmek is concerned about multiple-bounce irradiance.

Actually I'm not . I don't need it, but I tried to put myself in place of someone who expects it. What I really don't get is a way to treat photons in MP. I understand that I should generate photon maps in PBR and so I did. Then I was using VEX's standard photons function to open & integrate photon's samples but with no success. I mean, I was getting the image but photons were storing no reasonable data - whatever settings I tried, increasing, decreasing its number, changing filter setttings etc.

Perhaps I should dig further into it, but as Jason mentioned there are so many goodies of Mantra, and so lack of advanced Docs on it... (What does it mean one can realize analysing +300 page's pdf about rendering in mental ray coming with XSI. All you'll like to know about BSP Trees but you're afraid to ask... etc)

What I meant by saying about ridiculous of VEX GI Light Shader, is that simply applying point cloud's (pcwrite()) into it speeds up things 3-10 times! Add to it raytrace objects scope, sexy bend normals, ray weighting - all missing in present shader yet avaiable in VEX. and you have the picture.

People in studios know that and use it. Noobs are left alone with irradiance() brutal force...

Weird. SESI is mysterious as Houdini itself.

Edited March 17, 2008 by SYmek

[Ole]

Actually I'm not . I don't need it, but I tried to put myself in place of someone who expects it. What I really don't get is a way to treat photons in MP. I understand that I should generate photon maps in PBR and so I did. Then I was using VEX's standard photons function to open & integrate photon's samples but with no success. I mean, I was getting the image but photons were storing no reasonable data - whatever settings I tried, increasing, decreasing its number, changing filter setttings etc.

Perhaps I should dig further into it, but as Jason mentioned there are so many goodies of Mantra, and so lack of advanced Docs on it... (What does it mean one can realize analysing +300 page's pdf about rendering in mental ray coming with XSI. All you'll like to know about BSD Trees but you're afraid to ask... etc)

What I meant by saying about ridiculous of VEX GI Light Shader, is that simply applying point cloud's (pcwrite()) into it speeds up things 3-10 times! Add to it raytrace objects scope, sexy bend normals, ray weighting - all missing in present shader yet avaiable in VEX. and you have the picture.

People in studios know that and use it. Noobs are left alone with irradiance() brutal force...

Weird. SESI is mysterious as Houdini itself.

Thank you for bringing this up guys. I want to add my voice to the choir as I just recently had this discussion being worried about me not being able to utilise Houdini for rendering because of the lack of a quick and good looking GI sollution - or beeing able to utilise Mantra fully. I really want to go hard and heavy for Houdini, but as it is I don't have the capacity or know-how to write my own shaders. Therefore I have to either settle for shaders/sollutions that won't produce what I'm after, or send data to and from Houdini via other applications. This really annoys me because it seems so unneccessary! Jumping through hoops just to render the image - and loose data and flexibility along the way. It is nice to have the opportunity to build own shaders, but come on. It's like you have to breed own cows to get milk. A shader like mia_material (for example) for mental ray would cover 95% of my needs. I know it's controversial, but modo also has a good and fast rendering sollution even though it can't be compared to the features of Mantra.

So lights with more control (uberlights was mentioned. being able to easily color and export shadows/secondary bounces for example), shaders and GI please.

Thank you for your time.

Ole

[AdamJ]

This was brought up somewhere else but I'll echo it.. SESI should have hip examples of all those wonderfull features. A hip that you can just load and hit render and get a pretty image. With comments on the nodes, it would at least provide a start for the user.

Having the equivalent of Application Notes (PRman) but for mantra would also be awesome.

[symek]

I don't think this is so easy, and I don't expect from Mantra to give me a nice pictures with one button. Whereas PRMan Application Notes are better example I think. You simply cannot handle all specific details in features in a HIP. There is no sense in these stuff without knowing technical details of implementation. Generating photons, filtering these maps, number of way of integrating samples, dealing with PC in different spaces, detailed relation between dicing, coving, comprehensive overview of ray trace routines etc These are topic worth to clarify.

Edited March 17, 2008 by SYmek

[anamous]

ok, since I'm sick this took some time, but here it is. A very simple scene that is driving me crazy. Mind you, the speed of rendering isn't the issue (it's ok), it's the fact that as I mentioned, the high quality ROPs don't render at all (compared to for example the PBR_MP_LQ which is lightning fast but quite crappy of course).

Hopefully some of you have some hints etc.

cheers.

plasticroom.zip

[anamous]

As of 9.1.189 this issue has been resolved, the scenes now render in MP PBR perfectly. love this software.

[Jason]

As of 9.1.189 this issue has been resolved, the scenes now render in MP PBR perfectly. love this software.

Good to know!

[stevenong]

Hi,

I downloaded the file & noticed you had a lot of added parameters to your high quality micropolygon PBR ROP. For example, you added a Raytrace Motion blur parameter to the ROP which seems to defeat the purpose of using micropolygon.

I rendered with your ROP but after almost an hour and only half the frame done, I decided to put down my own, copied the relevant parameters & re-render.

I didn't set the same number of Min and Max Ray Samples (16 and 32 vs 64 and 768) but my render completed in maybe 10mins. It definitely has more noise in the image but more samples will probably be enough.

I recommend putting down a new mantra ROP and not add any parameters & see how the render goes again.

I did the render on my 1.83Ghz Core Duo laptop with 2GB RAM. From the verbose output: Memory: 597.08 MB of 634.70 MB arena size

Good luck!

Cheers!

steven