Anyone running Houdini with 64 or 128GB of RAM?

[magneto]

Hi,

I am not looking to upgrade anytime soon even though I only have 16GB, but I was curious to see what the max memory one can build using consumer motherboards. To my surprise, it seems like 128GB of RAM can be added to non-server builds?

Are there anyone here running Houdini with 64 or 128GB of RAM? I know there are some hardcore hardware experts here (Hi Mark, Luke )

If so what mobo are you using? I found this one that supports 128GB RAM for intel cpus:

http://www.newegg.co...N82E16813130626

Do you think that's a good one?

Thanks

[malexander]

I have an ASUS X79 motherboard with 8 slots that I'm pretty happy with. It can handle up to 64GB, though I only have it populated with 32GB at the moment. Another trick you can do (especially on Linux) is to buy a cheap 120GB SSD and use that as your swap partition, so you could conceivably double your memory without remorgaging your house (or worse, taking a roommate) to do it.

[magneto]

Thanks Mark. If you have the fastest SSD, would it be comparable to RAM? It seems like that mobo I posted also has 8 slots, so AFAIK 8gb is the max you can get without paying insane price per RAM stick. But if I am not wrong you would have to pay huge prices if you go the server route or buy amount of RAM per stick like 16gb per stick, right?

Because 32GB (4x8GB) seems to be around $200-300, which doesn't seem very bad IMO.

Next year 128GB should be more readily available for us just like 64GB is now?

[malexander]

Memory prices have actually gone up since the fall because memory makers are shifting to production of phone, tablet and laptop memory, leaving a bit of a deficit for desktops. I haven't seem any 16GB DIMMs available online, but I suppose they are an inevitability. Right now, it'll cost you ~$500 to fill 64GB at the cheapest 8GB DIMMs, while a 120GB SSD can be found for about $110-140. However, even an SSD running swap won't give you the same performance as memory, simply because there is a lot of overhead involved. Still, if you do go over your memory limit, with an SSD you'll actually be able to still use your machine without issue, whereas with a hard drive you'll either have to spend the next 15 minutes trying to kill the process or reboot the machine.

I suspect when 16GB DIMMs debut, they'll occupy the same price tier that 8GB sticks did when they came out, roughly $80-100/stick. So it doesn't hurt to have a motherboard that has 16GB DIMM compatibility.

I don't have any experience with that particular MSI motherboard, so I can't comment on it specifically, but I have owned a couple in the past and they were fine.

[magneto]

Thanks Mark, it's good that you know alot about hardware. I will probably have to wait till 2015 before I build a new system.

Btw that SSD trick you are talking about, would it work if your primary HDD is an SSD where the OS resides (Windows)? Or do you really need to have a separate SSD? If you do this setup, would going over your memory become acceptable at the cost of performance? Or is it just a fail-safe like mechanism just to get you out of a out of memory incidence quickly?

Because when my memory limit is reached and I don't realize it, the windows immediately become bolded, and in my experience whenever this happens, you are over your memory limit. All sorts of screen drawing issues. Although I still didn't have a hard time killing the app doing this (not Houdini for most cases), OS still seemed responsive enough. So not sure if that's the same thing.

[Follyx]

have a look at EVGA

[magneto]

Thanks, do you mean their SR2 mobo? I think it supports up to 48 and is very pricey but I assume it has way more features than just support for large amount of memory.

[dulo]

I am using a supermicro H8QGi-LN4F with 512 GB of ram. the mainboard hat 32 ram slots and we use it with 4, 8 and 16 Gig registered ecc dimms

You have to be carefull to use registered dimms !! or you will be limited to 64 gig.

Works like a charm. No problems till now. Use it as a workstation / render node.

[magneto]

At first I thought you wrote 512 MB of RAM but then I noticed GB That's alot of memory. I assume a system like that would cost $15-20k?

That's a kick ass setup either way

Edited April 16, 2013 by magneto

[exu]

Hi , Magneto,

today, is less than 20k,

try a quote

http://www.thinkmate.com/System/SuperServer_8046B-TRF

but, you will probably need think about power consumption !

So, in the future, everything that you need will be in the clouds, or cloud like...

cheers!

[magneto]

When I crank the amount of RAM to 512, the price jumps to $14k. I think at this point it becomes if you want to keep this system for 12 years or build a brand new $3k system every 2.5 years. I would also fear the system getting devalued very fast, like when DDR4, DDR5 comes out with more capacity per stick, faster timings, etc.

[static]

We're using dell T7600 workstations with dual xeon E5-2680s and 64gigs ram and k5000 gpus. I did a rough price match on dell's site and it came shy of $8000... And you could go heavier on the ram for not that much more.

I hardly ever come close to using this amount, unless I have 5-10 houdini's open and/or I'm dedicating resources to our render farm.

[magneto]

That's a pretty good system Then I guess 32-48GB should be enough for most. I find my 16GB running out every once in a while if I don't watch it.

[dulo]

At first I thought you wrote 512 MB of RAM but then I noticed GB That's alot of memory. I assume a system like that would cost $15-20k?

That's a kick ass setup either way

it was about 4000 Euro with 64 cores ( amd abu dabi) and 20 TB harddisc. I assembled it on my own, to make it as cheap as possible.

As a base system it runs on gentoo linux and in virtualization it runs windows 8 x64 and another gentoo x64 with KVM and native pcie graphics in both virtual machines !!

Also the sata controller and the network controller are native in the VM that means 100% performance in the virtual machine.

In the vm I can also change the topology of the machine that mean the VM just has One Socket with 64 processors. That way I can install a simple windows home on a quad socket machine.

Normally you would have to buy a HPC or server windows for about 1000$ to use it on a quad socket machine ..

[Oddball]

We have a single 256gb machine here (this wasn't a purely economic decision but more like an experiment) and what can I say.. you manage to fill that up quite quickly . The productivity increase does not justify the investment though, unless you have scenes that really require it, which we don't.

[magneto]

@dulo: How did you get that system for $4k euros? That still seems very cheap compared to what you can buy here.

@Oddball: Yeah I agree, but if you can get it for $4k euros, I don't actually mind paying that, even if I am not gonna use it, because here a solid brand new system costs around $3k anyway.