SSD for Caching Quesions

[shawn_kearney]

Sorry for such a long post on such a trivial issue. But I would appreciate input on this topic.

I currently have a an under-utylized 256gb SSD drive that I originally bought for file caching. I'd like to try setting it up for Houdini. A small portion of the SSD is partioned for applications, so I have something like 180gb for cache. 

I got the drive way before I was using Houdini so the cache partition probably isn't sufficient for this. Nonetheless, I'd like to experiment to see if this is a worthwhile direction.

I'm running Houdini on a Dell Precision t5610 on Windows (may switch to CentOS). The SSD drive is currently attached to the system's SATA port. I also have a nice Avago LSI 6G controller installed that I have set up as a RAID 1 mirror for the two SAS 1tb/7200 drives that came with the machine. This volume is further partitioned for WindowsOS and Work files. Currently I am caching to the default project locations on the "Work" partition. The raid controller has six lanes available on two ports, two lanes occupied by the SAS drives.

1) My first question is if there would be any benefit to attaching the SSD to the RAID controller, even if not using RAID? The controller does have an onboard cache of 512mb. I don't know if that would matter. If there is some advantage, can I mix a SATA with SAS drives on the same port, or will I need to use one port for SAS and the other for SATA?

2) Second, there's a lot of gamer-driven hype over SSDs, but one thing I've learned, you can't always trust a gamer for performance advise. Are SSD's even appropriate for caching very large files in sequence? If SSD is appropriate, I may set up 4-8, 256gb SATAs SSDs in RAID 0 or get a single, large SSD on PCIe (depending on price/performance/budget). But this only if it makes sense to do so in this first place. Used SAS 15K drives are pretty inexpensive from a price/gb standpoint and reliability for this application really isn't necessary (i'd be limited to four, 2.5" HDDs, unless external).

3) Also, is it possible to have two file caches write simultaneously from a single instance of Houdini? Could I, say, fork the output into two File Cache SOPs or Alembic ROPs and set each respectively to write even frames to one volume and odd frames to another? Would it even make sense from a performance standpoint?

Depending on if Houdini writes the file before or after sending the frame back through the simulation, I can see how this could be problematic for dynamics, but what about graphs that do not require previous frames to compute? What about reading the cache back? Would the files need to be composited into one directory?

4) Last, even if I don't write caches to a SSD it would be nice to have cache files in one location so I can just dump them in bulk. I recently unloaded nearly half a terrabyte of various moldy caches from various applications. Hunting them all down manually was a serious pain. There must be some way to change the default project locations in Houdini. How would this be best accomplished so that naming conventions are maintained?

Again. Sorry so long.

Edited October 7, 2016 by shawn_kearney

[Skybar]

It mainly depends on what you're going to cache out. A flip/pyro sim usually takes up a lot of space, so caching that to an SSD would be very expensive since you would need quite a lot of empty space. It also wouldn't make much sense, since typically the computing part is what takes time and not really the writing to disk itself. Sure it would then read in faster afterwards, but I reckon the netgain in overall time saved in the end won't make it worth it. The bulk of the time will be spent simulating and then rendering any way.

For other scenarios, where you mainly cache out animation/particles/geometry/usual SOP stuff, it might make more sense. It would give you faster turnarounds while working, and you could playback most things in realtime when reading that from disk. 

So I guess it depends on what your goal is. For myself I would probably run the system and applications of an SSD, and cache everything to a bunch of harddrives on raid 0. I suppose thats how most studios work as well, where you cache everything on the network (that usually have a lot of space, = many disks in raid). Sure it WOULD be better if everything was an SSD, but I don't think its very cost-effective for what you gain. Again depending on what you do, but in general no.

Do some testing. Does it speed up your workflow caching to an SSD? If yes, is it worth the cost?

 

[malexander]

Where SSDs truly shine over RAID-based HDs is random access. They can access lots of tiny files almost instantly, whereas the HD's read heads would be going nuts moving back and forth. They're serveral orders of magnitude faster. Sequential access is fast enough over RAID, though SSDs are generally better there too.

I use a small SSD (64GB) purely for swap, just in case my 32GB is full.  It prevents the machine from completely freezing up when it goes into a swap-frenzy.

[shawn_kearney]

Well. I could not find an aqueduct external RAID solution at a price I could afford at this point in time, so I went ahead with a single 4GB Constellation ES.3 SAS that I found refurbished with a Seagate warranty for about the price of a Desktop HD. I'll get a second one at some point and just set up for a 4-way capable RAID when I figure out what kind of enclosure is best suited.

Shouldn't be too hard, just some kind of enclosure with an internal to external SAS slot cover and a PSU.