Storage

[Justin K]

Hey folks,

I’m building a new workstation, first personal build, and I’m struggling a bit with understanding the best way to set up high speed storage for simulation work. Now in my current pc I use a small high speed ssd (128 gb) for my boot drive, and an additional mirrored raid for storage. That raid drive does not have nearly the read write capabilities of the ssd drive though, which in the end impacts performance. I’m upgrading and I want to find the most optimized way to work with large amounts (> 1 tb) of data, in a way that doesn’t require constantly transferring files and folder structures. 

So this is what I’m struggling to grasp. I’ve been told to get the lowest latency fastest read write combo for the boot drive. So I’m getting a Octane 905P (storage up for debate) for my boot drive coupled with a threadripper processor. That all makes sense. That in and of itself can have up to 1 tb of storage, which is a lot. However, does it really make sense to work off of that drive? I frankly don’t think so. Though it would fit a lot of simulations, if I had a 14 shot sequence, with iterations, then it would probably make sense to have a live storage capacity of let’s say 10 Tbs worth of storage to hold the data....... But then your speed is compromised right? If my Houdini files are housed in the same locale as my sim and render data, then I’m at the mercy of that drive’s read and write speeds regardless of how fast my boot drive is right? I guess that’s the key here. How do I pair a high profile boot drive with a blazing fast storage solution.  Ive done research into setting up nvme raids in pcie4 slots, which could get me say 4 tbs of storage, but anything higher than that is seagate nas level storage with I don’t know much about and doesn’t seem to be the fastest way to work live.... 
I don’t have an insane budget, around 6k and also know nothing about setting up something like a home nas. But usually that seems to be for long term storage not ‘live’ storage, though I have heard about fiber optic cable connections with (10 gb/s) transfer rates- I’m all ears for insight on that. 
Well.....trying not to be ranty here, but I hope this all makes sense. Absent a render farm, the multi core threadripper (3970 or 3990) with an optane boot drive seems to be the fastest home combo for Houdini sim and rendering, now I just need to understand this elephant in the room, STORAGE. 
thanks in advance for any insight!!!

Edited July 18, 2020 by Justin K

[symek]

If you want a <10TB of fast storage you have two options:

a) either really good NAS with even better network (like QSFP+ switch and network cards or IB) -  pretty much studio like environment. 

_b) or external RAID storage over thunderbold3 like this: https://shop.westerndigital.com/g-technology/g-speed-shuttle 

The bigger one offers bandwidth comparable with SSD drives and have 24TB+.  

40Gbit/s network isn't that expensive these days, but still requires quite a lot of knowledge and maintenance effort, so AFAIK for a single computer it's questionable move.

On the other hand external RAID is suitable only as a working scratch area. It's pretty easy to lost the data on them, so they shouldn't be used as persistent storage and should be backuped with NAS/tapes.

[Justin K]

Thank you Symek, thunderbolt 3 in conjunction with the ssd g-speed shuttle would be perfect, but it seems like getting it working with an AMD system might be a bit tricky, which is unfortunate.   I guess only two motherboards  for amd cpus are thunderbolt approved, and they  trade the 40 GB network spot on the motherboard for a thunderbolt 3, and so dont have both :(.    Might seem trivial, but it makes flexibility more of an issue--if I ever wanted to run things using a network.   The usb 3.2 gen 2x2 (usbc type) is the closest equivalent (20Gbs), and is about half as fast as thunderbolt until usb4 comes out, so perhaps that will be good enough....  At any rate, that speed shuttle is beautiful--seems like it would be perfect for storing footage on location, though as you say despite redundancy probably would need to have that stuff saved elsewhere.  

 Now for the network option, can I get away with not using one of those expensive switches if I only plan on directly connecting to one pc?--and then perhaps just being able to look at files here and there on a slower wif based laptop?   The actual NAS storage seems somewhat 'reasonable' otherwise-by that I mean around 3k.   Reading this article here anyways: https://nascompares.com/2019/10/07/thunderbolt-3-vs-40gbe-nas-in-2019/   It conveniently also showcases thunderbolt, but I am not going intel for this build largely for price reasons.......

 

 

 At any rate appreciate the points though.  

I might wind up getting a high speed 2 tb ssd to work locally off of, and then get a network nas to just back stuff up to on a daily basis.  It wont be live, but at least it will be safe (enough).  Im just a bit worried about the 2 tbs.  I feel like with iterations you can run out quite quickly :(.