Jump to content
Sign in to follow this  
magneto

Is it smart to use 2 M.2 SSDs in raid 0 for temporary caching?

Recommended Posts

That's not amazing though - in MegaBytes it's read 625, write 375

 

EDIT: Oh I see - the Bytes and Bits are mixed up. Not sure what the real data is. One is amazing, the other is okay.

 

Yeah the ignorant author made that mistake, I just had to quote verbatim :) I wouldn't hold my breath for the prices though, so Samsung 950 pro might be a better option for people who want great speeds IMO. But Mark knows better. Maybe Intel should send one to SESI so Mark can use it for profiling.

 

But compared to your RAM that you used to test, Intel's new SSD is more than twice faster? I thought RAMs had speeds like 30GB/s, etc. I have a hard time imagining an SSD that surpasses RAMs these days. I am not knowledgeable though so I might be off.

Share this post


Link to post
Share on other sites
Guest tar

But compared to your RAM that you used to test, Intel's new SSD is more than twice faster? I thought RAMs had speeds like 30GB/s, etc. I have a hard time imagining an SSD that surpasses RAMs these days. I am not knowledgeable though so I might be off.

 

Doubt it would be twice as fast in the same test. Those are theoretical numbers that would not translate into real-world tests - it's like bicycle tires or tubes- apparently the manufacturers has a margin of error and they cannot guarantee an exact circumference of it.  If a simple thing like that is off in the real world then I can understand that a complicated technical device will be not accurate too! 

Share this post


Link to post
Share on other sites

 Maybe Intel should send one to SESI so Mark can use it for profiling.

 

I generally just get graphics cards for testing (not that I'm complaining at all here :) ). I'm not sure what a good test, or set of tests, for Houdini would be. The way Houdini cooks is that most files get loaded and then processed heavily before doing file output, and so this doesn't tend to put a lot of pressure on the disk IO (unless you're doing this over a network, which is a completely different story).

 

However, if Intel sent me one, I certainly wouldn't turn it away!

  • Like 1

Share this post


Link to post
Share on other sites

I generally just get graphics cards for testing (not that I'm complaining at all here :) ). I'm not sure what a good test, or set of tests, for Houdini would be. The way Houdini cooks is that most files get loaded and then processed heavily before doing file output, and so this doesn't tend to put a lot of pressure on the disk IO (unless you're doing this over a network, which is a completely different story).

 

However, if Intel sent me one, I certainly wouldn't turn it away!

 

Then we should pester Intel until they give in :) Is there a way to improve file read speeds using these drives from programming point of view? What about saving the viewport ready processed state in cache files separately? Can this be done so that it's only about showing the geo and directly loading it on the graphics card?

 

That would be a neat trick. I am also not sure if these drives allow reading multiple big files at once or if they would struggle like the disk based drives where you can hear the needle(?) going back and forth between 2 spots on the drive.

Share this post


Link to post
Share on other sites
Guest tar

Then we should pester Intel until they give in :) Is there a way to improve file read speeds using these drives from programming point of view? What about saving the viewport ready processed state in cache files separately? Can this be done so that it's only about showing the geo and directly loading it on the graphics card?

 

 

 

Which part of the processing are you trying to speed up? 

Share this post


Link to post
Share on other sites

Which part of the processing are you trying to speed up? 

 

Viewport processing, geometry processing. I am not in front of Houdini right now but when you profile your network, it seems like viewport processing takes a significant amount of time, sometimes even more than the operation itself. I am not sure what that means in the profiler. Maybe a catch all name for everything but the named operations?

Share this post


Link to post
Share on other sites
Guest tar

I'm not sure - all I know is the viewport must available for geometry processing to happen, but to throw the images to the display would surely be  a few mili-seconds.

 

On a related note turning Trim on the SSD drive has already brought the write speeds back to 230+ MB/sec, up from ~95 MB/sec!

 

EDIT:

Read: 260 MB/s (was 269 MB/s)

Write: 235 MB/s (was 91.3 MB/s)

Rewrite 100 MB/s (was 89.4 MB/s)

 

Read latency: 11 ms (was 13 ms)

Write latency: 18ms (was 154 ms)

Rewrite latency: 52ms (was 86 ms)

Edited by tar

Share this post


Link to post
Share on other sites

I'm not sure - all I know is the viewport must available for geometry processing to happen, but to throw the images to the display would surely be  a few mili-seconds.

 

On a related note turning Trim on the SSD drive has already brought the write speeds back to 230+ MB/sec, up from ~95 MB/sec!

 

EDIT:

Read: 260 MB/s (was 269 MB/s)

Write: 235 MB/s (was 91.3 MB/s)

Rewrite 100 MB/s (was 89.4 MB/s)

 

Read latency: 11 ms (was 13 ms)

Write latency: 18ms (was 154 ms)

Rewrite latency: 52ms (was 86 ms)

 

Is this feature not turned on by default on SSDs? If so, why?

Share this post


Link to post
Share on other sites
Guest tar

Is this feature not turned on by default on SSDs? If so, why?

 

Some older models were:

http://www.macrumors.com/2015/07/01/os-x-trim-ssd/

HEED THE DATA LOSS WARNING! There are a number of extremely buggy SSD models out there (like almost all the Samsung 8* and various Crucial models) that will permanently delete the wrong data ('https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/')when issuing TRIM commands. The bug is not dependent on queued TRIM, it happens even when using the non-queued version. All the Samsung SSDs beginning with a "8" (840 and 850, both EVO and Pro) are known to have the data-destroying TRIM bug and TRIM is blacklisted on those drives on other platforms.

 

 

Edited by tar

Share this post


Link to post
Share on other sites

What about saving the viewport ready processed state in cache files separately? Can this be done so that it's only about showing the geo and directly loading it on the graphics card?

 

That would be possible and that's basically what game engines do. For example textures are stored in a video card ready format instead of something universally readable like EXR or PNG.

 

https://en.wikipedia.org/wiki/S3_Texture_Compression

 

I don't see it as something useful in a visual effects and computer animation setting though. In a studio it would double the IOPS on the storage servers, require more space to store things, more network throughput to handle the extra data, etc. It's much better (and simple) to just handle this on the client end in Houdini even if it adds a few seconds of processing time here and there.

 

If developers are going to spend time implementing features to improve performance I'd rather see performance improvements to rendering with Mantra where computations on a project can often be measured in machine years so even minimal improvements add up to meaningful cost savings.

  • Like 1

Share this post


Link to post
Share on other sites
Guest tar

That would be possible and that's basically what game engines do. For example textures are stored in a video card ready format instead of something universally readable like EXR or PNG.

 

https://en.wikipedia.org/wiki/S3_Texture_Compression

 

 

 

How different is that from the Texture limiting options in the Texture tab of the Display Options?

Share this post


Link to post
Share on other sites

How different is that from the Texture limiting options in the Texture tab of the Display Options?

 

As far as I know Houdini is creating everything for the viewport on the fly from existing data (including textures and volumes seen in the viewport). What magneto had suggested is creating dedicated viewport ready caches so loading things and seeing them in the viewport would be faster than the way things are right now where it creates viewport stuff on the fly. I'm not a fan of the idea but it's an interesting idea.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×