Jump to content

Is it smart to use 2 M.2 SSDs in raid 0 for temporary caching?


magneto

Recommended Posts

That's not amazing though - in MegaBytes it's read 625, write 375

 

EDIT: Oh I see - the Bytes and Bits are mixed up. Not sure what the real data is. One is amazing, the other is okay.

 

Yeah the ignorant author made that mistake, I just had to quote verbatim :) I wouldn't hold my breath for the prices though, so Samsung 950 pro might be a better option for people who want great speeds IMO. But Mark knows better. Maybe Intel should send one to SESI so Mark can use it for profiling.

 

But compared to your RAM that you used to test, Intel's new SSD is more than twice faster? I thought RAMs had speeds like 30GB/s, etc. I have a hard time imagining an SSD that surpasses RAMs these days. I am not knowledgeable though so I might be off.

Link to comment
Share on other sites

But compared to your RAM that you used to test, Intel's new SSD is more than twice faster? I thought RAMs had speeds like 30GB/s, etc. I have a hard time imagining an SSD that surpasses RAMs these days. I am not knowledgeable though so I might be off.

 

Doubt it would be twice as fast in the same test. Those are theoretical numbers that would not translate into real-world tests - it's like bicycle tires or tubes- apparently the manufacturers has a margin of error and they cannot guarantee an exact circumference of it.  If a simple thing like that is off in the real world then I can understand that a complicated technical device will be not accurate too! 

Link to comment
Share on other sites

 Maybe Intel should send one to SESI so Mark can use it for profiling.

 

I generally just get graphics cards for testing (not that I'm complaining at all here :) ). I'm not sure what a good test, or set of tests, for Houdini would be. The way Houdini cooks is that most files get loaded and then processed heavily before doing file output, and so this doesn't tend to put a lot of pressure on the disk IO (unless you're doing this over a network, which is a completely different story).

 

However, if Intel sent me one, I certainly wouldn't turn it away!

  • Like 1
Link to comment
Share on other sites

I generally just get graphics cards for testing (not that I'm complaining at all here :) ). I'm not sure what a good test, or set of tests, for Houdini would be. The way Houdini cooks is that most files get loaded and then processed heavily before doing file output, and so this doesn't tend to put a lot of pressure on the disk IO (unless you're doing this over a network, which is a completely different story).

 

However, if Intel sent me one, I certainly wouldn't turn it away!

 

Then we should pester Intel until they give in :) Is there a way to improve file read speeds using these drives from programming point of view? What about saving the viewport ready processed state in cache files separately? Can this be done so that it's only about showing the geo and directly loading it on the graphics card?

 

That would be a neat trick. I am also not sure if these drives allow reading multiple big files at once or if they would struggle like the disk based drives where you can hear the needle(?) going back and forth between 2 spots on the drive.

Link to comment
Share on other sites

Then we should pester Intel until they give in :) Is there a way to improve file read speeds using these drives from programming point of view? What about saving the viewport ready processed state in cache files separately? Can this be done so that it's only about showing the geo and directly loading it on the graphics card?

 

 

 

Which part of the processing are you trying to speed up? 

Link to comment
Share on other sites

Which part of the processing are you trying to speed up? 

 

Viewport processing, geometry processing. I am not in front of Houdini right now but when you profile your network, it seems like viewport processing takes a significant amount of time, sometimes even more than the operation itself. I am not sure what that means in the profiler. Maybe a catch all name for everything but the named operations?

Link to comment
Share on other sites

I'm not sure - all I know is the viewport must available for geometry processing to happen, but to throw the images to the display would surely be  a few mili-seconds.

 

On a related note turning Trim on the SSD drive has already brought the write speeds back to 230+ MB/sec, up from ~95 MB/sec!

 

EDIT:

Read: 260 MB/s (was 269 MB/s)

Write: 235 MB/s (was 91.3 MB/s)

Rewrite 100 MB/s (was 89.4 MB/s)

 

Read latency: 11 ms (was 13 ms)

Write latency: 18ms (was 154 ms)

Rewrite latency: 52ms (was 86 ms)

Edited by tar
Link to comment
Share on other sites

I'm not sure - all I know is the viewport must available for geometry processing to happen, but to throw the images to the display would surely be  a few mili-seconds.

 

On a related note turning Trim on the SSD drive has already brought the write speeds back to 230+ MB/sec, up from ~95 MB/sec!

 

EDIT:

Read: 260 MB/s (was 269 MB/s)

Write: 235 MB/s (was 91.3 MB/s)

Rewrite 100 MB/s (was 89.4 MB/s)

 

Read latency: 11 ms (was 13 ms)

Write latency: 18ms (was 154 ms)

Rewrite latency: 52ms (was 86 ms)

 

Is this feature not turned on by default on SSDs? If so, why?

Link to comment
Share on other sites

Is this feature not turned on by default on SSDs? If so, why?

 

Some older models were:

http://www.macrumors.com/2015/07/01/os-x-trim-ssd/

HEED THE DATA LOSS WARNING! There are a number of extremely buggy SSD models out there (like almost all the Samsung 8* and various Crucial models) that will permanently delete the wrong data ('https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/')when issuing TRIM commands. The bug is not dependent on queued TRIM, it happens even when using the non-queued version. All the Samsung SSDs beginning with a "8" (840 and 850, both EVO and Pro) are known to have the data-destroying TRIM bug and TRIM is blacklisted on those drives on other platforms.

 

 

Edited by tar
Link to comment
Share on other sites

What about saving the viewport ready processed state in cache files separately? Can this be done so that it's only about showing the geo and directly loading it on the graphics card?

 

That would be possible and that's basically what game engines do. For example textures are stored in a video card ready format instead of something universally readable like EXR or PNG.

 

https://en.wikipedia.org/wiki/S3_Texture_Compression

 

I don't see it as something useful in a visual effects and computer animation setting though. In a studio it would double the IOPS on the storage servers, require more space to store things, more network throughput to handle the extra data, etc. It's much better (and simple) to just handle this on the client end in Houdini even if it adds a few seconds of processing time here and there.

 

If developers are going to spend time implementing features to improve performance I'd rather see performance improvements to rendering with Mantra where computations on a project can often be measured in machine years so even minimal improvements add up to meaningful cost savings.

  • Like 1
Link to comment
Share on other sites

How different is that from the Texture limiting options in the Texture tab of the Display Options?

 

As far as I know Houdini is creating everything for the viewport on the fly from existing data (including textures and volumes seen in the viewport). What magneto had suggested is creating dedicated viewport ready caches so loading things and seeing them in the viewport would be faster than the way things are right now where it creates viewport stuff on the fly. I'm not a fan of the idea but it's an interesting idea.

  • Like 1
Link to comment
Share on other sites

  • 3 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...