Author Topic: Maximizing SSDs  (Read 2985 times)

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Maximizing SSDs
« on: 2014 December 20 14:26:30 »
Is there any guide to setting up multiple SSD drives for maximum efficiency for use in PI? I've got a new Samsung EVO Pro ITB drive on order and the system now has a Samsung 512 SSD used as the root drive. With the cost of SD drives falling it's a good idea to maximize performance but knowing how is crucial. Any preference to size on the scratch drives?

Thanks,

Steve
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline rgbtxus

  • Newcomer
  • Posts: 26
Re: Maximizing SSDs
« Reply #1 on: 2014 December 20 17:05:13 »
I cannot speak to PI, but I can give you a few pointers in general.  Since I don't know what OS you are running I can't really say anything specific but all of what I am about to mention is trivial in Linux using builtin software services and I'm pretty sure there are off the shelf boxes that can make these arrays for you with little fuss for connection to other types of machines.  On the MAC you can use ThunderBolt to achieve very high transfer rates in and out of your box and USB3 follows not all that far behind.  I don't know anything about the windows world but I assume all of these things can be done -- I just do not know if its built into the OS as it is in Linux or whether you need to buy special SW to create and manage these arrays.  To get maximum speed you stripe n SSDs of capacity c and each write/read is broken up into stripes that are spread across the n drives and then written/read in parallel so you get approximately n times the throughput of a single SSD and n*c the storage.  This is RAID 0. The thing to be aware of is any error kills the whole array.  When you want high availability and high speed the usual approach is a stripe of mirrors, RAID 1+0.  The minimal configuration uses 4 drives of the same size organized as 2 pairs.  The pairs are mirrored (RAID 1) and the mirrors are striped (RAID 0).  Now any drive can fail and the array continues unscathed and un degraded in speed. So, you get 2n speed and 2c capacity but at the cost of 4c drive space.  There are numerous other raid levels but they trade speed for space and so are probably not what you are talking about.  My home server, which is more optimized for space and reliability than speed, is built on ZFS running on Linux, which while different in subtle and important ways from traditional RAID, can be thought about in terms of RAID.  I run 2 stripes (like RAID 0) of 6 discs in the zfs analogue to RAID6.  So out of each of the two stripes I can suffer a total failure of any two drives in each stripe without losing any data.  In other words I can lose any 2 of my 12 discs and many different combinations of 3 or 4 discs without data loss.  The cost for this is trading time for space.  I have 12c of storage physically, but trade 4c of that away for peace of mind.  This configuration requires calculating two parity syndromes for each block in each stripe and writing to all 12 disks to write a block of user data.  BTW I use naked Samsung EVO 1TB (not Pro) drives for all my temporary storage needs -- just pop one in a TB or USB3 adapter and I get great performance out of my laptop.  If I understand correctly, there are two basic parameters distinguishing SSDs: what percent of cells are set aside for replacement when cells wear out and what technology (SLC, TLC,MLC) is used where the difference is bits per cell(1,2,3). The top end SSDs use SLC and have a high percentage of cells set aside.  I believe the Samsung 840s, which I use, use the "worst" of the technologies, TLC.  Where worst is from the perspective of endurance.  Common estimates put P/E cycles at 1K for TLC, 3K for MLC and 100K for SLC. The 840 Pros use, MLC.  So, you do buy endurance with the Pros.  If you look at http://www.anandtech.com/show/6459/samsung-ssd-840-testing-the-endurance-of-tlc-nand you will see that a "crappy" 1TB TLC 840 non Pro you should be able to write ~ 100GiB day in and day out for 23 years before you exhaust its lifetime.  So, I think I'm good with non Pros.  In my case the frequency of writes is low so there is no need to pay a premium and/or give up storage to use the Pros over non Pros.  I will confess that I have been using non Pro SSDs as caches in my ZFS configuration for years without issues -- but the load on my server is just me.   I'm sure there are work loads where it is crucial to have the SLC devices -- it is not in the least clear to me that PI will put the kind of load on your SSDs to justify the extra cost.  But I bet the PI wizards can clue us in.  While PI sloshes a lot of data around, that is generally mediated by a lot of computation of what to store and a lot of time we spend scratching our heads trying to decide what function to apply next <G>  To give you a sense of SSD lifetime with one real world example I just looked at one of the 2 120GB SSDs (mirrored, RAID1) that hold the swap partition, root partition, user partition and ZFS cache for my main file server.  This SSD has 14198 hours = 1.621 years of power up time, it has sinked 21TiB of data and sourced 13TiB  (36GiB/day).  It has had 1 reallocated event and estimates that it has 99% of its life left.  So, for my work loads cheap SSDs seem fine.   Now this SSD is a MLC.  If I had a TLC 128Gib installed a back of the envelope calculation would put it's life at about 9 years.

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Re: Maximizing SSDs
« Reply #2 on: 2014 December 20 17:10:37 »
Thanks for the reply. I should have mentioned that I use Windows 7 Pro 64 bit. I always seem to forget about the other various systems PI works with.

-Steve
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline Alejandro Tombolini

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1267
    • Próxima Sur
Re: Maximizing SSDs
« Reply #3 on: 2014 December 21 16:43:14 »
Hi Steve, this video could be useful. 

Saludos, Alejandro.