Server Storage Types: The Ultimate Purchasing Guide

Server storage is simply where data at rest is stored. Data in motion is stored in RAM, but that data is lost when the server powers off for any reason, planned or unplanned. So data needs to be backed up to a storage solution that is more permanent, such as direct-attached storage (DAS), network-attached storage (NAS), cloud storage, or a storage area network (SAN). These four enterprise storage solutions will be explored in depth below.

All customers will have data that needs to be stored at rest – from operating system data to configuration files, databases, and product inventory. All this data must be stored somewhere, and it doesn’t always make sense to store it on the server alone. 

Data that is not backed up can be lost anytime due to hardware failure, malware infection, or breach. All data should be backed up regularly, preferably off-site of the server where the original data is stored. 

Also, RAID is not a backup solution. RAID is covered more in depth below, but in short, it allows you to lose one or more hard drives in most instances and still maintain access to the data through mirroring and/or parity. It can also increase the performance of the enterprise storage solution. 

“RAID is not a backup solution” is thrown around a lot and is a common saying. What it means is that if, for example, your server or NAS was hit with a ransomware attack (which works by encrypting all data with an algorithm that is impossible to crack via brute force), all the data in your RAID array would be lost just the same as if it was on a single disk.

The four server storage types are direct-attached storage (DAS), network-attached storage (NAS),  storage area networks (SAN), and cloud storage. Each has pros and cons, and a typical customer could use two or even all these types of storage.

View Table

The direct-attached server storage type (otherwise known as direct storage) is storage that is directly attached to the computer server, typically via either the SAS or SATA ports or via PCIe for NVMe storage. Most servers will require some form of direct-attached storage, usually in the form of either a hard disk drive or a solid-state drive (HDD and SSD, respectively). It is possible to run a server without direct-attached storage via, for example, mounting a network share, such as in an iPXE boot environment. Still, most servers will, at the very least, employ the direct-attached storage type for the operating system.

Hard drives are one of the cheapest and highest-capacity server storage types available. Hard disks are composed of spinning disks housed in 2.5-inch or 3.5-inch wide metal jackets to protect them. These disks, called platters, are magnetically charged, and their content is read with one or more read arms floating over the platters. A tiny device called a read head touches the platter ever so slightly and reads the data on the hard disk drive. Because these read heads touch the platters, hard disk drives are subject to shock damage, meaning they are easily damaged if dropped. 

The hard drive’s revolutions per minute (RPM) determines how fast it can seek data. Enterprise drives are typically available in three RPM speeds: 7,200 RPM, 10,000 RPM, and 15,000 RPM. The faster the rotational speed, the faster the data can be read. 

Hard drive capacity varies, starting at about 300GB and going up to over 10TB. Older, high-speed hard drives can be found as small as 73GB, and modern, high-capacity ones approach 20TB using the latest technologies. 

SATA 6, SAS 6, and SAS 12 are typical connection speeds. The number behind the interface indicates how fast the maximum data transfer speed can be. It is measured in Gb/s or gigabits per second. SATA 6, for example, operates at a maximum of 6 Gb/s. 

Hard drives break down over time because the mechanical parts that spin the hard drive and read from it wear out. But unlike SSDs, the media doesn’t wear out the more you write to it. Hard drives are an affordable way to provision a dedicated server that will spend most of its time accessing data in memory that only occasionally needs to be written to disk. 

Cost varies based on capacity and speed, but hard drives are generally very affordable. For example, a SATA 6 model hard drive with 1TB that spins at 7,200RPM starts at about $60. Even large hard drives are only a few hundred dollars for the highest capacity drives. For direct access storage, hard drives are going to be the biggest bang for your buck.

When they came out in the mid-2000s, SSDs were revolutionary. They use NAND flash memory cells and are always faster than hard drives, primarily because they do not have to “seek” the data on the disk, and latency is low. The effect is very noticeable to the user. The system boots faster and is more responsive. 

SSDs historically connected via the same connection as modern hard drives – either a SAS or SATA connector. (More on this below.) Unlike hard drives that rarely, if ever, fully saturate the bus of the SATA or SAS interface, however, SSDs fully saturate them. There is even a new SAS 24 interface for allowing 24 Gb/s transfers in addition to the traditional SATA 6, SAS 6, and SAS 12 explored above. 

So why not always use SSDs, then? Two reasons: price and durability. When dealing with small drive sizes, hard drives and SSDs are about the same price; but as capacity grows, the price increases much faster for SSDs. For example, a 4TB hard drive will cost only a couple hundred dollars, but a 4TB SSD will cost triple or quadruple that amount. 

Lastly, there is a durability issue. SSDs wear out as they are written to. The memory cells can only withstand so many writes before they wear out. This is measured in total drive writes per day (DWPD). Most drives have a 5-year warranty, so this is how many times you could write the drive’s total capacity per day for five years and expect the drive to not wear out prematurely. The bigger the drive, the more DWPD it can endure. 

Also, SSDs are made up of different kinds of flash storage or NAND memory. Single-level cell (SLC) drives last the longest, and multi-level cell (MLC) is the next in longevity and durability. Finally, we have triple-level cell (TLC), which is the least durable and least expensive but wears out the fastest. 

SSDs make a great choice when your storage requirements are a few hundred gigabytes to a few terabytes, and you won’t be writing to the drive all day long. SSDs have, by and large, replaced hard drives on new servers.

NVMe storage is a sort of sub-class of SSDs that employs a different server storage type. The problem they solve is that the SAS and SATA bus interfaces were originally designed for slow, mechanical hard drives. NVMe storage can connect via a wide range of connector types from M.2 to U.2 to U.3 to newer standards such as E1.S, also known as “ruler” SSDs. What all of these connectors have in common is that they connect over the PCIe bus, typically an x4 connection, though x2 and x8 are also possible. The maximum speeds for Gen 3 PCIe and Gen 4 PCIe are 3.5GB/s and 7 GB/s, respectively. Note these are in BYTES vs BITs, and there are 8 bits in a byte, so an x4 PCIe Gen 4 connection is capable of 56Gb/s speed! Compare that to regular SSDs that generally top out at 12 Gb/s for the fastest SAS 12 drives! 

Almost everything else said about regular SSDs applies to NVMe SSDs as well. NVMe SSDs wear out with writes like SATA or SAS SSD drives. And like SATA or SAS SSD drives, they are also made up of SLC, MLC, and TLC NAND flash memory. 

There is a specific type of NVMe drive based on a new technology developed jointly by Intel and Micron called 3D Xpoint memory. 3D Xpoint memory is under the Intel brand name, Optane Memory. Optane NVMe enterprise storage devices wear out a lot slower than traditional NAND flash and maintain their speed for the full write of the drive – meaning they don’t get slower the more data you write to them, as frequently happens with NAND-based solutions. 

Optane NVMe SSDs are prohibitively expensive, however, and can cost thousands of dollars for just a few terabytes. Really, all NVMe storage is expensive, but it is currently the most modern and fastest storage option available and is great for databases that don’t fit entirely in RAM or other frequent reads from the drive. 

NAS stands for network-attached storage and is a disk storage solution. That may sound complicated, but it really isn’t. It is just a computer server whose sole purpose is to store and serve data over the network via protocols such as Samba for Windows and NFS for Linux and Unix. In the enterprise, this server will typically be a 2U storage server with lots of disk drives and make use of custom operating systems that are just for NAS units. It is far cheaper to build your own NAS out of regular server components, but ready-made NAS units exist. NASs that are built on 2U chassis will typically employ a NAS operating system such as TrueNAS (formally known as FreeNAS), which is a FreeBSD-based solution leveraging the ZFS file system. ZFS lets the NAS take advantage of the perks of HDD and SSD types of server storage, with SSDs employed as a cache and hard disk drives making up the bulk of the storage. NAS units serve as a server storage solution that can be a very turnkey solution for bulk data storage.

Alternatively, there are custom-built Linux distros just for NASs like Open Media Vault. NASs can be used for backup, but they are used for on-site backup, not off-site backup. A properly equipped NAS can fully saturate a 1GB Ethernet interface, thanks to RAID. RAID can be and is used in direct-attached storage but is almost, if not always, used in NASs. 

There are a variety of “levels” of RAID, including RAID 0, 1, 5, 6, 10, 50, and 60. The specific attributes of each level that make it advantageous to certain use cases are discussed below, but all levels of RAID (except RAID 0 “stripe”) provide increased redundancy. This means that each level can afford the loss of one or more disk drives before any data is lost.

RAID 0 stripes data across two or more disks so that, for example, a file is broken into slices of so many KB each, and that “slice of the file” is written to each drive. For example, in a three-drive RAID 0, each file is broken into three slices, and s1 (for slice 1) is written to drive 0, s2 to drive 1, and s3 to drive 2, and the cycle continues. 

Some enterprise workstations use this level of RAID for the increased speed because each drive you add effectively doubles the read and write performance of the disk, but this comes at a terrible price that makes it rarely used on servers. If a single disk drive in the array fails, all data – no matter how many disks it is composed of – is lost in its entirety!

RAID 1 mirrors the data across two or more disks. This means that each disk in the array contains a complete copy of the data. It has very low overhead and does not require the use of much processing power. It is expensive to implement, however, because you are effectively getting half of the storage you pay for, at a minimum! 

RAID 1 arrays of more than three drives are rare because, in a three-drive RAID 1, you are only getting the capacity of one drive in the array. Each drive you add slows the write performance of the array but adds to read speeds because data can be read from multiple disks at the same time in many implementations. 

RAID 1 is super common on servers because it is easy to implement, requires few CPU resources, and makes data loss due to a mechanical or write endurance failure less likely.

RAID 5 (referred to as RAID Z1 in ZFS nomenclature) is the first modern RAID level that tries to balance out the performance increases of RAID 0 and the reliability of RAID 1. It does this through something called parity. 

Data is split into slices just like with RAID 0 and is written to three or more disk drives, but with a catch. A mathematical operation called Exclusive OR or XOR is performed on the slices, and that data (parity) is written to the last drive. In a RAID 5 array of three drives, 66% of the total drive capacity is available. For an array of n drives, RAID 5 gives the capacity of n-1 drives.

RAID 5 requires a minimum of three disks to be used, where two would be for striping and one for parity. Any single drive in the array, no matter how big, could be lost safely. 

RAID 5 arrays of more than four disks are discouraged, though, because of something called the RAID 5 Write Hole. Drives tend to wear out at about the same time, and when one drive goes out in a RAID 5, all the remaining drives have to work harder to calculate the data that is missing. This means that a second drive can often go out before the first one can be replaced because they are about the same age. 

RAID 5 gets almost the same read and write performance as RAID 0 with some redundancy, but the performance on writes is reduced because of the parity calculation. This is why hardware RAID cards were developed to aid in the parity calculations and offload the calculations from the main CPU. 

RAID 5 used to be the gold standard for servers but has been eclipsed by RAID 6 and RAID 10.

RAID 6 (referred to as RAID Z2 in ZFS nomenclature) is simple to explain when you already know about RAID 0 and RAID 5. It tries to achieve a best-of-both-worlds approach to redundancy and performance. Instead of the XOR operation being done once and creating one parity stripe, it is done twice. This means that RAID 6 requires four drives at a minimum to implement, and you get the total capacity of the array minus two drives. 

RAID 6 was first implemented to solve the write hole problem of RAID 5 with higher capacity hard disk drives and is still used much more often than RAID 5. Read performance is similar to what a four-drive RAID 0 array would be, but write performance suffers compared to RAID 0 because of the parity calculations needed for each write. 

RAID 6 is arguably the best of both worlds in terms of performance balanced with reliability, with a guaranteed failure of two drives being allowed for. This means two drives can be guaranteed to be lost in a RAID 6 array in a server, making it common to use in four-bay, 1U servers in the enterprise.