Frequently Asked Questions for SnapRAID.

General

Setup

Configuration

Use

Recover

Tech

Performance

Known Problems

General

What is SnapRAID?

SnapRAID is an application that creates a partial backup of your disk array. If some disks in your array fail, even if they are completely broken, you can recover their contents. It provides only a partial backup because it does not allow recovery from a failure of the entire array, but only if the number of failed disks is below a predefined limit.

How is it different from RAID?

RAID solutions mirror data and propagate changes immediately to the parity data. In this sense, RAID systems are not backups because they do not allow restoration of an earlier state of the array.

SnapRAID, however, is more akin to a backup solution, allowing you to restore the state of the last backup. For example, with SnapRAID, if you accidentally delete a file, you can easily recover it.

Other advantages over traditional RAID5/6 solutions include:

RAID has advantages, such as better speed due to data striping. The best choice depends on your specific use case.

How is it different from ZFS/Btrfs snapshots and NTFS Shadow Copy?

SnapRAID creates a file-system snapshot conceptually similar to other snapshot solutions. However, its primary purpose is to recover a disk to its saved state after a complete disk failure.

SnapRAID is more akin to the RAID-Z/RAID functionality of ZFS/Btrfs. Compared to them, SnapRAID offers the following advantages:

ZFS and Btrfs have advantages, such as being real-time solutions. The best choice depends on your specific use case.

How is it different from other backup solutions?

With a typical backup solution, you need as much space as the entire disk array for the backup. With SnapRAID, you need only one additional parity disk to recover from the most common case of a single disk failure. Adding more parity disks allows recovery from additional disk failures.

Note that only a full backup allows recovery from a complete failure of the disk array, making it the preferred solution if you can afford it. SnapRAID is a good option when a full backup is not feasible.

What are the advantages of SnapRAID compared to similar solutions like unRAID, and Windows Storage Spaces?

Visit the Compare page to see a comparison with other similar solutions.

A key advantage of SnapRAID is its integrity and silent error management, which is at the same level as (or better than) ZFS and Btrfs file systems. In contrast, unRAID and Windows Storage Spaces lack integrity checks.

How much does it cost?

Nothing. SnapRAID is a free software application released under the GPLv3 license.

How can I get support?

Visit the Support page.

Does SnapRAID have a GUI?

No, SnapRAID is a command-line application. However, there is an external GUI called Elucidate, and a plugin for OpenMediaVault.

Does SnapRAID provide encryption, virtual views/storage pooling, SMART monitoring, or power control?

SnapRAID provides a basic virtual views/storage pooling solution but is also compatible with other pooling solutions.

For encryption, SMART monitoring, power control, and data recovery, you can use other tools. Here are some suggestions:

LinuxWindowsMac OS X
Encryption dm-crypt/LUKS
(tutorial)
VeraCrypt
Virtual views/storage pooling mergerfs (tutorial), Greyhole DrivePool ($)
SMART monitoring smartmontools, GSmartControl smartmontools, GSmartControl, CrystalDiskInfo smartmontools, GSmartControl, SMARTReporter
Power control hdparm hdparm
Data recovery ddrescue, safecopy, dd_rescue

Is SnapRAID suitable for backing up the home/My Documents directory?

No. If you have a limited amount of data that can be easily saved to an external USB drive or an online backup service, using SnapRAID is unnecessary. SnapRAID is best suited for backing up a very large amount of data, such as many terabytes, where a full backup copy is not feasible.

Is SnapRAID suitable for backing up the boot OS disk?

No. For the disk used to boot your machine and containing the OS files, it’s better to use another solution, such as RAID1 or a mirror copy. SnapRAID requires an operating system to run and cannot recover the OS if it is missing.

Can SnapRAID hash files without parity computation?

No, SnapRAID cannot be used solely to hash files. Its hashing is designed to work in combination with parity computation and is not optimized for hashing alone.

For a list of file verification tools, see: Wikipedia: Comparison of file verification software.

Setup

For a setup example in Ubuntu 14.04, see Zack's SnapRAID tutorial.

Do I need one or more parity disks?

As a rule of thumb, use one parity disk (RAID5) for up to four data disks, and then add one parity disk for each group of seven data disks, as shown in the table:

ParitiesData disks
1/Single Parity/RAID5 2–4
2/Double Parity/RAID6 5–14
3/Triple Parity 15–21
4/Quad Parity 22–28
5/Penta Parity 29–35
6/Hexa Parity 36–42

This table accounts for the fact that more parity levels help recover from an unsynced array and that disk failures may be correlated if all disks are in the same environment.

Multiple failures can occur. The worst case reported in SnapRAID’s history is a four-disk failure, successfully recovered using four parities.

For more information on parity levels and disk failure rates, see: Why RAID 5 stops working in 2009, Why RAID 6 stops working in 2019, Does RAID 6 stop working in 2019?, Triple-Parity RAID and Beyond, How long do disk drives last?.

Which file system is recommended for SnapRAID?

For data disks, consider these suggestions:

DataResult
Any OK. You can generally use any file system you prefer, with some exceptions noted below.
Btrfs OK. Ensure SnapRAID is built with the libblkid library for UUID support. You will be warned at each run if this is not the case. Note that multiple Btrfs snapshots are not supported.
ZFS Partial. UUID support is unavailable, but core SnapRAID functionalities remain usable. Note that multiple ZFS snapshots are not supported.
FAT Uncertain. There are reports of duplicate inode cases. NTFS is significantly faster and recommended instead.
ReFS No. ReFS uses 128-bit inodes, which SnapRAID does not yet support.

For parity disks, consider these suggestions:

ParityResult
ext4 OK if disks are not larger than 16 TB. The parity is stored in a single large file, and ext4 has a 16 TB file size limit.
XFS OK
Btrfs OK. Ensure SnapRAID is built with the libblkid library for UUID support. You will be warned at each run if this is not the case.
JFS No. It does not support the fallocate() command needed to allocate parity files. It also does not optimally implement the posix_fadvise() command.
NTFS OK in Windows. Not supported in Linux, as it does not support the fallocate() command needed to allocate parity files. You can use it if the parity disk contains only the parity file.
FAT No. Files larger than 4 GB cannot be stored. It is also not journaled. Use NTFS instead.
HFS+ OK

In Linux, to maximize space for the parity file system, format it with the -m 0 -T largefile4 options, like:

mkfs.ext4 -m 0 -T largefile4 DEVICE

On an 8 TB disk, this can save about 400 GB. It is also expected to be as fast as, or faster than, the default.

Which hardware configuration is recommended?

For optimal performance, connect all disks via SATA, not USB.

See Raj's Prototype Builds for examples of hardware setups.

For choosing hard drives, refer to Backblaze's What Hard Drive Should I Buy? and Backblaze's Enterprise Drives: Fact or Fiction?.

Is a multi-core CPU recommended?

No. SnapRAID is typically not CPU-bound; the limiting factor is disk read performance. A single-core CPU is usually sufficient.

For example, in my system, I achieve a combined read speed of 400 MiB/s, but SnapRAID can hash at 12,000 MiB/s on a single core.

Is a 64-bit operating system recommended?

Yes. SnapRAID performs better on a 64-bit operating system, as it can access more than 4 GiB of memory and use faster hashing and parity algorithms. However, for less than 30 TB of data, it performs well on a 32-bit operating system too.

How much memory does SnapRAID require to run?

Approximately, SnapRAID requires 1 GiB of memory for every 16 TB of data.

If you have insufficient memory, the recommended solution is to install more memory.

Alternatively, increase the blocksize option in the configuration file. Memory usage is inversely proportional to the selected block size. For example, using "blocksize 512" instead of the default 256 halves memory usage. However, a larger block size increases wasted space in parity files.

Running with memory swapped to disk is not recommended, as it significantly slows performance.

Should I check the SMART attributes of my disks?

Yes. Approximately 20% of disks fail within four years, as reported by Backblaze's How long do disk drives last?. Monitoring SMART attributes can help anticipate failures and prevent data loss.

If any of the SMART attributes Reallocated_Sector_Ct (5), Reported_Uncorrect (187), Command_Timeout (188), Current_Pending_Sector (197), or Offline_Uncorrectable (198) are non-zero, replace the disk, as recommended by Backblaze's Hard Drive SMART Stats.

See also Google's Failure trends in a large disk drive population, which found:

After the first reallocation, drives are
over 14 times more likely to fail within
60 days than drives without reallocation
counts.
After the first probational event, drives
are 16 times more likely to fail within
60 days than drives with zero probational
counts.
After the first offline reallocation, drives have over
21 times higher chances of failure within 60 days than
drives without offline reallocations.

These findings recommend replacing a disk after the first error in the SMART attributes Reallocated_Sector_Ct (5), Current_Pending_Sector (197), or Offline_Uncorrectable (198).

Is it possible to use SnapRAID in a dual-boot configuration, such as Windows and Linux?

Yes, SnapRAID can be used in a dual-boot configuration, regardless of the OS combination.

It is recommended to run SnapRAID from only one OS. While it may work if run from both OSes, this scenario is not thoroughly tested and risks issues due to different SnapRAID versions, configurations, or content files.

Configuration

What is the SnapRAID 'content' file?

The content file stores the list of all files in your array, including their checksums, timestamps, and other necessary information.

This file will be approximately a few GiB in size, depending on the array’s size. For 10 TB of data, the content file will be about 500 MiB.

What are the SnapRAID 'parity' and 'N-parity' files?

These files store the parity data used for recovery.

Their size grows to match the largest amount of data stored on any single disk in the array.

How do I configure SAMBA to share the pool directory?

To configure SAMBA to share the pool directory (e.g., /pool), add the following options to your /etc/samba/smb.conf:

# In the global section of smb.conf
unix extensions = no

# In the share section of smb.conf
[pool]
comment = Pool
path = /pool
read only = yes
guest ok = yes
wide links = yes

How can I use more disks than available drive letters in Windows?

Use Volume Mount Points to mount a disk in a directory without using a drive letter, similar to Linux.

Use

What are the most important steps to prevent data loss?

To minimize the risk of data loss, follow these steps:

How can I add a data disk to an existing array?

To add a new data disk to the array, add the new "disk" option in the configuration file, then run a "sync" command:

snapraid sync

If the disk is empty, the "sync" command will complete immediately.

How can I remove a data disk from an existing array?

To remove a data disk from the array, follow these steps:

The array will then have no reference to the removed disk.

How can I add or remove a parity disk in an existing array?

To add a new parity level, add the appropriate "N-parity" option in the configuration file, then run the "sync" command with the "-F, --force-full" option:

snapraid -F sync

The "-F" option instructs SnapRAID to recompute the full parity.

During this process, your existing parity remains unmodified, ensuring continued protection (note that this applies only from version 11.0).

To remove a parity level, remove the highest "N-parity" option from the configuration file and delete the parity file.

Note that after removing a parity file, it cannot be reused, as it becomes outdated after the next "sync" command.

How can I verify a single data disk?

The "scrub" command verifies the entire array and cannot be limited to a single data disk. To verify only one data disk, use the "check" audit command with the disk specified:

snapraid check -a -d DISK_NAME

This command verifies the checksums of all files on the specified disk. Other data and parity disks are not read or verified.

How can I replace a data disk?

If the disk is lost, refer to the recovery section in the manual. Otherwise, copy all files to the new disk, maintaining the same directory structure and names.

In Linux, use the command:

cp -av /from_dir/. /to_dir

WARNING: If using 'rsync' or 'mc' commands, ensure they are recent versions. 'rsync' must be at least version 3.1.0 (2013), and 'mc' at least version 4.8.19 (2017). Older versions may not copy sub-second timestamps.

In Windows, use the command:

robocopy F:\from_dir T:\to_dir /e /copyall

Update the SnapRAID configuration to point to the new mount point, then run a "diff" command to ensure everything is correct:

snapraid diff

If "diff" reports "added" or "removed" files, you may have copied files with a different directory structure. Correct this until "diff" reports only "equal", "restored", or "copied" files.

To verify that files were copied correctly, run a "check" audit command on the replaced disk to verify checksums:

snapraid check -a -d DISK_NAME

Finally, run "sync" to update the SnapRAID state. This command will complete almost immediately:

snapraid sync

How can I replace a parity disk?

If the disk is still accessible, ensure the array is synced by running:

snapraid sync

Copy the parity file to the new disk and update the parity location in the configuration file.

Run "sync" again to make SnapRAID recognize the new disk. This command will complete almost immediately:

snapraid sync

In Linux, a good copy command is ddrescue. For example:

ddrescue /OLD_DISK/parity /NEW_DISK/parity /tmp/copy.log

This copies the parity file from the old to the new location, continuing despite errors. To retry problematic sections, repeat the same command.

A partial parity file is still useful, as it allows recovery using the valid parts.

If the parity file is partially damaged or lost, use the "fix" command with the -d option to recreate it, targeting only the affected parity:

snapraid fix -d PARITY_NAME

How can I safely move files from one disk to another?

To safely move files between data disks in the array, follow these steps:

This process ensures that, even if disks fail during the sync process, you can recover all data. It also verifies that the hashes of moved files match the originals.

Can I defragment data and parity disks?

Yes, defragmentation does not affect SnapRAID.

How can I make SnapRAID less resource-intensive for other processes?

In Linux, use the ionice command with idle priority, like:

ionice -c 3 snapraid sync

How can I upgrade to a newer version of SnapRAID?

To upgrade, replace the old SnapRAID executable with the new one. The new version will use your existing configuration, content, and parity files.

Occasionally, the content file format changes, but newer SnapRAID versions can read all older formats. The content file is updated to the new format the next time it is written, if necessary.

Downgrading to an older SnapRAID version is generally not possible. If the old version cannot read the new content file, you will receive an appropriate message.

Recover

How can I undelete a recently deleted file?

Run the "fix" command with a filter for the specified file, like:

snapraid fix -f my_just_deleted_file

To undelete a directory, use:

snapraid fix -f my_just_deleted_dir/

To undelete all missing files, use:

snapraid fix -m

What happens if I add files and a disk fails before updating parity with "sync"?

Files added to the failed disk after the last "sync" command are lost. All other data is safe and can be recovered.

What happens if I delete files and a disk fails before updating parity with "sync"?

In the worst case, files deleted or modified on non-failed disks may prevent recovery of an equivalent amount of data on the failed disk. For example, deleting 10 GB of data may prevent recovery of 10 GB from the failed disk. The exact data loss depends on how much the deleted and failed data overlap in the parity.

Using two parity disks significantly improves recovery chances.

To ensure safety, avoid deleting files directly. Instead, move them to another directory on the same disk, outside the SnapRAID-checked directory tree. Run the "sync" command, then delete the files. If a disk fails during the "sync", run the "fix" command, specifying the location of the deleted files:

snapraid fix -i dir_of_deleted_files

What happens if a disk fails during a "sync"?

You can still recover data. In the worst case, you can recover as much data as if the disk failed before the "sync". If the "sync" process has progressed, SnapRAID uses the partially synced data to recover more. Using the "autosave" configuration option saves intermediate content files during the sync process, improving recovery.

What are "data errors" in "sync", and how do I fix them?

Syncing...
309505: Data error for file d4/data/index.db at position 40960
WARNING! Unexpected data error in a data disk! The block is now marked as bad!
Try with 'snapraid -e fix' to recover!

Data errors occur when SnapRAID detects that a file’s content has changed without an updated write timestamp. This should not happen and may indicate a silent disk error or a file system issue, such as an improper shutdown.

If a data error is found, the "sync" process continues, marking erroneous blocks as bad. These can be fixed with the "-e fix" command, which typically takes only a few seconds:

snapraid -e fix

If errors occur in multiple files, check for hardware issues. Test your PC’s memory with a tool like MemTest86. Also, check disk cabling and CPU temperature.

Another possibility is that a program modified files without updating their timestamps. In Linux, this is rare, except for memory-mapped files not yet closed. In Windows, timestamps are guaranteed to update only after closing a file. See MSDN WriteFile():

When writing to a file, the last write time is not fully updated until
all handles used for writing have been closed. Therefore, to ensure an
accurate last write time, close the file handle immediately after writing
to the file.

For memory-mapped files in Windows, the program must manually update the timestamp, as Windows does not do so automatically. See MSDN CreateFileMapping():

When modifying a file through a mapped view, the last modification timestamp
may not be updated automatically. If required, the caller should use SetFileTime
to set the timestamp.

To avoid these issues in Windows, run "sync" when no programs are accessing files in the array.

How can I store all messages from a check or fix command in a file?

Use the "-l" option to generate a log file:

snapraid check -l mylog.log

These log files can be large, potentially several GB. If too large for your editor, use HxD.

How can I recover the snapraid.conf file?

Use the "-C, --gen-conf" command to generate a template configuration file compatible with your content file. Manual adjustments are needed to set disk mount points, but comments in the file provide guidance:

snapraid -C snapraid.content > snapraid.conf.new

Tech

If the SnapRAID "check" command reports everything is OK, is it really OK?

Yes. SnapRAID verifies all files with a 128-bit checksum. If it reports they are correct, they are.

What integrity checksum does SnapRAID use?

SnapRAID uses Murmur3 and SpookyHash 128-bit hashes. Murmur3 is the default on 32-bit platforms, and SpookyHash is the default on 64-bit platforms.

Why are integrity checksums important?

Integrity checksums detect silent errors when a hard drive returns corrupt data without reporting an error.

For an introduction to this issue, see: Data corruption is worse than you know.

For a detailed analysis, see: An Analysis of Data Corruption in the Storage Stack:

A significant number (8% on average) of corruptions
are detected during RAID reconstruction, creating the
possibility of data loss. In this case, protection against
double disk failures is necessary to prevent data loss.

How can SnapRAID recover if a disk fails during a 'sync'?

Before starting a sync, SnapRAID saves a special content file describing both the pre- and post-sync states. This allows SnapRAID to attempt two recovery strategies.

For each block to recover, it first assumes the parity was correctly computed, as if the "sync" completed successfully. If this fails, it assumes the parity was not yet updated, ignoring any new files added to the array.

Using double parity improves recovery when there are overlapping changes in the array.

Can I access data disks during a 'sync'?

Yes, you can safely read from data disks during a 'sync'.

You can also write, and if this interferes with the 'sync', the process continues, skipping the written part.

This may affect potential recovery, but it is equivalent to writing just after the 'sync' completes.

Does SnapRAID retry reads or writes if a disk fails?

No, SnapRAID does not retry disk operations. It reports errors and attempts to continue. After 100 errors, it stops execution.

In such cases, it’s better to allow manual data recovery before the disk fails completely. Automatic retries could further damage the disk. To retry, repeat the command manually.

You can adjust the 100-error limit using the "-L, --error-limit" option.

For "fix" and "check" commands, you can resume from the point of interruption using the "-s, --start" option, specifying the block number that caused the error.

For the "sync" command, the latest state is saved before termination, so restarting it will continue from where it stopped.

What is the snapraid.content.lock file?

This file is used by SnapRAID to check if another instance is running. It is created where the first content file is stored.

What does the notice about zero sub-second timestamps mean?

In the 'status' command, you may see:

You have XXXXX files with zero sub-second timestamp.
Run the 'touch' command to set it to a non-zero value.

This indicates that some files have timestamps rounded to the nearest second, even though your file system supports sub-second precision.

While not a problem, this may limit SnapRAID’s ability to detect moved or copied files between disks.

You can set sub-second timestamps to a random value using the 'touch' command, which does not modify the second part of the timestamp.

Performance

What is the optimal hardware configuration for the best performance?

The most critical factor is using (E)SATA connections for all disks to avoid controller slowdowns. Avoid USB disks.

The second factor is using disks with similar performance, as the array’s read/write speed is limited by the slowest disk. After a sync, SnapRAID displays statistics with wait times for each disk, helping identify bottlenecks.

Finally, a fast CPU can improve performance with many disks. Check CPU usage during sync operations to determine if it’s a limiting factor.

What is the optimal block size for the best performance?

The default block size is typically optimal.

Increasing it may not improve performance and could reduce it in some cases.

What is the optimal Linux read-ahead size for the best performance?

The default Linux read-ahead size of 128 KiB is optimal for SnapRAID’s default block size of 256 KiB. If using a larger block size, set the read-ahead size to at least half the SnapRAID block size.

For example, with a SnapRAID block size of 512 KiB, configure a read-ahead size of 256 KiB for sdX using:

echo 256 > /sys/block/sdX/queue/read_ahead_kb

For a SnapRAID block size smaller than 256 KiB, use the default read-ahead size.

How fast are the hash and parity computations?

Murmur3 hash is very fast on 32-bit systems, and SpookyHash is even faster on 64-bit systems. Parity computation is efficient up to double parity, even without SSE support. Beyond double parity, a CPU with SSSE3 support is recommended. For low-end CPUs without SSE, a special, incompatible triple-parity option is available. See 'z-parity' in the manual for details.

The following results are from a Core i5-4670K @ 3.4 GHz. Check your values with the -T option:

root@redstar:/root# snapraid -T
snapraid v7.0 by Andrea Mazzoleni, http://www.snapraid.it
Compiler gcc 4.8.1
CPU GenuineIntel, family 6, model 60, flags mmx sse2 ssse3 sse42 avx2
Memory is little-endian 64-bit
Support nanosecond timestamps with futimens()

Speed test using 8 data buffers of 262144 bytes, for a total of 2048 KiB.
Memory blocks have a displacement of 1792 bytes to improve cache performance.
The reported values are the aggregate bandwidth of all data blocks in MiB/s,
not counting parity blocks.

Memory write speed using the C memset() function:
  memset   22233

CRC used to check the content file integrity:
   table    1261
   intel    9306

Hash used to check the data blocks integrity:
            best murmur3 spooky2
    hash spooky2    4715   14684

RAID functions used for computing the parity with 'sync':
            best    int8   int32   int64    sse2   sse2e   ssse3  ssse3e    avx2   avx2e
    gen1    avx2           13339   25438   45438                           50588
    gen2    avx2            4115    6514   19441   21840                   32201
    genz   avx2e            2337    2874    9803   10920                           18944
    gen3   avx2e     814                                    8934   10154           18613
    gen4   avx2e     620                                    6204    7569           14229
    gen5   avx2e     496                                    4777    5149           10051
    gen6   avx2e     413                                    3846    4239            8190

RAID functions used for recovering with 'fix':
            best    int8   ssse3    avx2
    rec1    avx2    1158    2916    3019
    rec2    avx2     517    1220    1633
    rec3    avx2     110     611     951
    rec4    avx2      71     395     631
    rec5    avx2      49     264     421
    rec6    avx2      36     194     316

Known Problems

Why does my machine reboot or crash while SnapRAID is running?

SnapRAID pushes the system to extreme conditions, which may expose latent hardware or driver issues, resulting in a system crash or reboot.

Although the issue occurs only when SnapRAID is running, it is undoubtedly a hardware or driver problem, as SnapRAID is a standard user application that cannot crash or reboot the system.

Check the following:

Why does SnapRAID abort without an error message in Linux?

This is likely due to insufficient free memory, causing the Linux out-of-memory (OOM) killer to terminate SnapRAID for using too much memory. Check available memory with:

free -m

If memory is low, install more memory in the machine.

Why do I get the error 'Failed to grow parity file 'xxx' to size xxx due to lack of space' during 'sync'?

This error indicates that SnapRAID needs to expand the parity file to a size that exceeds the available space on the parity disk.

Ensure no other data is stored on the parity disk, reserving it exclusively for the parity file. Alternatively, move files mentioned in the output to another data disk to reduce the required parity size.

In Linux, you can reformat the parity disk to increase available space using:

mkfs.ext4 -m 0 -T largefile4 DEVICE

The '-T largefile4' and '-m 0' options provide more space for the parity file.

Why do I get the error 'Internal inode xxx inconsistency...'?

This error occurs when SnapRAID detects two different files using the same inode, a condition that should theoretically never happen but can occur in specific cases.

Ensure the mentioned files were not modified while SnapRAID was running. Renaming, deleting, or recreating files during SnapRAID operation can cause inode reuse, triggering this error.

If using a FAT file system in Windows, convert to NTFS. Although inodes (FileIndex) are theoretically unique in FAT, this issue has been observed in practice.

Why do I get the error 'Data change at file...' with .xls files?

Excel modifies .xls files upon opening, even without saving, to store the name of the last user who accessed the file. This allows it to warn if another user tries to open the file simultaneously. Excel restores the original timestamp, but SnapRAID detects the modification, reporting a data error:

Data change at file '/xxx/xxx.xls' at position '0'
WARNING! Unexpected data modification of a file without parity!

To work around this, see How to prevent Excel from modifying the file on exit?, or switch to the .xlsx format.

Why are VeraCrypt containers never saved?

VeraCrypt (a fork of TrueCrypt) has a default option, Preserve modification timestamp of file containers, that prevents SnapRAID and other backup programs from detecting changes to file containers. Disable this option in VeraCrypt.