Frequently Asked Questions for SnapRAID.
General
- What is SnapRAID?
- How is it different from RAID?
- How is it different from ZFS/Btrfs snapshots and NTFS Shadow Copy?
- How is it different from other backup solutions?
- What are the advantages of SnapRAID compared to similar solutions like unRAID, and Windows Storage Spaces?
- How much does it cost?
- How can I get support?
- Does SnapRAID have a GUI?
- Does SnapRAID provide encryption, virtual views/storage pooling, SMART monitoring, or power control?
- Is SnapRAID suitable for backing up the home/My Documents directory?
- Is SnapRAID suitable for backing up the boot OS disk?
- Can SnapRAID hash files without parity computation?
Setup
- Do I need one or more parity disks?
- Which file system is recommended for SnapRAID?
- Which hardware configuration is recommended?
- Is a multi-core CPU recommended?
- Is a 64-bit operating system recommended?
- How much memory does SnapRAID require to run?
- Should I check the SMART attributes of my disks?
- Is it possible to use SnapRAID in a dual-boot configuration, such as Windows and Linux?
Configuration
- What is the SnapRAID 'content' file?
- What are the SnapRAID 'parity' and 'N-parity' files?
- How do I configure SAMBA to share the pool directory?
- How can I use more disks than available drive letters in Windows?
Use
- What are the most important steps to prevent data loss?
- How can I add a data disk to an existing array?
- How can I remove a data disk from an existing array?
- How can I add or remove a parity disk in an existing array?
- How can I verify a single data disk?
- How can I replace a data disk?
- How can I replace a parity disk?
- How can I safely move files from one disk to another?
- Can I defragment data and parity disks?
- How can I make SnapRAID less resource-intensive for other processes?
- How can I upgrade to a newer version of SnapRAID?
Recover
- How can I undelete a recently deleted file?
- What happens if I add files and a disk fails before updating parity with "sync"?
- What happens if I delete files and a disk fails before updating parity with "sync"?
- What happens if a disk fails during a "sync"?
- What are "data errors" in "sync", and how do I fix them?
- How can I store all messages from a check or fix command in a file?
- How can I recover the snapraid.conf file?
Tech
- If the SnapRAID "check" command reports everything is OK, is it really OK?
- What integrity checksum does SnapRAID use?
- Why are integrity checksums important?
- How can SnapRAID recover if a disk fails during a 'sync'?
- Can I access data disks during a 'sync'?
- Does SnapRAID retry reads or writes if a disk fails?
- What is the snapraid.content.lock file?
- What does the notice about zero sub-second timestamps mean?
Performance
- What is the optimal hardware configuration for the best performance?
- What is the optimal block size for the best performance?
- What is the optimal Linux read-ahead size for the best performance?
- How fast are the hash and parity computations?
Known Problems
- Why does my machine reboot or crash while SnapRAID is running?
- Why does SnapRAID abort without an error message in Linux?
- Why do I get the error 'Failed to grow parity file 'xxx' to size xxx due to lack of space' during 'sync'?
- Why do I get the error 'Internal inode xxx inconsistency...'?
- Why do I get the error 'Data change at file...' with .xls files?
- Why are VeraCrypt containers never saved?
General
What is SnapRAID?
SnapRAID is an application that creates a partial backup of your disk array. If some disks in your array fail, even if they are completely broken, you can recover their contents. It provides only a partial backup because it does not allow recovery from a failure of the entire array, but only if the number of failed disks is below a predefined limit.
How is it different from RAID?
RAID solutions mirror data and propagate changes immediately to the parity data. In this sense, RAID systems are not backups because they do not allow restoration of an earlier state of the array.
SnapRAID, however, is more akin to a backup solution, allowing you to restore the state of the last backup. For example, with SnapRAID, if you accidentally delete a file, you can easily recover it.
Other advantages over traditional RAID5/6 solutions include:
- All your data is hashed to ensure data integrity and prevent silent corruption.
- If too many disks fail to allow recovery, you lose data only on the failed disks; data on other disks remains safe.
- If you accidentally delete files on a disk, you can recover them.
- You can start with already filled disks.
- Disks can have different sizes.
- You can add disks at any time.
- It does not lock in your data. You can stop using SnapRAID at any time without needing to reformat or move data.
- To access a file, only a single disk needs to spin, saving power and reducing noise.
- You can have up to six parity levels, compared to one for RAID5 and two for RAID6.
RAID has advantages, such as better speed due to data striping. The best choice depends on your specific use case.
How is it different from ZFS/Btrfs snapshots and NTFS Shadow Copy?
SnapRAID creates a file-system snapshot conceptually similar to other snapshot solutions. However, its primary purpose is to recover a disk to its saved state after a complete disk failure.
SnapRAID is more akin to the RAID-Z/RAID functionality of ZFS/Btrfs. Compared to them, SnapRAID offers the following advantages:
- If too many disks fail to allow recovery, you lose data only on the failed disks; data on other disks remains safe.
- You can start with already filled disks.
- You can add disks at any time (not possible with ZFS, but possible with Btrfs).
- It does not lock in your data. You can stop using SnapRAID at any time without needing to reformat or move data.
- To access a file, only a single disk needs to spin, saving power and reducing noise.
- You can have up to six parity levels, compared to three for ZFS RAID-Z and two for Btrfs RAID-5/6.
ZFS and Btrfs have advantages, such as being real-time solutions. The best choice depends on your specific use case.
How is it different from other backup solutions?
With a typical backup solution, you need as much space as the entire disk array for the backup. With SnapRAID, you need only one additional parity disk to recover from the most common case of a single disk failure. Adding more parity disks allows recovery from additional disk failures.
Note that only a full backup allows recovery from a complete failure of the disk array, making it the preferred solution if you can afford it. SnapRAID is a good option when a full backup is not feasible.
What are the advantages of SnapRAID compared to similar solutions like unRAID, and Windows Storage Spaces?
Visit the Compare page to see a comparison with other similar solutions.
A key advantage of SnapRAID is its integrity and silent error management, which is at the same level as (or better than) ZFS and Btrfs file systems. In contrast, unRAID and Windows Storage Spaces lack integrity checks.
How much does it cost?
Nothing. SnapRAID is a free software application released under the GPLv3 license.
How can I get support?
Visit the Support page.
Does SnapRAID have a GUI?
No, SnapRAID is a command-line application. However, there is an external GUI called Elucidate, and a plugin for OpenMediaVault.
Does SnapRAID provide encryption, virtual views/storage pooling, SMART monitoring, or power control?
SnapRAID provides a basic virtual views/storage pooling solution but is also compatible with other pooling solutions.
For encryption, SMART monitoring, power control, and data recovery, you can use other tools. Here are some suggestions:
Linux | Windows | Mac OS X | |
---|---|---|---|
Encryption |
dm-crypt/LUKS (tutorial) |
VeraCrypt | |
Virtual views/storage pooling | mergerfs (tutorial), Greyhole | DrivePool ($) | |
SMART monitoring | smartmontools, GSmartControl | smartmontools, GSmartControl, CrystalDiskInfo | smartmontools, GSmartControl, SMARTReporter |
Power control | hdparm | hdparm | |
Data recovery | ddrescue, safecopy, dd_rescue |
Is SnapRAID suitable for backing up the home/My Documents directory?
No. If you have a limited amount of data that can be easily saved to an external USB drive or an online backup service, using SnapRAID is unnecessary. SnapRAID is best suited for backing up a very large amount of data, such as many terabytes, where a full backup copy is not feasible.
Is SnapRAID suitable for backing up the boot OS disk?
No. For the disk used to boot your machine and containing the OS files, it’s better to use another solution, such as RAID1 or a mirror copy. SnapRAID requires an operating system to run and cannot recover the OS if it is missing.
Can SnapRAID hash files without parity computation?
No, SnapRAID cannot be used solely to hash files. Its hashing is designed to work in combination with parity computation and is not optimized for hashing alone.
For a list of file verification tools, see: Wikipedia: Comparison of file verification software.
Setup
For a setup example in Ubuntu 14.04, see Zack's SnapRAID tutorial.
Do I need one or more parity disks?
As a rule of thumb, use one parity disk (RAID5) for up to four data disks, and then add one parity disk for each group of seven data disks, as shown in the table:
Parities | Data disks |
---|---|
1/Single Parity/RAID5 | 2–4 |
2/Double Parity/RAID6 | 5–14 |
3/Triple Parity | 15–21 |
4/Quad Parity | 22–28 |
5/Penta Parity | 29–35 |
6/Hexa Parity | 36–42 |
This table accounts for the fact that more parity levels help recover from an unsynced array and that disk failures may be correlated if all disks are in the same environment.
Multiple failures can occur. The worst case reported in SnapRAID’s history is a four-disk failure, successfully recovered using four parities.
For more information on parity levels and disk failure rates, see: Why RAID 5 stops working in 2009, Why RAID 6 stops working in 2019, Does RAID 6 stop working in 2019?, Triple-Parity RAID and Beyond, How long do disk drives last?.
Which file system is recommended for SnapRAID?
For data disks, consider these suggestions:
Data | Result |
---|---|
Any | OK. You can generally use any file system you prefer, with some exceptions noted below. |
Btrfs | OK. Ensure SnapRAID is built with the libblkid library for UUID support. You will be warned at each run if this is not the case. Note that multiple Btrfs snapshots are not supported. |
ZFS | Partial. UUID support is unavailable, but core SnapRAID functionalities remain usable. Note that multiple ZFS snapshots are not supported. |
FAT | Uncertain. There are reports of duplicate inode cases. NTFS is significantly faster and recommended instead. |
ReFS | No. ReFS uses 128-bit inodes, which SnapRAID does not yet support. |
For parity disks, consider these suggestions:
Parity | Result |
---|---|
ext4 | OK if disks are not larger than 16 TB. The parity is stored in a single large file, and ext4 has a 16 TB file size limit. |
XFS | OK |
Btrfs | OK. Ensure SnapRAID is built with the libblkid library for UUID support. You will be warned at each run if this is not the case. |
JFS | No. It does not support the fallocate() command needed to allocate parity files. It also does not optimally implement the posix_fadvise() command. |
NTFS | OK in Windows. Not supported in Linux, as it does not support the fallocate() command needed to allocate parity files. You can use it if the parity disk contains only the parity file. |
FAT | No. Files larger than 4 GB cannot be stored. It is also not journaled. Use NTFS instead. |
HFS+ | OK |
In Linux, to maximize space for the parity file system, format it with the -m 0 -T largefile4 options, like:
mkfs.ext4 -m 0 -T largefile4 DEVICE
On an 8 TB disk, this can save about 400 GB. It is also expected to be as fast as, or faster than, the default.
Which hardware configuration is recommended?
For optimal performance, connect all disks via SATA, not USB.
See Raj's Prototype Builds for examples of hardware setups.
For choosing hard drives, refer to Backblaze's What Hard Drive Should I Buy? and Backblaze's Enterprise Drives: Fact or Fiction?.
Is a multi-core CPU recommended?
No. SnapRAID is typically not CPU-bound; the limiting factor is disk read performance. A single-core CPU is usually sufficient.
For example, in my system, I achieve a combined read speed of 400 MiB/s, but SnapRAID can hash at 12,000 MiB/s on a single core.
Is a 64-bit operating system recommended?
Yes. SnapRAID performs better on a 64-bit operating system, as it can access more than 4 GiB of memory and use faster hashing and parity algorithms. However, for less than 30 TB of data, it performs well on a 32-bit operating system too.
How much memory does SnapRAID require to run?
Approximately, SnapRAID requires 1 GiB of memory for every 16 TB of data.
If you have insufficient memory, the recommended solution is to install more memory.
Alternatively, increase the blocksize option in the configuration file. Memory usage is inversely proportional to the selected block size. For example, using "blocksize 512" instead of the default 256 halves memory usage. However, a larger block size increases wasted space in parity files.
Running with memory swapped to disk is not recommended, as it significantly slows performance.
Should I check the SMART attributes of my disks?
Yes. Approximately 20% of disks fail within four years, as reported by Backblaze's How long do disk drives last?. Monitoring SMART attributes can help anticipate failures and prevent data loss.
If any of the SMART attributes Reallocated_Sector_Ct (5), Reported_Uncorrect (187), Command_Timeout (188), Current_Pending_Sector (197), or Offline_Uncorrectable (198) are non-zero, replace the disk, as recommended by Backblaze's Hard Drive SMART Stats.
See also Google's Failure trends in a large disk drive population, which found:
After the first reallocation, drives are over 14 times more likely to fail within 60 days than drives without reallocation counts.
After the first probational event, drives are 16 times more likely to fail within 60 days than drives with zero probational counts.
After the first offline reallocation, drives have over 21 times higher chances of failure within 60 days than drives without offline reallocations.
These findings recommend replacing a disk after the first error in the SMART attributes Reallocated_Sector_Ct (5), Current_Pending_Sector (197), or Offline_Uncorrectable (198).
Is it possible to use SnapRAID in a dual-boot configuration, such as Windows and Linux?
Yes, SnapRAID can be used in a dual-boot configuration, regardless of the OS combination.
It is recommended to run SnapRAID from only one OS. While it may work if run from both OSes, this scenario is not thoroughly tested and risks issues due to different SnapRAID versions, configurations, or content files.
Configuration
What is the SnapRAID 'content' file?
The content file stores the list of all files in your array, including their checksums, timestamps, and other necessary information.
This file will be approximately a few GiB in size, depending on the array’s size. For 10 TB of data, the content file will be about 500 MiB.
What are the SnapRAID 'parity' and 'N-parity' files?
These files store the parity data used for recovery.
Their size grows to match the largest amount of data stored on any single disk in the array.
How do I configure SAMBA to share the pool directory?
To configure SAMBA to share the pool directory (e.g., /pool), add the following options to your /etc/samba/smb.conf:
# In the global section of smb.conf unix extensions = no # In the share section of smb.conf [pool] comment = Pool path = /pool read only = yes guest ok = yes wide links = yes
How can I use more disks than available drive letters in Windows?
Use Volume Mount Points to mount a disk in a directory without using a drive letter, similar to Linux.
Use
What are the most important steps to prevent data loss?
To minimize the risk of data loss, follow these steps:
- Before running the first "sync", test your RAM with a program like MemTest86. Faulty RAM is the most common cause of data loss with SnapRAID.
- Run the "sync" command frequently, from once a day to once a week.
- Run the "scrub" command weekly.
- Use smartmontools or a similar utility to monitor SMART attributes. Replace a disk at the first sign of Reallocated_Sector_Ct, Current_Pending_Sector, or Offline_Uncorrectable, even if it still functions.
How can I add a data disk to an existing array?
To add a new data disk to the array, add the new "disk" option in the configuration file, then run a "sync" command:
snapraid sync
If the disk is empty, the "sync" command will complete immediately.
How can I remove a data disk from an existing array?
To remove a data disk from the array, follow these steps:
- Change the related "disk" option in the configuration file to point to an empty directory.
- Remove any "content" option pointing to that disk from the configuration file.
- Run a "sync" command with the "-E, --force-empty" option:
snapraid sync -E
The "-E" option allows SnapRAID to proceed even when detecting an empty disk. - After the "sync" command completes, remove the "disk" option from the configuration file.
The array will then have no reference to the removed disk.
How can I add or remove a parity disk in an existing array?
To add a new parity level, add the appropriate "N-parity" option in the configuration file, then run the "sync" command with the "-F, --force-full" option:
snapraid -F sync
The "-F" option instructs SnapRAID to recompute the full parity.
During this process, your existing parity remains unmodified, ensuring continued protection (note that this applies only from version 11.0).
To remove a parity level, remove the highest "N-parity" option from the configuration file and delete the parity file.
Note that after removing a parity file, it cannot be reused, as it becomes outdated after the next "sync" command.
How can I verify a single data disk?
The "scrub" command verifies the entire array and cannot be limited to a single data disk. To verify only one data disk, use the "check" audit command with the disk specified:
snapraid check -a -d DISK_NAME
This command verifies the checksums of all files on the specified disk. Other data and parity disks are not read or verified.
How can I replace a data disk?
If the disk is lost, refer to the recovery section in the manual. Otherwise, copy all files to the new disk, maintaining the same directory structure and names.
In Linux, use the command:
cp -av /from_dir/. /to_dir
WARNING: If using 'rsync' or 'mc' commands, ensure they are recent versions. 'rsync' must be at least version 3.1.0 (2013), and 'mc' at least version 4.8.19 (2017). Older versions may not copy sub-second timestamps.
In Windows, use the command:
robocopy F:\from_dir T:\to_dir /e /copyall
Update the SnapRAID configuration to point to the new mount point, then run a "diff" command to ensure everything is correct:
snapraid diff
If "diff" reports "added" or "removed" files, you may have copied files with a different directory structure. Correct this until "diff" reports only "equal", "restored", or "copied" files.
To verify that files were copied correctly, run a "check" audit command on the replaced disk to verify checksums:
snapraid check -a -d DISK_NAME
Finally, run "sync" to update the SnapRAID state. This command will complete almost immediately:
snapraid sync
How can I replace a parity disk?
If the disk is still accessible, ensure the array is synced by running:
snapraid sync
Copy the parity file to the new disk and update the parity location in the configuration file.
Run "sync" again to make SnapRAID recognize the new disk. This command will complete almost immediately:
snapraid sync
In Linux, a good copy command is ddrescue. For example:
ddrescue /OLD_DISK/parity /NEW_DISK/parity /tmp/copy.log
This copies the parity file from the old to the new location, continuing despite errors. To retry problematic sections, repeat the same command.
A partial parity file is still useful, as it allows recovery using the valid parts.
If the parity file is partially damaged or lost, use the "fix" command with the -d option to recreate it, targeting only the affected parity:
snapraid fix -d PARITY_NAME
How can I safely move files from one disk to another?
To safely move files between data disks in the array, follow these steps:
- Copy the files to the new location.
- Run a 'diff' command to verify that the copied files are identified as 'copy'.
- Run a 'sync' command.
- Delete the files from the original location.
- Run another 'sync' command.
This process ensures that, even if disks fail during the sync process, you can recover all data. It also verifies that the hashes of moved files match the originals.
Can I defragment data and parity disks?
Yes, defragmentation does not affect SnapRAID.
How can I make SnapRAID less resource-intensive for other processes?
In Linux, use the ionice command with idle priority, like:
ionice -c 3 snapraid sync
How can I upgrade to a newer version of SnapRAID?
To upgrade, replace the old SnapRAID executable with the new one. The new version will use your existing configuration, content, and parity files.
Occasionally, the content file format changes, but newer SnapRAID versions can read all older formats. The content file is updated to the new format the next time it is written, if necessary.
Downgrading to an older SnapRAID version is generally not possible. If the old version cannot read the new content file, you will receive an appropriate message.
Recover
How can I undelete a recently deleted file?
Run the "fix" command with a filter for the specified file, like:
snapraid fix -f my_just_deleted_file
To undelete a directory, use:
snapraid fix -f my_just_deleted_dir/
To undelete all missing files, use:
snapraid fix -m
What happens if I add files and a disk fails before updating parity with "sync"?
Files added to the failed disk after the last "sync" command are lost. All other data is safe and can be recovered.
What happens if I delete files and a disk fails before updating parity with "sync"?
In the worst case, files deleted or modified on non-failed disks may prevent recovery of an equivalent amount of data on the failed disk. For example, deleting 10 GB of data may prevent recovery of 10 GB from the failed disk. The exact data loss depends on how much the deleted and failed data overlap in the parity.
Using two parity disks significantly improves recovery chances.
To ensure safety, avoid deleting files directly. Instead, move them to another directory on the same disk, outside the SnapRAID-checked directory tree. Run the "sync" command, then delete the files. If a disk fails during the "sync", run the "fix" command, specifying the location of the deleted files:
snapraid fix -i dir_of_deleted_files
What happens if a disk fails during a "sync"?
You can still recover data. In the worst case, you can recover as much data as if the disk failed before the "sync". If the "sync" process has progressed, SnapRAID uses the partially synced data to recover more. Using the "autosave" configuration option saves intermediate content files during the sync process, improving recovery.
What are "data errors" in "sync", and how do I fix them?
Syncing... 309505: Data error for file d4/data/index.db at position 40960 WARNING! Unexpected data error in a data disk! The block is now marked as bad! Try with 'snapraid -e fix' to recover!
Data errors occur when SnapRAID detects that a file’s content has changed without an updated write timestamp. This should not happen and may indicate a silent disk error or a file system issue, such as an improper shutdown.
If a data error is found, the "sync" process continues, marking erroneous blocks as bad. These can be fixed with the "-e fix" command, which typically takes only a few seconds:
snapraid -e fix
If errors occur in multiple files, check for hardware issues. Test your PC’s memory with a tool like MemTest86. Also, check disk cabling and CPU temperature.
Another possibility is that a program modified files without updating their timestamps. In Linux, this is rare, except for memory-mapped files not yet closed. In Windows, timestamps are guaranteed to update only after closing a file. See MSDN WriteFile():
When writing to a file, the last write time is not fully updated until all handles used for writing have been closed. Therefore, to ensure an accurate last write time, close the file handle immediately after writing to the file.
For memory-mapped files in Windows, the program must manually update the timestamp, as Windows does not do so automatically. See MSDN CreateFileMapping():
When modifying a file through a mapped view, the last modification timestamp may not be updated automatically. If required, the caller should use SetFileTime to set the timestamp.
To avoid these issues in Windows, run "sync" when no programs are accessing files in the array.
How can I store all messages from a check or fix command in a file?
Use the "-l" option to generate a log file:
snapraid check -l mylog.log
These log files can be large, potentially several GB. If too large for your editor, use HxD.
How can I recover the snapraid.conf file?
Use the "-C, --gen-conf" command to generate a template configuration file compatible with your content file. Manual adjustments are needed to set disk mount points, but comments in the file provide guidance:
snapraid -C snapraid.content > snapraid.conf.new
Tech
If the SnapRAID "check" command reports everything is OK, is it really OK?
Yes. SnapRAID verifies all files with a 128-bit checksum. If it reports they are correct, they are.
What integrity checksum does SnapRAID use?
SnapRAID uses Murmur3 and SpookyHash 128-bit hashes. Murmur3 is the default on 32-bit platforms, and SpookyHash is the default on 64-bit platforms.
Why are integrity checksums important?
Integrity checksums detect silent errors when a hard drive returns corrupt data without reporting an error.
For an introduction to this issue, see: Data corruption is worse than you know.
For a detailed analysis, see: An Analysis of Data Corruption in the Storage Stack:
A significant number (8% on average) of corruptions are detected during RAID reconstruction, creating the possibility of data loss. In this case, protection against double disk failures is necessary to prevent data loss.
How can SnapRAID recover if a disk fails during a 'sync'?
Before starting a sync, SnapRAID saves a special content file describing both the pre- and post-sync states. This allows SnapRAID to attempt two recovery strategies.
For each block to recover, it first assumes the parity was correctly computed, as if the "sync" completed successfully. If this fails, it assumes the parity was not yet updated, ignoring any new files added to the array.
Using double parity improves recovery when there are overlapping changes in the array.
Can I access data disks during a 'sync'?
Yes, you can safely read from data disks during a 'sync'.
You can also write, and if this interferes with the 'sync', the process continues, skipping the written part.
This may affect potential recovery, but it is equivalent to writing just after the 'sync' completes.
Does SnapRAID retry reads or writes if a disk fails?
No, SnapRAID does not retry disk operations. It reports errors and attempts to continue. After 100 errors, it stops execution.
In such cases, it’s better to allow manual data recovery before the disk fails completely. Automatic retries could further damage the disk. To retry, repeat the command manually.
You can adjust the 100-error limit using the "-L, --error-limit" option.
For "fix" and "check" commands, you can resume from the point of interruption using the "-s, --start" option, specifying the block number that caused the error.
For the "sync" command, the latest state is saved before termination, so restarting it will continue from where it stopped.
What is the snapraid.content.lock file?
This file is used by SnapRAID to check if another instance is running. It is created where the first content file is stored.
What does the notice about zero sub-second timestamps mean?
In the 'status' command, you may see:
You have XXXXX files with zero sub-second timestamp. Run the 'touch' command to set it to a non-zero value.
This indicates that some files have timestamps rounded to the nearest second, even though your file system supports sub-second precision.
While not a problem, this may limit SnapRAID’s ability to detect moved or copied files between disks.
You can set sub-second timestamps to a random value using the 'touch' command, which does not modify the second part of the timestamp.
Performance
What is the optimal hardware configuration for the best performance?
The most critical factor is using (E)SATA connections for all disks to avoid controller slowdowns. Avoid USB disks.
The second factor is using disks with similar performance, as the array’s read/write speed is limited by the slowest disk. After a sync, SnapRAID displays statistics with wait times for each disk, helping identify bottlenecks.
Finally, a fast CPU can improve performance with many disks. Check CPU usage during sync operations to determine if it’s a limiting factor.
What is the optimal block size for the best performance?
The default block size is typically optimal.
Increasing it may not improve performance and could reduce it in some cases.
What is the optimal Linux read-ahead size for the best performance?
The default Linux read-ahead size of 128 KiB is optimal for SnapRAID’s default block size of 256 KiB. If using a larger block size, set the read-ahead size to at least half the SnapRAID block size.
For example, with a SnapRAID block size of 512 KiB, configure a read-ahead size of 256 KiB for sdX using:
echo 256 > /sys/block/sdX/queue/read_ahead_kb
For a SnapRAID block size smaller than 256 KiB, use the default read-ahead size.
How fast are the hash and parity computations?
Murmur3 hash is very fast on 32-bit systems, and SpookyHash is even faster on 64-bit systems. Parity computation is efficient up to double parity, even without SSE support. Beyond double parity, a CPU with SSSE3 support is recommended. For low-end CPUs without SSE, a special, incompatible triple-parity option is available. See 'z-parity' in the manual for details.
The following results are from a Core i5-4670K @ 3.4 GHz. Check your values with the -T option:
root@redstar:/root# snapraid -T snapraid v7.0 by Andrea Mazzoleni, http://www.snapraid.it Compiler gcc 4.8.1 CPU GenuineIntel, family 6, model 60, flags mmx sse2 ssse3 sse42 avx2 Memory is little-endian 64-bit Support nanosecond timestamps with futimens() Speed test using 8 data buffers of 262144 bytes, for a total of 2048 KiB. Memory blocks have a displacement of 1792 bytes to improve cache performance. The reported values are the aggregate bandwidth of all data blocks in MiB/s, not counting parity blocks. Memory write speed using the C memset() function: memset 22233 CRC used to check the content file integrity: table 1261 intel 9306 Hash used to check the data blocks integrity: best murmur3 spooky2 hash spooky2 4715 14684 RAID functions used for computing the parity with 'sync': best int8 int32 int64 sse2 sse2e ssse3 ssse3e avx2 avx2e gen1 avx2 13339 25438 45438 50588 gen2 avx2 4115 6514 19441 21840 32201 genz avx2e 2337 2874 9803 10920 18944 gen3 avx2e 814 8934 10154 18613 gen4 avx2e 620 6204 7569 14229 gen5 avx2e 496 4777 5149 10051 gen6 avx2e 413 3846 4239 8190 RAID functions used for recovering with 'fix': best int8 ssse3 avx2 rec1 avx2 1158 2916 3019 rec2 avx2 517 1220 1633 rec3 avx2 110 611 951 rec4 avx2 71 395 631 rec5 avx2 49 264 421 rec6 avx2 36 194 316
Known Problems
Why does my machine reboot or crash while SnapRAID is running?
SnapRAID pushes the system to extreme conditions, which may expose latent hardware or driver issues, resulting in a system crash or reboot.
Although the issue occurs only when SnapRAID is running, it is undoubtedly a hardware or driver problem, as SnapRAID is a standard user application that cannot crash or reboot the system.
Check the following:
- Test system memory with MemTest86.
- Verify hard drive cabling.
- Ensure the power supply can support all hard drives spinning simultaneously.
- Check that the system temperature is not too high.
- Update storage-related system drivers.
- Review system logs or Event Viewer for clues about the issue.
Why does SnapRAID abort without an error message in Linux?
This is likely due to insufficient free memory, causing the Linux out-of-memory (OOM) killer to terminate SnapRAID for using too much memory. Check available memory with:
free -m
If memory is low, install more memory in the machine.
Why do I get the error 'Failed to grow parity file 'xxx' to size xxx due to lack of space' during 'sync'?
This error indicates that SnapRAID needs to expand the parity file to a size that exceeds the available space on the parity disk.
Ensure no other data is stored on the parity disk, reserving it exclusively for the parity file. Alternatively, move files mentioned in the output to another data disk to reduce the required parity size.
In Linux, you can reformat the parity disk to increase available space using:
mkfs.ext4 -m 0 -T largefile4 DEVICE
The '-T largefile4' and '-m 0' options provide more space for the parity file.
Why do I get the error 'Internal inode xxx inconsistency...'?
This error occurs when SnapRAID detects two different files using the same inode, a condition that should theoretically never happen but can occur in specific cases.
Ensure the mentioned files were not modified while SnapRAID was running. Renaming, deleting, or recreating files during SnapRAID operation can cause inode reuse, triggering this error.
If using a FAT file system in Windows, convert to NTFS. Although inodes (FileIndex) are theoretically unique in FAT, this issue has been observed in practice.
Why do I get the error 'Data change at file...' with .xls files?
Excel modifies .xls files upon opening, even without saving, to store the name of the last user who accessed the file. This allows it to warn if another user tries to open the file simultaneously. Excel restores the original timestamp, but SnapRAID detects the modification, reporting a data error:
Data change at file '/xxx/xxx.xls' at position '0' WARNING! Unexpected data modification of a file without parity!
To work around this, see How to prevent Excel from modifying the file on exit?, or switch to the .xlsx format.
Why are VeraCrypt containers never saved?
VeraCrypt (a fork of TrueCrypt) has a default option, Preserve modification timestamp of file containers, that prevents SnapRAID and other backup programs from detecting changes to file containers. Disable this option in VeraCrypt.