Scrubbing

To periodically check the data and parity for errors, you can run the scrub command.

snapraid scrub

This command compares the data in your array with the hash computed during the sync command to verify integrity.

Each run of the command checks approximately 8% of the array, excluding data already scrubbed in the previous 10 days. You can use the -p, --plan option to specify a different amount and the -o, --older-than option to specify a different age in days. For example, to check 5% of the array for blocks older than 20 days, use:

snapraid -p 5 -o 20 scrub

If silent or input/output errors are found during the process, the corresponding blocks are marked as bad in the content file and listed in the status command.

snapraid status

To fix them, you can use the fix command, filtering for bad blocks with the -e, --filter-error option:

snapraid -e fix

At the next scrub, the errors will disappear from the status report if they are truly fixed. To make it faster, you can use -p bad to scrub only blocks marked as bad.

snapraid -p bad scrub

Running scrub on an unsynced array may report errors caused by removed or modified files. These errors are reported in the scrub output, but the related blocks are not marked as bad.

Pooling

Note: The pooling feature described below has been superseded by the mergerfs tool, which is now the recommended option for Linux users in the SnapRAID community. Mergefs provides a more flexible and efficient way to pool multiple drives into a single unified mount point, allowing seamless access to files across your array without relying on symbolic links. It integrates well with SnapRAID for parity protection and is commonly used in setups like OpenMediaVault (OMV) or custom NAS configurations.

To have all the files in your array shown in the same directory tree, you can enable the pooling feature. It creates a read-only virtual view of all the files in your array using symbolic links.

You can configure the pooling directory in the configuration file with:

pool /pool

or, if you are on Windows, with:

pool C:\pool

and then run the pool command to create or update the virtual view.

snapraid pool

If you are using a Unix platform and want to share this directory over the network to either Windows or Unix machines, you should add the following options to your /etc/samba/smb.conf:

# In the global section of smb.conf
unix extensions = no

# In the share section of smb.conf
[pool]
comment = Pool
path = /pool
read only = yes
guest ok = yes
wide links = yes
follow symlinks = yes

In Windows, sharing symbolic links over a network requires clients to resolve them remotely. To enable this, besides sharing the pool directory, you must also share all the disks independently, using the disk names defined in the configuration file as share points. You must also specify in the share option of the configuration file the Windows UNC path that remote clients need to use to access these shared disks.

For example, operating from a server named darkstar, you can use the options:

data d1 F:\array\
data d2 G:\array\
data d3 H:\array\
pool C:\pool
share \\darkstar

and share the following directories over the network:

\\darkstar\pool -> C:\pool
\\darkstar\d1 -> F:\array
\\darkstar\d2 -> G:\array
\\darkstar\d3 -> H:\array

to allow remote clients to access all the files at \\darkstar\pool.

You may also need to configure remote clients to enable access to remote symlinks with the command:

fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1

Undeleting

SnapRAID functions more like a backup program than a RAID system, and it can be used to restore or undelete files to their previous state using the -f, --filter option:

snapraid fix -f FILE

or for a directory:

snapraid fix -f DIR/

You can also use it to recover only accidentally deleted files inside a directory using the -m, --filter-missing option, which restores only missing files, leaving all others untouched.

snapraid fix -m -f DIR/

Or to recover all the deleted files on all drives with:

snapraid fix -m

Recovering

The worst has happened, and you have lost one or more disks!

DO NOT PANIC! You will be able to recover them!

The first thing you must do is avoid further changes to your disk array. Disable any remote connections to it and any scheduled processes, including any scheduled SnapRAID nightly sync or scrub.

Then proceed with the following steps.

data d1 /mnt/disk1/

data d1 /mnt/new_spare_disk/

snapraid -d NAME -l fix.log fix

snapraid -d NAME -a check

snapraid sync

status

Prints a summary of the state of the disk array.

It includes information about parity fragmentation, how old the blocks are without checking, and all recorded silent errors encountered while scrubbing.

The information presented refers to the latest time you ran sync. Later modifications are not taken into account.

If bad blocks were detected, their block numbers are listed. To fix them, you can use the fix -e command.

It also shows a graph representing the last time each block was scrubbed or synced. Scrubbed blocks are shown with *, blocks synced but not yet scrubbed with o.

Nothing is modified.

smart

Prints a SMART report of all the disks in the system.

It includes an estimation of the probability of failure in the next year, allowing you to plan maintenance replacements of disks that show suspicious attributes.

This probability estimation is obtained by correlating the SMART attributes of the disks with the Backblaze data available at:

https://www.backblaze.com/hard-drive-test-data.html

If SMART reports that a disk is failing, FAIL or PREFAIL is printed for that disk, and SnapRAID returns with an error. In this case, immediate replacement of the disk is highly recommended.

Other possible status strings are:

If the -v, --verbose option is specified, a deeper statistical analysis is provided. This analysis can help you decide if you need more or less parity.

This command uses the smartctl tool and is equivalent to running smartctl -a on all devices.

If your devices are not auto-detected correctly, you can specify a custom command using the smartctl option in the configuration file.

Nothing is modified.

probe

Prints the POWER state of all disks in the system.

Standby means the disk is not spinning. Active means the disk is spinning.

This command uses the smartctl tool and is equivalent to running smartctl -n standby -i on all devices.

If your devices are not auto-detected correctly, you can specify a custom command using the smartctl option in the configuration file.

Nothing is modified.

up

Spins up all the disks of the array.

You can spin up only specific disks using the -d, --filter-disk option.

Spinning up all the disks at the same time requires a lot of power. Ensure that your power supply can sustain it.

Nothing is modified.

down

Spins down all the disks of the array.

This command uses the smartctl tool and is equivalent to running smartctl -s standby,now on all devices.

You can spin down only specific disks using the -d, --filter-disk option.

To automatically spin down on error, you can use the -s, --spin-down-on-error option with any other command, which is equivalent to running down manually when an error occurs.

Nothing is modified.

diff

Lists all the files modified since the last sync that need to have their parity data recomputed.

This command doesn't check the file data, but only the file timestamp, size, and inode.

After listing all changed files, a summary of the changes is presented, grouped by:

If a sync is required, the process return code is 2, instead of the default 0. The return code 1 is used for a generic error condition.

Nothing is modified.

sync

Updates the parity information. All modified files in the disk array are read, and the corresponding parity data is updated.

You can stop this process at any time by pressing Ctrl+C, without losing the work already done. At the next run, the sync process will resume where it was interrupted.

If silent or input/output errors are found during the process, the corresponding blocks are marked as bad.

Files are identified by path and/or inode and checked by size and timestamp. If the file size or timestamp differs, the parity data is recomputed for the entire file. If the file is moved or renamed on the same disk, keeping the same inode, the parity is not recomputed. If the file is moved to another disk, the parity is recomputed, but the previously computed hash information is retained.

The content and parity files are modified if necessary. The files in the array are NOT modified.

scrub

Scrubs the array, checking for silent or input/output errors in data and parity disks.

Each invocation checks approximately 8% of the array, excluding data already scrubbed in the last 10 days. This means that scrubbing once a week ensures every bit of data is checked at least once every three months.

You can define a different scrub plan or amount using the -p, --plan option, which accepts: bad - Scrub blocks marked bad. new - Scrub just-synced blocks not yet scrubbed. full - Scrub everything. 0-100 - Scrub the specified percentage of blocks.

If you specify a percentage amount, you can also use the -o, --older-than option to define how old the block should be. The oldest blocks are scrubbed first, ensuring an optimal check. If you want to scrub only the just-synced blocks not yet scrubbed, use the -p new option.

To get details of the scrub status, use the status command.

For any silent or input/output error found, the corresponding blocks are marked as bad in the content file. These bad blocks are listed in status and can be fixed with fix -e. After the fix, at the next scrub, they will be rechecked, and if found corrected, the bad mark will be removed. To scrub only the bad blocks, you can use the scrub -p bad command.

It's recommended to run scrub only on a synced array to avoid reported errors caused by unsynced data. These errors are recognized as not being silent errors, and the blocks are not marked as bad, but such errors are reported in the output of the command.

The content file is modified to update the time of the last check for each block and to mark bad blocks. The parity files are NOT modified. The files in the array are NOT modified.

fix

Fixes all the files and the parity data.

All files and parity data are compared with the snapshot state saved in the last sync. If a difference is found, it is reverted to the stored snapshot.

WARNING! The fix command does not differentiate between errors and intentional modifications. It unconditionally reverts the file state to the last sync.

If no other option is specified, the entire array is processed. Use the filter options to select a subset of files or disks to operate on.

To fix only the blocks marked bad during sync and scrub, use the -e, --filter-error option. Unlike other filter options, this one applies fixes only to files that are unchanged since the latest sync.

SnapRAID renames all files that cannot be fixed by adding the .unrecoverable extension.

Before fixing, the entire array is scanned to find any files moved since the last sync operation. These files are identified by their timestamp, ignoring their name and directory, and are used in the recovery process if necessary. If you moved some of them outside the array, you can use the -i, --import option to specify additional directories to scan.

Files are identified only by path, not by inode.

The content file is NOT modified. The parity files are modified if necessary. The files in the array are modified if necessary.

check

Verifies all the files and the parity data.

It works like fix, but it only simulates a recovery and no changes are written to the array.

This command is primarily intended for manual verification, such as after a recovery process or in other special conditions. For periodic and scheduled checks, use scrub.

If you use the -a, --audit-only option, only the file data is checked, and the parity data is ignored for a faster run.

Files are identified only by path, not by inode.

Nothing is modified.

list

Lists all the files contained in the array at the time of the last sync.

With -v or --verbose, the subsecond time is also shown.

Nothing is modified.

dup

Lists all duplicate files. Two files are assumed equal if their hashes match. The file data is not read; only the precomputed hashes are used.

Nothing is modified.

pool

Creates or updates a virtual view of all the files in your disk array in the pooling directory.

The files are not copied but linked using symbolic links.

When updating, all existing symbolic links and empty subdirectories are deleted and replaced with the new view of the array. Any other regular files are left in place.

Nothing is modified outside the pool directory.

devices

Prints the low-level devices used by the array.

This command displays the device associations in the array and is mainly intended as a script interface.

The first two columns are the low-level device ID and path. The next two columns are the high-level device ID and path. The last column is the disk name in the array.

In most cases, you have one low-level device for each disk in the array, but in some more complex configurations, you may have multiple low-level devices used by a single disk in the array.

Nothing is modified.

touch

Sets an arbitrary sub-second timestamp for all files that have it set to zero.

This improves SnapRAID's ability to recognize moved and copied files, as it makes the timestamp almost unique, reducing possible duplicates.

More specifically, if the sub-second timestamp is not zero, a moved or copied file is identified as such if it matches the name, size, and timestamp. If the sub-second timestamp is zero, it is considered a copy only if the full path, size, and timestamp all match.

The second-precision timestamp is not modified, so all the dates and times of your files will be preserved.

rehash

Schedules a rehash of the entire array.

This command changes the hash kind used, typically when upgrading from a 32-bit system to a 64-bit one, to switch from MurmurHash3 to the faster SpookyHash.

If you are already using the optimal hash, this command does nothing and informs you that no action is needed.

The rehash is not performed immediately but takes place progressively during sync and scrub.

You can check the rehash state using status.

During the rehash, SnapRAID maintains full functionality, with the only exception that dup cannot detect duplicated files using a different hash.

locate

Locate files stored in the parity disks. For each matching file, it prints its location within the parity file and the number of fragments it occupies.

You can use the -t, --tail option to restrict the operation to files occupying the specified tail portion of the parity.

If you want to reallocate these files, you can then use the -W, --force-realloc-tail option. Be aware that such files will not be protected by parity during the reallocation process.

parity FILE [,FILE] ...

Defines the files to use to store the parity information. The parity enables protection from a single disk failure, similar to RAID5.

You can specify multiple files, which must be on different disks. When a file cannot grow anymore, the next one is used. The total space available must be at least as large as the largest data disk in the array.

You can add additional parity files later, but you cannot reorder or remove them.

Keeping the parity disks reserved for parity ensures that they do not become fragmented, improving performance.

In Windows, 256 MB is left unused on each disk to avoid the warning about full disks.

This option is mandatory and can be used only once.

(2,3,4,5,6)-parity FILE [,FILE] ...

Defines the files to use to store extra parity information.

For each parity level specified, one additional level of protection is enabled:

• 2-parity enables RAID6 dual parity.
• 3-parity enables triple parity.
• 4-parity enables quad (four) parity.
• 5-parity enables penta (five) parity.
• 6-parity enables hexa (six) parity.

Each parity level requires the presence of all previous parity levels.

The same considerations as for the parity option apply.

These options are optional and can be used only once.

z-parity FILE [,FILE] ...

Defines an alternate file and format to store triple parity.

This option is an alternative to 3-parity, primarily intended for low-end CPUs like ARM or AMD Phenom, Athlon, and Opteron that do not support the SSSE3 instruction set. In such cases, it provides better performance.

This format is similar to but faster than the one used by ZFS RAIDZ3. Like ZFS, it does not work beyond triple parity.

When using 3-parity, you will be warned if it is recommended to use the z-parity format for performance improvement.

It is possible to convert from one format to another by adjusting the configuration file with the desired z-parity or 3-parity file and using fix to recreate it.

content FILE

Defines the file to use to store the list and checksums of all the files present in your disk array.

It can be placed on a disk used for data, parity, or any other disk available. If you use a data disk, this file is automatically excluded from the sync process.

This option is mandatory and can be used multiple times to save multiple copies of the same file.

You must store at least one copy for each parity disk used plus one. Using additional copies does not hurt.

data NAME DIR

Defines the name and mount point of the data disks in the array. NAME is used to identify the disk and must be unique. DIR is the mount point of the disk in the file system.

You can change the mount point as needed, as long as you keep the NAME fixed.

You should use one option for each data disk in the array.

You can rename a disk later by changing the NAME directly in the configuration file and then running a sync command. In the case of renaming, the association is done using the stored UUID of the disks.

extra NAME DIR

Defines the name and mount point of additional disks to monitor with the smart and probe commands.

This is useful for monitoring disks that are not part of the array but are required for the system to function, such as the boot disk.

Note that such disks are not affected by the up and down commands because they are expected to be always spinning.

nohidden

Excludes all hidden files and directories. In Unix, hidden files are those starting with .. In Windows, they are those with the hidden attribute.

exclude/include PATTERN

Defines the file or directory patterns to exclude or include in the sync process. All patterns are processed in the specified order.

If the first pattern that matches is an exclude one, the file is excluded. If it is an include one, the file is included. If no pattern matches, the file is excluded if the last pattern specified is an include, or included if the last pattern specified is an exclude.

See the PATTERN section for more details on pattern specifications.

This option can be used multiple times.

blocksize SIZE_IN_KIBIBYTES

Defines the basic block size in kibibytes for the parity. One kibibyte is 1024 bytes.

The default blocksize is 256, which should work for most cases.

WARNING! This option is for experts only, and it is highly recommended not to change this value. To change this value in the future, you will need to recreate the entire parity!

A reason to use a different blocksize is if you have many small files, on the order of millions.

For each file, even if only a few bytes, an entire block of parity is allocated, and with many files, this may result in significant unused parity space. When you completely fill the parity disk, you are not allowed to add more files to the data disks. However, the wasted parity does not accumulate across data disks. Wasted space resulting from a high number of files on a data disk limits only the amount of data on that data disk, not others.

As an approximation, you can assume that half of the block size is wasted for each file. For example, with 100,000 files and a 256 KiB block size, you will waste 12.8 GB of parity, which may result in 12.8 GB less space available on the data disk.

You can check the amount of wasted space on each disk using status. This is the amount of space you must leave free on the data disks or use for files not included in the array. If this value is negative, it means you are close to filling the parity, and it represents the space you can still waste.

To avoid this issue, you can use a larger partition for parity. For example, if the parity partition is 12.8 GB larger than the data disks, you have enough extra space to handle up to 100,000 files on each data disk without any wasted space.

A trick to get a larger parity partition in Linux is to format it with the command:

mkfs.ext4 -m 0 -T largefile4 DEVICE

This results in about 1.5% extra space, approximately 60 GB for a 4 TB disk, which allows about 460,000 files on each data disk without any wasted space.

hashsize SIZE_IN_BYTES

Defines the hash size in bytes for the saved blocks.

The default hashsize is 16 bytes (128 bits), which should work for most cases.

WARNING! This option is for experts only, and it is highly recommended not to change this value. To change this value in the future, you will need to recreate the entire parity!

A reason to use a different hashsize is if your system has limited memory. As a rule of thumb, SnapRAID typically requires 1 GiB of RAM for each 16 TB of data in the array.

Specifically, to store the hashes of the data, SnapRAID requires approximately TS*(1+HS)/BS bytes of RAM, where TS is the total size in bytes of your disk array, BS is the block size in bytes, and HS is the hash size in bytes.

For example, with 8 disks of 4 TB, a block size of 256 KiB (1 KiB = 1024 bytes), and a hash size of 16, you get:

RAM = (8 * 4 * 10^12) * (1+16) / (256 * 2^10) = 1.93 GiB

Switching to a hash size of 8, you get:

RAM = (8 * 4 * 10^12) * (1+8) / (256 * 2^10) = 1.02 GiB

Switching to a block size of 512, you get:

RAM = (8 * 4 * 10^12) * (1+16) / (512 * 2^10) = 0.96 GiB

Switching to both a hash size of 8 and a block size of 512, you get:

RAM = (8 * 4 * 10^12) * (1+8) / (512 * 2^10) = 0.51 GiB

autosave SIZE_IN_GIGABYTES

Automatically saves the state when syncing or scrubbing after the specified amount of GB processed. This option is useful to avoid restarting long sync commands from scratch if interrupted by a machine crash or any other event.

temp_limit TEMPERATURE_CELSIUS

Sets the maximum allowed disk temperature in Celsius. When specified, SnapRAID periodically checks the temperature of all disks using the smartctl tool. The current disk temperatures are displayed while SnapRAID is operating. If any disk exceeds this limit, all operations stop, and the disks are spun down (put into standby) for the duration defined by the temp_sleep option. After the sleep period, operations resume, potentially pausing again if the temperature limit is reached once more.

During operation, SnapRAID also analyzes the heating curve of each disk and estimates the long-term steady temperature they are expected to reach if activity continues. The estimation is performed only after the disk temperature has increased four times, ensuring that enough data points are available to establish a reliable trend. This predicted steady temperature is shown in parentheses next to the current value and helps assess whether the system's cooling is adequate. This estimated temperature is for informational purposes only and has no effect on the behavior of SnapRAID. The program's actions are based solely on the actual measured disk temperatures.

To perform this analysis, SnapRAID needs a reference for the system temperature. It first attempts to read it from available hardware sensors. If no system sensor can be accessed, it uses the lowest disk temperature measured at the start of the run as a fallback reference.

Normally, SnapRAID shows only the temperature of the hottest disk. To display the temperature of all disks, use the -A or --stats option.

temp_sleep TIME_IN_MINUTES

Sets the standby time, in minutes, when the temperature limit is reached. During this period, the disks remain spun down. The default is 5 minutes.

pool DIR

Defines the pooling directory where the virtual view of the disk array is created using the pool command.

The directory must already exist.

share UNC_DIR

Defines the Windows UNC path required to access the disks remotely.

If this option is specified, the symbolic links created in the pool directory use this UNC path to access the disks. Without this option, the symbolic links generated use only local paths, which does not allow sharing the pool directory over the network.

The symbolic links are formed using the specified UNC path, adding the disk name as specified in the data option, and finally adding the file directory and name.

This option is required only for Windows.

smartctl DISK/PARITY OPTIONS...

Defines custom smartctl options to obtain the SMART attributes for each disk. This may be required for RAID controllers and some USB disks that cannot be auto-detected. The %s placeholder is replaced by the device name, but it is optional for fixed devices like RAID controllers.

DISK is the same disk name specified in the data option. PARITY is one of the parity names: parity, 2-parity, 3-parity, 4-parity, 5-parity, 6-parity, or z-parity.

In the specified OPTIONS, the %s string is replaced by the device name. For RAID controllers, the device is likely fixed, and you may not need to use %s.

Refer to the smartmontools documentation for possible options:

https://www.smartmontools.org/wiki/Supported_RAID-Controllers
https://www.smartmontools.org/wiki/Supported_USB-Devices

For example:

smartctl parity -d sat %s

smartignore DISK/PARITY ATTR [ATTR...]

Ignores the specified SMART attribute when computing the probability of disk failure. This option is useful if a disk reports unusual or misleading values for a particular attribute.

DISK is the same disk name specified in the data option. PARITY is one of the parity names: parity, 2-parity, 3-parity, 4-parity, 5-parity, 6-parity, or z-parity. The special value * can be used to ignore the attribute on all disks.

For example, to ignore the Current Pending Sector Count attribute on all disks:

smartignore * 197

To ignore it only on the first parity disk:

smartignore parity 197

Examples

An example of a typical configuration for Unix is:

parity /mnt/diskp/snapraid.parity
content /mnt/diskp/snapraid.content
content /var/snapraid/snapraid.content
data d1 /mnt/disk1/
data d2 /mnt/disk2/
data d3 /mnt/disk3/
exclude /lost+found/
exclude /tmp/
smartctl d1 -d sat %s
smartctl d2 -d usbjmicron %s
smartctl parity -d areca,1/1 /dev/sg0
smartctl 2-parity -d areca,2/1 /dev/sg0

An example of a typical configuration for Windows is:

parity E:\snapraid.parity
content E:\snapraid.content
content C:\snapraid\snapraid.content
data d1 G:\array\
data d2 H:\array\
data d3 I:\array\
exclude Thumbs.db
exclude \$RECYCLE.BIN
exclude \System Volume Information
smartctl d1 -d sat %s
smartctl d2 -d usbjmicron %s
smartctl parity -d areca,1/1 /dev/arcmsr0
smartctl 2-parity -d areca,2/1 /dev/arcmsr0