tl;dr Set up e-mail alerts so that you are alerted to when things (start to) go wrong.
This morning, I got an email from my FreeNAS server with the following:
New alerts: * Boot Pool Status Is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.
Logging in to the server, I ran zpool status freenas-boot to see the state of the pool:
# zpool status pool: freenas-boot state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://illumos.org/msg/ZFS-8000-9P scan: scrub repaired 98.5K in 0 days 01:08:27 with 0 errors on Wed May 27 04:53:27 2020 config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/44bef123-fee5-11e4-9e92-0cc47a4a5aff ONLINE 0 0 1 ada0p2 ONLINE 0 0 0 block size: 512B configured, 4096B native errors: No known data errors
Luckily, this pool is made up of a mirrored vdev: a USB flash drive (gptid/44bef123-fee5-11e4-9e92-0cc47a4a5aff) and an SSD (ada0/p2). Per zpool status, I ran zpool clear to clear the error. The next scrub will be towards the end of the month (June), so we will see if this error comes back. In the meantime, I have started to look into replacing the flash drive with a SATA DOM.
# zpool clear freenas-boot # zpool status freenas-boot pool: freenas-boot state: ONLINE status: One or more devices are configured to use a non-native block size. Expect reduced performance. action: Replace affected devices with devices that support the configured block size, or migrate data to a properly configured pool. scan: scrub repaired 98.5K in 0 days 01:08:27 with 0 errors on Wed May 27 04:53:27 2020 config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/44bef123-fee5-11e4-9e92-0cc47a4a5aff ONLINE 0 0 0 ada0p2 ONLINE 0 0 0 block size: 512B configured, 4096B native errors: No known data errors
The moral of this story: set up email alerts or some kind of alerting system to alert you when things (start to) go wrong.