General

pros

  • better performance for most workloads excluding random IO from multiple sources

cons

  • no easy add or remove drive
  • needs same disks
  • can’t use already populated disk
  • write pattern for all disks is the same, which is problematic for all SSD array - it’s a bigger chance for failure of more than one disk at once

Tuning

https://news.ycombinator.com/item?id=38484708

https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#postgresql

Blocksize in Proxmox??
How it relates to ashift and single disk pool?
Performance, write multiplication?

image

https://forum.proxmox.com/threads/adding-zfs-pool-with-ashift-12-which-block-size-option.46594/post-473950

The ashift defines what the smallest blocksize that can be used will be. If you got 8 disks in a striped mirror with disks that use native 512B/512B logical/physical sectors but you use a ashift of 12 (=4k blocks used per drive) you still got a minimum volblocksize of 4x 4kb = 16k. If you use ashift=9 (512B blocksize per disk) it would be 4x 512B = 2KB minimum volblocksize.

Also keep in mind when using SSDs that they will lie about the physical/logical blocksize. Internally they all should use something like 8K, 16k or even bigger but not a single manufacturer will tell you whats actually used internally because it would make them look bad. So there it makes no sense to use a ashift of below 12 as you just move the write amplificaion from the host to the SSD.

Tips

zpool status
 
zfs list -t snapshot
 
arc_summary

Unmirror mirror.
Remove one disk from mirror.
Just use detach
https://forum.proxmox.com/threads/split-zfs-mirror-rpool-into-2-single-disks.109680/

zpool detach POOLNAME DISKID

Encryption

ForProxmox usage, it’s better to use LUKS than native encryption.
Speed isn’t that different but compatibility is much better with LUKS.
https://old.reddit.com/r/zfs/comments/wdrfxp/testing_and_comparing_io_performance_with_and/

RAIDZ expansion in ZFS will arrive in OpenZFS 2.3

https://github.com/openzfs/zfs/pull/15022#issuecomment-1802428899