General
pros
- better performance for most workloads excluding random IO from multiple sources
cons
- no easy add or remove drive
- needs same disks
- can’t use already populated disk
- write pattern for all disks is the same, which is problematic for all SSD array - it’s a bigger chance for failure of more than one disk at once
Tuning
https://news.ycombinator.com/item?id=38484708
https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#postgresql
Blocksize in Proxmox??
How it relates to ashift and single disk pool?
Performance, write multiplication?
The ashift defines what the smallest blocksize that can be used will be. If you got 8 disks in a striped mirror with disks that use native 512B/512B logical/physical sectors but you use a ashift of 12 (=4k blocks used per drive) you still got a minimum volblocksize of 4x 4kb = 16k. If you use ashift=9 (512B blocksize per disk) it would be 4x 512B = 2KB minimum volblocksize.
Also keep in mind when using SSDs that they will lie about the physical/logical blocksize. Internally they all should use something like 8K, 16k or even bigger but not a single manufacturer will tell you whats actually used internally because it would make them look bad. So there it makes no sense to use a ashift of below 12 as you just move the write amplificaion from the host to the SSD.
Tips
zpool status
zfs list -t snapshot
arc_summaryUnmirror mirror.
Remove one disk from mirror.
Just use detach
https://forum.proxmox.com/threads/split-zfs-mirror-rpool-into-2-single-disks.109680/
zpool detach POOLNAME DISKIDEncryption
ForProxmox usage, it’s better to use LUKS than native encryption.
Speed isn’t that different but compatibility is much better with LUKS.
https://old.reddit.com/r/zfs/comments/wdrfxp/testing_and_comparing_io_performance_with_and/
RAIDZ expansion in ZFS will arrive in OpenZFS 2.3
https://github.com/openzfs/zfs/pull/15022#issuecomment-1802428899