PCIe M.2 NVMe for storage array

Ever since the release of the Flashstor and this mini NAS PoC I’ve been thinking about using M.2 (gen3) for my home server, if I build it myself and don’t buy a Synology.

Would you think that’s safe for data storage? I know that U.2 is also an option, but it’s still too expensive, unless I buy the U.2 SSDs on the used market.

I’d go for the (apparently still unreleased) Icy Dock ToughArmor MB873MP-B V2 with eight hotswappable M.2: https://global.icydock.com/product_363.html

It’s OCuLink on the backplane, so I’d use the following adapters & HBA to connect to it:

With this setup I’d be able to attach eight gen3 or six gen4 M.2 SSDs. The Asus board also has two more PCIe 3.0 x4 slots (in x16 physical), so theoretically I could even go for ten gen3 M.2 SSDs. (But the Icy Dock cage has only 8 slots anyway.)

I guess it would be a really fast setup with RAIDZ1, and the PCH should be able to handle it (12 x 4.0, 16 x 3.0 PCIe lanes, DMI bandwidth at PCIe 4.0 x8).

But the main question is what you think about using M.2 for data storage. Money is a slight issue, so I’d probably go for Team Group MP34 gen3 SSDs (2,400TBW for the 4TB model).

PS: with such a fast storage, would there be a need for an L2ARC? At any rate, I would use L2ARC for metadata only, but the RAID speed for read operations would probably exceed even gen4 M.2 L2ARC cache speeds (and saturate the dual 25GbE NIC anyway).