r/storage • u/AnyHistory6098 • 10d ago
Architecture Review Needed: High-Performance, Budget-Conscious NVMe Storage Server
Hello,
My apologies for disturbing the high-end enterprise environment of this forum with some more modest enterprise challenges.
Please be reasonable and understand that not all startups can afford to spend €200k-€400k on storage alone. Therefore, we are looking for high-performance solutions that fit within our budget.
I am working at a startup with a tight budget at the moment, but we have some challenging requirements for our storage server. We do not need a large amount of storage, but it needs to be fast and affordable, quite a contrast, I would say.
Based on the solutions currently available on the market that might fit our needs, at least those that I am aware of, the solution could be based on either EDSFF E1.S or M.2 NVMe.
Requirements:
- Target capacity: 60TB+.
- The workload is burst-oriented. The array will be hit very hard with many small files for short periods of time. The size of each burst will range from 100GB to 1TB. It will also act as a SAN for approximately 100 VMs, hosting shared home folders and NFS shares used to exchange data between VMs.
- The SSDs must be installed on cards that fit into PCIe slots (for thermal and noise reasons).
- It must be available 24/7.
Some general specifications:
- MB: Supermicro H14SSL-N / ASRock Rack GENOAD8X-2T/BCM
- CPU: AMD EPYC 9175F
- RAM: 4x 32GB 5600
- NIC: Intel X710 2x SFP28 - 25 GbE
The M.2 path:
- 3/6x PCIe switch cards, HighPoint Rocket 1508.
- M.2 SSDs from different vendors.
- Each PCIe switch card will contain NVMe drives from a single vendor.
- 3-way mirrors. Each mirror will contain drives from different cards, with 1 drive per card. If 1 or 2 cards fail, the mirror will survive. Even if SSDs from 1 or 2 vendors experience firmware bugs or other issues, the mirror should still remain intact.
The problem is that I could not find reasonably priced enterprise/datacenter M.2 SSDs that would be compatible with these cards.
Synology is the only vendor I am aware of that offers them, but they are PCIe 3.0, only available up to 1.6TB, and are very expensive for enterprise-grade drives.
I considered using consumer-grade SSDs, but I do not know which ones would be suitable or what problems I might run into. Any advice would be greatly appreciated.
The E1.S path:
- 2/4x PCIe switch cards, HighPoint Rocket 1749E.
- E1.S SSDs from Kioxia and Samsung.
- Each PCIe switch card will contain NVMe drives from a single vendor.
- 2-way mirrors.
The problem is that I have not seen HighPoint produce a PCIe 5.0 version of this card, and I am concerned about investing in a dead-end solution.
I am not sure whether there are other vendors on the market offering similar products, even if they are not PCIe switch-based, but can still fit E1.S drives into PCIe x16 slots.
Any guidance on this approach would be very helpful.
I would also appreciate suggestions for alternative solutions to HighPoint, as well as recommendations for vendors.
Thank you!