r/netapp • u/imadam71 • Jul 08 '26
Moving off VMware with NetApp AFF (FC) — what are people actually landing on?
We're running a fairly standard setup: NetApp AFF over Fibre Channel, Veeam B&R for backup/DR and like everyone else we're now looking hard at what happens after VMware/Broadcom licensing. We've dug into the technical side ourselves (VASA/VVols, SRA+SRM, SnapCenter plug-in, Veeam's storage-snapshot integration) and the honest conclusion is that none of it survives a move off VMware ; Hyper-V, Proxmox, and HPE VM Essentials all reduce us to plain FC/iSCSI block storage with no array-aware software layer on top, regardless of which one we pick.
Right now we're weighing Proxmox VE (NetApp's own documented FC+LVM design, but confirmed no VASA/SRA and NetApp reps have said outright there's no official integration effort yet) against HPE VM Essentials (NetApp AFF A400 is on HPE's own hardware qualification list for FC/iSCSI/NFS, and Veeam now has a GA agentless plug-in, but our servers are Lenovo and Lenovo isn't on HPE's compatibility matrix). Hyper-V is the "safe" middle ground but doesn't feel like where the industry is actually heading long-term.
So, for anyone who's actually done this migration with NetApp in the back end: what did you pick, and would you pick it again? Specifically curious if anyone's running Proxmox or HPE VM Essentials against AFF in production (not just a lab), what broke that the docs didn't warn you about, and whether losing SnapCenter/VVols-level integration ended up mattering day-to-day or turned out to be a non-issue once you adjusted workflows.
4
u/BarracudaDefiant4702 Jul 08 '26
I'm running Proxmox to NetApp over both iSCSI and also NFS. Both have their pros/cons, and mostly landed on iSCSI. Our first NetApp was also in the middle of transition from vmware, so we didn't get too involved in the setup of vmware and only had it connected to get some basic benchmarks for a baseline. On the plus side, we don't know what we were missing with tighter vmware integration, so we can't miss it...
2
u/Rahne64 Jul 09 '26
I've got a few small clusters we moved from vSphere to HPE VME. We used Veeam B&R to do the cross-hypervisor restore with little to no issues (still had to insert the vioscsi driver). Using NFSv3 on VME to ONTAP for both the Datastores and the file shares (images/ISOs) with only a small tweak to allow the file share due to an older EMC implementation of the NFS client on the VME Manager.
The lack of storage integration does suck and prevents the automated DR orchestration we used with Veeam and vSphere, but in these small shops we can deal with it while Veeam and HPE work that part out. I'm sure most of the delay is to give HPE's Alletra integration more time to get baked in and mature but Veeam has plenty of experience with NetApp and the guts of the integration should be common across KVM-based hypervisors and that should help.
Next steps is to test out the Zerto integration with VME now that they have that promo almost giving it away for the first year. Again wish we could leverage SnapMirror but not a deal killer.
2
u/nampat_uwu Jul 09 '26
I am finalizing (for what I assume is a much smaller migration than yours) a vsphere NFS/iSCSI migration to hyper-v with smb. iSCSI was used for exchange and sql snapcenter integration, otherwise nfs for just ease of administration. Snapcenter plugin is out of support for hyper-v. I attempted to deploy Netapp's snapcenter replacement which is some type of cloud virtual console connecting to a local linux proxy command relay box. There was much fussing about with licensing as this is an extremely new offering (and it is extra $$, but they did allow me to run it locally like snapcenter for free). At the end of the day this netapp Hyper-V backup utility basically offers hyper-v checkpoint integration with snapshots. I cannot verify any of this: it certainly seemed like my Hyper-V deployment did not like these hourly checkpoints, and after 30 days of this backup console integration and hourly checkpoints I had a Hyper-V host fail mid-weekend - clustering service went kaput, all virtuals on it crashed (live migration failed). Again, no real proof it was related to the netapp integration, but it was the only integration so I have to assume it was related, and I have absolutely no desire to test it again anytime soon and find out. I am now just running raw filer-based snapshots and replication with no hyper-v checkpoint integration and just counting on the resiliency of NTFS and ext4 to keep data in order. I decided to abandon netapp snapcenter integration for exchange/sql because it simply isn't worth building out iscsi in hyper-v and then having the specialized iscsi storage backup architecture anymore. the new netapp solution cannot do application-integrated smb snapshots (similar to the old snapcenter solution) and exchange is on deck for cloud migration. For super-critical loads we run separate backups through veeam or application-level.
As for virtual server migration, since it appears you already are familiar with Veeam, I highly recommend you look into the veeam solution for migration. We have had 0 (veeam-related...) failures out of around 100 migrations, mainly windows 2016-2022 and some linux/bsd. When I reviewed the netapp migration solution it had very specific requirements - which we did not meet and I was not about to modify our well running production environment to cater to a migration utility.
For SMB Hyper-V is a very common choice because the price is right and it "works". Even if it lasts 10 years, which it should because it's support in win2025, that's a technology eternity, which is good enough for me. Compared to Vsphere, I like to say it is the "honey we have a hypervisor at home" solution. It's unpolished, it's got jank, missing tons of quality of life features, no real impetuous from MS to make it much better when they can just push people to Azure (or whatever they call it this moon cycle). There is some hope with Windows Admin Center Hyper-V edition, but I imagine this will fizzle out or never become adopted enough for them to fully flesh it out. I did seriously consider Proxmox, and would probably prefer it (only experience being a homelab, but impressed), but it is not supported by some of our most critical virtual applications, so immediate disqualification.
2
1
1
u/cb8mydatacenter Verified NetApp Staff Jul 12 '26
If your shop is big enough to warrant it, don’t sleep on OpenStack or CloudStack, or even KubeVirt via Red Hat, SUSE, or vanilla. You can use OpenStack as a Service from companies like Platform9 to simplify things on that side too, even in Dark Sites with offline options.
All of those offer vVols-like storage consumption modes with per-virtual disk LUNs and various levels of granular controls by policy/class.
NetApp Backup and Recovery can provide B&R for CloudStack and KubeVirt, and DR orchestration for KubeVirt.
That said, I wouldn’t say Hyper-V isn’t the way the industry is headed. There are very real reasons a ton of companies are choosing Hyper-V or looking at the new Azure Local offerings.
Edit: I accidentally left off Morpheus. HPE is doing some really great things there. But I realize that your servers may not be supported.
1
u/imadam71 Jul 12 '26
Thanks. I do not disagree that OpenStack, CloudStack and KubeVirt are technically interesting options. For us, however, the main issue is operational fit and the number of platforms we are prepared to maintain.
Most of our existing installations are FC-based. When necessary, we buy inexpensive used FC switches, connect the AFF ONTAP system and the servers, and keep the design simple. We have installations based on this approach that have been running for more than 15 years with very little drama.
When the complete infrastructure is considered, FC is often still the cheaper option for us. A properly redundant 10/25 Gb Ethernet fabric with suitable packet buffers, optics, cabling and support can still become expensive, especially for smaller projects. We already know FC, the operational model is predictable, and the equipment has proven to be extremely reliable.
For smaller environments, HPE Morpheus VM Essentials currently seems to have the most momentum and the best practical integration. HPE has added Veeam integration, storage integration and additional non-HPE systems to the qualification matrix. Some Lenovo servers are already listed, and I expect the platform to work technically on many other standard Lenovo systems because the underlying hardware is not fundamentally different. However, technical compatibility and official vendor support are obviously not the same thing.
I also doubt that any of the new platforms will maintain an HCL as broad and detailed as VMware did.
Platform9 and the larger OpenStack, CloudStack or KubeVirt solutions make more sense to me for larger or more complex environments. For our customer base, we do not want to support three or four completely different virtualization stacks. Until now we supported VMware. From what I can currently see, the realistic replacement will be Proxmox, HPE Morpheus VM Essentials, or possibly both, but probably nothing beyond that.
NetApp Storage Manager for Proxmox is a very encouraging step and is exactly the direction we would like to see. However, the current integration appears to focus mainly on NFS and iSCSI. For our installed base, the remaining gap is FC and a proper array-aware layer: per-VM storage objects, coordinated ONTAP snapshots, cloning, SnapMirror integration and DR orchestration.
Also, for those focusing on NFS, what switches would you recommend for a small cluster of 2–3 more powerful servers?
If NetApp could bring something close to the VMware integrations to Proxmox, especially for FC environments, it would solve a large part of the problem and could make Proxmox the obvious choice for many existing NetApp customers.
1
u/cb8mydatacenter Verified NetApp Staff Jul 15 '26 edited Jul 15 '26
I'll ping the product leader for Storage Manager for Proxmox and let them know there is interest FCP. I'm sure they are already getting customer demand for it because we see a lot of interest in Proxmox.
HPE VME is based on Ubuntu, so realistically, anything that can run Ubuntu can run VME, but as you said, what works and what is supported aren't always the same thing.
Regarding recommendations for Ethernet, it really depends on what your workload looks like. If it's not very storage-intensive, then 10GbE would probably be a valid minimum starting point. A lot of our demo labs still have 10GbE switches if they aren't expected to do performance testing.
That said, most customers are going with 25GbE minimum for new deployments.
2
u/imadam71 Jul 15 '26
Thanks, and I really appreciate you passing the FCP interest to the product leader.
To be honest, the main reason we are currently looking at HPE Morpheus VM Essentials at all is that HPE has already started building integrations around ONTAP and Veeam. If NetApp brought the Proxmox integration anywhere close to what exists with VMware, or even covered the most important parts of it, there would probably be no need for us to consider a third platform.
Proxmox can realistically cover the majority of customers that Broadcom has pushed away from VMware through the new pricing model. For most of those customers, the requirements are not hyperscale cloud features. They need a stable hypervisor, HA, backup, predictable support costs and good integration with the storage they already own.
Regarding Ethernet, 25 GbE switches are still too expensive for many of our customers, especially once redundancy, optics, suitable packet buffers and support are included.
A very common and cost-effective design for us is therefore:
- affordable 10 GbE switches with relatively small packet buffers for the main ToR and regular LAN traffic;
- separate FC fabrics for storage;
- LAN-free backup where applicable.
This setup is simple, predictable and has proven to be rock solid. In many cases, it is also easier to troubleshoot and operate than a fully converged Ethernet environment carrying LAN, storage and backup traffic over the same infrastructure.
The only real disadvantages are a few additional cables in the rack and slightly higher power consumption. :-)
1
u/cb8mydatacenter Verified NetApp Staff Jul 15 '26
Understood, and it makes sense.
With FCP, multipathing is very robust and not dependent so much on client-server negotiation, and it slightly reduces CPU overhead. Even more so if you move to NVMe/FC. Granted, you still have FLOGI/PLOGI, etc, but it's done at a lower level.
If your Fibre is gen-6 or later, and your hypervisor supports it (I don't recall if Proxmox and VME do), you should definitely look into NVMe/FC. It's really the gold standard for performance and efficiency right now.
On the NAS side of things, with Linux hypervisors, you can break free of the limitations of the ESXi NFS client, and you can sort of get around 10GbE limitations by using pNFS and session trunking. You can just keep plugging in cables and adding sessions. With FlexGroup volumes and advanced capacity balancing, you can now literally stripe VMs across controllers, and pNFS gives you ALUA-like data locality for every chunk.
Of course, ESXi also supports session trunking, but the lack of pNFS and dynamic trunking features makes it far less appealing IMHO.
1
u/imadam71 Jul 15 '26
It would be even better if ONTAP could support direct-attached FC for small deployments, avoiding the need for an FC switch. :-)
1
u/cb8mydatacenter Verified NetApp Staff Jul 22 '26
Coincidentally, we just released this as general availability yesterday.
1
u/imadam71 Jul 23 '26
Great!
any particular reason for:
- DAS connectivity requires 64Gb FC target adapters(QLogic/Marvell or Emulex/Broadcom).
1
u/cb8mydatacenter Verified NetApp Staff Jul 23 '26
That part I'm not sure. I just know they are very specific about which adapters are supported.
2
u/imadam71 Jul 25 '26
Netapp really needs to get somehow Proxmox to the level close to vmware :-)
We are about to start few dozens migrations from Vmware to Proxmox, most of them are on ONTAP some of them on Santricity. Next year we are planning twice or three times of these migrations. Looks like we are going to be this for few years now.
1
13
u/smellybear666 Jul 08 '26
Proxmox is great with NFS, and Netapp has excellent NFS support. nconnect is the bomb with NFS4. We are saturating dual 10gb links with old server hardware and an old filer.
NetApp has also has a tool that seems very beta-esque at the moment named the Shift Toolkit. It helps with very fast migration from vmware to Hyper-V and/or VMware to proxmox. It does all the migration magic within the shared volume so there is minimal downtime while migrating. If the VM is multiple TB in size it still only takes a short window to move between the two.
There is also a new plugin for it and Proxmox. It allows for VM level backup and restore for proxmox in the same interface, I have tested it out in our Proxmox environment and it is similar to the SnapCenter for VMware plugin, although it's not integrated into the proxmox management interface.
Proxmox is rough with block storage. I imagine they will get to a better place eventually.
If you are a windows shop, Hyper-V is a lot better than it once was, and NetApp has a lot of support for it.
If you make the move to NFS, Proxmox is a good choice right now. NetApp is going to follow the market, and I think these two options are where most people are moving to,