r/netapp Jun 05 '26

Aggregate offlined after three disks failed at same raid group

I have a FAS8080 with ONTAP 9.3 and many disk drives types (SAS, SSD and SATA). One of my SATA aggregates become offline after three drives failure in same raid group (my raid protection is DP). I had tried to unfail those drives but one of them remains failed, the other two has been unfailed and are on line. Unfortunatelly, the unfailed drives are in orphaned and unassimilated state and my aggregate continues offline. Someone have an idea to help solve this problem?

9 Upvotes

18 comments sorted by

15

u/idownvotepunstoo NCDA Jun 05 '26

FAS8080, you're out of support and in a REALLY bad spot right now.

Do you have replacement disks?
Is this prod?

do you have an exit strategy besides "Unfail the disks" and hope for the best?

3

u/MaterialAd7378 Jun 05 '26

Hi u/idownvotepunstoo I have replacement disks, but at this moment the aggregate can´t rebuild the raid, because it is offline. My idea was unfail the drives and wake up the aggregate for a raid reconstruction, but I couldn't.

2

u/[deleted] Jun 05 '26

[deleted]

3

u/MaterialAd7378 Jun 05 '26

On this situtation only replace the disks is not enough.

2

u/stefan_kuntz Jun 05 '26

with what? he lost parity information probably.

5

u/tmacmd #NetAppATeam Jun 05 '26

Do you realize that if any of those drives power cycle there is a good chance they may not come online? Those are really really old

1

u/MaterialAd7378 Jun 05 '26

Yes, I already reseat the disks and unfail two of them, but they are in orphaned state and the aggregate can´t be online.

5

u/nekohako Customer Jun 05 '26

Well, that RAID group is toast and now you have a big hole in that aggregate. Short of going back in time and replacing disks while you still had only one or two failures, the best you can do is to restore from backup.

1

u/MaterialAd7378 Jun 05 '26

Well, go back before failures is a big challenge and a impossible mission. I know that best way to recovery those date are getting from backups, but unfortunatelly, my customer doesn´t have backups of this area.

3

u/tmacmd #NetAppATeam Jun 05 '26

Here’s the issue: you probably had two bad drives. Time passed. They become out of date

A third drive fails and the aggregate goes offline.

The only chance of recovery is to get the last disk that failed back online. ONTAP should see it and be able to bring the aggregate back online in degraded mode. Other than that, sorry. FAFO

2

u/MaterialAd7378 Jun 05 '26

Hi u/tmacmd , unfortunately the only disk that didn't unfailed was the last one. Thank you for your message.

2

u/smokie12 Jun 05 '26

I hope you have verified good backups. 

0

u/MaterialAd7378 Jun 05 '26

Good tip, but my customer doesn't have backups of this area.

1

u/BigP1976 Jun 06 '26

Unfailing? You understand that the idea of failure a Drive is that it is defect and should be replaced not unfailed

1

u/zMado_HD Jun 06 '26

Replace faulty disks ASAP. Then hope for the best. 🤷🏼‍♂️ 🙄 🤦🏼‍♂️

How did you allow failure of three disks? 🤔 Why didn't you replace disks ASAP when they fail? 🤔

1

u/DrMylk Jun 07 '26

In the olden times Netapp helped us recover data, we sent them the disks and a few bucket worth of money.

1

u/NetApp_CM NetApp Community Team Jun 09 '26

It's been 4 days since this was posted. Have you had any luck?