The question: what do you do when the one disk in a hypervisor host starts throwing SMART warnings, and how do you move the guests off with the least downtime? I have an order of operations, but I would like to compare it with yours.
Say a standalone Proxmox box, one disk, no mirror, no second node. The warning is a climbing reallocated or pending sector count, not a dead drive. That is the worst time to be clever. Every hour of heavy reading on a sick disk is a gamble, so I would rather copy once and copy cleanly.
My lean:
- Check the physical layer first. A loose or bad SATA cable can raise CRC errors that look like a disk problem. Check the counters, reseat the cable, and see whether the number stops moving. Reallocated sectors are the disk. CRC errors are usually the cable.
- Run a fresh backup of every guest to a different disk or machine before anything else. Not a migration, a backup. Then confirm you can read it back.
- Get the replacement disk in, or a second box, and restore or migrate from there. On Proxmox that is
vzdumpandqmrestoreorpct restore. - Shut the guests down for the final copy. A cold copy of a guest is boring and reliable. Live migration without shared storage is possible, but it reads the whole disk while the guest keeps writing to it, and that is more stress than I want to put on a failing drive.
So my answer to "least downtime" is a short, planned outage per guest, not zero. I would take twenty minutes of known downtime over a live move that dies at 80 percent.
Where I am unsure is the order of the guests. I would move DNS first and the least important last, but I have not found a rule I trust beyond that.
Do you clone the whole disk first with something like ddrescue, or go guest by guest? And has anyone had a drive with rising warnings last for months?
Dwayne