You try to save a config file, restart a service, or just run apt update, and the server snaps back with Read-only file system. Nothing's full, you're root, and five minutes ago everything worked. It's one of the scarier errors on a VPS because it usually means the kernel decided your disk wasn't safe to write to. The good news: most of the time it's recoverable without losing data, as long as you don't panic-reboot first.
Symptom: what it looks like
You'll usually spot it in one of these ways:
touch /root/testreturnstouch: cannot touch '/root/test': Read-only file system- MySQL/MariaDB crashes or refuses to start, with
Errcode: 30 "Read-only file system"in its log - Nginx or Apache can't write logs, PHP sessions fail, and cpanel" class="auto-link">WordPress shows database or upload errors
aptordnffails withE: Could not open lock file ... (30: Read-only file system)- Cron jobs silently stop producing output
Confirm it in one command:
mount | grep ' / '
# or
findmnt -no OPTIONS /
If you see ro at the start of the options (instead of rw), the root filesystem has been remounted read-only.
Cause: why Linux flips a disk to read-only
Linux doesn't do this for fun. ext4 is usually mounted with errors=remount-ro, which means: "if you detect corruption or an I/O error, stop writing immediately so you don't make it worse." It's a safety brake. The common triggers:
| Trigger | What you'll see in dmesg | Typical fix |
|---|---|---|
| Filesystem metadata corruption (unclean shutdown, kernel bug) | EXT4-fs error ... Remounting filesystem read-only | Run fsck from rescue mode |
| Underlying storage I/O errors (host-side hiccup or failing disk) | blk_update_request: I/O error, Buffer I/O error on dev vda1 | Contact support, then fsck |
| Journal problems | JBD2: ... journal aborted | fsck, sometimes journal rebuild |
Someone mounted it ro on purpose | Nothing alarming; /etc/fstab has ro | Fix fstab, remount |
| XFS shutdown (AlmaLinux/Rocky default) | XFS (vda1): Corruption detected ... Filesystem has been shut down | xfs_repair from rescue mode |
So step one is always: find out which of these it is.
Fix: step by step
1. Read the kernel log before you touch anything
dmesg -T | grep -iE 'ext4|xfs|i/o error|remount|jbd2' | tail -40
If dmesg doesn't work because the log is huge, try journalctl -k -b | tail -100. Copy whatever you find somewhere off the server (your laptop, a support ticket). If the box goes down during repair, that output is the only evidence of what happened.
2. Get your data out while you still can
A read-only filesystem is still readable. That's your window. Before trying any repair, pull off the stuff you can't lose:
# databases: if MySQL is still running, dump to another machine over SSH
mysqldump --all-databases --single-transaction | ssh user@backup-host 'cat > all-dbs.sql'
# site files and configs
rsync -avz /var/www/ /etc/ user@backup-host:/backups/vps-ro/
Notice the dump streams straight over SSH. You can't write a local file right now, so don't try mysqldump > /root/backup.sql. It'll just fail. If you're on SkyServer and have snapshots enabled, check when the last good one was taken too.
3. Rule out the easy case: a bad fstab entry
cat /etc/fstab
If the root line literally says ro (or someone edited it recently), and dmesg is clean, you can just remount:
mount -o remount,rw /
Then fix the fstab line so it reads defaults or defaults,errors=remount-ro and save it.
Don't runmount -o remount,rw /to "make the error go away" ifdmesgshows EXT4/XFS errors or I/O errors. Forcing writes onto a corrupted filesystem is how small problems turn into lost directories.
4. Repair the filesystem from rescue mode
You can't safely fsck a mounted root filesystem. You need it unmounted, which means booting the VPS into rescue mode (or a recovery ISO) from your client panel. Once you're in the rescue system:
# find your VPS disk; it's usually vda1 or sda1
lsblk -f
# ext4: check and auto-fix
fsck.ext4 -f -y /dev/vda1
# XFS: dry run first, then real repair
xfs_repair -n /dev/vda1
xfs_repair /dev/vda1
A few notes from experience:
-fforces a full check even if the filesystem claims to be clean. Use it.- If
fsckfixes errors, run it again until it comes back clean with no changes. - If
xfs_repaircomplains about a dirty log, mount and unmount the partition once to replay it. Only usexfs_repair -Las a last resort, because it throws the log away. - Check
lost+foundon the partition afterwards. Files that fsck couldn't place end up there with numeric names.
Exit rescue mode, boot normally, and confirm with findmnt -no OPTIONS / that you're back to rw.
5. Bring services back carefully
After a corruption event, start things one at a time and read their logs:
systemctl status mariadb nginx php*-fpm
journalctl -u mariadb -n 50 --no-pager
mysqlcheck --all-databases --check
If InnoDB complains on startup, deal with that separately. InnoDB crash recovery is a whole topic of its own, and we have a separate guide for it on this blog.
6. If it happens again within days, it's not your filesystem
One-off corruption after a hard power-off happens. If it keeps coming back with I/O error lines in dmesg, the problem is below your OS: storage on the host side. Open a ticket with the exact dmesg output and timestamps. On a VPS you can't fix that by yourself, and running fsck in a loop only hides it.
Prevention
- Shut down cleanly. Use
shutdown -r noworreboot, not a hard reset from the panel, unless the box is truly frozen. - Keep off-server backups. A snapshot on the same host is nice, but a nightly
rsyncor database dump to another location is what saves you when storage itself is the problem. - Watch the kernel log. A simple cron that greps
dmesgforEXT4-fs errororI/O errorand mails you gives you a head start before the remount happens. - Don't let the disk hit 100%. A full disk doesn't cause read-only remounts by itself, but crashes during a full disk leave messier filesystems behind.
- Keep the kernel updated. A few historical ext4 corruption bugs were kernel-side and have long been patched.
Frequently Asked Questions
Will rebooting fix a read-only file system?
Sometimes, because many distros run fsck automatically at boot when the filesystem is marked dirty. But if the repair needs confirmation, the server can get stuck at an emergency shell and you'll lose SSH. Back up your data first, then reboot or go into rescue mode on purpose.
Is it safe to just run mount -o remount,rw /?
Only if dmesg shows no filesystem or I/O errors, for example when the cause was a bad /etc/fstab entry. If the kernel flagged corruption, forcing read-write can make the damage worse.
Can I run fsck on a live, mounted root partition?
No. Running a repairing fsck on a mounted filesystem can corrupt it further. Use rescue mode so the partition is unmounted. fsck -n (read-only check) is the only mode that's reasonably safe on a mounted disk, and even that can report false errors.
I'm on shared cPanel hosting and see this error. What do I do?
On shared hosting you don't control the filesystem, so it's a server-level issue. Open a support ticket with the exact error and the time it started. Don't keep retrying uploads or updates in the meantime.
Did I lose data?
Usually not, or very little. The remount exists to stop damage from spreading. Check lost+found after the repair and verify your databases with mysqlcheck. If anything is missing, restore it from the backup you took in step 2.
