Patch for ext2 online growth
PLEASE NOTE: THIS PAGE HAS MOVED
The online ext2 resize patch allows you to "safely" growing of an
ext2 filesystem WHILE IT IS MOUNTED. The growth of
the filesystem can happen while the FS is in use, with open files,
and no impact to programs reading from or writing to the filesystem.
I only quote "safely" because the patch is not in widespread use, so
there may be some unresolved issues with the patch. I'm especially not
responsible for people without a backup - always a good idea even when
not doing things like this.
Please note: The online-ext2 code has become part of the
ext2resize project
at SourceForge. All current releases of online-ext2 are at SourceForge.
Full code for 2.2.18 ext3 online resizing
here. This code is not fully
complete or tested, and does not have any support from the ext2prepare
program (some support is in ext2resize in CVS). This is a full fs/ext3
tree, so you should move your existing one out of the way. It does not
contain any of the other parts of ext3 (e.g. jfs, core kernel changes)
so you would have to apply the ext3-0.0.7a patch in advance. It also
has LFS support, so it is likely you need to have LFS support patched
into your kernel for this to work.
The changes for online resizing are confined to two files
so far, fs/ext2/super.c and fs/ext2/balloc.c, about 300 more lines
including comments and debugging. The full ext2 growth functionality
is in the kernel, but to really use it you need to have some user-space
tools, ext2prepare and ext2online, which are now part of the ext2resize
utilities (starting with devel version v1.1.6 and later).
There are patchs available for 2.0, 2.2, and 2.4. The actual functionality
of the patch hasn't changed much since its inception.
This patch DOES NOT give you the ability to resize
DOS partitions (i.e. partitions you made with fdisk or
similar). You need to be using something like a Logical Volume Manager(
LVM)
for Linux, or MD RAID devices, or hardware RAID units (or for testing,
loopback and not-full-size filesystems on larger partitions).
Examples of using ext2online with a mounted filesystem
under load and with LVM.
- Right now (with a patched kernel) you can:
Grow ANY mounted ext2 FS to the end of a group (8MB for 1k blocks,
128MB for 4k blocks) without any tools besides the kernel patch (using
mount -o remount,resize=size device).
- Right now with the ext2online tool + patched kernel you can:
Grow any mounted ext2 FS to the end of EXT2_DESC_PER_BLOCK groups
(normally 256MB for 1k blocks, 16GB for 4k blocks) without having to
unmount the filesystem. This again works on any normal ext2 filesystem
when you have the patched (19991014 or later, or 20000226 or later for
sparse filesystems) kernel running. The ext2online tool will create
group descriptors, and groups in the new areas of the disk without
touching the running filesystem, and then the online kernel resize will
start using these new groups.
- Right now the ext2prepare + ext2online tools + patched kernel you can:
Grow a mounted prepared filesystem to the limits of what you prepare for
(up to 2 TB). The reason for the "prepared" part is the need to add a
group descriptor block after the next multiple of 32/128 groups are used
up. The ext2prepare filesystem moves the metadata around in an unmounted
filesystem so that you can increase the number of group descriptor blocks
in a mounted filesystem when you need to. Since each reserved block
will give 256MB, 4GB, or 16GB for 1k, 2k or 4k block sizes respectively,
not much space needs to be reserved in order to grow for most needs. It
is good to note that a "prepared" filesystem is still a legal ext2
filesystem in every way, so it is possible to use it with an unpatched
kernel.
In order to be certain that the future GDT blocks are not used by other
files, they need to be assigned somewhere, so they are marked against a
special inode (#7 = EXT2_RESIZE_INO), and this requires a small
patch to e2fsck (versions less than 1.23 only)
so it doesn't complain about this inode being used. The patch assumes
you have a suitably modified ext2_fs.h, which happens if you apply the
online-ext2 patch. If you don't have the patch, you should say "n" when
e2fsck asks "Reserved inode 7 has bad mode. Fix?" and all will be OK.
If an unmodified e2fsck is run against a modified filesystem at boot
time, the only thing that will happen is that the filesystem will become
"unprepared", and you won't be able to resize past the normal group
blocks limit (per case 2 above).
BUGS/CAVEATS:
- For 1k block filesystems, there is a current limit of growth at 45GB if
you start with a size smaller 23GB. Similarly, there is a limit for 2k
block filesystems at 362GB if you start with a filesystem smaller than
181GB. This is due to the requirement of reserving the GDT blocks in an
inode, and I don't want to use triple indirect blocks for this (laziness
mostly). You probably don't want to use a 1k block size for filesystems
that large anyways, as there is a huge amount of wasted space in the
metadata and fsck times are very slow. However, if you grow a filesystem
incrementally (ie don't double the size at the above limits), then it is
possible to grow the filesystem arbitrarily.
- You can't resize a filesystem with ext2online when it is unmounted, but
who cares at that point, since you can always use ext2resize. In the
future, it may be that ext2online is integrated with ext2resize, so
there is a single tool for resizing filesystems whether they are mounted
or unmounted.
- If testing the patch, you should mount with "debug,check=strict" options,
so that the debug messages are printed to the kernel syslog. You need
something like "kern.*<tab>/var/log/kern.log" in your
/etc/syslog.conf to output all the kernel messages, and then
"tail -f /var/log/kern.log" in another window so you can watch what's
going on while you resize.
- I haven't tested on anything other than an x86 platform. I think I have
all of the endian conversions for the 2.2 patch on big-endian platforms,
but I have no way of testing this. The user tools definitely do not
work on non-x86 platforms.
Let me know of any issues at all with the patch. Testing wise,
I've been running resizes every few seconds on a filesystem while
simultaneously copying 11 MB/1500 files into the filesystem into a new
directory each time. This goes on for several hours (thousands of +2MB
or +random resizes) until I fill the disk with many thousands of files
(1GB worth). No problems with the latest patches.
The 2.0 patch is not currently maintained. If there is real demand for it,
a new patch could be made. However, it would also be easy (if you really
need it) to manually apply the 2.2 patch to the 2.0 kernel, since they are
nearly identical for the code in question.
Latest 2.0 patch: online-ext2-2.0-19991014.diff
Old patches available
(and why they were replaced)
Thanks to Miguel de Icaza for his
ext2volume patch
for the basis for this work. While he worked with whole disks, I can use LVM
or MD to handle the growth for me (meaning less kernel mods), and I'm left
with a single normal filesystem afterwards. The other plus of my patch is that
the resize code path is only used at resize time, whereas Miguel's was used for
each block access, much like LVM, so there was more overhead.
Back to Andreas' home page.
You are visitor No:
77,016
<adilger at shaw dot ca>