Logical Volumes

Motivation

When you setup your system you will partition your hard-drive and reserve some space for it. Later you will find that the space is to small. Especially on server systems it is a good idea to have separate partition so that e.g. an user in his/her home-directory can not fill the space for the / and /var partitions, etc.. But on important server systems you also want to be able to increase that space without rebooting. You might also come across the case that you want to create a consistent snapshot of a partition or you want a volume that stretches over more then one physical device.

For all of this LVM (Logicial Volume Management) has a solution. Usually it is a good idea to setup your system with LVM even if it is just a personal laptop.

LVM concepts and terms

There are 3 main entities in LVM:

1. pv - physcial volume: where you store your data: usually a partition on a hard drive e.g. /dev/sda7 or a complete drive e.g.: /dev/sdg
2. vg - volume group: pv's grouped together to form one pool where you can store your data. Each vg has a name. E.g. you could use rootvg as the name of your main vg.
3. lv - logical volume - a block-device where you can create your files-system. Each lv takes up a space in one vg. E.g. you could have rootlv as the lv for your root file-system. If the name of the volume group would be rootvg then that would be /dev/rootvg/rootlv

Thick vs Thin

1. thick volumes are the default, they reserve all blocks in advance. Initially LVM offered only thick volumes. The main downside here are snapshots which then are computationally expensive: If you have a snapshot on a thick volume it slows down your write speed by a factor of about 3.
2. thin volumes only take up space as data is written to it. Snapshots of thin volumes are cheap and have only little impact on performance.

When to use what? For normal volumes it is a good choice to stick with the thick volumes. If you have a server that hosts a lot of virtual machines that you want to backup and snapshot then thin volumes are a better choice.

Creating a LVM, extending it and creating a snapshot

To create an vg we first not a few pv:

# those partitions below should be unused:
pvcreate /dev/sda7
pvcreate /dev/md5
pvcreate /dev/sdf6
# now they are marked as pv and are being able to be used in LVM.
vgcreate myvg /dev/sda7 /dev/md5
# if we later find there is not enough spaced we can extend the vg:
vgextend myvg /dev/sdf6
# now we are ready to create a logical volume:
lvcreate -L 5G -n testlv myvg
# the above creates a lv named "testlv" in our myvg volume group
# with a size of 5GB

After this we have a device /dev/myvg/testlv or /dev/mapper/myvg-testlv. This can be used as a normal block device. E.g. we can format it with an ext4 filesystem and mount it:

mkfs.ext4 /dev/myvg/testlv 
mkdir /mnt/test
mount /dev/myvg/testlv /mnt/test

If we now find we want to make the testlv bigger we can do this online:

lvextend -L +2G myvg/testlv

While the device is now bigger we also want the file-system to be able to use that additional space. For this we need to resize the file-system. E.g. by:

resize2fs /dev/myvg/testlv 

Let's say we have a database that writes to that volume and we do not want to stop it for too long in order to create a consistent backup. One way to do this would be to create a snapshot. But beware that if it is a thick volume that this will cost you some significant of performance.

lvcreate -L 500M -s -n testsnap myvg/testlv

The above would create a logcial volme testsnap which is a snapshot of testlv. There is 500M space for differences to the original volume. So during the time the snapshot exists you should not write more then 500M to your original volume, otherwise the snapshot becomes invalid. (In a real world example you would reserve more space).

After you have done the backup from your snapshot you should remove that snapshot (as it impacts performance of the main volume as well).

umount /dev/myvg/testsnap
lvremove myvg/testsnap

Scan and Active vs Inactive

LVM knows 2 states of your vg and lv: active and inactive. E.g. when you boot with a live CD and you have LVM configured on your hard drive the kernel of the live-system might not see the LVM on the disk as active.

In order to search your disks for signatures of LVM you can use:

pvscan
vgscan
lvscan

Useful for information are also:

pvs
vgs
lvs

Which list information about PV, VG and LVs.

This will give you information about the LVM on that system and all disks that it sees. If an entity is inactive you can activate it. E.g.:

vgchange -a y some_vg

Change horses while riding them

Let's assume you bought an external disk-shelf for your server and you find that it is too slow and you want to replaced it with a newer model and you need to transfer all your data to the new shelf but you do not want any downtime for your users as this must be up 24x7.

If we have used LVM this can be done. We connect the additional disk shelf to our server (e.g. via SAS cable) and then we create a RAID on that shelf and add the disks to our vg:

vgextend datalv /dev/md7 

Now we would like to remove our old disk shelf which is on e.g. /dev/md6. In order to be able to remove a pv from a vg we need to make sure that no data is stored on that pv.

pvmove /dev/md6

This will move all data stored on that pv to other places. (and could take along time).

After the pvmove is complete and if it was able to completely move data to the other pv in the system then we can remove that pv:

vgreduce datalv /dev/md6
# if this was successfull we can remove the pv signature from /dev/md6
pvremove /dev/md6

After that we could stop the RAID and then remove (physically disconnect) the shelf from the server.

So we changed our horse while riding it. Needless to say that you should have a good backup before doing this!

Creating a thin volume

A thin volume is created within a special thick volume that was created with the -T option. This is the "pool" where your thin LVs reside in:

lvcreate -L 30G -T myvg/thinpool1

Now we can create thin volumes inside our thinpool1: E.g.:

lvcreate -V 100G -n thin1lv  myvg/thinpool1

The -V specifies the "virtual" size of your thin volume. This is how big it looks like.

Note that the size of our thin1lv is larger then the total size of the pool. This will give you a warning when cerating the LV but is allowed since actual space on the thinlv will only be when data is written to the device. In this case this does not make much sense but if you have a lot of thin LVs which all only use a part of their space you could do this kind of over provisioning. You should monitor your share and maybe set an automatic extentend threshold for your pool.

Now you can create snapshots and do not need to worry about performance. E.g. if you have a lot of virtual machines running on your server they could all be cloned from a base image and take up very little space.