Библиотека сайта rus-linux.net
|The Linux System Administrator's Guide: Version 0.7|
|Prev||Chapter 6. Using Disks and Other Storage Media||Next|
A filesystem is the methods and data structures that an operating system uses to keep track of files on a disk or partition; that is, the way the files are organised on the disk. The word is also used to refer to a partition or disk that is used to store the files or the type of the filesystem. Thus, one might say ``I have two filesystems'' meaning one has two partitions on which one stores files, or that one is using the ``extended filesystem'', meaning the type of the filesystem.
The difference between a disk or partition and the filesystem it contains is important. A few programs (including, reasonably enough, programs that create filesystems) operate directly on the raw sectors of a disk or partition; if there is an existing file system there it will be destroyed or seriously corrupted. Most programs operate on a filesystem, and therefore won't work on a partition that doesn't contain one (or that contains one of the wrong type).
Before a partition or disk can be used as a filesystem, it needs to be initialised, and the bookkeeping data structures need to be written to the disk. This process is called making a filesystem.
Most UNIX filesystem types have a similar general structure, although the exact details vary quite a bit. The central concepts are superblock, inode, data block, directory block, and indirection block. The superblock contains information about the filesystem as a whole, such as its size (the exact information here depends on the filesystem). An inode contains all information about a file, except its name. The name is stored in the directory, together with the number of the inode. A directory entry consists of a filename and the number of the inode which represents the file. The inode contains the numbers of several data blocks, which are used to store the data in the file. There is space only for a few data block numbers in the inode, however, and if more are needed, more space for pointers to the data blocks is allocated dynamically. These dynamically allocated blocks are indirect blocks; the name indicates that in order to find the data block, one has to find its number in the indirect block first.
UNIX filesystems usually allow one to create a
hole in a file (this is done with the
lseek() system call; check the manual page),
which means that the filesystem just pretends that at a particular
place in the file there is just zero bytes, but no actual disk
sectors are reserved for that place in the file (this means that the
file will use a bit less disk space). This happens especially often
for small binaries, Linux shared libraries, some databases, and a
few other special cases. (Holes are implemented by storing a
special value as the address of the data block in the indirect block
or inode. This special address means that no data block is
allocated for that part of the file, ergo, there is a hole in the
Linux supports several types of filesystems. As of this writing the most important ones are:
The oldest, presumed to be the most reliable, but quite limited in features (some time stamps are missing, at most 30 character filenames) and restricted in capabilities (at most 64 MB per filesystem).
A modified version of the minix filesystem that lifts the limits on the filenames and filesystem sizes, but does not otherwise introduce new features. It is not very popular, but is reported to work very well.
The most featureful of the native Linux filesystems, currently also the most popular one. It is designed to be easily upwards compatible, so that new versions of the filesystem code do not require re-making the existing filesystems.
An older version of ext2 that wasn't upwards compatible. It is hardly ever used in new installations any more, and most people have converted to ext2.
A more robust filesystem. Journalling is used which makes data loss less likely. Journalling is a mechanism whereby a record is kept of transaction which are to be performed, or which have been performed. This allows the filesystem to reconstruct itself fairly easily after damage caused by, for example, improper shutdowns./glossentry>
In addition, support for several foreign filesystem exists, to make it easier to exchange files with other operating systems. These foreign filesystems work just like native ones, except that they may be lacking in some usual UNIX features, or have curious limitations, or other oddities.
Compatibility with MS-DOS (and OS/2 and Windows NT) FAT filesystems.
Extends the msdos filesystem driver under Linux to get long filenames, owners, permissions, links, and device files. This allows a normal msdos filesystem to be used as if it were a Linux one, thus removing the need for a separate partition for Linux.
This is an extension of the FAT filesystem known as FAT32. It supports larger disk sizes than FAT. Most MS Windows disks are vfat.
The standard CD-ROM filesystem; the popular Rock Ridge extension to the CD-ROM standard that allows longer file names is supported automatically.
A networked filesystem that allows sharing a filesystem between many computers to allow easy access to the files from all of them.
A networks filesystem which allows sharing of a filesystem with an MS Windows computer. It is compatible with the Windows file sharing protocols.
The OS/2 filesystem.
SystemV/386, Coherent, and Xenix filesystems.
The choice of filesystem to use depends on the situation. If compatibility or other reasons make one of the non-native filesystems necessary, then that one must be used. If one can choose freely, then it is probably wisest to use ext2, since it has all the features but does not suffer from lack of performance.
/procdirectory, which is not really a filesystem at all, even though it looks like one. The proc filesystem makes it easy to access certain kernel data structures, such as the process list (hence the name). It makes these data structures look like a filesystem, and that filesystem can be manipulated with all the usual file tools. For example, to get a listing of all processes one might use the command
Note that even though it is called a filesystem, no part of
the proc filesystem touches any disk. It exists only in the
kernel's imagination. Whenever anyone tries to look at any part of
the proc filesystem, the kernel makes it look as if the part existed
somewhere, even though it doesn't. So, even though there is a
/proc/kcore file, it doesn't
take any disk space.
There is usually little point in using many different filesystems. Currently, ext2fs is the most popular one, and it is probably the wisest choice. Depending on the overhead for bookkeeping structures, speed, (perceived) reliability, compatibility, and various other reasons, it may be advisable to use another file system. This needs to be decided on a case-by-case basis. 
Filesystems are created, i.e., initialised, with the
mkfs command. There is actually a separate
program for each filesystem type. mkfs is just a
front end that runs the appropriate program depending on the desired
filesystem type. The type is selected with the
-t fstype option.
The programs called by mkfs have slightly different command line interfaces. The common and most important options are summarised below; see the manual pages for more.
Select the type of the filesystem.
Search for bad blocks and initialise the bad block list accordingly.
- -l filename
Read the initial bad block list from the name file.
-noption prevents validation, i.e., bad block checking). Then bad blocks were searched with badblocks, with the output redirected to a file,
bad-blocks. Finally, the filesystem was created, with the bad block list initialised by whatever badblocks found.
-coption could have been used with mkfs instead of badblocks and a separate file. The example below does that.
-coption is more convenient than a separate use of badblocks, but badblocks is necessary for checking after the filesystem has been created.
The process to prepare filesystems on hard disks or partitions is the same as for floppies, except that the formatting isn't needed.
Before one can use a filesystem, it has to be mounted. The operating system then does various bookkeeping things to make sure that everything works. Since all files in UNIX are in a single directory tree, the mount operation will make it look like the contents of the new filesystem are the contents of an existing subdirectory in some already mounted filesystem.
For example, Figure 6-3 shows three
separate filesystems, each with their own root directory. When the
last two filesystems are mounted below
/usr, respectively, on the first
filesystem, we can get a single directory tree, as in
/usrdirectories, respectively. One would then say that ``
/dev/hda2is mounted on
/home'', and similarly for
/usr. To look at either filesystem, one would look at the contents of the directory on which it has been mounted, just as if it were any other directory. Note the difference between the device file,
/dev/hda2, and the mounted-on directory,
/home. The device file gives access to the raw contents of the disk, the mounted-on directory gives access to the files on the disk. The mounted-on directory is called the mount point.
-t fstypeoption to specify the type directly; this is sometimes necessary, since the heuristics mount uses do not always work. For example, to mount an MS-DOS floppy, you could use the following command:
The mounted-on directory need not be empty, although it
must exist. Any files in it, however, will be inaccessible by name
while the filesystem is mounted. (Any files that have already been
opened will still be accessible. Files that have hard links from
other directories can be accessed using those names.) There is no
harm done with this, and it can even be useful. For instance, some
people like to have
/var/tmp synonymous, and make
/tmp be a symbolic link to
/var/tmp. When the system is booted, before
/var filesystem is mounted, a
/var/tmp directory residing on the root
filesystem is used instead. When
mounted, it will make the
on the root filesystem inaccessible. If
/var/tmp didn't exist on the root filesystem,
it would be impossible to use temporary files
If you don't intend to write anything to the filesystem, use
-r switch for mount to do a
read-only mount. This will make the kernel
stop any attempts at writing to the filesystem, and will also stop
the kernel from updating file access times in the inodes. Read-only
mounts are necessary for unwritable media, e.g., CD-ROMs.
The alert reader has already noticed a slight logistical problem. How is the first filesystem (called the root filesystem, because it contains the root directory) mounted, since it obviously can't be mounted on another filesystem? Well, the answer is that it is done by magic.  The root filesystem is magically mounted at boot time, and one can rely on it to always be mounted. If the root filesystem can't be mounted, the system does not boot. The name of the filesystem that is magically mounted as root is either compiled into the kernel, or set using LILO or rdev.
The root filesystem is usually first mounted read-only. The startup scripts will then run fsck to verify its validity, and if there are no problems, they will re-mount it so that writes will also be allowed. fsck must not be run on a mounted filesystem, since any changes to the filesystem while fsck is running will cause trouble. Since the root filesystem is mounted read-only while it is being checked, fsck can fix any problems without worry, since the remount operation will flush any metadata that the filesystem keeps in memory.
On many systems there are other filesystems that should
also be mounted automatically at boot time. These are specified
/etc/fstab file; see the fstab man
page for details on the format. The details of exactly when the
extra filesystems are mounted depend on many factors, and can be
configured by each administrator if need be; see
See the man page for further instructions on how to use the command. It is imperative that you always unmount a mounted floppy. Don't just pop the floppy out of the drive! Because of disk caching, the data is not necessarily written to the floppy until you unmount it, so removing the floppy from the drive too early might cause the contents to become garbled. If you only read from the floppy, this is not very likely, but if you write, even accidentally, the result may be catastrophic.
Mounting and unmounting requires super user privileges, i.e.,
only root can do it. The reason for this is that if any user can
mount a floppy on any directory, then it is rather easy to create a
floppy with, say, a Trojan horse disguised as
/bin/sh, or any other often used program.
However, it is often necessary to allow users to use floppies, and
there are several ways to do this:
Give the users the root password. This is obviously bad security, but is the easiest solution. It works well if there is no need for security anyway, which is the case on many non-networked, personal systems.
Use a program such as sudo to allow users to use mount. This is still bad security, but doesn't directly give super user privileges to everyone. 
Make the users use mtools, a package for manipulating MS-DOS filesystems, without mounting them. This works well if MS-DOS floppies are all that is needed, but is rather awkward otherwise.
List the floppy devices and their allowable mount points together with the suitable options in
/dev/fd0 /floppy msdos user,noauto 0 0
noautooption stops this mount to be done automatically when the system is started (i.e., it stops mount -a from mounting it). The
useroption allows any user to mount the filesystem, and, because of security reasons, disallows execution of programs (normal or setuid) and interpretation of device files from the mounted filesystem. After this, any user can mount a floppy with an msdos filesystem with the following command:
/dev/fd0 /dosfloppy msdos user,noauto 0 0 /dev/fd0 /ext2floppy ext2 user,noauto 0 0
umaskfilesystem options, described in detail on the mount manual page. If you aren't careful, mounting an MS-DOS filesystem gives everyone at least read access to the files in it, which is not a good idea.
Filesystems are complex creatures, and as such, they tend to be somewhat error-prone. A filesystem's correctness and validity can be checked using the fsck command. It can be instructed to repair any minor problems it finds, and to alert the user if there any unrepairable problems. Fortunately, the code to implement filesystems is debugged quite effectively, so there are seldom any problems at all, and they are usually caused by power failures, failing hardware, or operator errors; for example, by not shutting down the system properly.
Most systems are setup to run fsck
automatically at boot time, so that any errors are detected (and
hopefully corrected) before the system is used. Use of a corrupted
filesystem tends to make things worse: if the data structures are
messed up, using the filesystem will probably mess them up even
more, resulting in more data loss. However, fsck
can take a while to run on big filesystems, and since errors almost
never occur if the system has been shut down properly, a couple of
tricks are used to avoid doing the checks in such cases. The first
is that if the file
/etc/fastboot exists, no
checks are made. The second is that the ext2 filesystem has a
special marker in its superblock that tells whether the filesystem
was unmounted properly after the previous mount. This allows
e2fsck (the version of fsck
for the ext2 filesystem) to avoid checking the filesystem if the
flag indicates that the unmount was done (the assumption being that
a proper unmount indicates no problems). Whether the
/etc/fastboot trick works on your system
depends on your startup scripts, but the ext2 trick works every time
you use e2fsck. It has to be explicitly bypassed
with an option to e2fsck to be avoided. (See
the e2fsck man page for
details on how.)
The automatic checking only works for the filesystems that are mounted automatically at boot time. Use fsck manually to check other filesystems, e.g., floppies.
If fsck finds unrepairable problems, you need either in-depth knowledge of how filesystems work in general, and the type of the corrupt filesystem in particular, or good backups. The latter is easy (although sometimes tedious) to arrange, the former can sometimes be arranged via a friend, the Linux newsgroups and mailing lists, or some other source of support, if you don't have the know-how yourself. I'd like to tell you more about it, but my lack of education and experience in this regard hinders me. The debugfs program by Theodore Ts'o should be useful.
fsck must only be run on unmounted filesystems, never on mounted filesystems (with the exception of the read-only root during startup). This is because it accesses the raw disk, and can therefore modify the filesystem without the operating system realizing it. There will be trouble, if the operating system is confused.
When a file is written to disk, it can't always be written in consecutive blocks. A file that is not stored in consecutive blocks is fragmented. It takes longer to read a fragmented file, since the disk's read-write head will have to move more. It is desirable to avoid fragmentation, although it is less of a problem in a system with a good buffer cache with read-ahead.
The ext2 filesystem attempts to keep fragmentation at a minimum, by keeping all blocks in a file close together, even if they can't be stored in consecutive sectors. Ext2 effectively always allocates the free block that is nearest to other blocks in a file. For ext2, it is therefore seldom necessary to worry about fragmentation. There is a program for defragmenting an ext2 filesystem called, strangely enough, defrag  .
There are many MS-DOS defragmentation programs that move blocks around in the filesystem to remove fragmentation. For other filesystems, defragmentation must be done by backing up the filesystem, re-creating it, and restoring the files from backups. Backing up a filesystem before defragmenting is a good idea for all filesystems, since many things can go wrong during the defragmentation.
Some other tools are also useful for managing filesystems. df shows the free disk space on one or more filesystems; du shows how much disk space a directory and all its files contain. These can be used to hunt down disk space wasters. Both have manual pages which detail the (many) options which can be used.
sync forces all unwritten blocks in the buffer cache (see Section 7.6) to be written to disk. It is seldom necessary to do this by hand; the daemon process update does this automatically. It can be useful in catastrophes, for example if update or its helper process bdflush dies, or if you must turn off power now and can't wait for update to run. Again, there are manual pages. The man is your very best friend in linux. Its cousin apropos is also very useful when you don't know what the name of the command you want is.
In addition to the filesystem creator (mke2fs) and checker (e2fsck) accessible directly or via the filesystem type independent front ends, the ext2 filesystem has some additional tools that can be useful.
tune2fs adjusts filesystem parameters. Some of the more interesting parameters are:
A maximal mount count. e2fsck enforces a check when filesystem has been mounted too many times, even if the clean flag is set. For a system that is used for developing or testing the system, it might be a good idea to reduce this limit.
A maximal time between checks. e2fsck can also enforce a maximal time between two checks, even if the clean flag is set, and the filesystem hasn't been mounted very often. This can be disabled, however.
Number of blocks reserved for root. Ext2 reserves some blocks for root so that if the filesystem fills up, it is still possible to do system administration without having to delete anything. The reserved amount is by default 5 percent, which on most disks isn't enough to be wasteful. However, for floppies there is no point in reserving any blocks.
dumpe2fs shows information about an ext2 filesystem, mostly from the superblock. Figure 6-5 shows a sample output. Some of the information in the output is technical and requires understanding of how the filesystem works (see appendix XXX ext2fspaper), but much of it is readily understandable even for layadmins.
Figure 6-5. Sample output from dumpe2fs
dumpe2fs 0.5b, 11-Mar-95 for EXT2 FS 0.5a, 94/10/23
Filesystem magic number: 0xEF53
Filesystem state: clean
Errors behavior: Continue
Inode count: 360
Block count: 1440
Reserved block count: 72
Free blocks: 1133
Free inodes: 326
First block: 1
Block size: 1024
Fragment size: 1024
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 360
Last mount time: Tue Aug 8 01:52:52 1995
Last write time: Tue Aug 8 01:53:28 1995
Mount count: 3
Maximum mount count: 20
Last checked: Tue Aug 8 01:06:31 1995
Check interval: 0
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
Block bitmap at 3, Inode bitmap at 4, Inode table at 5
1133 free blocks, 326 free inodes, 2 directories
Free blocks: 307-1439
Free inodes: 35-360
debugfs is a filesystem debugger. It allows direct access to the filesystem data structures stored on disk and can thus be used to repair a disk that is so broken that fsck can't fix it automatically. It has also been known to be used to recover deleted files. However, debugfs very much requires that you understand what you're doing; a failure to understand can destroy all your data.
dump and restore can be used to back up an ext2 filesystem. They are ext2 specific versions of the traditional UNIX backup tools. See Chapter 12 for more information on backups.
Currently there are several filesystems vying for replacement of ext2, these include reiserfs and ext3. They include ``journalling''. A definition and explanation of journalling is outside the (current) scope of this book, but put very simply it is a mechanism whereby the filesystem is more robust against power failure, or other inelegant shutdowns. This makes data loss far less likely and so not surprisingly it is looking like it will be the standard in Linux filesystems eventually.
For more information, see the kernel source or the Kernel Hackers' Guide.
It should of course be unmount, but the n mysteriously disappeared in the 70s, and hasn't been seen since. Please return it to Bell Labs, NJ, if you find it.
It requires several seconds of hard thinking on the users' behalf. Furthermore sudo can be configured to only allow users to execute certain commands. See the sudo(8), sudoers(5), and visudo(8) manual pages.