Home –  filesystem
Tag Archives: filesystem

NO DEFRAGMENTATION NEEDED IN LINUX

defragment in linux

We all must have heard about defragmenting our hard drives to speed up performance. Sound of that similar to windows users but we don’t have any defragment utility, hell, we don’t need to defragment! And why is that? Lets find out-

I remember when I was new to Linux, a friend of mine came with a hard disk and asked me to defragment it (yeah, we all have faced it).  It was easy in windows, how hard it could be in Linux?  I connected the spare bus to his hard disk and then started a mission to defragment via Linux. I can’t stress this enough that I was new to Linux and didn’t know much about how things work. After trying every bit I knew, looked up the mighty google and it came like a bolt from the blue- NO DEFRAGMENTATION NEEDED IN LINUX. </chitchat>

What do you mean by fragmentation of hard disk?

I’ll cover it in as short as possible; your hard disk is divided into sectors. These sectors are capable of storing small pieces of data. So, when the size of file is more than size of a sector then the file is saved over multiple sectors, as simple as that. As the new file comes, it is saved next to previous sector. But, when the size of first file increases, it might not find free sectors near original file so it will be saved few sectors apart, this goes on and on. While reading this file, the head will have to skip physical locations which ultimately will make things slower. Defragmentation is the process in which file is put back in continuous fashion so that there is no skipping of sectors.

Note: Defragmenting is not recommended for SSD drives. They have a different theory, doing so will reduce their life.

How it works in case of Linux?

Linux’s file systems deal with fragmentation in an intelligent way. Instead of placing files next to each other, it keeps a lot of space between the sectors of two files therefore leaving space for these files to grow. This keeps fragmentation far away, yet not impossible. Even if there are fragments, file system attempts to move the files thereby reducing the fragmentation. The system is designed to avoid fragmentation in normal use. You might face some fragmentation when your space fills up, may be 80% or 90%. The more suitable method to deal with it is to get a new hard drive with more space else you can copy your files to other drive, delete original files and then copy them back- they will be arranged automatically.

Beginners guide to Linux directory structure

Have you ever looked in your / directory, you’ll see a lot of directories. Here we are presenting beginners guide to linux directory structure explaining what they mean and what are the contents of these directories.

Screenshot of contents of root directory:

linux directory structure

 

/

This is called root partition. All files and directories start with root partition. Write privileges under this directory are avaible with root user only. Not to confuse it with root user’s home directory, know the difference, “/” is root partition while root user’s home directory is “/root”.

/bin

This directory has binary executable files. Linux commands used in single user mode are found in this directory. It also holds commands that are used by all users. Examples: ls, ping, cp.

/sbin

Like /bin, /sbin also contains binary executable files but the commands held by this directory are used by system administrators with the prime purpose of system maintenance. Examples: iptables, reboot, fdisk.

/etc

It holds all the configuration files which are required by all programs. Shell scripts needed by programs to start or stop them are held by this very directory. Examples are /etc/resolv.conf, /etc/logrotate.conf

/dev

/dev contains device files. In Linux, everything is a file. Included are terminal device or usb or any other device connected to the computer. For example: /dev/usbmon0

/proc

All the system process information is held in /proc. It is a pseudo filesystem as it contains information about running processes. For an instance, /proc/ is the directory which holds information of the process with . It contains information about the system resources. /proc/uptime is one such directory.

/var

var denotes variable files. Those files are kept in this directory that are supposed to grow. Some of the files that reside in here are- /var/log (system log files), /var/lib (package files), /var/mail (emails), /var/spool (print queues), /var/tmp (temporary files that are needed across reboots).

/tmp

System generated and user generated temporary files are kept in this category. Important files should not be saved in here because contents of /tmp are flushed every time system boots.

/usr

/usr is the one that holds user programs. It contains documentations, libraries and source-code for all the second level applications. /usr/bin holds binaries for user programs. While looking for a binary after /bin also look in /usr/bin. Examples of binaries you mighht find in /usr/bin are awk, less, cc. Similarly binary files for system administrators are kept in /usr/sbin. Examples are cron, sshd, useradd. Libraries for /usr/bin and /usr/sbin are kept in /usr/lib. /usr/local holds user programs that a user installs from source.

/home

All user’s personal files are kept in their respective home directories.

/boot

This directory has boot loader files. While booting, files needed are found in /boot. Kernel initrd, grub and few more files reside in this directory.

/lib

Binaries located in /bin and /sbin are supported by library files kept in /lib. Library nomenclature goes like ld* or lib*.so.* . For example: ld-2.11.1.so

/opt

/opt is for add-on optional applications from individual vendors. All applications that are optional should be installed in /opt or any of its subdirectory.

/mnt

/mnt is our mount directory. It is the temporary mount location where where system administrators can mount temporary filesystems.

/media

It serves as the temporary mount point for removable devices. All removable devices are mounted in this directory. Example: /media/cdrom or /media/floppy.

/srv

srv denotes service. All the service related data used for servers is saved in this directory.

Types of Filesystem, differences and dealing with them using Perl

If you are looking to write code in Perl which will work on multiple platforms then you must have the knowledge of filesystem of each platform. Filesystems are categoriesed in 3 parts.

1. Filesystem of Unix

FFS (Berkeley Fast File System) is the ancestor of all current filesystems of Unix variants. The filesystem has been extended by different vendors according to their needs. Like for better security they have been extended to provide support for POSIX ACL (access control list).

In Unix filesystem the root is denoted by / (forward slash). The path of any directory in Unix filesystem starts from / and goes deeper in the system. The directory names or file names in Unix filesystem are case-sensitive.

Path Example: /root/home/user/Desktop/file.txt

2. Filesystem of Windows

Operating systems which are based on Windows supports 3 filesystems: FAT (File Allocation Table), NTFS (NT FileSystem), FAT32 (Advance version of FAT).

FAT filesystem is not case-sensitive. It uses \\ (backward slash) as path separator. In the FAT systems the direcotries and files contains some flags, these flags are known as attributes like “Read Only”. NTFS supports unicode feature.

Path Example: C:\\home\\user\\Desktop\\file.txt

3. Filesystem of Mac OS

Classic Mac OS used HFS (Hierarchical File System). The version 8.1 of Mac OS uses HFS+ which is an advance version of HFS. The current filesystem looks similar to Unix filesystem. The method of showing paths is same in both OS. The difference between these is that Unix file system is case-sensitive whereas HFS+ is not case-sensitive.

Summary

OS and FileSystem Path Separator Filename length Absolute path format Relative path format Unique features
Unix (Berkeley FFS and others) / OS-dependent number of chars /dir/file dir/file OS-variant dependent additions
Mac OS (HFS+) / 255 Unicode chars /dir/file dir/file Mac OS legacy support, BSD extended attributes
Windows based OS (NTFS) \\ 255 Unicode chars Drive:\\dir\\file dir\\file File encryption and compression
DOS (basic FAT) \\ 8.3 Drive:\\dir\\file dir\\file Attributes

 

Using Perl to deal with different Filesystems

By now we know all the filesystems differ from each other and to deal with these in Perl we use File::Spec module. This module is used to hide the differences between these filesystems. Let us learn how we use this module. We will use the catfile method to get the path of the file.

Example:

use File::Spec;
my $path= File::Spec->catfile(qw{home user Desktop file.txt});

In this way the scalar variable $path get sets to home\user\desktop\file.txt in a Windows operating system. and in Unix based system it gets set to home/user/Desktop/file.txt. The File::Spec module also contains methods like curdir and updir to get the current directory (.) and up-directory (..).

You may also use the moudle Path::Class which is an another good wrapper. You will have to install it before using.

Below is an example of how it works:

use Path::Class;
my $file = file (qw{home user Desktop file.txt});
my $dir = file (qw{home user Desktop});

The $file and $dir are scalar variable which contains the path to the file.txt file and Desktop directory respectively.

Here’s the path

print $file;
print $dir;

The output will depend on operating system. If it is Unix then it will yield

home/user/desktop/file.txt and home//user//desktop

and in Windows based systems it will give

home\\user\\desktop\\file.txt and home\\user\\desktop

Here $file and $dir are objects which have several methods which can be applied on them for some use. Like absoulte method gives the absolute path and slurp method slurps through the file’s contents.

my $abspath = $file->absolute;
my $content = $file->slurp;
$file->remove(); #delete the file

When you want to write a code for a particular system and you want to make this system understand the path of other operating system then you may use foreign_file() and foreign_dir() methods. These will return the path based the argument which you have to specify explicitly.

use Path::Class qw(foreign_file foreign_dir);
my $foreignfile = foreign_file('Win32', qw{home user Desktop file.txt});
my $foreigndir = foreign_dir('Win32', qw{home user});

Now $foreignfile will contain home\\user\\desktop\\file.txt even if the code is run from a Unix based system. This approach is very handy and useful. In the next article we will continue and expand our discussion.