Introducing Squasher!

Assalamualaikum semua! Apa khabar? Saya sihat… anda juga… etc, etc.

Okay, so have I mention before that running ubuntu live from usb stick feels actually faster than an installed Ubuntu? Yes, there are several reason for that, and one of those are the installation is compressed into a squashfs image. A squashfs image should be about three times smaller than an uncompressed directory. Canonical use squashfs in the live image so that the whole ubuntu installation could fit into a CD (its DVD now). The problem with squashfs image is that, it is a read only image, meaning you can’t install anything the live image. Of course, there are ways to overcome this, like unetbootin’s persistant storage, but that is a different story. So, bottomline, when you install Ubuntu, what the installer does actually is it uncompress the squashfs image to your hard disk. Because the uncompressed file is about three times bigger, the hard disk will need to read three time more data to start your system. You can, of course, manually set up an ubuntu intallation into using a partial squashed installation by following this guide, but that for me is quite a hassle to do manually. So, I’ve come up with a script I called Squasher written in bash that will automatically compress your /usr into an image and set up fstab and help in updating the squashfs image.

So, here is how you use it. First download the script from its github repo. Copy it to wherever you want, just not in /usr. Then, install the package squashfs-tools and aufs-tools. You can use the command “sudo apt-get install squashfs-tools aufs-tools” to install it or you can click this link which supposedly will open ubuntu software center and install the package.

For example, I’ve put the in my home directory. Don’t put it too many folder because we will execute it in command prompt.

 Then, right click on it, and press properties. Make sure you checked “allow executing file as program”. This will come in handy later.

 Next, restart and enter recovery mode. If you use ubuntu 12.10, the recovery mode is in the “Advanced option for Ubuntu”

Then, on the Recovery Menu, choose “Drop to root shell prompt”.

Notice, on the bottom of the screen, you can now type stuff. Navigate to where you keep the script, and run it. If it says something like “access denied”, you probably did not check “allow executing file as program”. As a workaround, type “mount -o remount,rw /” and “chmod +x”.

The script will prompt you to disable AppArmor. This will prevent AppArmor to start at startup. Disabling AppArmor is recommended because many thing may not work if you leave it. Even so, after disabling it, some profile (namely cupsd) may still be loaded at startup, meaning printers may not work. You may want to remove AppArmor completely to fix that.

 Next, it will prompt to squash /usr. The script will start modifying the system after this, so it asked you to continue.

 If you pick Yes, it will start by compressing /usr into an image.

 And if everything goes well, you should be prompted to restart your system.

 By the way, the old /usr is not deleted. It had only been renamed to /usr.backup/. Delete it manually if you are sure everything is ok.

Now, your partial squashfs system has already been set up. You could see some load time improvement when starting applications. You can install new application, and do stuff normally. Be advised though, that new application will not be automatically compressed. In this setup, any write to /usr will be written in /var/squashed/rw. While you squashed image is mounted in /var/squashed/ro. Aufs then will merge both of these forder into /usr.

As for example, the screenshot above is right after I squashed my system. The /var/squashed/rw is empty because I have no installed anything.

Now, for example sake, I install gparted. Notice that the gparted binary is in /var/squashed/rw/bin. But it is not compressed.

To recompress it, repeat the procedure to start the script. the script will automatically detect your configuration. Make sure the parameter is about the same as above. If you did not do anything with your fstab, this should work just well.

Select continue to recompress /usr. You should see a similar progress status as before, as when you compress /usr for the first time. When it finish, restart your system. Beware that the script does not delete your old squashfs image and read write branch. For security, you MUST delete them manually before running the script again.

Notice on the screenshot above there are two image with size about 600MB. Delete the one with the ‘.old’ postfix, once you are sure everything is ok.

The same is also true for the read write branch. On the screenshot above, the read write branch is empty. The old rw branch has been renamed to “rw.old”. Delete it and you should see that your newly installed application is still available and compressed.

Performance gain.
On a Virtualbox I’ve notice that it does start faster, but I have no number to prove that. On my laptop, and a stopwatch, I did not see any start up loading time improvement. Uncompressed or compressed, it took 55 second to boot and start chromium. But the number do strip down to 40 second once I’ve reconfigure ureadahead. I do notice however, is that nautilus and common application do load faster in this configuration. Unfortunately it is still not as fast as using a thumbdrive. Probably because the script only compress /usr.

So if you are still using a conventional hard disk, squasher may be worth a look. Until btrfs become optimized, a squashfs image is probably a solution to speed up loading time by compression.

WARNING: If at some time you dist-upgrade your system, AppArmor profile for some of the updated file may be loaded after you restart. I’m still not sure how did this happen. So you could try removing apparmor completely, or ‘invoke-rc.d apparmor teardown’ for temporary solution.


Why people use Ubuntu(or its derivative) instead of other distro

Remember the old Ubuntu? Before plymouth or KMS, around Ubuntu 9.10, with it slick boot screen, as if you notebook is turning into some kind of transformer robot, and then you boot into gnome 2 desktop, top and bottom panel just swiftly but with nice timing slide into the screen with it black elegent theme. And you could enter you Facebook/Twitter/Google Talk account and chat right from the desktop with new message notification which is just… damn! You can even post picture right from nautilus!

No, you have to see it for yourself. The logo fade in and out like it was… booting.
Now look at Ubuntu today… damn… (say it sadly)… What happen? If you are not the kind of person to keep up to date with these stuff, you probably not know what happen. Let me recap it for you, KMS enter the world, many bugs, ubuntu’s theme started to be more.. “mature” or “static” if I may say so, then Gnome 3 happen, the community retaliate, Ubuntu retaliate but in an even more different way than the community hoped for. KDE4 just… seems to not learn fast enough . Compiz is dying, then MATE, Cinnamon, Unity, Elementary, gnome has never been so fragmented. And here we are, in a strange distant land. 
Not to say that Gnome 3 is bad, don’t get me wrong, its great. Its my favorite desktop environment (with extensions of course), but if I have to choose between gnome2 in ubuntu 9.10 or gnome3 or unity, I would pick gnome 2 hands on. No because of the software, but because of the integration.
Ubuntu started of as a linux distro aims for the ‘normal’, mere mortal human being instead of being server oriented or even a universal distro. Sadly right now Canonical may have put the definition of ‘normal’ human being too far by introducing Unity to the land of cluttered gnome, with its lack of customization. Probably the only thing that you can change in the desktop(without third party apps) is the wallpaper (IOS anyone?). Because of this, Ubuntu’s throne in the DistroWatch ranking right now has plummeted to the number 3, even surpassed by the relatively young, Mandriva based Mageia.
Despite that, other old-time distro such as Fedora and OpenSuse still has trouble catching up to Ubuntu even though ubuntu is handicapped the notoriously infamous Unity. Some may like it, some may not,old timers definitely not, the Unity desktop environment main problem is probably customization and not so intuitive way of task switching, but one thing for sure, most people do not like it (at least at first). Other than that, we have KDE or Gnome3 which is quite universal for all distro. And yet Ubuntu based distro (see Linux Mint over there), overall still a win. Why is that so?
1. The installation, it actually work.
Recently, due to boredom I’ve decided to replace my ubuntu 12.04 for OpenSuse 12.2, you know, just for fun. But the catch is, I have no CD-R. Who use them nowadays anyway? With thumbdrive being more reliable and faster. So, like always, I ran the Startup USB Disk Creator, in Ubuntu, and it fail to open the ISO (no warning whatsoever). So, like a normal human being, I blame it on the USB Disk Creator, cause I know it is buggy. Then I restart to Windows 7 (please, I just being human), use the trusty unetbootin, and tada! Disc created. Rebooted, “fail to find gfxboot”. I’ve done this with Ubuntu/Mint ISO so many times, but today my faith on unetbootin has been tainted. And so, like many other stupid geeks, I love to do it the hard way, boot Windows 7, run VirtualBox where my (real)partition has been setup and installed it from there. I knew there would be some boot error(my bad), but hey, How hard is it to reinstall grub? I was wrong… In Ubuntu, “grub-install –bootdirectory=’stuff’ /dev/sda” would work, in OpenSuse, grub-install open the famous Yast2 tools to set the bootloader, which did not work. And I’m stuck with manually typing “linux (hd0,3)..and stuff” on grub terminal. I’ve tried both grub2 and grub-legacy, grub-install.unsupported in opensuse and yet no menu whatsoever. Just a grub terminal. Right then I change to fedora. Ok, sure, its my fault. I should have known that the iso would not work using unetbootin. I’m suppose to dd-it. But Ubuntu ISO works just fine right?

Ubuntu installer does not always work for me, but compared with other distro, Ubiquity  works relatively well.

2. Synaptics and apt-get rules!

You know, whenever you use Yast2 or yum to install a package, it will first download the index even if you just update it 5 minute ago. Sure, it only download the header, but still, that could take 15 second of non-interactivity which basically do nothing. Things are even more horrible on fedora, where most of the GUI frontend depends on packagekit. Now imagine, at some inappropriate time, yum is doing an index update, and you want to install, lets just say… an applet. Apper will hang until the backround yum is done. Not to say that the same thing does not happen in Ubuntu, but it happen a lot less, and it’s easy to set it off. Now, about Apper, have you tried searching for a package using it before? It basically call packagekit which then call yum, which I don’t know what it does, it seems to be doing nothing for the first 3 second, then my disk start to get busy. Happen every single time you search. Now imagine if yum is doing an update in the middle of your package browsing. Compare that to synaptics experience and ubuntu is starting to really shine. One thing I don’t like about apt-get (in ubuntu) is that it does not have delta updates like Fedora or OpenSuse or even Debian. From what I understand, they did not implement it because of something not suitable with Ubuntu’s infrastructure. This cause and Ubuntu update to be unnecessarily large. Just imagine, having to re-downloads kde-wallpapers (70MB) for  minor version update. 

3. Personal Package Archive (PPA) System

xorg-edgers, bringing new xorg since 10.04

Some may say that Ubuntu does not contribute much back to the opensource community. They mostly use whatever they think is suitable, than refine it into a distro. After all, that is what a distro is for. But there are great things that Canonical make, for example, UpStart and the PPA system. Personally, I think PPA is one of Ubuntu’s greatest strength. For a long time, the problem about a non-rolling-release distribution is getting up to date software. Look at Debian for example. A rolling-release distro in another hand tend to get a bit unstable. The PPA system fix the problem (up to certain point) of a non-rolling-release distro by making it easy for other contributor to publish their own package without the hassle of putting it in the official repository. Not to say that other distro does not have their own solution, Opensuse has its own third party repo, while Fedora has RpmFusion. But Canonical make their PPA System much easier to use by implementing it with the Launchpad system. As a result, if you want to try the newest Xorg, or Linux Kernel, or some other bleeding edge software, there is a ppa for it. This increase the number of application you can install by a whole lot, even though the number of  packages from Debian itself has reach more than 20,000. Now, if only it has one click installation from browser…

4. Popularity

DistroWatch may put Ubuntu on the third place, but Its no secret Ubuntu is in fact, if you count person new to this Linux stuff, a very famous distribution even though the person you are asking does not know what is a ‘distro’. When people say about Linux, I mean, normal people who does not work in the IT sector, they will probably say “Linux? That is Ubuntu right?”. Yes, I know, Linux and Ubuntu are not the same thing, but those people probably think so, and they probably don’t care much. The point is if they need something other than Mac or Windows, the first choice would be Ubuntu. And because of this traction, other people in the IT sector, who depends on these people (let say… for living) would probably support Ubuntu as well. This give Canonical a very competitive edge in terms of free support. In addition other hardware vendors would most likely target their driver to work well with Ubuntu releases, much like AMD or Nvida drivers being scheduled for Ubuntu release. Proprietary software developer will also most likely to provide binaries for Ubuntu or Fedora (because of Red Hat), than other distribution such as Gentoo, although Gentoo user are most probably competent enough to work around it. Plus, Ubuntu Software Center has support for paid software. Because ubuntu-based distribution like Linux Mint are simply Ubuntu, plus some additional repositories, things that work in Ubuntu are most likely to also work with no or little changes. In some way, the biggest contribution from Canonical is probably their marketing effort for Ubuntu which indirectly also affect other distributions.  

OMG! Its on the cover page!