Tuesday, May 3, 2011

Speeding up Ubuntu 11.04 boot time

Just because I have nothing else to do on a lazy, rainy Sunday morning I thought it would be fun to see how much I can slash off the startup time for Ubuntu 11.04 64Bit on my HP Probook 4720s. The OS boots from a class 6 SD card (16 Mbyte/sec read) and 6 GB RAM.Apart from the standard setup the laptop is also running Virtualbox, Mysql and winbind.


Measuring progress

Bootchart is a nice package that protocols meticulously the entire boot process and creates graphical charts of the times and where they are spent. So the first step is getting bootchart:

sudo apt-get install bootchart

After rebooting, you can get aforementioned chart in /var/log/bootchart. A word of caution: the first chart is non-representative, as ureadahead is in profiling mode and slows things down. After another reboot the chart would look much better. And that is how it looks like:


 The chart shows three milestones: the X-Window manager starts at 19 seconds after boot, Gnome at 41seconds and the desktop is visible at 59 seconds, which I'll consider also the entire time of the boot. That's the time to beat.

Parallelizing Ureadahead

Ureadahead takes about 10 seconds to finish filling the cache, during that time nothing else loads. This makes sense for a regular, rotational harddisk as the frequent head movements induced by processes  booting in parallel would slow things down. But since this installation is booting from an SD card, it might be worth exploring a way to run ureadahead in parallel. My first take was to put ureadahead in a script and launch it in the background:

/etc/init.d/bootopts

#!/bin/sh
(/sbin/ureadahead --daemon)&


and change /etc/init/ureadahead.conf to call this script instead of /sbin/ureadahead

The reboot is 3 seconds faster which are gained somewhere between booting X and booting Gnome. Disappointingly, the new chart shows that ureadahead is still running sequentially stalling everything after it. Some investigation shows that the I/O scheduler thinks that the SD card is a rotational device, so I'm changing the bootopts script to correct that by adding:

echo 0 > /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.0/host6/target6:0:0/6:0:0:0/block/sdb/queue/rotational
 

Note that the SD card is /dev/sdb

After rebooting, the chart looks like this:

Much better, ureadahead runs now in parallel. Unfortunately it made the boot process 1 sec slower, mainly delaying X and Gnome. But I feel I am on to something,  so let's not just give up. 

Tweaking flashcard readahead

Some bechmarks with a simple hdparm -t -T on the flash card shows that a sequential read times with 14 MB/sec, thus 2 MB/sec short of it's nominal classification. The reason seems to be a read-ahead setting of 256,  reducing it to 128 gets the maximum of 16 MB/sec. Thus another addition to my bootopts script:

hdparm -a128 /dev/sdb 

Correcting the I/O scheduler

Another reboot later, I'm back to 56sec. I suspect that the I/O scheduler gives ureadahead the same priority as to other processes, which it really shouldn't so it should be hidden behind an ionice, thus a I modify bootopts:

(/usr/bin/ionice -c 3 /sbin/ureadahead --daemon)&


This improves boot time by another 2 sec.  The chart reveals that the flash card is still not utilised fully, so I follow these instructions  to disable journaling on the boot partition and I also disable plymouthd which is the animated bootscreen. And that does the trick, time is now down to 45 sec:


[update 2011.05.22]
I also made an experiment in the opposite direction, namely starting ionice with -c 1 which corresponds to realtime I/O which reduced startup time by another second. While at first this seems counterintuitive since fewer processes can run in parallel, at a second glance it makes sense because parallel I/O introduces some overhead.

Deferring services

Now let's defer services not needed for the boot process to after login. My candidates:

mysqld
nmbd
smbd
cupsd
modem-manager
bluetoothd
alsa-store [update 2011.05.22]
alsa-restore [update 2011.05.22]
pulseaudio [update 2011.05.22]
rsync [update 2011.05.22]
apparmor - I apt-get removed that one [update 2011.05.22]

plymouthd [update 2011.05.22] I deleted the /etc/init/plymouthd*.conf files - arguably this will affect logging and the ability of services to interact with the user in case something goes wrong. Use at your own risk
ufw (firewall, there is a small security risk when setting up later) [update 2011.05.22]
acpi-support 
All of these have upstart jobs and can be deferred by changing the start event in the respective /etc/init/xyz.conf files to something like "stuff-that-can-wait". 
apport [update 2011.07.07]
 

A new script in /etc/init.d/stuff-that-can-wait can then execute these  scripts. Mine looks like this:


#!/bin/sh

/etc/init.d/mysql $1
/etc/init.d/samba $1
/etc/init.d/acpi-support $1


And of course not to forget: 

chmod a+x /etc/init.d/stuff-that-can-wait

I added it as an upstart script but it is probably easier to just add it as a startup script to the gnome session ("Startup Applications")

I then removed similar services from /etc/rcX.d with this command:

update-rc.d -f servicename remove

and set them up instead as upstart scripts (stuff-that-can-wait)

Running init scripts in parallel

Nice post here [update 2011.05.22]
The essence is to modify /etc/init.d/rc to

CONCURRENCY=shell


Taming laptop mode tools [update 2011.05.22]

There is more to do: near the end of the boot sequence, bootchart shows a long waiting mount (12 seconds) which I located in /usr/lib/pm-utils/power.d/journal-commit. I am modifying this file as such:




#!/bin/sh
exit0

(I probably could just have deleted it)

This one not only slashed a whopping 10 seconds off the bootchart but also the Unity Desktop built up noticeably faster.

There is also pulseaudio starting much earlier than it should (see Deferring Services)

Blacklisting modules [update 2011.07.07]

This step eliminates unnecessary (for my setup) modules. Edit /etc/modprobe.d/blacklist.conf and append the lines

blacklist rt2800pci
blacklist rt2x00pci
blacklist parport_pc
blacklist vboxnetado
blacklist vboxnetflt
blacklist bvoxdrv
blacklist joydev
blacklist lp
blacklist hp_accel

and comment the lp module in /etc/modules

Which leaves us with 28sec boot time:

12 comments:

  1. hello there, nice guide, I arrived here looking to improve my boot time, right now, I booting in 8.12 seconds (with a SSD) with no modifications, so I think that I can get less time modifying the boot process, but, nothing improve that time :(
    any idea?

    ReplyDelete
  2. I forgot: I running Ubuntu 11.04 64 bits.

    ReplyDelete
  3. Buy a chrome and get a life :)

    ReplyDelete
  4. @D: now, where is the fun in that? :-)

    @MeduZa: A good start is the bootchart package. It will create a graph of all components involved in the boot process and how much time they take.

    ReplyDelete
  5. So in the end, how much does it take to boot?

    ReplyDelete
  6. Hello Sonay,

    On my HP it is down to 31sec

    ReplyDelete
  7. Howdy,

    Nice :) Would you like to do a test with systemd and blog about the results? I wonder how it compares to your hand-made parallelization.

    I am an Arch user at the moment, as I am not happy with the state of Unity. Tried systemd with native configurations but it doesn't change much, maybe because I don't have an ssd.

    ReplyDelete
  8. That sounds like some work to do :-) I don't expect too much from systemd: my current setup is highly parallelized, making good use of the available I/O capacities and CPU. As a matter of fact ureadahread slows it down which is imo an indication towards that I'm approaching a limit which can't be exceeded by simply rearranging startup procedures in a smarter way - which is what systemd would be doing.

    I feel that the X-Server and Unity are taking long to load and as they are built on monolithic binaries so I think that further investigation should go into slimming them down.

    Another target should be the boot image and the kernel which are loading more modules than needed on a laptop and are delaying the intial boot step by around 7 seconds.

    ReplyDelete
  9. Excellent article..

    Although some more pointers would be good - e.g. which files you are editing and where they are.

    I have Ubuntu 11.04 on HP 7635s (HDD is a 7200 RPM disk), and need to get the boot time down from 37 seconds.

    ReplyDelete
  10. Thank you Rod

    I tried to mention any changes in the respective files. If you spot any omission please let me know and I'll update the post.

    ReplyDelete
  11. Hi. Very useful instructions, thanks! I would appreciate if you could simply them a bit more, by giving direct shell commands to execute, say like a recipe.

    ReplyDelete