Friday, June 30, 2006
preload and LSB compliance webpage
I started to try another hotspot: preload. Unfortunately, I noticed no change on the boot time after installing preload. Not even after changing its boot order from /etc/rc2.d/ to /etc/rcS.d/ after the dependencies were met: in the init script it claims to require $time, $remote_fs and $local_fs. I believe seeing it in ubuntu at just the beginning of /etc/rcS.d/. The bootcharts and test procedures can be seen here. This is an issue that has to be further investigated.
On the other side, the lsb check script was debugged a little bit more. A list with the compliance of the init scripts in sid is available here. The few LSB compliant scripts are on the top, the big incomplete majority in the middle and those without LSB headers at the bottom.
On the other side, the lsb check script was debugged a little bit more. A list with the compliance of the init scripts in sid is available here. The few LSB compliant scripts are on the top, the big incomplete majority in the middle and those without LSB headers at the bottom.
Thursday, June 29, 2006
Debugging the LSB check in lintian/linda.
With run-time dependencies in the init scripts (as LSB headers) we can verify the correct boot order of the system. Currently, lintian just checks the init scripts to provide start, stop, restart and force-reload actions. A small modification could allow it to check for LSB compliance as well and decrease the big amount of scripts lacking run-time dependencies until now.
Currently we have a script to verify LSB compliance and is on a debugging phase. One of the issues is when to warn about a missing information on the stop phase, like Required-Stop, Should-Stop and Default-Stop. Changes we made were not to warn (at least for Default-Stop) when the system default start is only runtime S (initialization).
Finally, there are now some way to track the bugs related to this project with BTS usertags. Most of them have been submitted by Petter Reinholdtsen. Follow this link to see the bugs.
Currently we have a script to verify LSB compliance and is on a debugging phase. One of the issues is when to warn about a missing information on the stop phase, like Required-Stop, Should-Stop and Default-Stop. Changes we made were not to warn (at least for Default-Stop) when the system default start is only runtime S (initialization).
Finally, there are now some way to track the bugs related to this project with BTS usertags. Most of them have been submitted by Petter Reinholdtsen. Follow this link to see the bugs.
Wednesday, June 28, 2006
Script for checking LSB compliance (SoC 2006)
Based on a script by Petter Reinholdtsen to check the correct init script order, I've made a script to check for LSB compliance. It is available on the project webpage. The script checks for the existance of each of the LSB headers and considers 4 main output cases:
+ No LSB headers (when the START INIT INFO header is not found)
+ Basic LSB compliance (when Should-Start, Should-Stop and the Description have no arguments.
+ Full LSB compliance (when all headers have arguments)
+ Partial compliance (in the other cases).
Besides, it sends to to STDERR comments on missing headers or missing arguments in them.
+ No LSB headers (when the START INIT INFO header is not found)
+ Basic LSB compliance (when Should-Start, Should-Stop and the Description have no arguments.
+ Full LSB compliance (when all headers have arguments)
+ Partial compliance (in the other cases).
Besides, it sends to to STDERR comments on missing headers or missing arguments in them.
Monday, June 26, 2006
startpar and insserv
Yesterday I tried insserv together with startpar according to the post from Petter Reinholdtsen although the results were not what we expected as just a 2 second reduction was observed. The bootchart is here. This behaviour seems to be due to a deficient ordering of the init scripts by insserv as it may be observed by comparing the init scripts order before and after ordering with insserv.
In the order of the initscripts it can be observed that:
+there are repeated versions of one initscript in the symbolic link farms (/etc/rcS.d and /etc/rc2.d) like udev.
+some known errors in the order were not corrected (bugs #375388 and #375389).
To correct this behavior requires some modifications to the code for ordering the initscripts in insserv. This would be the next step in the project (to implement a promising hotspot) when dependency information (LSB-compliance) can be checked.
In the order of the initscripts it can be observed that:
+there are repeated versions of one initscript in the symbolic link farms (/etc/rcS.d and /etc/rc2.d) like udev.
+some known errors in the order were not corrected (bugs #375388 and #375389).
To correct this behavior requires some modifications to the code for ordering the initscripts in insserv. This would be the next step in the project (to implement a promising hotspot) when dependency information (LSB-compliance) can be checked.
Sunday, June 25, 2006
BTS and Lintian
While working on the script to check for LSB compliance, and as a recommendation from Petter Reinholdtsen, I commited two bug reports to the BTS related to a wrong order of some scripts in the sysvinit package. Specifically alsa-utils and udev (Bug reports #375388 and #375389, respectively).
The reason for this, besides correcting the bug itself, is to raise the issue of correct script ordering. Currently, a developer cannot know if their package is correctly placed in the /etc/rc?.d symbolic pools until a bug is reported. If lintian and/or linda included some check for init script dependency information, i.e. basic LSB-compliance, the maintainers would remember to include this information and a script could correct the order for each individual system. The script needs to be developed for debian and this is one of the next steps on the current boot project.
The reason for this, besides correcting the bug itself, is to raise the issue of correct script ordering. Currently, a developer cannot know if their package is correctly placed in the /etc/rc?.d symbolic pools until a bug is reported. If lintian and/or linda included some check for init script dependency information, i.e. basic LSB-compliance, the maintainers would remember to include this information and a script could correct the order for each individual system. The script needs to be developed for debian and this is one of the next steps on the current boot project.
Saturday, June 24, 2006
dash and LSB check
One new boot process hotspot was tested yesterday: using dash instead of bash. Got a good reduction of 3 seconds in the boot time. The results and method used are in the project webpage. Nevertheless, this was different from the 6 seconds obtained by Margarita Manterola in her slides from debconf6.
On the other side, with the aim of having a script for checking lintian rules and LSB compliance, I've started to take a look to the script written by Petter Reinholdtsen. It is written in perl so I took a quick tutorial of perl first :)
On the other side, with the aim of having a script for checking lintian rules and LSB compliance, I've started to take a look to the script written by Petter Reinholdtsen. It is written in perl so I took a quick tutorial of perl first :)
Friday, June 23, 2006
Background and LSB compliance (SoC 2006)
Yesterday I tried another hotspot: networking on the background. In the Sarge networking script it is pretty easy to run in the background. Just have to add an ampersand after the line:
ifup -a
Couldn't be easier I guess. Well, there is the problem that the message "done." will appear afterwards at some random place out of context. Now, the sid version of the script is much nicer and seems to be LSB compliant!!! Grand! Well, the problem comes as "ifup -a" is inside an IF statement such that sending to the background makes no sense. So maybe it could be sent to the background in a new shell and send a message like "Configuring network interfaces...done".
Well, as my intention was just to test the boot speed I removed the IF statement. There was no change on time! That is not what should happen. I'll have to double check. The results and scripts are available here
Finally, I tried removed discover from the scripts as udev is already doing the same job. This change was motivated by a comment from Marco D'Itri. Also in this case I noticed no change on the boot time. This should be hardware dependent.
ifup -a
Couldn't be easier I guess. Well, there is the problem that the message "done." will appear afterwards at some random place out of context. Now, the sid version of the script is much nicer and seems to be LSB compliant!!! Grand! Well, the problem comes as "ifup -a" is inside an IF statement such that sending to the background makes no sense. So maybe it could be sent to the background in a new shell and send a message like "Configuring network interfaces...done".
Well, as my intention was just to test the boot speed I removed the IF statement. There was no change on time! That is not what should happen. I'll have to double check. The results and scripts are available here
Finally, I tried removed discover from the scripts as udev is already doing the same job. This change was motivated by a comment from Marco D'Itri. Also in this case I noticed no change on the boot time. This should be hardware dependent.
Thursday, June 22, 2006
First hotspots tried (SoC 2006)
Debian sid was installed and the installation log can be found here. There is a link to a list of the packages installed with the distribution update from etch and the resulting init scripts. Besides, bootcharts and other resources will be available in this section.
Using the same sid installation, from the hotspots yesterday I tried to load the hwclock on the background with a 2 seconds reduction in the boot time. The bootcharts and scripts are availabe here as well.
Finally, depmod is not used any more in sid's module-init-tools and this is one of the hotspots. This was removed as a result from a discussion started by Margarita Manterola in debian-devel. Just by adding it again from an old script, the 2 second reduction from hwclock was lost.
Using the same sid installation, from the hotspots yesterday I tried to load the hwclock on the background with a 2 seconds reduction in the boot time. The bootcharts and scripts are availabe here as well.
Finally, depmod is not used any more in sid's module-init-tools and this is one of the hotspots. This was removed as a result from a discussion started by Margarita Manterola in debian-devel. Just by adding it again from an old script, the 2 second reduction from hwclock was lost.
Wednesday, June 21, 2006
Sid bootcharts (SoC 2006)
Hi, another rainy summer day in ireland and another bootchart in the webpage Now it was sid's turn. So now we have the time from grub to Kde startup for the four releases:
Woody(32s)
Sarge(44s)
Etch(34s)
Sid(33s)
All of them use kde with autologin although boothchart stops at least 6 seconds before kde stops loading. That means that all of the times here are at least 6 seconds shorter so Sid takes 39 seconds from grub. An attempt to measure until kde finishes loading was made by changing the /sbin/bootchartd wait_boot function such that bootchart waits until the program ktip (the one that gives you a usage tip at startup). Nevertheless, in this case bootchart takes around 6 seconds to stop logging. The bootchart is on the webpage as well.
Woody(32s)
Sarge(44s)
Etch(34s)
Sid(33s)
All of them use kde with autologin although boothchart stops at least 6 seconds before kde stops loading. That means that all of the times here are at least 6 seconds shorter so Sid takes 39 seconds from grub. An attempt to measure until kde finishes loading was made by changing the /sbin/bootchartd wait_boot function such that bootchart waits until the program ktip (the one that gives you a usage tip at startup). Nevertheless, in this case bootchart takes around 6 seconds to stop logging. The bootchart is on the webpage as well.
Tuesday, June 20, 2006
/sbin/bootchartd and KDE
After finding out that bootchart wasn't considering the whole time for KDE to start up, we are trying to modify bootchart so it ends up with another program. The changes doesn't seem to be working for the moment as the program selected to end logging can't be found in bootchart.tgz. Anybody familiar with this?
On the other side, we are considering to add ELF prelinking as one of the hotpots to decrease the boot process time. Probably it could help when big programs start that need to link to many libraries, e.g. KDE.
Well, as a remark, anonymous comments are now enabled in the blog and a contact address has been added to both, the webpage and the blog.
On the other side, we are considering to add ELF prelinking as one of the hotpots to decrease the boot process time. Probably it could help when big programs start that need to link to many libraries, e.g. KDE.
Well, as a remark, anonymous comments are now enabled in the blog and a contact address has been added to both, the webpage and the blog.
Sunday, June 18, 2006
D1:State-of-the-art in the boot process and promising hotspots (SoC2006)
Finally the first deliverable (D1) is ready for revision here. It can be modified according to comments and extended if it is considered that some subject should be further investigated. Besides, some hotspots were identified and, as stated in the proposal it is time to implement some of them. At least the 5 more promising for the second deliverable. Comments about the order will be appreciated.
Together with this 5 hotspots, the second deliverable will deal with the evolution of the boot process in Debian: from woody to sid. Already, initial results were presented through this blog during the last week concerning boot times from woody, sarge and etch.
Together with this 5 hotspots, the second deliverable will deal with the evolution of the boot process in Debian: from woody to sid. Already, initial results were presented through this blog during the last week concerning boot times from woody, sarge and etch.
Saturday, June 17, 2006
Bootchart vs Stopwatch (SoC2006)
Today I did some serious polishing of the first deliverable of the boot process project and made it available here. One of the things added were some hotspots ordered by how promising they are. The final version of the deliverable will be ready tomorrow evening.
One question that I had today was about the relationship between Debian's update-rc.d and Fedora's chkconfig command. Both of them seem to arrange the order of execution of the init scripts. The other question was about the acceptance of file-rc and it seems that, at least for debian, it is used quite seldom (1).
Finally, as a result from Petter Reinholdtsen's feedback to the previously published bootcharts of woody, sarge and etch, I compared the bootcharts time using a stopwatch. Petter considered that the times were too small and it looked like KDE autologin time was too short. The time results (in seconds) for Sarge and Etch are:
Sarge 0:44 (bootchart) and 1:06 (stopwatch). Difference = 0:22
Etch 0:34 (bootchart) and 0:49 (stopwatch). Difference = 0:15
Big difference. The stopwatch was activated when the key ENTER is pressed at the GRUB selection screen and stopped until the KDE in-progress window disappeared.It should be noticed that bootchart doesn't account for the time of initrd.
Besides, from the time that the message of starting kdm appears until the KDE stops loading there is an important time difference:
Sarge ~0:06 (bootchart) and 0:23.5 (stopwatch). Difference=0:17.5
Etch ~0:05.5 (bootchart) and 0:19 (stopwatch). Difference=13.5
Then, most of the time difference seems to be because of bootchart stopping to count the time before the KDE stops loading.
Just as a final remark, the PC time from pressing the ON botton to the grub is around 11.5 second. Then the total startup times of Sarge and Etch end up being as big as 1:17.5 and 1:00.5, respectively. Both above one minute.
(1)http://qa.debian.org/developer.php?popcon=file-rc
One question that I had today was about the relationship between Debian's update-rc.d and Fedora's chkconfig command. Both of them seem to arrange the order of execution of the init scripts. The other question was about the acceptance of file-rc and it seems that, at least for debian, it is used quite seldom (1).
Finally, as a result from Petter Reinholdtsen's feedback to the previously published bootcharts of woody, sarge and etch, I compared the bootcharts time using a stopwatch. Petter considered that the times were too small and it looked like KDE autologin time was too short. The time results (in seconds) for Sarge and Etch are:
Sarge 0:44 (bootchart) and 1:06 (stopwatch). Difference = 0:22
Etch 0:34 (bootchart) and 0:49 (stopwatch). Difference = 0:15
Big difference. The stopwatch was activated when the key ENTER is pressed at the GRUB selection screen and stopped until the KDE in-progress window disappeared.It should be noticed that bootchart doesn't account for the time of initrd.
Besides, from the time that the message of starting kdm appears until the KDE stops loading there is an important time difference:
Sarge ~0:06 (bootchart) and 0:23.5 (stopwatch). Difference=0:17.5
Etch ~0:05.5 (bootchart) and 0:19 (stopwatch). Difference=13.5
Then, most of the time difference seems to be because of bootchart stopping to count the time before the KDE stops loading.
Just as a final remark, the PC time from pressing the ON botton to the grub is around 11.5 second. Then the total startup times of Sarge and Etch end up being as big as 1:17.5 and 1:00.5, respectively. Both above one minute.
(1)http://qa.debian.org/developer.php?popcon=file-rc
Benchmarking other distrubutions (SoC2006)
Hi, for the debian boot process project, the last days I looked at the SUSE, Mandriva and Fedora distributions to analyse their boot process. The most interesting so far was SUSE being LSB compliant and having insserv/startpar to set and ajust the dependencies for parallalel booting. The insserv/chkconfig combinations seems to do the same as FreeBSD's rcorder although rcorder checks dependencies in two stages: with root filesystem and with all mounted filesystems.
Fedora looks like a regular sysvinit process although it contains LSB information in the initscripts that is read by chkconfig to set dependencies in the /etc/rc.d directories. Nevertheless, the dependencies don't adjust to added/substracted scripts.
Finally Mandriva 2006 doesn't seem to have pinit/prcsys in their default configuration so it seems to have a standard sysvinit.
BTW, the first deliverable is being written in latex (with pdf and html versions generated together) and available here.
Fedora looks like a regular sysvinit process although it contains LSB information in the initscripts that is read by chkconfig to set dependencies in the /etc/rc
Finally Mandriva 2006 doesn't seem to have pinit/prcsys in their default configuration so it seems to have a standard sysvinit.
BTW, the first deliverable is being written in latex (with pdf and html versions generated together) and available here.
Tuesday, June 13, 2006
Woody / Sarge / Etch boot processes compared
Debian distributions booting times (up to kde with autologin) in the same PC:
The input-output data was just available for Etch as it requires a 2.6 linux kernel.
The input-output data was just available for Etch as it requires a 2.6 linux kernel.
Monday, June 12, 2006
Benchmarking between boot processes
For the project to improve debian boot process, I've installed debian woody and debian sarge to see the changes in debian releases. Currently installing debian etch.
The first bootcharts are here for woody and sarge with a clean installation with autologin to kde. Funny to see woody being faster with 32 seconds while sarge has 44 seconds.
For debian woody I had some problems to use bootchart but they were solved thanks to the kind help of Baruch Even. The errors were:
Finally, I'm checking out SUSE's implementation of startpar together with insserv for parallel execution. It is interesting to see a boot process that looks to be already LSB-compliant.
The first bootcharts are here for woody and sarge with a clean installation with autologin to kde. Funny to see woody being faster with 32 seconds while sarge has 44 seconds.
For debian woody I had some problems to use bootchart but they were solved thanks to the kind help of Baruch Even. The errors were:
- bootchart requires tmpfs filesystem, while the kernel 2.2 that comes with woody doesn't have tmpfs. It was solved by installing kernel 2.4 and substituting tmpfs with ramfs in the /sbin/bootchartd script.
- initrd ignores the kernel call to bootchartd, as it ignores the arguments given to the kernel. It was solved by renaming /sbin/init (e.g. /sbin/init-moved) and making the file /sbin/init link to /sbin/bootchartd. At the end of the latter, there renamed /sbin/init is called. It's perhaps an ugly tweak but worked for our means.
Finally, I'm checking out SUSE's implementation of startpar together with insserv for parallel execution. It is interesting to see a boot process that looks to be already LSB-compliant.
Friday, June 09, 2006
Margarita's guidelines
Another post about google's SoC project to improve the boot process written from sunny Ireland!
With the first deliverable deadline nearby, I've been preparing the computer I'll use for playing with the boot process with different distributions. Iv'e lately installed debian woody and will be comparing it with Etch and Sarge (as freshly installed).
From the blog of Margarita Manterola got some interesting guidelines of what to do for the beginning of the project [1]. As she had already tested boot times for several processes and got a 20 second improvement for Etch (reduced boot time to 60%!!) it looks like there is hope. It would be great to be able to watch the recorded presentation (hopefully soon to be available).
So new hotspots identified:
[2] http://www.fastboot.org
With the first deliverable deadline nearby, I've been preparing the computer I'll use for playing with the boot process with different distributions. Iv'e lately installed debian woody and will be comparing it with Etch and Sarge (as freshly installed).
From the blog of Margarita Manterola got some interesting guidelines of what to do for the beginning of the project [1]. As she had already tested boot times for several processes and got a 20 second improvement for Etch (reduced boot time to 60%!!) it looks like there is hope. It would be great to be able to watch the recorded presentation (hopefully soon to be available).
So new hotspots identified:
- setting up the hardware clock in the background
- improve cpu use when starting the desktop manager.
- make the boot less verbose
- serel -> claims to get at least an 8% faster booting process (using parallelizaton). An interesting choice documented in [2].
- rcpar -> claimed to have the same functionality as the shell concurrency of /etc/init.d/rc with better handling of the output.
[2] http://www.fastboot.org
Wednesday, June 07, 2006
Project SVN repository is up
Used part of my 6 hours in Malmo aiport to prepare the summer project webpage. Just 2 buses per day to Sturup airport (Malmo) from Copenhagen!!
Well, so the webpage will be again available soon as it will be updated automatically from the SVN repository thanks to Petter.
http://initscripts-ng.alioth.debian.org/soc2006-bootsystem
For hotspots, reading Henrique de Moraes [1] got interested in serel [2] and will consider to include it in the things to test. Besides, got to read some months of initscripts-ng mailing list being interesting a comment from Erich Schubert about readahead (use it just at the very beginning and while bringing up the gdm/kde desktop) among others.
[1] http://alioth.debian.org/docman/view.php/30730/38/debconf2-initscripts-bkg.pdf
[2] http://www.fastboot.org
Well, so the webpage will be again available soon as it will be updated automatically from the SVN repository thanks to Petter.
http://initscripts-ng.alioth.debian.org/soc2006-bootsystem
For hotspots, reading Henrique de Moraes [1] got interested in serel [2] and will consider to include it in the things to test. Besides, got to read some months of initscripts-ng mailing list being interesting a comment from Erich Schubert about readahead (use it just at the very beginning and while bringing up the gdm/kde desktop) among others.
[1] http://alioth.debian.org/docman/view.php/30730/38/debconf2-initscripts-bkg.pdf
[2] http://www.fastboot.org
Monday, June 05, 2006
reading, hotspots and repository
This is the second post of the blog for improving the boot process. There are some logistic news and hotspots being identified together with some readings.
Logistics:
[1]svn+ssh://svn.debian.org/svn/initscripts-ng
[2] http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/
[3] http://wiki.debian.org/SummerOfCode2006
Logistics:
- SVN repository is up at [1]
- place for webpage was decided and will be ready soon at [2]
- Project added to the debian wiki at [3]
- reordering bootscripts
- parallel booting (like with startpar)
- preloading (like with dynamic readahead)
- using internal functions in init scripts or dash
- don't run depmod at run time
[1]svn+ssh://svn.debian.org/svn/initscripts-ng
[2] http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/
[3] http://wiki.debian.org/SummerOfCode2006
Thursday, June 01, 2006
Fist message
It is exciting to be in the summer of code 2006 with debian!
I will use this blog to include new advances, my current work and ideas and links to the deliverables. I still don't kow where will be the webpage used to host the project itself and the deliverables but that should be decided soon.
In the next days I'll prepare the first deliverable with the state of the art of linuces boot process and identifying some hotspots to increase the boot speed.
For the moment I'm looking at suse, ubuntu and FreeBSD. I'll write down some summary so anybody interested can send me some comments before preparing the deliverable.
pos eso es todo por hoy :D
I will use this blog to include new advances, my current work and ideas and links to the deliverables. I still don't kow where will be the webpage used to host the project itself and the deliverables but that should be decided soon.
In the next days I'll prepare the first deliverable with the state of the art of linuces boot process and identifying some hotspots to increase the boot speed.
For the moment I'm looking at suse, ubuntu and FreeBSD. I'll write down some summary so anybody interested can send me some comments before preparing the deliverable.
pos eso es todo por hoy :D
