Month: November 2011

=====================

Distro-neutrally creating a liveUSB.

My goal in this writing this is to find a better way to use practically *any* Linux distribution to create a nice, bootable liveUSB drive.

By *any* Linux distro, I mean
– NO requirement for MS-Windows XP/Vista/7/…etc. (hope that was obvious!)
– NO specific requirement for a Ubuntu-based Linux distribution (very simply, I  DISMISS the presumption made all-too-frequently that Ubuntu or another Ubuntu-based distro is the chief Linux distro that “everybody uses” or that “everyone WILL use”.  I simply think that such a presumption is false.)

As a bold FYI, a  half-dozen popular non-Ubuntu Linux distros include:

• Fedora, http://fedoraproject.org/
• OpenSUSE, http://www.opensuse.org/en/
• Debian, http://www.debian.org/
• Arch Linux, http://www.archlinux.org/
• Puppy Linux, http://www.puppylinux.com/
• Slackware, http://www.slackware.com/
  

ca sera suffi…

In general, creating any sort of bootable device requires three overall actions:
A. Partitioning and then Formatting the device.
B. Copying to the device 1) the startup files and 2) various utility and extraneous/productivity files.
C. Extra steps taken to effectively make the device bootable.

 

Post alert: History digression here.<start_brief_history_lesson>

Going way back to the late 1980’s, you’d create one of those older bootable 3.5″ floppy-disks (and check that floppy too) by using the historic MS-DOS command ‘FORMAT /S /C A:’
In one step, that command string did the formatting step of A, and then for B-C it sufficiently copied the MS-DOS command-interpreter/startup files of COMMAND.COM, IO.SYS, and MSDOS.SYS.

More recently, but still for those older bootable 3.5″ floppy-disks, was Tom’s Root and Boot Linux floppy disk, see http://www.toms.net/rb/
Once the necessary file was downloaded, the file uncompressed, and the uncompressed README-type file’s instructions followed, an install script performed the above steps A through C onto the floppy disk. It should be noted for tomsrtbt that a) the install script used the Linux ‘fdformat’ command for step A, which roughly corresponds with the ‘FORMAT /C’ command for DOS, and b) the floppy was made bootable in step C using the LInux LOader (LILO) boot loader roughly corresponding to MS-DOS’s ‘SYS’ command and its resultant trio of “boot-loading” files previously mentioned in the paragraph above.

Besides the Tom’s Root and Boot Linux floppy disk, there were (and are) other Linux bootdisk creation methods for those older 3.5″ floppies.
These methods perform step B by the process of copying a single pre-created image file (.img) onto a DOS-formatted floppy disk using the Linux ‘dd’ or ‘cat’ command-line commands.  And instead of using LILO, many if not most of these methods now use GRUB (GRand Unified Boot loader) for step C.
A pair of current examples of such GRUB-based boot-floppy creators are
– The All In One Boot Floppy, website http://schierlm.users.sourceforge.net/bootdisk/index.html
– Super Grub2 Disk, website http://www.supergrubdisk.org/super-grub2-disk/

====> And as another FYI most appropriate here in this mention of the Linux command-line, Grant Bowman has a key post at this very BerkeleyLUG.com site entitled ‘Learning the Command Line’, http://www.berkeleylug.com/?p=834.  A nice collection of ‘Learning the Command Line’ links is in his very own http://www.grantbow.com/command_line.html ; very immediately applicable to top-level steps A through C.

In any case, bootable CD’s have been called “liveCDs” for awhile now, and they have an order of magnitude more utility files and productivity applications than on floppy disks.

Instead of any necessity to carry out steps A and C to create bootable liveCDs, these steps are ALREADY carried out when downloaded as .iso image files (the ISO-9660 format).
The only remaining step to carry out for creating bootable liveCDs is step B in the process of “burning” the filename.iso image file and the file-contents contained within, onto a blank CD.
Common CD-burning applications using Linux’s X11 graphical user interface are Brasero, K3b, Xfburn, and X-CD-Roast. The commands ‘wodim’, a.k.a. ‘cdrecord’, serve to burn an .iso image onto a blank CD from the Linux command-line.
The tutorial at http://www.yolinux.com/TUTORIALS/LinuxTutorialCDBurn.html primarily covers in much further detail, how to use both X-CD-Roast and ‘cdrecord’.

Enough of the history digression; now back to creating liveUSBs. </end_brief_history_lesson>

 

To rehash this again, creating any sort of bootable device requires three overall actions:
A. Partitioning and then Formatting the device.
B. Copying to the device 1) the startup files and 2) various utility and productivity files/apps.
C. Extra steps taken to effectively make the device bootable.

 

A. .. Partitioning.
Almost every USB pendrive of 1 GB capacity or higher comes partitioned and formatted for use in Windows, meaning that by factory default, it comes in FAT32 format, a.k.a., “vfat” in Linux terminology.  One of the key Linux utilities to 1st check USB pendrive partitioning is ‘fdisk’ (yes, I know others also suggest ‘cfdisk’ or ‘parted’… they’re both excellent and may even be BETTER than ‘fdisk’!)  Among other  decent references around for using ‘fdisk’ are the Geek Stuff’s ‘7 Linux fdisk Command Examples to Manage Hard Disk Partition’ at http://www.thegeekstuff.com/2010/09/linux-fdisk/ and the Linux Documentation Project’s ‘USB Flash Memory HOWTO’ (it’s for pendrives) at http://tldp.org/HOWTO/html_single/Flash-Memory-HOWTO/ .

To tell which device Linux calls the USB pendrive, you would run ‘fdisk -l’ from the command-line. The entire pendrive would then be  displayed near the bottom of the output as /dev/sdX with the X here typically written as “c”, “d”, “e”, or “f” depending upon what other storage devices you concurrently have connected.

A source of confusion is which “partition identifier” is the best one to use for pendrives. Two good but fairly brief references on partition identifiers are the SystemRescueCD’s ‘Sysresccd-Partitioning-EN-Partitions-attributes’ at http://www.sysresccd.org/Sysresccd-Partitioning-EN-Partitions-attributes   and the Linux Documentation Project’s ‘Partitions-Mass-Storage-Definitions-Naming-HOWTO — Partition Types’ at http://tldp.org/HOWTO/Partition-Mass-Storage-Definitions-Naming-HOWTO/x190.html .
The FAT32 partition identifiers “b” and “c” (“0B” and “0C” respectively for ‘cfdisk’) are easily the most appropriate to use for people who have USB pendrives of 1 GB or 2 GB capacity, and who SOLELY want to use these pendrives to boot up a single liveUSB with little to NO need for any extra file-storage space. I have used both identifiers without mishap when partitioning liveUSB pendrives.

An extra step to carry out at this partitioning-stage is to designate the first USB-pendrive partition as the active bootable partition.  Within ‘fdisk’, you’d use the ‘a’ selection for this, and the bootable partition is then designated by an asterix under the “Boot” column.  Within ‘cfdisk’, you’d use the bottom ‘[ Bootable ]’ selection for this, and the bootable partition is then designated by the “Boot” label under the “Flags” column.

A. .. Formatting.
Once USB pendrives are properly partitioned with either of the above FAT32 partition identifiers, they then must be effectively formatted.
The Linux command-line command for this is normally ‘mkdosfs’ , a.k.a. ‘mkfs.vfat’, and the command string most comparable to the old DOS ‘FDISK /C A:’ for this task would probably be ‘mkdosfs -F 32 -cv /dev/sdX1’ where that X is going to be the device letter of the USB pendrive;the X typically substituted with the appropriate “c”, “d”, “e”, or “f”.  Section 6.6 of the above ‘USB Flash Memory HOWTO’  reference http://tldp.org/HOWTO/html_single/Flash-Memory-HOWTO/ also mentions ‘mkdosfs’ here.  Specifically UNlike the DOS ‘FDISK’ command, however, ‘mkdosfs’ CANNOT create boot-able file systems; a bootable liveUSB must be created through additional methods as described below for steps B and C.
The next mini-step to carry out following successfull USB-pendrive formatting is to make the drive available for copying files to it through the Linux command-line command ‘mount’.  As the Linux Documentation Project’s ‘Linux Filesystem Hierarchy’ webpage http://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/mnt.html describes it; “Mounting is the process by which you make a filesystem available to the system. After mounting your files will be accessible under the mount-point.”  Oftentimes, if you just type ‘mount’ at the command-line, the output shows that the USB pendrive’s mount-point is something like /media/disk, /media/disk-1, or /mnt/sdX1 (again, X here is likely going to be one of the lower-case letters “c”, “d”, “e”, or “f”) and thus the USB pendrive is ALREADY automatically mounted and accessible for copying files to it. If such  “automounting” is not the case, then you’d have to manually use the command-line ‘mount’ command in a string such as ‘mount -t vfat /dev/sdX1 /mnt/sdX1’ (note that Section 7.4 of the above ‘USB Flash Memory HOWTO’ reference http://tldp.org/HOWTO/html_single/Flash-Memory-HOWTO/ uses an alternative mount-point name /mnt/memstick instead of the /mnt/sdX1 in this particular command string)

 

B. Copying to the device 1) the startup files and 2) various utility and productivity files/apps.

Okay, let’s say that you have now successfully partitioned, formatted, and mounted for use your USB-pendrive.
There are currently a few non-Windows and non-Ubuntu options to use for copying files to the pendrive.

One excellent option is to use UNetbootin. The UNetbootin website is http://unetbootin.sourceforge.net/ and this Linux method of creating a liveUSB works quite effectively for a wide number of popular and even less-popular Linux distributions.  It should be noted that UNetbootin not only copies the startup files and various utility/extraneous files to the mounted USB-pendrive, but it also carries out most of step C through its use of the SYSLINUX boot-loader and its (syslinux’s) startup/configuration files. For those who are interested, there is a good webpage entitled ‘How UNetbootin Works’ at http://sourceforge.net/apps/trac/unetbootin/wiki/howitworks which attempts to explain what UNetbootin really does.
I happen to think that UNetbootin for Linux is far and away the BEST and easiest-to-use non-Windows and non-Ubuntu method of creating a liveUSB.

Another option is to use the LiveUSB Install USB Creator tool created by Krasimir S. Stefanov, website at http://live.learnfree.eu/
Have not used LiveUSB Install myself, but this is probably fine for creating a liveUSB.

There are by now other good automatic and distro-independent liveUSB creation options BESIDES UNetBootin and LiveUSB Install. Maybe there are some much better bash, perl, or Python scripts/apps aroundfor ALL current Linux distros??

The second major option — other than using any of these automated methods — is to use a decidedly MANUAL method of copying to that previously mounted USB pendrive both 1) the startup files and 2) a varied mixture of utility and productivity files/apps.  Although this manual option is without a doubt HIGHLY complex, its advantage is in offering somewhat more flexibility and fine-tuning.
DIY’ers, are any of you seeing this??!!

Almost invariably, the manual option will require that you download and use the SYSLINUX commands and files, currently available through the kernel.org webpage http://www.kernel.org/pub/linux/utils/boot/syslinux/ . To obtain the full and latest SYSLINUX commands and files at the time of this writing, you would download from this site either syslinux-4.04.tar.bz2 or syslinux-4.04.tar.gz to your current working directory, and then run respectively either ‘tar -xjvf syslinux-4.04.tar.bz2’ or ‘tar -xzvf syslinux-4.04.tar.gz’.
When you have carried out either of these last two steps from the Linux command-line, you will have a subdirectory in your current working directory appropriately called “syslinux-4.04” in which to navigate as needed.

Now with your mounted USB-pendrive, you will have to discover the location of your downloaded Linux distro that you wish to make a liveUSB out of.

Sometimes, a few Linux distro creators nicely make available an archived and compressed “.tar.gz” file, a.k.a. a “.tgz” file as  a liveUSB form of their distro. This filename.tar.gz (or filename.tgz) is then downloaded and copied-over to the mounted USB pendrive .  The process of carrying this out is fairly simple: Download the archived and compressed file (ending in .tar.gz or .tgz) to your current working directory, do a straight copying of this file via the command-line ‘cp’ command to the mounted USB pendrive, and then uncompress this file at the very topmost part (the “root” directory) of the drive. The latter two steps are probably going to be something very similar to ‘cp -v ./filename.tgz /mnt/sdX1’ where ./ is your current working directory and then ‘tar -xzvf /mnt/sdX1/filename.tgz’.

Unfortunately, the vast majority of Linux distros are primarily available as the downloable .iso image files (the ISO-9660 format) that I wrote about way above for CDs, e.g., the fairly nice and small 30 MB slitaz-3.0.iso for SliTaz Linux. (And btw, this SliTaz Linux happens to have its own particular  automatic liveUSB-creating utility ‘TazUSB’ ; further info at http://hg.slitaz.org/tazusb/raw-file/tip/doc/tazusb.en.html )  In any case, the extra and likely CUMBERSOME(!) tasks involved with manually copying over the necessary files from the .iso image to the USB pendrive are the following:
..1. Loop-mounting the .iso image
..2. Copying the loop-mounted file-contents of the image onto the USB pendrive
..3. Optional: Editing the configuration file(s) now on the USB pendrive.

You’d first log in as the superuser, a.k.a., user “root” (yes, it’s a bit of a PITA!)  UNetbootin and distro-native liveUSB-creating utilities such as the ‘TazUSB’ one I just mentioned above make the first two parts of this process VASTLY simpler!!

The manual copying tasks.
..1. Loop-mounting the .iso image.
Most of this step is described in such references as LinuxHelp.net’s  ‘LinuxHelp.net’ webpage at http://www.linuxhelp.net/linux_downloads/ and Example 17-7 of the Linux Documentation Project’s ‘Advanced Bash-Scripting Guide: System and Administrative Commands’ webpage at http://tldp.org/LDP/abs/html/system.html
You’d then create a loop mount-point such as /media/iso or /mnt/iso via the ‘mkdir’ command-line command, i.e., ‘mkdir /media/iso’ or ‘mkdir /mnt/iso’
Navigate to the directory where your downloaded filename.iso distro image is located.
Loop-mount the .iso image by issuing the appropriate and longer ‘mount’ command string ‘mount -rt iso9660 filename.iso -o loop /media/iso’ or alternatively ‘mount -rt iso9660 filename.iso -o loop /mnt/iso’

..2. Copying the loop-mounted file-contents of the image onto the USB pendrive.
You’d navigate to the loop mount-point /media/iso (or /mnt/iso).
You’d then copy the entire contents (*) of this directory to the mounted USB pendrive, for example, using the command string ‘cp -v /media/iso/* /mnt/sdX1’ or alternatively  ‘cp -v /mnt/iso/* /mnt/sdX1’
The copying process could take a long while. When it appears that the copying has finally finished, you’d issue the command ‘sync’ to synchronize copying completion.
You’d then navigate back to a previous working directory OTHER THAN the loop-mounted directory /media/iso (or /mnt/iso) and then attempt to disassociate the .iso image’s temporary loop mount-point by running ‘umount /media/iso’ or alternatively ‘umount /mnt/iso’.

Now did I already mention that these steps are HIGHLY complex or what?!

..3. Optional: Editing the configuration file(s) now on the USB pendrive.
Most of the time, this simply involves checking and MAYBE editing such USB pendrive startup configuration files as isolinux.cfg in the /mnt/sdX1/boot/isolinux directory and/or grub.lst in the /mnt/sdX1/boot/grub directory.
I’ve found on extremely rare occasions that one or more of the following SYSLINUX files have had to be copied from the syslinux-4.04 directory created above to the appropriate directory of the mounted USB pendrive:
– syslinux-4.04/com32/menu/menu.c32
– syslinux-4.04/com32/menu/vesamenu.c32
– syslinux-4.04/com32/modules/chain.c32
– syslinux-4.04/com32/modules/cmd.c32
– syslinux-4.04/com32/modules/reboot.c32

 

C. Extra steps taken to effectively make the device bootable.

UNetbootin seems to handle liveUSB boot-activation quite well, and I’m certain that LiveUSB Install and other automated USB creation tools performs similarly. There is a caveat to this that I’ll note below, see note #1.

The first thing you’d have to do to MANUALLY activate liveUSB device booting, before carrying out any other steps here, is to unmount the USB pendrive’s mount-point used in Step B . This would be carried out while still logged on as the superuser / “root”-user by running the command-line command string ‘umount /mnt/sdX1’ or alternatively ‘umount /media/sdX1’

The next step is to navigate to the appropriate SYSLINUX directory (I navigate to the subdirectory syslinux-4.04/linux) and to then run ‘syslinux -s /dev/sdX1’ That alone “should” make the USB pendrive bootable.

If the USB pendrive is STILL not bootable, then a brute-force option to carry out here is to navigate to the syslinux-4.04/mbr subdirectory and to then run the command-string ‘cat mbr.bin > /dev/sdX’ without the ending “1”.

Note #1: I had to run ‘install-mbr -p1 /dev/sdX1’ after completing UNetbootin in order to effectively fix my USB pendrive’s Master Boot Record (MBR), and to then successfully boot the pendrive.
This particular command-line ‘install-mbr’ command happened to be available in the non-Ubuntu Linux distro I was using to create the liveUSB pendrive.  The command string ‘install-mbr -p1’ works by re-writing a MBR to the pendrive and then activating the pendrive’s first partition — through the “-p1” option — as the bootable one.

Following the use of ‘cat mbr.bin > /dev/sdX’ or ‘install-mbr -p1 /dev/sdX1’, you might have to re-run the prior command string ‘syslinux -s /dev/sdX1’ to ensure that the pendrive device can now successfully act as a bootable liveUSB.  Once you have completed all three of the above steps A thru C, then the best proof that you have successfully created a liveUSB on your pendrive is to actually use your pendrive to boot-up your computer.

A key note: You should definitely reboot your computer at this point in the creation of a liveUSB and check your BIOS settings to make certain that your boot-order has something like a “USB” or “Removable drive” boot option placed at the very top of the boot-order list. Among other good references for figuring out how to carry out this last step is the webpage ‘How To Boot From a USB Device’ at http://pcsupport.about.com/od/tipstricks/ht/bootusbflash.htm

The SystemRescueCD liveCD project has a useful manual webpage in English entitled ‘How to install SystemRescueCd on an USB-stick’, http://www.sysresccd.org/Sysresccd-manual-en_How_to_install_SystemRescueCd_on_an_USB-stick
I found that some of the non-SystemRescueCD-specific information near the bottom of this SystemRescueCD page is helpful, but I also think that webpages OTHER than this and OTHER than the many Ubuntu-specific ones are just as helpful in successfully creating a bootable liveUSB without the specific requirement to use MS-Windows or to use a Ubuntu-based Linux distro.

 

I think it is appropriate to remind readers again here near the bottom of this post, what I wrote near the top. A nice collection of ‘Learning the Command Line’ links is renown BerkeleyLUG contributor’s own webpage  http://www.grantbow.com/command_line.html ; very applicable to steps A through C that I’ve already spent most of the body of this post writing about.

Of course any of you reading this are perfectly welcome to provide below, your newly-revised and hopefully BETTER suggestions for distro-neutrally creating bootable liveUSBs 🙂

=====================