dCore Test Drive on a USB Stick (Beginners 101)

Take dCore for a spin to understand how it works! Each step is performed manually first before it is eventually automated. Altogether it will lead to a more or less complete installation and deeper understanding of dCore.

dCore will be installed on a USB-stick. The system basically consists of the following files:

1. Linux kernel and initial RAM disk, typically named 'vmlinuz' and 'core.gz'
2. an archive 'mydata.tgz' containing persistent user data
3. a folder 'sce/' containing extensions (sce-archives)
4. lists of files for loading on demand or on boot, respectively

All these files or folders may well reside on any type of file system, even ones not respecting Linux permissions.

The user data in '/opt/' or '/home/' could well be stored on a partition of its own with a Linux file system. We shall not endeavour to doing so here, however.

As explained, dCore consists of a Linux kernel, a root file system in RAM, an init-process that governs the behavior of the kernel, BusyBox utilities, a choice of extensions and some persistent user data. What all is needed to get started with a basic desktop installation?

1. In order to install applications, one will need network connectivity. The easiest, of course, is wired but also wireless can be used.
2. dCore by itself provides a shell running in a virtual console. In order to start a graphical desktop, one must, of course, first have X11 with the necessary video drivers.
3. X11 would allow one to start graphical programs from the command line. In order to have some sort of graphical starting mechanism as well as resizeable windows, a window manager needs to be installed. FLWM is used, like in TinyCore, as basic standard but of course this can be changed at will.
4. Furthermore, a configurable dock allows to start applications from their icons; Wbar is used as basic standard.
5. Some applications demand a tray (or notification area) for which Stalonetray is a suitable lightweight candidate.
6. International users will need to take care of localisation questions.

All the above should start automatically so the system boots into a graphical desktop. This necessitates the appropriate boot codes as well as some persistent storage for configuration files.

Additional software, including system software, is available by importing and loading extensions.

These notes were created by using computers already with standard Ubuntu Linux installations; notably Grub2 as boot-loader. It should be trivial to adapt the outline to other boot loaders or environments.

1. Check the forum for recent release announcements. Look out for kernel version (3.16.6 for Trusty in this case here).
2. Download files from 'http://tinycorelinux.net/dCore/x86/release/dCore-trusty/':
   1. Kernel: 'vmlinuz-trusty'
   2. InitRD: 'dCore-trusty.gz'

As usual, double-checking MD5-sums is worthwile.

Use a regular USB-drive with FAT32 file system. The choice of file system is purposeful as it should provide insights into the dependency of dCore on the file system. This example uses the Grub2 boot-loader of the harddisk. Other set-ups will work just the same, of course.

¡Warning: dCore will detect and use an existing swap partition on harddisk. So make sure that there is no hibernation file saved in swap from some other OS on the computer!

1. Put the kernel and the initrd in the root of the USB-drive.
2. Boot your computer into a Grub2 shell and make it load dCore-Trusty by the following commands.
   1. set root=(hd1,msdos1)
   2. linux /vmlinuz-trusty
   3. initrd /dCore-trusty.gz
   4. boot

Adapt the first step to your specific situation; ls gives information about available drives.

Using Syslinux 3.63-3.70 from the USB-stick, the boot stanza is /vmlinu~1 initrd=/dcore-~1.gz (cf. explanation for the 8.3-file names.)

And voilà! You are presented with a GNU/Linux command line system. This all happens incredibly fast (5 seconds on a legacy netbook).

Some note-worthy facts that will be changed in the definitive set-up:

• There is one single virtual terminal tty1.
• There is a standard user “tc” who can gain root privileges by 'sudo'.
• The host name is “box”.
• Language and keyboard are set to US-English.
• There is no persistence at all. Changing any file will happen only on a temporary file system in RAM which will be discarded at shut-down. Rebooting will give you the absolutely identical system.
• Yes, the beeping when pressing the TAB-key is annoying, indeed! (You can get rid of it temporarily by issueing rmmod pcspkr.)

Let's play around a bit in order to get better acquainted with some of the guiding principles of the way dCore operates! As a first thing, one probably wants to have something more than just a bare command line system.

For simplicity a wired network connection is used. Assuming a working Ethernet port, the connection is automatically set up at boot via DHCP.

The command sce-import sl will …

1. check the repos for packages whose name begins with “sl” and then ask you to specify the right one.
2. ask you to certify that you, being of sound mind and body, do indeed mean to import that specific package.
3. gather the dependency information from the repos and allow you to review the list of packages that will be included into the extension 'sl.sce'
4. ask you to confirm that now the time has come to finally add some steam to dCore!
5. download the packages and merged them into a dCore extension. The appropriate file will be saved under '/tmp/tce/sce/' and the down-loaded Ubuntu packages under '/tmp/tce/debs/'. (As said before, this is only on a temporary file system in RAM.)

From the sce-directory, the extension can be loaded by the command sce-load sl. Now you can let the steam blow by the newly added command sl; also try the options -a, -l, -F while you are at it or also mis-spell it as ls … . (In case you should wonder what the name “sl” signifies, check out the man page.)

NB: adding the option -n to sce-import makes it proceed in a non-interactive fashion.

What does “loading” an extension mean? Each extension consists of a SquashFS-file with ending '.sce' that is stored under 'sce/' in the tce-directory. The command sce-load mounts this file system under '/tmp/tcloop/' as is easily detected by mount.

Let's explore the contents for once! '/tmp/tcloop/sl/' contains

etc/: ld.so.conf.d/
lib/: i386-linux-gnu/  ld-linux.so.2
usr/: games/  lib/ local/

Clearly, this whole file system is layed over the root file system, making all files available.

'/tmp/tcloop/sl/usr/games/' contains LS, sl and sl-h which all have their executable bit set. Luckily, '/usr/games' is included in the PATH. In case you should wonder about the commands LS and sl-h, just try them out!

There are two kinds of persistence to deal with. First there are the extensions. As explained, extensions come with their own file system (à la SquashFS), so they should work even with FAT32; however, there is a twist to it as we shall find out soon.

Home storage seems to be a different case as it does depend on a POSIX compatible file system being available. We shall deal with that case later.

After a reboot, the extension sl is not available anymore because it had been stored under '/tmp' in RAM which got discarded. To be cautious, you may want to verify that the command sl does not work anymore, indeed.

Let's set up persistent storage for extensions! The command tce-setdrive sets up persistent storage for the extensions; it will …

1. allow you to choose the hard-drive (choose the USB-drive for the moment).
2. create the directory 'tce/' in the root of the partition, together with sub-directory 'sce/' and two files named 'mydata.tgz' and 'sceboot.lst'.

From now on, the imported extensions will be stored under 'tce/sce/'.

Trying to import sl by a simple sce-import sl quits with an error message, complaining that the tce-directory resides on a Windows filesystem. Furthermore it suggests to try again with the -r flag to unpack files in RAM. Indeed, 'tce' may well reside on a Windows filesystem, but in order to unpack and repack packages, a POSIX compatible file system is necessary. Given sufficient RAM, this can be provided in an ephemeral fashion.

sce-import -r sl will import sl as an extension into the tce-directory. sce-load sl will mount the extension and provide you with the 'sl' command to play with.

The extension graphics-<KERNEL>-tinycore adds some resolution types which is nice for eg. netbooks. Try it out and see if it changes your resolution or that of your computer to the better! In case it does, treat is as part of the standard installation even if it is not always explicitly mentioned.

Let's venture on to get a graphical desktop up and running from our USB-dCore! Import and load the extensions xorg-all and flwm one by one (also graphics-<KERNEL>-tinycore if desired). xorg-all is a huge meta-package, consisting of all available X-drivers; later on, one may want to reduce that.

As an aside, take note that the Xprogs extension – which is mentioned in earlier documentation – has become a dependency of any xorg-extension.

The command startx &, however will fail with the rather uninformative message that the connection to the X server was lost. For a comparison, launching sudo startx will present a graphical Xterm in full-screen without decoration but with a functional mouse cursor. Exit with Ctrl+D back to the command line and see if we can shoot the trouble!

Inspecting 'startx', one can see that it calls '~/.xinitrc' which …

1. reads in the variable 'DESKTOP' from the file '/etc/sysconfig/desktop', specifying the chosen desktop manager (Flwm in our case).
2. starts the desktop manager, dumping errors to '/tmp/wm_errors'.
3. calls '/usr/bin/setupdesktop' as well as '~/.setbackground', '~/.mouse_config' provided the latter exist; the same it does with all files in the directory '~/.X.d/', waiting until all of the called programs have terminated.

Obviously, the information which desktop manager to call is missing for the moment. So create a file with the appropriate content by issueing

sudo su -c "echo flwm > /etc/sysconfig/desktop" 

Now try again with startx & and admire a shiny dCore desktop! At the moment, it does not contain much but by clicking anywhere on the desktop, an Flwm menu opens, giving you access to terminal, editor etc. After a playful moment, exit back to the command line prompt.

The previous steps have to be repeated after every boot:

1. set the tce-directory
2. load the desired extensions
3. edit configuration files as needed
4. issue startup commands for applications

More specifically:

1. tce-setdrive
2. tce-load to load the extensions xorg-all, flwm (optionally also graphics-<KERNEL>-tinycore)
3. '/etc/sysconfig/desktop': add flwm
4. call startx &

The only step so far in direction of persistence has been to set up a tce-directory in order to avoid having to down-load every extension anew. Let's tackle the other issues one-by-one! Later on, we shall also provide persistent home storage.

Just for the graphical environment, these steps can be automated very easily.

The boot parameter tce specifies the partition containing the tce-directory (in its root directory or else the directory must be explicitly specified), eliminating the need for calling tce-setdrive manually. So one wants to enhance the kernel boot stanza by adding tce=sdXn which stands for the device (sdb1 in my case). Take note that the syntax does omit the usual /dev/ from the device name!

In the case of a USB-drive, however, one has to add the additional code waitusb=5 to the boot stanza because otherwise dCore will simply rush trough too fast, even if the USB device needs more time to get ready. It is possible that some slower USB devices need a longer time-out; adjust the value as necessary!

All in all the Syslinux boot stanza has grown to /vmlinu~1 waitusb=5 tce=sdX# initrd=/dcore-~1.gz. As a result, the partition is mounted and the tce-directory is linked as '/etc/sysconfig/tce'.

In our approach here it is necessary to call tce-setdrive manually because we had started without a tce-directory. After having created one, we point dCore to it by means of the tce boot-code.

If one quickly wants to get to a dCore-installation, one will set up boot-loader, kernel and initrd and include the tce boot-code at the first start. dCore then recognizes that the tce-directory does not exist yet in the specified location and thus calls tce-setdrive by itself.

For the sake of completeness, we would like to point out that the partition can also be specified by uuid or label (cf. the FAQ about pendrives). This is highly recommended for USB-installations as one cannot know under what name the USB-stick will be recognized by the system in general. To keep matters simple, however, we leave this out.

The tce-directory contains a text file 'sceboot.lst' which specifies extensions that should automatically be loaded after boot, eliminating the need for calling sce-load manually. So let's put a list of desired extensions into that (persistent) file by the following command! (The option '-e' forces 'echo' to correctly interpret the escaped newline characters.)

echo -e "xorg-all\nflwm" > /etc/sysconfig/tcedir/sceboot.lst

The folder '/etc/sysconfig/' contains files whose respective content is taken from boot codes. Among others, there can be the following files (shown here with their respective boot code and, if applicable, standard values in parentheses):

• 'desktop': desktop; window manager, as seen above
• 'tcuser': user (“tc”)

In an analogous fashion, the hostname in '/etc/hostname' may be set by the host boot-code (with standard value “box”).

Combining these options into a single (here: Syslinux) boot stanza does make it desirable to have it included in the boot-loader menu, indeed. You may do that or also keep on typing it in manually a few more times only (promised!).

/vmlinu~1 waitusb=5 tce=sdX# desktop=flwm user=<USER> host=<HOST> initrd=/dcore-~1.gz

This boots into a graphical desktop (startx is automatically called). The boot time is prolonged by having each extension loaded, especially xorg-all.

Clearly, the codes user and host can be omitted in the future but try them out once just for kicks! Omitting the 'desktop' entry will drop one back to the command line after an unsuccessful launch of the graphical desktop (as explained above).

In case one does not need an extension anymore, one could simply delete its respective file from the tce-directory. However the command sce-remove is specifically designed for this task and does some more house-keeping than simply removing the files. Try it with the sl extension (even though, it is a pitty, isn't it?)!

Other than persistence for extensions there are two more cases of persistence, namely home storage and system configuration files. Both cases pertain to regular files as opposed to extensions which consist of an entire file system themselves.

Thus a simple solution for persistent home storage is to have it on a regular file system that gets mounted; the same may be done with '/opt/'. This is the regular approach for a fully fledged dCore installation. It necessitates a POSIX compatible file system, of course.

Let's venture on with our minimalistic set-up, however, where only a FAT32 system is available! As promised, this adventure will teach us more about dCore. Furthermore, our dCore set-up already provides us with all the necessary mechanisms for persistent user data; we just have to start making use of them.

Providing a POSIX compatible file system is not a problem; as usual, it is wrapped into an archive that can be extracted to RAM which can be stored on any kind of file system. Since /home/ and /opt/ essentially contain user data, these folders are combined in one file system.

There is one major draw-back to this approach; namely that files actually “live” in RAM and are only copied to persistent storage upon request, for instance at shutdown. Seen the other way, one gains the choice if a session should be stored (backed up) or not.

If one wanted to keep persistent storage on a separate file system, the Kernel would have to know about its location already during the boot process. For specifying a file system, the boot codes home and opt are used.

Why prefer this approach? One major advantages is that it avoids having to un-wrap yet another, possible quite large SquashFS for the user data.

Oftentimes, the usage of persistent file systems is coined “persistent home” or “persistent opt”, respectively. We use the term persistent here in the sense of not ephemeral.

All user files are wrapped up in the archive file 'mydata.gz' that resides in the root of the tce-directory. This file does not need to be touched as its content is un-packed by dCore, providing the home- as well as the opt-directory.

The choice of files to be extracted is stored in the file '/opt/.filetool.lst'. Initially, it contains only the name of the folders 'opt' and 'home'. Two remarks about this content:

• Notice the missing slash ('/') in front of the folders. Currently it seems there is no need for this anymore as also files with absolute path are backed-up, indeed (cf. a forum entry from 2010, however). So better be safe than sorry and leave out the initial slash. It does save energy, too!
• Also notice that white-space is not stripped from the end of a line.
• The seeming self-reference, namely telling to backup '/opt/' in the file '.filetool.lst' which is stored under '/opt/ itself does no harm. (Think about it, draw a flow-chart or take a break!)

Complementary, there is '.xfiletool.lst' which lists files to be excluded from the backup. This file obeys to different semantics than '.filetool.lst'. Indeed, the entries are patterns as used by the command tar. Briefly put, they describe the ending of a file, including its path.

When exiting from the graphical desktop, a selection window is presented with options to shutdown, reboot and a drop-down menu for the backup options. We shall discuss these in a moment.

Exit back to the command line (with no backup being performed) and let's create two files in order to see this all in action!

echo 'Personal file' > ~/Pers-test
echo 'Opt file' > /opt/Opt-test
sudo su -c "echo 'Configuration file' > /etc/Conf-test"

If one were to reboot now, these files would be lost because backup does not happen automatically at shutdown but must be triggered by the user. Since 'Pers-test' and 'Opt-test' are stored under '/home/' or '/opt/', respectively, they are already marked for backup. The file '/etc/Conf-test', however, must be included into '.filetool.lst' anew by

echo etc/Conf-test >> /opt/.filetool.lst

Launching backup will …

1. ask if the backup location is correct.
2. back up the files into 'mydata.tgz'

Since /usr/bin/backup simply is a wrapper around /usr/bin/filetool.sh, we can take a closer look at its proceedings by calling filetool.sh -bv which will backup and produce a verbose listing. As expected, all files under '/home/' and '/opt/' are listed as well as '/etc/Conf-test'. These file now persist across a reboot.

NB: do not mistake 'filetool.sh' for 'filetool', the latter being a graphical wrapper of the former.

If a file is erroneously deleted then it can be restored from 'mydata.gz', provided no backup has happened yet. Let's for example rm ~/Pers-test and restore it by filetool.sh -rv.

Shutting down the graphical desktop, the backup command can be triggered in the selection window.

In the graphical desktop, one can choose “Safe” as backup option. This means that an already existing 'mydata.tgz' is copied to 'mydatabk.tgz' before overwriting it with the new state. This is equivalent to filetool.sh -s.

If need be, the old version 'mydatabk.tgz' can be substituted for the new version or stored away under a different name.

For files in '/home/' or '/opt/' it suffices to simply delete them and trigger a backup. Files like '/etc/Conf-test' must be excluded from '.filetool.lst' in addition. This can be done by launching vi /opt/.filetool.lst, moving to the desired line with the arrow keys and hitting the 'D'-key twice. Save and quit by hitting Esc : w q or quit without saving by hitting Esc : q !. Of course, one may achieve the same in the graphical desktop by launching the editor (which is provided by the Xprogs extension, BTW; cf. the aside above).

Now get rid of the three test files for good and go take some lessons in Vi if necessary!

The location or, more precisely, the full path and name of the file used for storing persistent user data, is read from '/etc/sysconfig/mydata' which can be specified through the boot-code 'mydata'.

Similarily, the choice of files to be extracted from persistent user data into the root file system can be specified by an arbitrary file named 'yyy.lst' within the given tce-directory through the boot-code 'lst'.

The above catalogue of desirable features has only been partially completed. The crucial point being “issue startup commands for applications”.

Before taking a closer look at it, let us make clear that there are several distinct situations where one may find need for automatically running commands.

1. During system startup: essential system commands (with root privileges) that serve for “circumventing” some peculiarities of dCore imposed by its ephemeral philosphy. dCore provides a special mechanism for this case, taking the role of '/etc/rc.local' in standard Debian.
2. Similarly during system shutdown.
3. When launching a shell.
4. When starting a graphical desktop: auto-starting applications (with user privileges).

dCore provides the following mechanism adapted to these situations:

1. '/opt/bootsync.sh' and '/opt/bootlocal.sh' (cf. below for an thorough discussion)
2. '/opt/shutdown.sh' (ditto)
3. '~/.ashrc' can be used as usual
4. '~/.xinitrc' calls alls cripts under '~/X.d/', so individual auto-start settings can be stored there.

These files, residing in '/opt/', are shell scripts, interpreted by an Almquist shell ('ash').

• 'bootsync.sh' is executed at system startup. Its commands are meant be terminated before moving on; i.e. they should not be made running in the background by adding a ampersand sign ('&'). As such they may influence the duration of the boot process.
  • 'bootlocal.sh' is meant to be the last command called by 'bootsync.sh' and it is run in the background. As such the system does not wait for them to complete before it moves on with the boot-process.
• 'shutdown.sh' is similar to 'bootsync.sh' but it does in addition do some house cleaning of files matching the patterns from '/opt/.xfiletool.lst'.

So system startup commands that need to be completed before other processes can be called go into 'bootsync.sh'. Commands that have to run during startup but where nothing depends on them go into 'bootlocal.sh'. Individual shutdown commands go into 'shutdown.sh'.

These files run with root privileges and, accordingly, can only be manipulated as root user, so one must use sudo vi (or sudo editor in the GUI).

Let's, for example, generate a symbolic link '/tce' in the root of the file system to the tce-directory '/etc/sysconfig/tcedir/'. Furthermore, we want to remove the pcspkr kernel module in order to get rid of that bleepin' beeping! The following commands will add appropriate lines to '/opt/bootlocal.sh'.

sudo su -c "echo 'ln -s /etc/sysconfig/tcedir/ /tce' >> /opt/bootlocal.sh"
sudo su -c "echo 'rmmod pcspkr' >> /opt/bootlocal.sh"

Don't forget to trigger a backup before rebooting!

The black-listing of kernel modules can also be achieved via boot codes. In case one should not have been able to say aloud “bleepin' beeping” five times in a row without mis-spelling, one should include blacklist=pcspkr (also compare the discussion in the forum) into the boot-stanza.

This sample set-up addressed the following issues:

• Running a basic dCore installation from a FAT32 file system
• Persistently importing extensions into the tce-directory
• Booting into a minimal graphical desktop
• Using boot codes to set system variables (especially desktop)
• Automatically loading extensions via 'sceboot.lst'
• Persistently storing user data in 'mydata.tgz' for selected files listed in '.filetool.lst'
• Automatically running system command during startup or shutdown

Where to head from here? Check out the page about installations and see if there is one that suits your needs. If not, you might want to produce one of your own and document it here in the wiki. Contributions are very welcome!

This sample set-up leaves desire, among other things, for …

• handling the boot-loader from within dCore
• localisation (in the command line as well as in the X environment)
• wireless connections
• updating extensions
• upgrading the system

There are several explanations in the wiki that provide very informative reading after having followed all along. A recommended track consists of a basic desktop installation followed by a switch to LXDE. A study of the techniques for handling extensions is pretty mandatory.