You are here: welcome » dcore » development

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
dcore:development [2015/01/11 16:54] – [Package management scripts guide:] LichenSymbiontdcore:development [2015/05/21 16:32] (current) – page is replaced by [[dcore:sce-import]]; left un-linked for reference by original author sm8ps
Line 1: Line 1:
-===== Development: =====+===== Development =====
 ==== Package management scripts guide ==== ==== Package management scripts guide ====
-The scripts have quite a lot of fluff, without sufficient explinations.+The scripts have quite a lot of complicated stuff, without sufficient explanations.
 So here is my (LichenSymbiont) guide to the scripts. So here is my (LichenSymbiont) guide to the scripts.
  
Line 7: Line 7:
 It also uses some more complex awk programs, so you should learn a bit about the [[http://www.grymoire.com/Unix/Awk.html|Awk language]]. It also uses some more complex awk programs, so you should learn a bit about the [[http://www.grymoire.com/Unix/Awk.html|Awk language]].
 And of course you need to know bash scripting, so here is a [[http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html|guide]]. And of course you need to know bash scripting, so here is a [[http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html|guide]].
 +And understand some pitfalls in Bash (some applicable to Ash): [[http://mywiki.wooledge.org/BashPitfalls|Bash Pitfalls]].
  
-**Starting with a general outline:**+** First a quick summary of the most important parts of the scripts: ** 
 + 
 +importsce is the command-line package manager (script), which calls deb2sce for getting the .deb files, and collecting them into the .sce package. 
 +It also calls the update function for the indices of Debian packages (debGetEnv). 
 + 
 +As you should expect, it must handle the regular Debian and Ubuntu package repository mirrors. 
 +Beyound that, it also downloads the dCore specific dependencies, and excluded packages, for the .deb packages (contained in /dCore/$build/import/deb2sce.tar.gz on the FTP). 
 +It also contains specific packages compiled for dCore (fltk library, flwm_topside and others). 
 + 
 +The scripts are complicated, unless you have a desire to learn, a good general understanding, and/or at least study them consistently. 
 + 
 +**SCE packages:**
  
 A sce package is a [[http://squashfs.sourceforge.net/|SquashFS]] file, which can compress data using multiple compression algorithms. A sce package is a [[http://squashfs.sourceforge.net/|SquashFS]] file, which can compress data using multiple compression algorithms.
Line 15: Line 27:
 But you can add new files to an already created file-system. Read about it [[http://www.tldp.org/HOWTO/html_single/SquashFS-HOWTO/|Here]]. But you can add new files to an already created file-system. Read about it [[http://www.tldp.org/HOWTO/html_single/SquashFS-HOWTO/|Here]].
 This allows for the creation of SquashFS files that stores all your favorite programs, in one file, by just incrementally adding them (this is not a feature of importsce yet). This allows for the creation of SquashFS files that stores all your favorite programs, in one file, by just incrementally adding them (this is not a feature of importsce yet).
 +
 +Then sce packages are mounted as [[https://en.wikipedia.org/wiki/Loop_device|loop-devices]] through loadsce (which just uses "mount -t squashfs -o loop..."), and located in the file-system at /tmp/tcloop/.
 +loadsce also loads all the dependencies of a package.
 +
 +** importsce: **
  
 Now I will walk you through the importsce script, so download the latest scripts, [[http://git.tinycorelinux.net/index.cgi?url=dCore-scripts.git/|here]]. Now I will walk you through the importsce script, so download the latest scripts, [[http://git.tinycorelinux.net/index.cgi?url=dCore-scripts.git/|here]].
Line 20: Line 37:
 And follow along, looking at the actual importsce script. And follow along, looking at the actual importsce script.
  
-First it sets global variables for the script, and sets up the /tmp directory. +  - First it sets global variables for the script, and sets up the /tmp directory. 
- +  - Then removes temporary files (potentially left from a previous run of the scripts). 
-Then importsce sets a bunch of variables from the options used when calling the command. +  - Then it sets the mirrors to the dCore specific packages and the file with extra dependency information for packages. 
-Then from its config file, if it exists (though it doesn't presently check this, just grep's the options). +  - Then importsce sets a bunch of variables from the options used when calling the command. Then from its config file, if it exists (though it doesn't presently check this, just grep's the options). So the config file takes precedence over the command-line options. 
-So the config file takes precedence over the command-line options. +  - Then decreases OPTIND by one, making the first argument the package name (instead of the one-letter options) 
- +  - Then sets the target .sce file to be created (just sets the TARGET variable to the package name used when calling it). 
-Then sets the target .sce file to be created. +    - And enters the /tmp directory. 
- +  - Then downloads the special dCore packages (like Xprogs and flwm_topside), and special dependencies for certain packages (which don't show up in a regular Debian system (as they would already be installed)). There are also other things, like dependencies to exclude. You can study the file yourself, by downloading it from [[http://www.tinycorelinux.net/dCore/x86/import/deb2sce.tar.gz|here]] (extract it into a new directory, as the files are in the root of the package). 
-Then acquires the mirror to the special dCore packages (like Xprogs and flwm_topside), and special dependencies for certain packages (which don't show up in a regular Debian system (as they would already be installed)). +    - Then extracts its content into a bunch of text-files. 
-There are also other things, like dependencies to exclude. You can study the file yourself, by downloading it from [[http://www.tinycorelinux.net/dCore/x86/import/deb2sce.tar.gz|here]] (extract it into a new directory, as the files are in the root of the package). +  - Then checks if the target package is to be excluded (and exits if it's on the excluded list). 
-Then downloads the latest file containing these packages. +  - Then it checks if a .lst file (file containing a list of packages to download) is being used, and changes the targetted .sce accordingly. 
-And extracts its content into a bunch of text-files. +  - Then it checks for more options from the config file, and then checks if you are trying to use a Windows file-system to extract files to (and if so, it informs the user of this, and exits). 
- +  - Then it let's you enter a package name, if no target it specified (why is this so far down the script?). And exits the script if you don't specify anything. 
-Then it checks if a .lst file (file containing a list of packages to download) is being used, and changes the targetted .sce accordingly. +  - Then it checks if it can find the package in the package indices (both regular repos, and meta-packages and special dCore packages). This is done through the debGetEnv script, which also stores the relevant repo in the DEBINX variable. 
- +  - And finally it calls the deb2sce script (through the mksce function) after a bunch of checks, which does the real work of downloading the package(s) (with dependencies and all).
-Here I skip a few checks... +
- +
-Then it let's you enter a package name, if no target it specified (why is this so far down the script?). And exits the script if you don't specify anything. +
- +
-Then it checks if it can find the package in the package indices (both regular repos, and meta-packages and special dCore packages). +
-This is done through the debGetEnv script, which also stores the relevant repo in the DEBINX variable. +
- +
-And finally it calls the deb2sce script (through the mksce function) after a bunch of checks, which does the real work of downloading the package(s) (with dependencies and all).+
  
 Then after deb2sce has downloaded all of the required packages, it runs the mksquashfs program, which takes all of the downloaded deb files and makes the sce package. Then after deb2sce has downloaded all of the required packages, it runs the mksquashfs program, which takes all of the downloaded deb files and makes the sce package.
Line 54: Line 63:
 First skim over the functions, to understand generally what they do. First skim over the functions, to understand generally what they do.
  
-Then we start at #Main, where it first checks the options that deb2sce is called with. +  - Then we start at #Main, where it first checks the options that deb2sce is called with. And where it exits the script if no target is specified. 
-And where it exits the script if no target is specified. +  - Then it displays the repository (DEBINX) and mirror to the user. 
- +  - After it removes the unused debinx files, it checks if the user is using a package-list file for making a collection in an sce file. 
-Then it displays the repository (DEBINX) and mirror to the user. +  - Then checks if the package collection matches already existing meta-packages, and sets the META variable if it is matched. And if it's not, the META variable is set at the top of the if-statement block, so META will always be set when using your own package list. The /tmp/.targetfile is set by importsce, and is just a copy of your custom package list. 
- +  - If it matches a regular Debian package, it sets the mirrors and other variables pertaining to the repo containing it, and displays this information. 
-Then after it removes the unused debinx files, it checks if the user is using a package-list file for making a collection in an sce file. +  - Then it creates the directory for the stored deb files, and sce files. And changes the import directory to the in-RAM one, and enters it, if you have set this option. 
- +  - Then creates and enters the temporary directory for extracting your package(s). 
-Then checks if the package collection matches already existing meta-packages, and sets the META variable if it is matched. +  - Then it gets the dependencies for your package(s), using debGetDeps -- iterating through the contained packages in your package list -- if that's what you are using. 
-And if it's not, the META variable is set at the top of the if-statement block, so META will always be set when using your own package list. +  - Then it checks if there are blocked/excluded packages, which is set by debGetDeps. 
-The /tmp/.targetfile is set by importsce, and is just a copy of your custom package list. +  - Then it gets extra dependent packages from /tmp/.importdep, which is set by importsce. 
- +  - Then checks if the target sce already exists (why so far down?). 
-If it matches a regular Debian package, it sets the mirrors and other variables pertaining to the repo containing it, and displays this information. +  - Then it checks if a .dep file already exists for you target package, and informs the user of them. 
- +  - Then it checks if there is enough space to extract into. 
-Then it creates the directory for the stored deb files, and sce files. +  - Then it adds the new dependencies to the temporary dependency file (/tmp/.pkglist). 
-And changes the import directory to the in-RAM one, and enters it, if you have set this option. +  - Then the interactive import logic. 
- +  - Then it gets the package description file for packages, and extracts it into the temporary target extraction directory. This is a big block of code. 
-Then creates and enters the temporary directory for extracting your package(s). +  - And finally, in this big loop, it downloads the .deb file as well, using the fetchDeb() function. And extracts it into the temporary target dir. 
- +  - And finally, after a lot of operations (a bit too complicated to put here now), it uses mksquashfs to compress the temporary target directory into a SquashFS file -- the sce package.
-Then it gets the dependencies for your package(s), using debGetDeps -- iterating through the contained packages in your package list -- if that's what you are using. +
- +
-Then it checks if there are blocked/excluded packages, which is set by debGetDeps. +
- +
-Then it gets extra dependent packages from /tmp/.importdep, which is set by importsce. +
- +
-Then checks if the target sce already exists (why so far down?). +
- +
-Then it checks if a .dep file already exists for you target package, and informs the user of them. +
- +
-Then it checks if there is enough space to extract into. +
- +
-Then it adds the new dependencies to the temporary dependency file (/tmp/.pkglist). +
- +
-Then the interactive import logic. +
- +
-Then it gets the package description file for packages, and extracts it into the temporary target extraction directory. +
-This is a big block of code. +
-And finally, in this big loop, it downloads the .deb file as well, using the fetchDeb() function. +
-And extracts it into the temporary target dir. +
- +
-And finally, after a lot of (a bit too complicated to put here now) operations, it uses mksquashfs to compress the temporary target directory into a SquashFS file -- the sce package.+
Print/export
QR Code
QR Code dcore:development (generated for current page)