DESIGN

In essence, Cauldron consists of two sub-components: cauldron (which is for
creating installation ISOs, and manages building packages for the ISO and
making sure everything fits together), and enchantment (which is for the
installer interfaces that run during the ISO run-time). The two are related to
each other, but perform very distinct tasks, and are used at different times
(cauldron is for build-time, enchantment is for run-time). To think of it
another way, cauldron could be likened to "make cdrecord", while enchantment
would be a front-end to cdrecord.

===
ISO Generation - cauldron
===

What follows below is a rough outline of the process Cauldron needs to go
through in order to create ISOs. For the implementation, each separate part of
the process should be as modular as possible. Additionally, although each part
should be written so that user intervention is not needed, manual processing
should be allowed for at every stage. This is to allow for fixes/workarounds to
be applied in the event of bugs in the code (which will invariably happen), as
well as to allow for more complicated processing that we either can't handle in
an automated way, or haven't had the foresight to account for ahead of time.

The basic toolchain can/should be gotten from the running host system if
possible. This can be accomplished by using binary caches from
/var/cache/sorcery. If the cache file for a given toolchain spell does not
exist, it should not be taken from a repository somewhere, or from a basesystem
tarball, unless absolutely necessary (see next paragraph). Nor should the iso
builder have to re-cast one of the tool-chain components in order to generate
the cache file for use. Rather, it should be taken from the already existing
installed files on the system. This will require sorcery to generate binary
tarballs of installed spells. If it doesn't do this already, a solution can be
had by grabbing files from the installwatch logs (/var/state/sorcery?) and
rolling those into a tarball. (I believe devel sorcery handles this situation.)

In the event that the host system does not have a build toolchain itself (an
embedded system, for example, or a system geared for security), then a
basesystem tarball will be needed. This may also be the case if cauldron is
being run from a non-SMGL host system. However, for the intents and purposes of
Cauldron, that basesystem will then be considered the host system, so the same
discussion as above still applies.

Once the basic tool-chain tarball is generated/found/copied, it can be unpacked
inside of the build directory. This, along with a copy of sorcery, should
provide everything needed to generate the iso aside from the spell packages
themselves, which should be gotten over the net or copied in manually from
pre-downloaded files. For grabbing over the net, a proper resolv.conf will be
needed inside the iso build chroot.

The settings for sorcery should be set for the target system (target cpu,
etc.). For optimizations, strip should be used. The question of whether -Os
should be used or not depends on how stable the resulting builds will be for
the minimal architecture to be supported by the iso being built (e.g. i486,
x86_64). If -Os will cause problems for some machines to boot the iso which the
iso was intended to boot on, then it obviously should not be used. If -Os can
be used without causing any problems for the booting of the iso on the target
systems, -Os should be used in addition to --strip in order to reduce the size
of the iso.

Afterwards, a basic system can be built inside the build chroot using
sorcery and the configured tool-chain. This basic system should be absolutely
minimal, as it will be what is non-optionally installed onto the target system.
After the base system is generated/built, the chroot should be exited, and the
cache files for all the spells that are to be included in the target install
(non-optional spells) should be unpacked into a separate directory. That
directory will then be archived and compressed as system.tar.bz2 to be placed
on the ISO. Don't forget to clean out unnecessary files from the new separate
directory, _not_ the original build directory, before making the bzipped
tarball. [system.tar.bz2] (unnecessary files will include the host
(cross-compile) tool-chain used to build the base system, as well as [possibly]
sorcery files in /var, anything in /usr/src, etc.)

Chroot back into the main build dir (i.e. not the system install copy).
Build the iso system by casting additional spells on top of the base system
generated in the previous step. This includes a kernel (config for the kernel
SHOULD be taken from Cauldron itself), filesystem tools (reiserfstools,
xfstools, e2fstools, etc.), and anything else required not only to boot the
target system but manipulate and install onto it. This means that all features
desired in the installer must be present or provided for through the spells
cast in this step or the previous step (though very little should come from the
previous step, as the previous step should really only provide an absolute
minimal tool-chain generated for the target system).

Exit the iso build chroot. Unpack the necessary spell caches to a separate
directory for the iso system. Remove any unnecessary files/packages/etc. from
the iso (i.e., run the cleaners on it). Copy in the bzipped tarball of the
system install (system.tar.bz2).

Generate an initrd/initramfs to be used to boot the ISO, and place it in boot/
in the iso system directory just created and populated from the build spell
caches. Configure the boot/init process for the iso. This includes modifying
isolinux/isolinux.cfg, as well as various scripts/configs in /etc, as well as
generating an initrd/initramfs for the ISO. Pay particular attention to the
init system (including what will happen "before init"). (Whether or not other
bootloaders, such as grub, can be used as viable alternatives to isolinux will
be researched later.)

Copy whatever installers are going to be used during the ISO runtime into the
ISO. Make sure that it is set to run the installer(s) upon booting.

Generate a bootable iso using mkisofs (or whatever other ISO creation tool you
prefer). Compress the iso using bzip2. [e.g. smgl-devel.iso.bz2]


===
Installers - enchantment
===

All common functionality for installers should be abstracted into libraries
that can then be sourced by each installer. This is especially true for any
back-end processing (any processing which is not directly related to
interfacing with the user - compressing a file, keeping track of the current
installation step, etc. are all back-end processes). Output messages should, to
the extent possible, be separated from code to allow for easier translation and
integration into i18n.

To achieve the above goals, the installers are set up as follows. The common
library is libenchantment, which is responsible for tracking the current step,
maintaining the history of steps (both forward and backward), getting the list
of modules, etc. For state tracking, this is left to lib.potion.

The modules (steps) themselves are placed in their own directory, which will be
common to all installers. Each step will have a short summary description as
$module.info, while the actual instructions are in $module. Because there may
be processing specific to a module that really isn't shared with other modules,
or we may need to do processing specific to the start of a module (which would
be too tedious to test for each time in libenchantment), the each module will
also have it's own enter, exit, and potion subdirectories, which can contain
scripts for processing each of those events. Any scripts placed in those
directories should be named based on the function of the script (fdisk if it is
to run fdisk one state information, for example). We may eventually want to
make a common directory for such scripts, and then symlink them into the module
subdirectories, and only have actual script files in the module subdirectories
when they are truly unique to that module.

Each installer will have its own directory. There will be required files that
each installer is required to have (such as the list of modules it uses and
their ordering), though this may be changed later if they are all forced to
have the file contents in common. No installer should conflict with other
installers (they should all be able to run in parallel, so that the user can
switch from one to the other seamlessly).

All installer state files are to be flat files. That means they should simply
list filenames, spell names, mount points, etc. directly in the file, one per
line. For organization, either use a directory or a separate file. Do not
implement file formats unless you really, really have to, and even then don't
do it without permission from the Cauldron Lead.

Everything that could conceivably be user-configurable, should be. This
includes what compression program is used, what language the installer is
running in, and even where the installer files are located. This should all
also be configurable at run-time, via the enchantment config file. If a user
implemented their own installer, but didn't want to make a whole ISO just to
include the couple of shell scripts comprising their new installer, as long as
they can mount a disk (or other device) containing the new scripts it should be
easy for the user to point enchantment at them and run them from the ISO. If
the user speaks a language other than English, seeing their native language
should require something as easy as changing ENCHANT_LANGUAGE="en" to
ENCHANT_LANGUAGE="zh", for example. Ensuring user customizability will require
a little extra effort initially, but will make the installers far more flexible
and powerful, which will in the long-run help minimize the effort to add new
code/features as well as make users more happy.