[Tinkerphones] makesd (was Re: [Gta04-owner] New LetuxOS Kernels and some tricks and thoughts)

H. Nikolaus Schaller hns at goldelico.com
Tue May 28 14:27:44 CEST 2019


Hi Paul,

> Am 28.05.2019 um 13:20 schrieb Paul Boddie <paul at boddie.org.uk>:
> 
> On Monday 27. May 2019 20.46.32 H. Nikolaus Schaller wrote:
>> 
>> The pre-built images are just the result of a complete debootstrap in a
>> fresh chroot on some ARM or MIPS SBC. Either bare bone with some minimal
>> set of required or generally useful packages or with some more, e.g. a full
>> GUI like Xfce or QtMoko.
> 
> Right. So they are already fully configured.

Yes.

> This is the challenge when cross-
> bootstrapping, and with the CI20 they recommend using qemu for the second 
> stage:

I had experimented with qemu some years ago in the hope to make things
easier but it was a dead end... The solution was to use the Letux Cortex 8
(basically the same as a PocketBeagle) for <50€ and power and connect through
an USB cable to the Goldelico server. Some shell scripts can remote control
the debootstrap process and copy results from the SBC to the web-server's
file system. The Letux Cortex 8 also runs a copy of Letux OS for that purpose
on a 200GB µSD card (another 50€) so that it has a lot of space for all these
debootstrapped images...

This concept is running smoothly for ca. 3 years now.

Now, I have a LetuxOS image for the CI20 and can just do the same. Maybe
over Ethernet because USB/OTG seems to not yet be available with upstream
kernels. But the concept is exactly the same. I do not even have to modify
my remote control scripts significantly (just telling them to debootstrap
MIPSEL and use a different IP address).

> https://www.elinux.org/How_to_make_a_debian_rootfs_for_MIPS_CI20
> 
> I never got that to work, and given that a lot of the package configuration 
> should be machine-independent (it shouldn't need to see specific hardware), I 
> would rather see some kind of generic configuration getting done, leaving only 
> a small amount of machine-specific stuff, if any, to the actual hardware 
> itself.

> 
> (Emulation is just a hack because there isn't a nice way of running host 
> programs to do the configuration work within the target environment. So one 
> ends up running emulated binaries that do the same things as host binaries, 
> discarding the portability and interoperability benefits of having a set of 
> programs that behave the same way and can act on filesystems and other things 
> regardless of the system architecture.)
> 
> So, I decided to use my own tool to make a root filesystem instead:
> 
> https://hg.boddie.org.uk/qi-emdebian
> 
> This runs the second stage on the CI20 itself.

Well, I can run first and second on CI20... By eliminating the cross-debootstrapping
step.

> 
>> Configuration of the LetuxOS is now completely done by some
>> letux-something.deb packages. So no single file is touched (well, there is
>> some fallback at the moment that should be eliminated soon) besides through
>> installing a .deb package. So it can also be removed or replaced (there are
>> usually preinst and postrm scripts to make backups). The same for the
>> kernel package.
> 
> This part starts to sound a bit like Jonas's Boxer tool and its objectives.

Yes, a little.

But I think the task here is much simpler and a set of .deb packages just does
what is needed. IMHO RaspBian has invented the same concept independently. It is
not intended to become a swiss army knife tool like I interpret Boxer.

Here, the task is to make SD cards boot an fill with a useful rootfs...
There is no dependency on what hardware is used (which is sometimes not easy).
And almost no dependency on the GUI. Basically most packages are meta-packages
just listing which useful standard Debian packages are to be installed. A slightly
complicated thing is that not all packages have the same name in wheezy, jessie,
stretch, buster etc. or do not exist in all architectures (e.g. x11 for omap3).

So the main task of that is not to invent a new tool but just avoid having
someone type a list of parameters for apt-get install after first boot and
setting up a network access. Keep it simple is the idea...

> 
> [remakesd]
> 
>> Yes, I noticed that you are trying to make it a little more object-oriented
>> by this approach, especially in the aspect of modularization and
>> encapsulation of implementations.
>> 
>> Unfortunately the bash isn't the best OO language :)
> 
> If I were really going for object orientation, I would probably use Python 
> instead.

Indeed.

> 
> I did think a bit more about the formulation of systems in makesd yesterday, 
> trying to understand how some of the options work, and it occurred to me that 
> if I were describing the partitioning scheme

That is an assumption which makesd tries to avoid. A user might not even need
to know that a specific board needs two partitions and another one a single and
some invisible disk area. So why should he initially care about that partitions
exist?

Therefore the basic syntax is:

	makesd $device [ -r $gui ]

This may end up in single or dual partition SD cards, depending on what the $device
needs. And a general SD card that can even be swapped between devices (assuming they
have the bootloader in some NAND flash and therefore do not need a FAT partition)
can be built by

	makesd debian - r $gui

This is like removing a HDD from one PC and plugging into an USB adapter box and
connecting to another one.


> then I might use a notation like 
> this:
> 
> ----
> 
> partition debian
> type: ext4
> root: debian
> kboot: latest
> dboot: latest
> modules: latest
> config: latest
> 
> partition lxde
> $debian
> root: lxde
> 
> partition lc8-boot
> type: fat
> size: 5
> boot: Letux-Cortex-8/latest
> kernel: latest
> devices: latest
> 
> partition lc8-root
> $lxde
> kernel: none
> devices: none
> size: 95
> 
> system lc8
> $lc8-boot
> $lc8-root
> 
> ----
> 
> The motivation here is that I want to be sure of how many partitions there 
> are, which is not immediately clear from the macro argument syntax.

Which is intended as described above...

BR,
Nikolaus

> So, 
> partitions are described (using the argument names from makesd in the above as 
> parameters) and can be specialised. Thus, the "debian" partition is a general 
> description of "vanilla" Debian, but the "lxde" definition incorporates the 
> "debian" definition and specialises it.
> 
> What I would want to enforce is the separation of partitions and other things. 
> Here, I define the "lc8" system as incorporating two partitions. It would not 
> be allowed to just reference "lxde" and then add some overriding parameters:
> 
> ----
> 
> system lc8
> $lc8-boot
> $lxde
> kernel: none
> devices: none
> size: 95
> 
> ----
> 
> This is because it would become difficult to see how many partitions there 
> might be, and the semantics would need to be considered about what happens 
> when partitioning parameters (like "size") are mentioned: do they declare 
> another partition?
> 
> This kind of notation reminds me of other things, which I suspect might be 
> these "orchestration" tools people use for cloud computing and provisioning. 
> However, I think multistrap also has some similar notion of specialising 
> definitions:
> 
> ----
> 
> [Debian]
> packages=udev busybox-static
> packages=openssh-server openssh-client
> ...
> 
> ----
> 
> Here, from this extract of my CI20 configuration, the existing "Debian" 
> definition (not shown) is augmented by additional packages. Naturally, 
> multistrap focuses on root filesystem population, not partitioning, but the 
> customisation principle involved seems to be very similar.
> 
> Paul
> 
> P.S. In the wider Debian realm, I remembered that there is a Linaro-
> originating project that tests single-board computers, and I had thought that 
> there might be some tools of interest within that. Here is the documentation:
> 
> https://lava.collabora.co.uk/static/docs/v2/index.html
> 
> But it seems like a very heavyweight thing, aimed at continuous integration. 
> And the documentation does not make the parts of interest to us particularly 
> obvious.
> _______________________________________________
> Community mailing list
> Community at tinkerphones.org
> http://lists.goldelico.com/mailman/listinfo.cgi/community
> http://www.tinkerphones.org



More information about the Community mailing list