[Letux-kernel] Organization U-Boot for retail OS

[H. Nikolaus Schaller]

Hi Patrick,

> Am 18.06.2018 um 00:57 schrieb patrick <patrick at wylight.de>:
> 
> On 16.06.2018 13:28, H. Nikolaus Schaller wrote:
>> Hi Patrick,
>> 
>>> Am 16.06.2018 um 01:18 schrieb patrick <patrick at wylight.de>:
>>> 
>>> On 15.06.2018 17:42, H. Nikolaus Schaller wrote:
>>>>> Am 15.06.2018 um 00:36 schrieb patrick <patrick at wylight.de>:
>>>> 
>>>> So this was a good point in time rebase and fix and extend the code. But
>>>> then we did leave it as is. Especially because the config system was in
>>>> the middle of being upgraded and there were experiments with device tree
>>>> in u-boot. So we decided to "never touch a running system".
>> 
>>>> That's an interesting topic to discus, because my opinion is quite the
>>> opposite. But I don't want to start an argument on my second day on this
>>> list so let's keep that discussion for the release party and in the mean
>>> time I try to persuade by action.
>> 
>> Yes, let's do it!
>> 
>> My argument is not a technical one but an organizational one...
>> Or weighting usefulness vs. efforts.
>> 
>> And there I see that rebasing u-boot to every new release from upstream
>> doesn't add many significant features, but takes a lot of testing time.
>> Maybe I am a bit naive here but besides from booting, what do you have
> to test? In the end the only u-boot feature you are using is one
> straight boot path.

No. There are upstream changes that may break your system and your board
may boot, but not everything has been initialized in the same way the kernel
relies on. For example we have some special initialization for the
charger chip.

And the key problem we have with kernel development is that kernel
maintainers do not accept everything. So there is a zombie state where
upstream does not care about private work, because they don't see it.

> 
>> Therefore, if we wait until a big set of new features have accumulated
>> and then do the rebase and testing once, it is a much better improvement
>> for the invested time.
>> 
> In my personal experience I made a big mistake, waiting to long to
> rebase so that rebasing costs me a lot of time.

Yes, it did cost us ca. 2 months to rebase 2013 to 2016.

> Since all my code is
> mainline.

Ok, if you are happy to get to that point. We are hunting for this
for years to get in the same state for the kernel but it is impossible.

> All I do is compile and boot test once in a while. I don't
> have to ship new versions, but I know that if I had to I could
> immediately start without rebasing multiple iterations of u-boot first.
> Working on the latest version, has the same benefits as for the kernel:
> - you have community support
> - you can send your changes immediately to mainline
> While getting them accepted can sometimes be a real pain, it is worth it
> in the end. You get code review for free! And in my experience we
> reduced my initial code to 30%. Since my board is in mainline it
> received 7 commits. Only two of them are from me. The rest was bug-fixes
> and global cleanups made by the community. All for free!

My key concern is that you don't get everything upstream. If you get it
it is nice.

> 
>> For the kernel I see the reverse situation. Every new release brings
>> significant improvements - either from upstream or by our own work to
>> fill gaps (e.g. missing device drivers). For U-Boot we have almost no
>> missing features because it does what it is designed for.
>> 
>> Therefore for the kernel, the rolling rebase scheme has turned out to
>> be better manageable than taking a snapshot, develop and fix missing
>> things the next years. But not be able to rebase it to a newer kernel
>> because kernel APIs have diverged too much.
>> 
>> Any my final though is that u-boot belongs to the basis we want to build
>> something on. So it should be stabilized - but not changed too often...
>> 
> In my opinion every software is the same. They all have bugs. So all of
> them need to be update able.

Well, this contradicts your idea of that there are no bugs if it passes
the compile&boot test :)

> If I update software, I have to make sure
> the update isn't making anything worse. So I have to test. If I master
> my testcases I don't have to care about frequent updates anymore. If my
> test process is good, I don't have to make a distinction between small
> bugfix updates or big version upgrades. They are both best tested and
> fulfill the same quality standards.

Yes, that is correct.

> 
>> Where I see the next good situation for upgrading u-boot source code
>> is when there is no longer a (horrible) mixture of old u-boot config
>> and Kconfig where it sometimes needs experimentation to find out what
>> should be configured where. I don't know how the status of 2018.05 is
>> but 2017 u-boot wasn't much better than 2016.
>> 
> While getting into mainline, they will tell you exactly what config
> option you have to move to Kconfig, now. If you have your board in
> mainline the community will convert the rest of config files to Kconfig
> for you!

Yes, that would be nice.

Anyways, if you think it is the best way and can do it I am fine with this.
I just want to make sure that we are working on the most important things
(from my overall view we have more than 10 times as many issues with kernel
than with u-boot) and they do not end in a cul-de-sac.

>>>> 
>>>> if you need write access at some point, please contact me.
>>>> 
>>>>> or github
>>>> 
>>>> git://git.goldelico.com/gta04-uboot.git is mirrored to
>>>> https://github.com/goldelico/gta04-uboot
>>> 
>>> I have a compiling tree, now. I am pretty sure it won't boot, but at
>>> least the compiler is happy.
>>> 
>>> I tried to login/register on https://dev.pyra-handheld.com but didn't
>>> got a mail till, now.
>> 
>>> 
>>> So I forked your repo on github and pushed a new branch with my rebase:
>>> https://github.com/bruenn/gta04-uboot/tree/v2018.05-pyra-v1
>> 
>> Nice. If I find time, I'll take a look into it.
>> 
>> Maybe we should pull it into http://git.goldelico.com/?p=gta04-uboot.git;a=heads
>> so that it shows up as
>> 
>> 	letux-2018.05
>> 
> Well, if that quick & dirty rebase is working for your other boards,
> sure take it.

Well, I always wonder why there is some "my board" and "your boards"
notion. Letux OS is "our boards" and Pyra is one of ours. Of course
everyone has favourites and not everyone has the same set of boards
to really work and do tests.

>> or alternatively as
>> 
>> 	work/patrick/letux-2018.05
>> I would prefer this. Because my target is to ultimately get rid of any
> out tree patches for the Pyra. To reach there I will have to remove
> everything not Pyra related from my intermediate branch and then regroup
> the changes in small commits which I can send upstream one by one.

> On that way I will create a lot of new branches and rebases.

That is how we do it for the letux kernel as well. There we have ca. 30
feature branches and merge them together for every release. Similar to
linux-next.

These branches are cleaned up and tried to get upstream making them empty
after a while.

Unfortunately some are not empty since 5 years...

> You can get an idea from the names.
> <u-boot-base-version>-pyra-<rebase-version>: f.e.:
> v2018.05-pyra-v1 -> v2018.05-pyra-v2... v2018.07-rc1-pyra-v1....
> Today I begun divide and conquer and removed beagle and gta files from
> v2018.05-pyra-v2.
> so my diff was shrinked from:
> 197 files changed, 13598 insertions(+), 74 deletions(-)
> to:
> 56 files changed, 2335 insertions(+), 74 deletions(-)

I wonder if it would be possible to do the same for the gta/beagle files
while you are alredy working on this, so that the whole letux u-boot can
finally be upstreamed? Letux OS is not a private project, but for the
community.

> 
> To continue I have some questions:
> Which is the correct boot-scr/dtb/defconfig for the Pyra?
> uEnv-letux-cortex15+pyra.txt

this one.

> uEnv-letux-cortex15.txt

Letux Cortex 15 is the CPU board (which is to be used for different
projects as well and can even be booted stand-alone) and + pyra is
for the combination with the Pyra mainboard.

The idea is to be able to replace the CPU board by another one
as easyly as possible and not have to dig into the pyra components.

Therefore, please keep lc15 and pyra separated! Having just a single
pyra board upstream but not lc15 makes this upgrade path more painful.

> uEnv-letux-cortex8.txt

this is an am3358 board.

> uEnv-omap5-uevm+pyra.txt
> uEnv-omap5-uevm.txt

this is the omap5evm and there was a prototype of the pyra
mainboard connected to the omap5evm by cables.

> 
> OSD3358 is not relevant for the Pyra is it?

No, that is also the AM3358 chip (inside a module).

> 
> What is board/goldelico/cmd_monitor.c? I didn't found any reference
> within the u-boot repository.

That is a tool built into SPL which allows you to break into the
serial boot process and have some very primitive console commands.

This is useful if a board can not access DRAM properly. But we don't
need that regularity.

> 
> 
>> and give you write access so that you can simply push.
>> 
>> For getting access, please register to http://projects.goldelico.com/p/gta04-uboot/
>> provide your public key in the user's settings and I can add you to the
>> "Happy Crew" which gives you write access to the git repo.
>> 
> Done.

Ok, you are now member of the happy crew!

> Small note: the webinterface doesn't except my ed25519 key. Maybe
> it's just a config option or upgrade away.

Seems to be too new for Indefero. Unfortunately, Indefero isn't maintained
for a while.

> For now, I generated a rsa one.

So please test. If it does not work, I have a debug procedure and we could
always solve it.

BR and thans,
Nikolaus