[Lenny400] Un-piling git commits of letux-base

H. Nikolaus Schaller hns at goldelico.com
Wed Oct 4 21:12:03 CEST 2017


Hi,
some of you may remember that we initially had a git
development pattern that simply piles up patches
in a single letux-kernel branch.

This is a good approach in some cases:
* easy to understand
* multiple contributors possible
* it is clear what the latest version is
* never rebase
* no git push --force and git pull --force needed

But it has a big drawback:
* features are not packed into easily identifiable "development branches"
* it is not possible to git format-patch for upstreaming a single feature
* it is not possible to rebase on top of linus/master or linux-next
* it is not possible to work with feedback by upstream reviewers

The reason is that Linux has a different, non-linear development
model:

* there is linus/master
* new features (e.g. device drivers, fixes, defconfig, device tree) are always on top of linus/master or linux-next
* there is one patch set per feature change
* clean patches (they even discuss commas in commit messages)
* rebase often

So at some point of Letux-kernel development we did switch to
the Linux development model.

Well, not completely. We already had piled up ca. 3-4 years
of GTA04 development using the linear model.

So we introduced our own letux-base branch. This is the base
for all current feature development.

And every Monday or so, I merged the latest linus/master into
letux-base and rebased all relevant feature branches.

This worked quite well for all the stuff we have developed for
e.g. the Pyra or the GTA04A5 to name two of them.

And some of these feature branches could easily be rebased to
either letux-base, linus/master or linux-next to submit patches
upstream.

But we still had a lot of legacy commits in letux-base which
made it impossible to rebase letux-base itself.

Now it would have been too easy if git would provide a simple

	git cherry-pick linus/master letux-base -- interesting-files

Such a command does not exist :(

Therefore I wrote and debugged in the past weeks a script
that:

a) reads a control file (which says which source files should go
into which new feature branch)
b) does a git blame to find out when these files have been changed
c) works with git diff, git am and git commit -C to replay the
filtered changes into a new feature-branch that is on top of the
latest linus-master

There were some difficult aspects to solve: how to handle new
files and removed files and renames. How to handle permission
changes or permissions for new files (some scripts have +x mode).
And how to keep original Author and Author date so that proper
attribution can be maintained.

After a lot of debugging it now works and the letux git has been
reworked.

The result is a set of work/letux-base branches that exactly
copy the current letux-base if merged on top of v4.14-rc3.

All needs some more review, but I think we can soon switch
Letux-kernel completely to the official Linux git process.

Then we can review and reduce the new work/letux-base patch
sets since there are some spurious additions and removals
which have no function. They come from this effect:

            /--- A ---------- C ---------\
           /                              \
common-base                                E ------ head
           \                              /
            \---------- B --------- D ---/

If A adds some feature and we try to make a sequence of

common-base - A - B - C - D - E --- head

out of it, the algorithm will add the feature in A, remove it
in B, add in C, remove in D and finally add in E.

And in some cases almost all additions by A and B have
been removed by C and D so that a single line of change
may remain from 10 commits.

This needs manual work to squeeze all these patches into a
single one, but is not very difficult or urgent to do now.

The result of such a cleanup will not change anything in
our letux tree, but make it more obvious which patches we
have inherited from ancient time. And it makes it easier
to check if they are still good or hidden time bombs...

So please look at the latest git.

For reference, let me try to write branches as formulae:

old:

	letux-base := letux-base + v4.14-rc1
	rebase([all feature branches], letux-base)
	letux-4.14-rc1 := letux-base + [all feature branches]

new (two stages):

	letux-base := v4.14-rc3
	rebase([all letux-base branches], letux-base)
	letux-intermediate = letux-base + [all letux-base branches]
	rebase([all feature branches], letux-intermediate)
	letux-4.14-rc3 := letux-intermediate + [all feature branches|

So everything which makes up letux is placed on top of v4.14-rc3
instead of letux-base where v4.14-rc3 was merged in.

Anyways, development of letux-kernel continues almost unchanged.
Just in a slightly different set of branches.

BR,
Nikolaus




More information about the Lenny400 mailing list