[Openpvrsgx-devgroup] New version letux-pvrsrvkm-v5.6.0-rc1 available

H. Nikolaus Schaller hns at goldelico.com
Tue Feb 18 19:08:36 CET 2020


Hi Tony,

> Am 18.02.2020 um 16:27 schrieb Tony Lindgren <tony at atomide.com>:
> 
> * H. Nikolaus Schaller <hns at goldelico.com> [200218 14:31]:
>> Hi Tony,
>> 
>>> Am 18.02.2020 um 15:06 schrieb Tony Lindgren <tony at atomide.com>:
>>> 
>>> * H. Nikolaus Schaller <hns at goldelico.com> [200218 05:42]:
>>>> Hi Tony,
>>>> 
>>>>> Am 17.02.2020 um 23:19 schrieb Tony Lindgren <tony at atomide.com>:
>>>>> 
>>>>> * H. Nikolaus Schaller <hns at goldelico.com> [200217 21:48]:
>>>>>> letux-pvrsrvkm-v5.6.0-rc2 will be different of course but also
>>>>>> stay stable.
>>>>> 
>>>>> Argh no.. Please just stop doing this pointless weekly respinning
>>>>> of branches stuff. There's absolutely no point setting up a new
>>>>> git branch for every Linux -rc kernel every week. That's just
>>>>> silly.
>>>> 
>>>> No. I have explained the reasons.
>>> 
>>> Sorry this rebuilding of git trees does not work for me for Linux
>>> kernel driver development.
>> 
>> Can you please explain, since it seems I still have not yet understood
>> your problem and how a "standard way" would be better.
> 
> I just want to be able to do this type git standard stuff:
> 
> $ cd linux_openpvrsgx
> $ git checkout my-dev-branch-i-last-worked-on-two-weeks-ago
> $ git fetch linux_openpvrsgx
> $ git diff linux_openpvrsgx/letux-pvrsrvkm-v5.6
> ...

iMac:master hns$ git checkout -B my-dev-branch-i-last-worked-on-two-weeks-ago letux-pvrsrvkm-v5.6.0-rc1
iMac:master hns$ git diff --stat letux-pvrsrvkm-v5.6.0-rc2 drivers/gpu/drm/pvrsgx/
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/Makefile                                    |  29 ++++++++++++++---------------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/eurasiacon/build/linux2/buildvars.mk        |   8 ++++++--
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/eurasiacon/build/linux2/config/core.mk      |  16 ++++++++--------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/eurasiacon/build/linux2/omap_linux/Makefile | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/include4/pvrversion.h                       |   4 ++--
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/dmabuf.c          |   3 +--
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/dmabuf.h          |   4 ++--
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/mmap.c            |  17 +++--------------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/module.c          |  23 +++++++++++++++++------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/mutils.h          |   8 --------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/osfunc.c          |  28 +++++++++++++++++++++-------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/private_data.h    |   3 ---
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/proc.c            |  11 -----------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/pvr_bridge_k.c    |   4 ----
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/pvr_debug.c       |   2 +-
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/pvr_drm.c         |  38 ++++++++++++++++++++++++++++++++++----
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/pvr_drm.h         |   3 ---
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/pvr_linux_fence.c | 100 ++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/srvkm/env/linux/pvr_uaccess.h     |   4 ++--
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/system/omap/syslocal.h            |   1 +
 drivers/gpu/drm/pvrsgx/1.17.4948957/eurasia_km/services4/system/omap/sysutils_linux.c      |   2 --
 drivers/gpu/drm/pvrsgx/Makefile                                                            |   8 ++------
 22 files changed, 206 insertions(+), 233 deletions(-)
iMac:master hns$ 

Ok, someone has upgraded the 1.17.4948957 tree and modified the
master Makefile.

> looks fine, nice new fixes, i want to update to that

Now I see differences to your expectation: a difficult thing is
that you have to specify the subtree or you get all other upstream
changes as well.

And it is not that easy to run a git log to list the diff patches.

git log --oneline linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc1..linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc2 drivers/gpu/drm/pvrsgx/

is quite ugly...

And shows that rc1 and rc2 do not have a common ancestor where expected.

This is obviously not what you expect.

> $ git merge linux_openpvrsgx/letux-pvrsrvkm-v5.6

we just have to replace the merge by a rebase.

> ...
> hmm where was i? let's see
> $ git diff linux_openpvrsgx/letux-pvrsrvkm-v5.6
> ...

iMac:master hns$ git checkout -B my-dev-branch-i-last-worked-on-two-weeks-ago linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc1
Branch 'my-dev-branch-i-last-worked-on-two-weeks-ago' folgt nun Remote-Branch 'letux-pvrsrvkm-v5.6.0-rc1' von 'linux_openpvrsgx'.
Setze Branch 'my-dev-branch-i-last-worked-on-two-weeks-ago' neu
Ihr Branch ist auf demselben Stand wie 'linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc1'.
iMac:master hns$ echo '#' >>drivers/gpu/drm/pvrsgx/Makefile 
iMac:master hns$ git commit -m "do some new work" -a
[my-dev-branch-i-last-worked-on-two-weeks-ago 1d4757cc034d] test
 1 file changed, 1 insertion(+)
iMac:master hns$ git diff linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc1
diff --git a/drivers/gpu/drm/pvrsgx/Makefile b/drivers/gpu/drm/pvrsgx/Makefile
index 83ab04f912d3..cadea1e9986b 100644
--- a/drivers/gpu/drm/pvrsgx/Makefile
+++ b/drivers/gpu/drm/pvrsgx/Makefile
@@ -159,3 +159,4 @@ ifneq ($(CONFIG_SGX),)
 always += pvr
 clean-dirs := $(obj)/$(SUBPATH)
 endif
+#
iMac:master hns$ grep pvr_drv drivers/gpu/drm/pvrsgx/Makefile
iMac:master hns$ git rebase -i linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc2
Successfully rebased and updated refs/heads/my-dev-branch-i-last-worked-on-two-weeks-ago.
iMac:master hns$ git diff linux_openpvrsgx/letux-pvrsrvkm-v5.6.0-rc2
diff --git a/drivers/gpu/drm/pvrsgx/Makefile b/drivers/gpu/drm/pvrsgx/Makefile
index d1354d2b985a..0a5b348874d5 100644
--- a/drivers/gpu/drm/pvrsgx/Makefile
+++ b/drivers/gpu/drm/pvrsgx/Makefile
@@ -163,3 +163,4 @@ ifneq ($(CONFIG_SGX),)
 always += pvr
 clean-dirs := $(obj)/$(SUBPATH)
 endif
+#
iMac:master hns$ grep pvr_drv drivers/gpu/drm/pvrsgx/Makefile
pvr_drv=no
pvr_drv=yes
pvr_drv=yes
ifeq ($(pvr_drv),yes)
else # pvr_drv
iMac:master hns$ 

Seems to work as I would expect. This solves at least the problem how you
can get the latest "upstream" (not meaning linus/master but our project)
version to your lokal workspace.

So your local changes are still there (and even stay at the tip of the
branch making it easier to locate or edit them).

And (my) upstream changes (the latest pvr_drv mechanism) have arrived in
your working branch.

Ok, got it. You have to remember the -rc1 -rc2 things...

So in summary there are these issues:
* one has to remember / look up the current -rc
* git diff or git log is difficult to apply as used to
* one has to restrict git diff/log to the subproject directory

Let me think about how to add this to the requirements
and how a solution could look like.

Well, the best solution would be if all this were in kernel.org...
Then Linus would provide us all with this linear history as a common
basis and we could contribute independently.

> And no, I do not want to rebase.

Yes, it is different to what text books say, but I learned that it
is more convenient - at least for me. Sometimes, text books are misleading
and innovation is done by not believing in and breaking with tradition :)
But only for good reasons...

> Ideally the master branch would be
> used with immutable commit id's, but in reality some kernel specific
> topic branches are needed to work on let's v5.4-lts and current kernel.

> 
> And now you're going to got rant about me being able to use whatever
> branch your cronjob created.. No I don't want to do that because
> finding applied fixes to two new branches you already generated
> is such a pain with the same patches appearing with different commit
> IDs.

Hm. I think git rebase isn't based on commit hashes to find duplicates
but on content.

> 
>>> It's too much of a pain for me already and I only have two sets of
>>> patches so far (for ddk 1.9 and ddk 1.17).
>> 
>> So what is the pain?
> 
> This constant rebase, trying to figure out what other patches got
> applied to n-number of more recent branches.

Ok, you expect that commits are just piled up in a feature branch.

Well, I have to handle the expectation of upstream maintainers that patches
are cleaned up (i.e. voluntarily wiped out history) and therefore they
are /not/ piled up. And they must be based on a recent upstream version
or they complain that it does not apply.

BTW: I internally even have versioned feature branches (topic-v2 -v3 etc.)
to easily handle the [PATCH v3] mechanism without loosing track...

This also tries to keep feature branches separated by subsystem.

Another issue is that merging in upstream releases into such a tree
usually works, but the changes become buried and unidentifiable between
other patches. So it violates the "my changes are at the tip of the branch"
concept.

All this did lead directly to the rebase-weelky development model
because the "pile up during development and clean up if finished"
doesn't work.

> 
>> In my understanding it suffices to git checkout letux-pvrsrvkm-v5.6.0-rc1
>> and add new commits to it. And let me know if you have new ones. Then I
>> can cherry-pick them into the feature branches (like I have done this
>> time).
>> 
>> If you want to take patches from letux-pvrsrvkm-v5.6.0-rc2 just cherry-pick
>> them or decide to rebase.
> 
> No I don't want to work like that. It's too much of a pain to keep
> up with these weekly branches.

Ok. In understand your reasons now.

> 
>>>>> Let's make this git tree behave like a normal Linux development
>>>>> git tree so we can use git in a standard way.
>>>> 
>>>> You can use it in the way you have explained that you want to use it.
>>>> 
>>>> Just take letux-pvrsrvkm-v5.6.0-rc1 and letux-pvrsrvkm-v5.7.0-rc1
>>>> as your basis and ignore the others.
>>>> 
>>>> I don't see where the problem is with this on your side.
>>> 
>>> Sorry to hear that. I'll just set up a new normal style git tree
>>> for the driver development then.
>> 
>> How would that look like?
> 
> Well I'm thinking to set up a tree for working on the new pvr-drv.c
> parts with a goal of getting it into usable state for merging into
> Linux kernel. Initially with 2d acceleration based on the gma500
> code in the kernel.
> 
>> If that is easy enough, I once again do not understand your problem.
>> And you do not seem to accept that I have a problem with other
>> approaches than we have.
> 
> I really only have two issues with your current approach:
> 
> 1. You are building a new topic branch every -rc once a week
>   while it should be only done once per kernel version for
>   topic branches

How does that work with upstream patches and fixes coming in?

Let's assume we only have rc1. Then local patches are added.

Then let's assume we (or someone else) finds a bug in some
subsystem and submit a hot-fix which is applied within days
to the next -rc. This happens especially between -rc1 and -rc2.
 
So we have to merge rc2 rc3 rc4 etc. So you end up with your
patches hidden by merges from upstream.

So you can't take your branch any more for git-format-patch for
submission. You have to clean up. And in that moment you rewrite
history.

Then you POST and get feedback and massage the patch set even more.
This may need several weeks depending on work-load of everyone.

In the meantime some co-developer may have added some more patches
to the original queue. But they are no longer compatible to the rewritten
version.

Then you spend a lot of time fixing up merge/rebase conflicts.

I had once tried to rebase the AUFS drivers for v4.19 to v4.20.
Merging with v4.20 did work, but rebasing to v4.20 was impossible
(too many conflicts).

So it was not possible to move all the patches on top of v4.20.
They were too deeply buried and mixed with other patches.

My conclusion from this was to rebase as often as possible.

Well, the pvrsgx we are talking here is quite independent of
other subsystems and upstream changes.

And AFAIR there is no need for us doing many changes outside of
drivers/gpu/drm/pvrsgx. Maybe for jz4780 SoC support but OMAP is
(almost) done. I think two of Tero's patches have not yet arrived
as you can see here:

https://github.com/openpvrsgx-devgroup/linux_openpvrsgx/commits/letux/omap-sysc-prm-gfx

So it is soon time to close this feature branch and remove
from rebasing/merging.

We just have to get the DTS changes (letux/omap-pvr-soc-glue-v6)
upstream...

I don't remember why the review process got stuck. AFAIR I did send
some questions for the YAML part (which is still a Chinese dialect
for me) but didn't get answers.

> 
> 2. You are using -rc specific branch names while mainline kernel
>   specific name would be sufficient

Of course. If 1. is not done.

> 
> But please see below about the separate integration tree, maybe
> that's the solution here :)
> 
>> It is for reasons and based on bad experiences with other concepts.
>> We are running a factory of kernels that are ahead of Linus/master
>> but must work with always changing feature branches, because the review
>> process requires us to permanently rewrite the history to derive
>> and submit clean patches. And this sometimes takes months or even
>> years until it is settled. So we can't wait to integrate.
> 
> OK so that's usable for your internal testing. How about set up
> another git tree for integration testing? And just make itclear
> that it's generated from the linux_openpvrsgx development tree
> for integration testing?

Yes.

Hm. This could mean that it is a branch that continuously adds
new commits. Sometimes reverting older ones and replacing by
newer ones.

I had tried to automate this to result in the following branch:

https://github.com/openpvrsgx-devgroup/linux_openpvrsgx/commits/letux-pvrsrvkm

but it seems that the algorithm isn't doing what is expected.

Hm. Here it could be possible to start a new one at -rc1 and
then just collect new commits restricted to drivers/gpu/drm/pvrsgx/

That way there would be a branch that starts at a well defined
basis but only adds pvrsgx related commits. So this could be
the branch where you start and diff your
my-dev-branch-i-last-worked-on-two-weeks-ago
to.

Would that be what you are looking for?

My gut feeling says that it looks as if I might be able to do
and automate that with some rebase --onto while keeping the
existing scheme in parallel...

Basically the idea would be to transplant v5.6-rc2..letux-pvrsrvkm-v5.6.0-rc2
onto this branch. A problem may be patches that are removed
or where the message is edited. Those should be reverted first
and committed in the new version. Which is far beyond rebase capabilities.
But scripting can do everything...

> 
> That would leave out all the churn from linux_openpvrsgx development
> tree and we can keep linux_openpvrsgx development tree usable for
> other developers.

How should these changes come back into the letux tree for integration
testing?

> 
>> Another thing to consider is the GPL requirement to provide a
>> source code for each binary which means to have an archive.
> 
> Yes.. But having tens or of different versions of git history sure
> does not help with keeping track of that :)

Well, in Linux there is only the linus/master history. The history
of how patch series have been developed and discussed is already gone
there.

I just want to keep that for reference. Imagine a tree where the
branches all start at the same position of the tree trunk which
happens to be some Linus release... And we just do not cut it down
for archiving purposes.

> 
>> So I really want to understand and learn but I do not see how
>> it keeps my problems solved (which we have for a while).
> 
> How about something like this:
> 
> 1. Normal style git tree for linux_openpvrsgx with topic branch
>   for each major kernel version as needed

I wonder who is integrating my changes into that and how is it done?
Can it be automated?

> 
> 2. Separate weekly cronjob integration testing tree for your
>   purposes like your're doing now
> 
> 3. I'll set up a minimal tree for pvr-drv.c only that can be
>   be merged as needed into #1 above

That one exists as: 

https://github.com/openpvrsgx-devgroup/linux_openpvrsgx/commits/letux/pvrsrvkm

The only difference to your plan is that it is rebased (often). And that
this branch alone it is not a complete pvrsrvkm solution and doesn't
compile/work stand-alone.

Well, it is difficult because there are different requirements.
Everything would be much easier if we had all code upstream...

Hm. This mail again got longer and longer...

So let me think about all this again. At least I now have a hopefully
good understanding in your need and an idea for a solution.

Rome wasn't built in a day.

BR,
Nikolaus



More information about the openpvrsgx-devgroup mailing list