[Openpvrsgx-devgroup] [PATCH] Update pvrsgx 1.14.3759903 to latest kernel

[H. Nikolaus Schaller]

Hi Lucas,

> Am 13.11.2021 um 05:26 schrieb Lucas Fryzek <lucas.fryzek at hazeco.xyz>:
> 
> For my next steps I'll take a look at resolving some of the compiler warning as those typically indicate bigger problems.

Indeed.

> 
>> Booting unfortunately fails with a segfault (see below).
>> OSInvalidateCPUCacheRangeKM/dma_sync_phys indicates that some fix does not handle
>> the processor cache properly. Maybe the changes for 1.17.4948957 were too OMAP specific
>> or we are missing some detail. Or it is even related to the va_list things.
>> Or some 1.17.4948957 fixes are not 100% compatible to the 1.14.3759903.
>> Maybe a comparison with the fixes for the 1.14.3699939 will show. That one
>> also works on omap3/am335x (e.g. Beaglebone).
>> Unfortunatley I have not the spare time to look deeper into it, although it is tempting :)
>> Anyways this is a really big step forwards. For the first time we can compile a
>> jz4780 kernel driver for a modern kernel with working HDMI *and* have a (potentially)
>> matching user-space available.
>> BR and thanks,
>> Nikolaus
> Which kernel version are you working from? Do you have any patches applied beyond what I provided?

Not much or anything obviously changing the game...

> Did you modify the kernel config from `letux_defconfig` beyond just switching to the `1.14.37539903` version of the pvr module? I was able to build and boot the kernel with this patch and without crashes. I can take a look at why that `OSInvalidateCPUCacheRangeKM` triggered a crash, but it helps if I'm able to reproduce the problem on my end :P

First I thought that I am not able myself to reproduce...
But I had defined /etc/modprobe.d/pvrsrv.conf to blacklist the kernel driver :)

I have to boot and manually load the driver + run pvrsrvctl.

The crash happens when running modprobe *or* pvrsrvctl. Sometimes modprobe
succeeds (a log of such a case is attached) and then pvrsrvctl fails.

Interestingly it says:

[   96.138541] PVR_K: UM DDK-(3759903) and KM DDK-(3759903) match. [ OK ]

which is already a good sign. We have passed the first entrance check :)

Only /proc/pvr/version saying "Version ... omap_sgx" is questionable why it
says "omap_sgx". But that may just be a missing #define somewhere without
deeper reason or effect.

I have pushed my working tree as:

	https://git.goldelico.com/?p=letux-kernel.git;a=shortlog;h=refs/heads/work-demo-pvrsgx-jz4780-1.14.3759903

Rootfs is:

	 makesd -v latest ci20 -r http://download.goldelico.com/letux-debian-rootfs/2021/20210826-stretch-9.13-mipsel-minimal.tbz

plus the kernel/dtb/modules from the kernel build.

And I finally remembered to have

	apt-get install letux-pvrsgx-1.14

to automatically ftp the user space code and install by its postinst script [1].

BR and thanks,
Nikolaus

[1]: https://git.goldelico.com/?p=letux-rootfs.git;a=tree;f=packages/letux-pvrsgx-1.14;h=00383037bf8d4a6ce7907d6b502b5800af43af85;hb=refs/heads/master

Linux letux 5.15.0-letux-ci20+ #7331 PREEMPT Sat Nov 13 08:42:59 CET 2021 mips GNU/Linux
root at letux:~# ls -l /etc/modprobe.d/pvrsrvkm.conf
-rw-r--r-- 1 502 root 37 Nov 13 07:36 /etc/modprobe.d/pvrsrvkm.conf
root at letux:~# cat /etc/modprobe.d/pvrsrvkm.conf
blacklist pvrsrvkm_jz4780_sgx540_130
root at letux:~# modprobe pvrsrvkm_jz4780_sgx540_130
[   66.535748] pvrsrvkm 13040000.gpu: Enabling quirks 00000000
[   66.542678] CheckExecuteCacheOp
[   66.546218] CheckExecuteCacheOp 1
[   66.552450] CheckExecuteCacheOp 2
[   66.556199] CheckExecuteCacheOp 6
[   66.561023] CheckExecuteCacheOp 12
+++ here modprobe did fail at least once +++
[   66.564861] CheckExecuteCacheOp 13
[   66.569695] CheckExecuteCacheOp 15
[   66.573602] CheckExecuteCacheOp
[   66.577125] CheckExecuteCacheOp 1
[   66.582218] CheckExecuteCacheOp 2
[   66.585951] CheckExecuteCacheOp 6
[   66.590697] CheckExecuteCacheOp 12
[   66.594542] CheckExecuteCacheOp 13
[   66.599440] CheckExecuteCacheOp 15
[   66.609493] [drm] Initialized pvr 1.14.3759903 20110701 for 13040000.gpu on minor 1
root at letux:~# ls -l /proc/pvr/
total 0
-r--r--r-- 1 root root 0 Nov 13 09:01 nodes
-r--r--r-- 1 root root 0 Nov 13 09:01 queue
-r--r--r-- 1 root root 0 Nov 13 09:01 version
root at letux:~# cat /proc/pvr/version
Version SGX_DDK_Linux_XOrg sgxddk MAIN at 3759903 (release) omap_sgx
System Version String: None
root at letux:~# 
root at letux:~# pvrsrvctl --start --no-module
[   96.138541] PVR_K: UM DDK-(3759903) and KM DDK-(3759903) match. [ OK ]
[   96.150000] CheckExecuteCacheOp
[   96.153564] CheckExecuteCacheOp 1
[   96.157279] CheckExecuteCacheOp 2
[   96.162538] CheckExecuteCacheOp 6
[   96.166272] CheckExecuteCacheOp 12
[   96.171117] CheckExecuteCacheOp 13
[   96.174945] CheckExecuteCacheOp 15
[   96.179961] CheckExecuteCacheOp
[   96.183503] CheckExecuteCacheOp 1
[   96.187218] CheckExecuteCacheOp 2
[   96.192255] CheckExecuteCacheOp 6
[   96.195987] CheckExecuteCacheOp 12
[   96.200821] CheckExecuteCacheOp 13
[   96.204647] CheckExecuteCacheOp 15
[   96.209674] CheckExecuteCacheOp
[   96.213221] CheckExecuteCacheOp 1
[   96.216935] CheckExecuteCacheOp 2
[   96.229294] CheckExecuteCacheOp 6
[   96.240614] CheckExecuteCacheOp 12
[   96.251782] CheckExecuteCacheOp 13
[   96.262327] CheckExecuteCacheOp 15
[   96.272382] CheckExecuteCacheOp
[   96.282267] CheckExecuteCacheOp 1
[   96.292179] CheckExecuteCacheOp 2
[   96.301969] CheckExecuteCacheOp 6
[   96.311735] CheckExecuteCacheOp 12
[   96.321804] CheckExecuteCacheOp 13
[   96.331749] CheckExecuteCacheOp 15
[   96.341885] CheckExecuteCacheOp
[   96.351580] CheckExecuteCacheOp 1
[   96.361346] CheckExecuteCacheOp 2
[   96.371108] CheckExecuteCacheOp 6
[   96.380832] CheckExecuteCacheOp 12
[   96.390715] CPU 0 Unable to handle kernel paging request at virtual address 40300000, epc == 8011a838, ra == 8011ac58
[   96.408976] Oops[#1]:
[   96.418073] CPU: 0 PID: 1775 Comm: pvrsrvctl Not tainted 5.15.0-letux-ci20+ #7332
[   96.432911] $ 0   : 00000000 00000001 ffffffe0 00000002
[   96.445379] $ 4   : 40300000 40303fe0 84b06c00 00000001
[   96.458030] $ 8   : fffce04e 84152410 00007000 0000003a
[   96.470919] $12   : 770fec58 77dc2020 00000000 00000001
[   96.483990] $16   : 40300000 00004000 84b06c00 00000002
[   96.497265] $20   : 40300000 80c70000 80c80000 80c70000
[   96.510689] $24   : 80b7723c 8017d250                  
[   96.524433] $28   : 8522e000 8522f968 80c70000 8011ac58
[   96.538497] Hi    : 00180600
[   96.550240] Lo    : 000c0300
[   96.562045] epc   : 8011a838 blast_inv_dcache_range+0x10/0x24
[   96.577502] ra    : 8011ac58 ingenic_xburst_dma_cache_inv+0x98/0xa0
[   96.593968] Status: 10000403 KERNEL EXL IE 
[   96.608628] Cause : 00800008 (ExcCode 02)
[   96.623332] BadVA : 40300000
[   96.636956] PrId  : 3ee1024f (Ingenic XBurst)
[   96.652514] Modules linked in: pvrsrvkm_jz4780_sgx540_130 usb_f_ecm g_ether usb_f_rndis u_ether libcomposite configfs cfg80211 snd_soc_hdmi_codec dw_hdmi_i2s_audio ci20_audio gpio_ir_recv rc_core gpio_keys ingenic_drm ingenic_dw_hdmi ohci_platform dw_hdmi ehci_platform dwc2 roles ohci_hcd display_connector drm_kms_helper ehci_hcd cfbfillrect syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea dm9000 mii nvmem_jz4780_efuse drm rtc_pcf8563 drm_panel_orientation_quirks snd_soc_jz4780_codec snd_soc_jz4740_i2s phy_ingenic_usb ipv6 autofs4
[   96.778108] Process pvrsrvctl (pid: 1775, threadinfo=12f2878b, task=8d8dcfb8, tls=77ea0280)
[   96.804976] Stack : c0304000 80255628 00000000 00004000 00004000 00004000 00004000 8011a57c
[   96.832714]         7bf6040c ffffffbf 86bf8700 00000007 80c70000 00000000 c04b0000 00007000
[   96.860920]         c02fd000 808b4584 8452b188 8452b188 c04d0000 8452b1b4 00000000 c04b0270
[   96.889662]         0000000c c04b7300 80ba8170 00000001 00000dc0 c04aca80 8452b188 02024200
[   96.918952]         00007000 86bf878c 8452b188 00004000 00007000 86bf878c 02024200 00007000
[   96.948654]         ...
[   96.972343] Call Trace:
[   96.996094] [<8011a838>] blast_inv_dcache_range+0x10/0x24
[   97.023658] [<8011ac58>] ingenic_xburst_dma_cache_inv+0x98/0xa0
[   97.052234] [<8011a57c>] dma_sync_phys+0x10c/0x1e4
[   97.079961] [<c04b0270>] CheckExecuteCacheOp+0x25c/0x340 [pvrsrvkm_jz4780_sgx540_130]
[   97.112424] [<c04b2028>] OSInvalidateCPUCacheRangeKM+0x24/0x30 [pvrsrvkm_jz4780_sgx540_130]
[   97.145166] [<c04acaec>] NewVMallocLinuxMemArea+0x94/0x100 [pvrsrvkm_jz4780_sgx540_130]
[   97.177200] [<c04af9a4>] OSAllocPages_Impl+0x9c/0x12c [pvrsrvkm_jz4780_sgx540_130]
[   97.208775] [<c0494ac8>] BM_ImportMemory+0x3c8/0x6b0 [pvrsrvkm_jz4780_sgx540_130]
[   97.240231] [<c04a2340>] RA_Alloc+0x18c/0x418 [pvrsrvkm_jz4780_sgx540_130]
[   97.271444] [<c0495218>] BM_Alloc+0x468/0x628 [pvrsrvkm_jz4780_sgx540_130]
[   97.302497] [<c0495900>] AllocDeviceMem.isra.3+0x140/0x238 [pvrsrvkm_jz4780_sgx540_130]
[   97.335288] [<c04963b8>] _PVRSRVAllocDeviceMemKM+0x130/0x2cc [pvrsrvkm_jz4780_sgx540_130]
[   97.368743] [<c048c4ac>] PVRSRVAllocDeviceMemBW+0x2bc/0x444 [pvrsrvkm_jz4780_sgx540_130]
[   97.402704] [<c048f9e8>] BridgedDispatchKM+0x298/0x2b0 [pvrsrvkm_jz4780_sgx540_130]
[   97.436521] [<c04b32c0>] PVRSRV_BridgeDispatchKM+0x258/0x438 [pvrsrvkm_jz4780_sgx540_130]
[   97.471325] [<c03e3398>] drm_ioctl_kernel+0xc4/0x148 [drm]
[   97.503745] [<c03e3bb0>] drm_ioctl+0x360/0x3c0 [drm]
[   97.535675] [<80294478>] vfs_ioctl+0x28/0x40
[   97.565866] 
[   97.592782] Code: 24a5ffff  00822024  00a22824 <bc910000> 1485fffe  24840020  03e00008  00000000  03e00008 
[   97.629280] 
[   97.657327] ---[ end trace 733e002d434d8512 ]---
[   97.688866] Kernel panic - not syncing: Fatal exception
[   97.721195] Rebooting in 10 seconds..