[Letux-kernel] MIPS: bug: gettimeofday syscall broken on CI20 board?

H. Nikolaus Schaller hns at goldelico.com
Thu Nov 14 11:04:41 CET 2019


Hi all,
after trying tho understand how the VDSO code is working by inspecting
ARM and MIPS versions in elixir, I observed that there have been bigger
changes recently.

v5.3-rc1	introduced lib/vdso/gettimeofday.c
v5.3		has arch/mips/vdso/gettimeofday.c
v5.4-rc2 	has it deleted
v5.4-rc1	introduced arch/mips/vdso/vgettimeofday.c

So there was a significant refactoring of the code and probably a
harmonization with other platforms in lib/vdso.

The relevant changes seem to be:

iMac:master hns$ git log --oneline v5.3-rc1..v5.4-rc5 -- arch/mips/vdso/gettimeofday.c arch/mips/vdso/vgettimeofday.c lib/vdso/gettimeofday.c
1638b8f096ca lib/vdso: Make clock_getres() POSIX compliant again
90800281e761 MIPS: VDSO: Remove unused gettimeofday.c
5c6bd5de3c2e Merge tag 'mips_5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux
c60a32ea4f45 lib/vdso/32: Provide legacy syscall fallbacks
502a590a170b lib/vdso: Move fallback invocation to the callers
a9446a906f52 lib/vdso/32: Remove inconsistent NULL pointer checks
1f66c45db330 mips: Add clock_gettime64 entry point
abed3d826f2f mips: Add clock_getres entry point
24640f233b46 mips: Add support for generic vDSO
iMac:master hns$ 

Then I have done some tests with older kernel builds from our archive:

a) v5.3 still works
b) v5.4-rc1 up to rc5 have problems with console login and can't be tested
c) v5.4-rc6 is broken

Net question was if there is some influence of my cross-compiler setup
but I was able to rebuild v5.3 and it still works. SO it is unlikely
(although the new generic vDSO could use a compiler feature that was
never used before).

This means:
1. my compiler-setup is no influencing factor
2. bad thing happened between v5.3 and v5.4-rc5
3. there has been major rework
4. maybe some corner case was overlooked

I tried to make poor-mans bisect by reverting one of these patches
after the other, but it failed to revert 24640f233b46 because there
is a merge involved.

I have to check if I can really set up a bisect.

BR,
Nikolaus


> Am 13.11.2019 um 17:46 schrieb Andreas Kemnade <andreas at kemnade.info>:
> 
> On Wed, 13 Nov 2019 14:48:12 +0100
> "H. Nikolaus Schaller" <hns at goldelico.com> wrote:
> 
>>> Am 12.11.2019 um 12:37 schrieb Paul Boddie <paul at boddie.org.uk>:
>>> 
>>> On Monday 11. November 2019 16.52.25 H. Nikolaus Schaller wrote:  
>>>> Hi Paul,
>>>> looks as if there is nobody active on linux-mips or the mipscreator list...  
>>> 
>>> I did chat with Ralf Baechle a while back on IRC, but I think he is a very 
>>> busy person.
>>> 
>>>> Do you see a similar effect on your CI20?
>>>> 
>>>> Or do you have an idea how to track it down?  
>>> 
>>> I haven't tried a recent kernel on the CI20 in the last few weeks because I've 
>>> been using it as my main working machine. There may be some complicating 
>>> issues with the CI20 that might affect things like DHCP, though: there is some 
>>> odd behaviour with MAC identifiers where it assigns a new one every time it 
>>> boots. This is documented on the eLinux wiki:
>>> 
>>> https://www.elinux.org/CI20_upstream#Ethernet_MAC_Address
>>> 
>>> [...]
>>> 
>>>>> I have done some strace and the first significant difference
>>>>> is that with v5.4-rc6 there is no gettimeofday syscall.
>>>>> 
>>>>> Another symptom pointing in the same direction is that
>>>>> after manually assigning an IP address I can run ping
>>>>> but get strange time values.
>>>>> 
>>>>> So it may be that
>>>>> 
>>>>> 24640f233b46 mips: Add support for generic vDSO
>>>>> 
>>>>> did break gettimeofday when used with latest Debian Stretch
>>>>> libraries. I tried to git revert but there are conflicts.
>>>>> 
>>>>> Just a side-note: both kernels work with Debian Jessie,
>>>>> which likely has an older gettimeofday wrapper that
>>>>> is not influenced by some subtle change.  
>>> 
>>> I can imagine that some system call and C library incompatibility might have 
>>> arisen. For various other reasons, I have been looking at things like these, 
>>> and I did stray into vDSO territory having seen the term mentioned a few 
>>> times. Unless I am mistaken, it is a way of accessing kernel functionality 
>>> from user space and can be used to implement some system calls as normal 
>>> functions accessing kernel resources:
>>> 
>>> https://en.wikipedia.org/wiki/VDSO  
>> 
>> Also helpful:
>> 
>> http://man7.org/linux/man-pages/man7/vdso.7.html
>> 
>>> 
>>> https://web.archive.org/web/20190927141756/https://www.linuxjournal.com/content/creating-vdso-colonels-other-chicken
>>> 
>>> Interestingly, the second article above talks about gettimeofday as being a 
>>> candidate for this technology. So it is possible that the later kernel no 
>>> longer has gettimeofday as a system call, but the C library still expects one.  
>> 
>> Yes, that is likely the reason.
>> 
>> The strace doesn't show gettimeofday() system calls:
>> 
>> ioctl(6, SIOCGIFHWADDR, {ifr_name="eth0", ifr_hwaddr=1e:e2:4e:82:c1:d5}) = 0
>> close(6)                                = 0
>> setsockopt(5, SOL_PACKET, PACKET_AUXDATA, [1], 4) = 0
>> setsockopt(5, SOL_SOCKET, SO_ATTACH_FILTER, "\v\0\0\0\204\307kU", 8) = 0
>> fcntl64(5, F_SETFD, FD_CLOEXEC)         = 0
>> socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP) = 6
>> setsockopt(6, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
>> bind(6, {sa_family=AF_INET, sin_port=htons(68), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
>> fcntl64(6, F_SETFD, FD_CLOEXEC)         = 0
>> getpid()                                = 1531
>> uname({sysname="Linux", nodename="letux", ...}) = 0
>> time(NULL)                              = 1478193445
>> getpid()                                = 1531
>> send(3, "<30>Nov  3 17:17:25 dhclient[153"..., 101, MSG_NOSIGNAL) = 101
>> write(5, "\377\377\377\377\377\377\36\342N\202\301\325\10\0E\20\1H\0\0\0\0\200\0219\226\0\0\0\0\377\377"..., 342) = 342
>> write(2, "../../../../lib/isc/unix/time.c:"..., 37../../../../lib/isc/unix/time.c:200: ) = 37
>> write(2, "Operation not permitted", 23Operation not permitted) = 23
>> 
>> But the source code (I hope it is the right one) shows the
>> call at line 199:
>> 
>> http://users.isc.org/~each/doxygen/bind9/isc_2unix_2time_8c-source.html
>> 
>> 00197 
>> 00198         if (gettimeofday(&tv, NULL) == -1) {
>> 00199                 isc__strerror(errno, strbuf, sizeof(strbuf));
>> 00200                 UNEXPECTED_ERROR(__FILE__, __LINE__, "%s", strbuf);
>> 00201                 return (ISC_R_UNEXPECTED);
>> 00202         }
>> 00203 
>> 
>> So either the dhclient for MIPS on Debian 9.9 isn't using the latest system
>> libraries or the VDSO implementation is broken (endianness, address offsets?).
>> 
>> The same Debian 9.9 and kernel built from same kernel tree works on ARM.
>> 
>> So an alternative could be my kernel build system for MIPS doesn't properly cross-compile
>> the vDSO and breaks gettimeofday() operation.
>> 
> hmm, what about comparing on a system (qemu-mips(el) e.g.) where a
> standard debian kernel works, then bisecting from there towards letux
> kernel.
> 
> Regards,
> Andreas



More information about the Letux-kernel mailing list