[Letux-kernel] Fixing Audio driver

H. Nikolaus Schaller hns at goldelico.com
Thu May 23 19:26:58 CEST 2019


> Am 23.05.2019 um 14:29 schrieb Michael Mrozek <EvilDragon at openpandora.org>:
> 
> Am Donnerstag, den 23.05.2019, 07:31 +0200 schrieb H. Nikolaus
> Schaller:
> 
> Hi,
> 
>>> Ignoring AESS for the moment (we can add that later), the setup of
>>> the audio devices should be in the twl6040 driver file?
>> Only the twl6040 audio devices are there.
>> To throw our AESS patches out, you could git checkout the mainline
>> twl6040 driver.
>> This should also make the rogue null ports go away.
> 
> If that helps in getting bare audio to work until someone finds the
> time to get AESS to work, then that's okay.
> 
> aTc made a quick test yesterday playing audio with software mixing
> through the headset - and it took 1,5% CPU power.

That is not really much and shows how powerful the OMAP5 is.

> So software mixing is not as bad as I feared and it would be good
> enough for the beginning.

I know, because that was working for quite some time. I just can't
tell you why it is currently broken for you. Or with 5.1/5.2 kernels.

> 
> We don't need any advanced Bluetooth, LTE-Modem or other features at
> the beginning, those can always be delivered later.

They do not harm if we find out why handsfree audio is broken for you.

Because they are independent drivers sitting inactively around.

> But we need working speakers and headsets (and the headset already
> works).
> 
>>> I expected them to be in the board files.
>>> And the first step would be to create proper devices instead of
>>> having the speakers on channel 3 and 4 of the headset port.
>> As far as I understand you need AESS to make it different. So we
>> either have the channel 3+4 thing or have to fix AESS. No way through
>> the middle...
> 
> That wouldn't make any sense in my opinion.
> According to the TWL6040 feature list, the chip has multiple stereo
> outputs - not one output with four channels.

That may be a matter of wording and perspective. 2 stereo channels
may be seen as four outputs with 1 channel. Or one output with 4
channels. It is always 4 wires to 4 speakers.

> 
> They even have different power drivers.

Yes, why not?

> So why should the speakers be channel 3 and 4? 

Because data is sent from the OMAP5 to the twl6040 in PCM packets
over a single PCM interface.

And as far as I know the PCM is hard formatted with 4 channels. The
twl6040 takes this PCM stream, splits it up and sends it to the 2
audio DACs for Headset and 2 for Handsfree speakers. This mapping
can not be modified.

And the twl4030 drivers simply maps that to Linux user space as a
single sound card with 4 channels. No other magic behind.

If that is to be changed, either the driver must be reprogrammed
to be able to reshuffle channel numbers or handle multiple sound
cards. Or AESS must be made working.

> 
> That would be weird.

No. It is how the twl6040 interface works. At least if I remember
correctly what I read some years ago. We had such discussions before:

https://www.pyra-handheld.com/pipermail/kernel/2014-September/000417.html

There was a good description on omappedia but it looks as if it has gone :(

> 
> Here's the standard TWL6040 driver (for Android, not sure if it's
> different than the Linux one):
> https://android.googlesource.com/kernel/omap/+/glass-omap-xrr02/sound/soc/codecs/twl6040.c

I have not looked into details but it is basically the same, resp. a very
old version. The most recent driver code is here:

	https://elixir.bootlin.com/linux/v5.2-rc1/source/sound/soc/codecs/twl6040.c

> 
> I'm no dev as you know, but from what I can see there, the outputs of
> the TWL are clearly defined here and routed to different output devices
> (handsfree, etc.)

The tables you find in the code define the amixer controls and handsfree is
for gain/loudness control. But there is none to change the channel mapping
because the twl6040 driver can't do that (without ABE extensions and AESS).

> I don't see a reason why the speakers shouldn't get their own output
> device even with a driver that doesn't use AESS.

As explained, to my knowledge there is only one sound card with 4 channels.
Instead of two with 2 channels each as you might expect. This was always the
setup of the Pandaboard and OMAP5 drivers make no basic difference.

It can be probably done to provide two separate sound cards and multiplex
the sound into the single PCM connection - if clock synchronization is possible.
But that means to rewrite a new driver from scratch. This needs even more
knowledgable people than fixing bugs in the existing driver.

> As headset output is working fine and speaker output shouldn't be that
> much different (both are analog outputs), fixing this weird channel
> issue might be enough to have a basic working audio system on the Pyra.

I just took some minutes (well more) and installed letux-4.19.44 on the 2GB Pyra.

And for me, sound works!

Well, not optimal , but it works. When playing an MP3 I get noise after
ca. 10 seconds. But before that, music is crisp and loud and volumed
damon works as expected. This was better with some older kernel, so
it is a pure software issue.

So it is "just" the idle/clock bug to be fixed. And not a fundamental
issue like you seem to believe. Of course such a simple idle/clock bug
may need a lot of time to identify and fix.

Or there is something in PyraOS which interferes with this basic setup
and makes it non-operating.

Here are all steps to reproduce my setup:

1. use "makesd pyra -r lxde" ("makesd pyra-4gb -r lxde" for 4GB RAM model)
   to build a LetuxOS SD card with some badly configured LXDE
2. (re)mount the card and copy some MP3 file to "/root/somefile.mp3"
   of the SD card
3. umount the card
4. insert into Pyra and boot
5. login as root (no password)
6. run "./twl somefile.mp3" through console
7. turn volume wheel

If that works for you, the big issues are not in kernel (unless non-LPAE vs. LPAE
plays a role). It might be in hardware of course (speakers have no contact
to the main PCB). Or PyraOS doing setup differently from the /root/twl script.

And yes, there are kernel messages which should not exist and may impact
audio reliability and quality:

[    5.584689] ------------[ cut here ]------------
[    5.590606] WARNING: CPU: 0 PID: 1360 at drivers/base/devres.c:899 snd_soc_card_new_dai_links+0x5c/0x94 [snd_soc_omap_abe_twl6040]
[    5.590817] asoc-simple-card sound_fmradio: W2CBW003 <-> 40124000.mcbsp mapping ok
[    5.615096] Modules linked in: snd_soc_omap_abe_twl6040(+) snd_soc_simple_card(+) snd_soc_simple_card_utils snd_soc_twl6040 leds_gpio wwan_on_off omapdss snd_soc_gtm601 encoder_tpd12s015 connector_hdmi cec omapdss_base pwm_omap_dmtimer pwm_bl snd_soc_w2cbw003_bt generic_adc_battery ehci_omap wlcore_sdio dwc3_omap bmp280_spi snd_soc_ts3a227e leds_is31fl319x crtouch_mt hci_uart tsc2007 btbcm bq2429x_charger ina2xx bq27xxx_battery_i2c bq27xxx_battery tca8418_keypad as5013 gpio_twl6040 twl6040_vibra palmas_pwrbutton bluetooth ecdh_generic palmas_gpadc usb3503 bmc150_magn_i2c bmg160_i2c bmc150_accel_i2c bmc150_magn bmg160_core bmc150_accel_core bmp280_i2c bno055 industrialio_triggered_buffer bmp280 kfifo_buf industrialio snd_soc_omap_aess snd_soc_omap_mcbsp snd_soc_omap_mcpdm snd_soc_sdma
[    5.756558] usb 1-2: new high-speed USB device number 2 using ehci-omap
[    5.803815] asoc-simple-card sound_telephony: gtm601 <-> 40126000.mcbsp mapping ok
[    5.807251] CPU: 0 PID: 1360 Comm: udevd Not tainted 4.19.44-letux+ #140
[    5.818788] Hardware name: Generic OMAP5 (Flattened Device Tree)
[    5.825124] [<c0110c30>] (unwind_backtrace) from [<c010c0b0>] (show_stack+0x10/0x14)
[    5.833246] [<c010c0b0>] (show_stack) from [<c07c8e34>] (dump_stack+0x7c/0x9c)
[    5.840812] [<c07c8e34>] (dump_stack) from [<c01329ac>] (__warn+0xd8/0x108)
[    5.848111] [<c01329ac>] (__warn) from [<c0132aec>] (warn_slowpath_null+0x38/0x44)
[    5.856060] [<c0132aec>] (warn_slowpath_null) from [<bf3b2468>] (snd_soc_card_new_dai_links+0x5c/0x94 [snd_soc_omap_abe_twl6040])
[    5.868312] [<bf3b2468>] (snd_soc_card_new_dai_links [snd_soc_omap_abe_twl6040]) from [<bf3b291c>] (omap_abe_probe+0x3f0/0x63c [snd_soc_omap_abe_twl6040])
[    5.882799] [<bf3b291c>] (omap_abe_probe [snd_soc_omap_abe_twl6040]) from [<c051f740>] (platform_drv_probe+0x48/0x98)
[    5.893922] [<c051f740>] (platform_drv_probe) from [<c051d890>] (really_probe+0x148/0x2ac)
[    5.902586] [<c051d890>] (really_probe) from [<c051dc48>] (driver_probe_device+0x110/0x158)
[    5.911337] [<c051dc48>] (driver_probe_device) from [<c051dd20>] (__driver_attach+0x90/0xc4)
[    5.920180] [<c051dd20>] (__driver_attach) from [<c051bdf8>] (bus_for_each_dev+0x64/0xa0)
[    5.928744] [<c051bdf8>] (bus_for_each_dev) from [<c051ce20>] (bus_add_driver+0xcc/0x1e0)
[    5.937306] [<c051ce20>] (bus_add_driver) from [<c051e96c>] (driver_register+0xb0/0xf8)
[    5.945695] [<c051e96c>] (driver_register) from [<bf3b8070>] (omap_abe_init+0x70/0x1000 [snd_soc_omap_abe_twl6040])
[    5.956642] [<bf3b8070>] (omap_abe_init [snd_soc_omap_abe_twl6040]) from [<c0102ecc>] (do_one_initcall+0xc0/0x268)

[    5.967488] [<c0102ecc>] (do_one_initcall) from [<c01b02f8>] (do_init_module+0x58/0x1d0)
[    5.975965] [<c01b02f8>] (do_init_module) from [<c01af030>] (load_module+0x1154/0x1244)
[    5.984342] [<c01af030>] (load_module) from [<c01af324>] (sys_finit_module+0x94/0xb4)
[    5.992541] [<c01af324>] (sys_finit_module) from [<c0101000>] (ret_fast_syscall+0x0/0x54)
[    6.001104] Exception stack(0xed2fbfa8 to 0xed2fbff0)
[    6.006405] bfa0:                   b6d737d4 00051f68 00000007 b6d729f8 00000000 b6d7331c
[    6.014980] bfc0: b6d737d4 00051f68 4960d000 0000017b 00020000 00037f78 00050048 00055750
[    6.023559] bfe0: bebf2ef0 bebf2ee0 b6d6cc4b b6e77a42
[    6.070176] ---[ end trace 8c0db6e65608fbbc ]---
[    6.094981] aess 401f1000.aess: ASoC: Failed to create component debugfs directory
[    6.103401] aess 401f1000.aess: ASoC: invalid header size for type 0 at offset 0x0 size 0x1a5e8.
[    6.112744] aess 401f1000.aess: request for AESS FW failed -22
[    6.122474] dwc3 4a030000.dwc3: Failed to get clk 'ref': -2
[    6.129806] omap-abe-twl6040 sound: ASoC: unknown pin AFML
[    6.137354] omap-abe-twl6040 sound: ASoC: unknown pin AFMR
[    6.144229] OF: graph: no port node found in /ocp/ocp2scp at 4a080000/usb2phy at 4a084000
[    6.146911] usb 1-2: New USB device found, idVendor=0424, idProduct=3503, bcdDevice=a1.a0
[    6.160872] dwc3 4a030000.dwc3: otg: idle->device
[    6.166876] usb 1-2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    6.176642] hub 1-2:1.0: USB hub found
[    6.181048] dwc3 4a030000.dwc3: changing max_speed on rev 5533202a
[    6.181639] omap-abe-twl6040 sound: twl6040-legacy <-> 40132000.mcpdm mapping ok
[    6.195386] hub 1-2:1.0: 3 ports detected
[    6.195525] omap-abe-twl6040 sound: snd-soc-dummy-dai <-> MultiMedia1 mapping ok
[    6.207514] omap-abe-twl6040 sound: snd-soc-dummy-dai <-> MultiMedia2 mapping ok
[    6.215446] omap-abe-twl6040 sound: snd-soc-dummy-dai <-> Voice mapping ok
[    6.223492] omap-abe-twl6040 sound: snd-soc-dummy-dai <-> Tones mapping ok
[    6.280207] omap-abe-twl6040 sound: snd-soc-dummy-dai <-> MODEM mapping ok
[    6.296665] omap-abe-twl6040 sound: snd-soc-dummy-dai <-> MultiMedia1 LP mapping ok
[    6.305178] omap-abe-twl6040 sound: twl6040-legacy <-> 40132000.mcpdm mapping ok
[    6.314062] omap-abe-twl6040 sound: Connecting non-supply widget to supply widget is not supported (Headset Mic -> Headset Mic Bias)
[    6.327755] omap-abe-twl6040 sound: ASoC: no dapm match for Headset Mic --> (null) --> Headset Mic Bias
[    6.343579] omap-abe-twl6040 sound: ASoC: Failed to add route Headset Mic -> direct -> Headset Mic Bias
[    6.355682] omap-abe-twl6040 sound: Connecting non-supply widget to supply widget is not supported (Main Handset Mic -> Main Mic Bias)
[    6.369050] omap-abe-twl6040 sound: ASoC: no dapm match for Main Handset Mic --> (null) --> Main Mic Bias
[    6.377199] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    6.379827] omap-abe-twl6040 sound: ASoC: Failed to add route Main Handset Mic -> direct -> Main Mic Bias
[    6.395809] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    6.397617] omap-abe-twl6040 sound: Connecting non-supply widget to supply widget is not supported (Sub Handset Mic -> Main Mic Bias)
[    6.404684] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[    6.417230] omap-abe-twl6040 sound: ASoC: no dapm match for Sub Handset Mic --> (null) --> Main Mic Bias
[    6.426551] cfg80211: failed to load regulatory.db
[    6.436197] omap-abe-twl6040 sound: ASoC: Failed to add route Sub Handset Mic -> direct -> Main Mic Bias
[    6.451421] omap-abe-twl6040 sound: ASoC: no source widget found for PDM_DL1
[    6.458976] omap-abe-twl6040 sound: ASoC: Failed to add route PDM_DL1 -> direct -> Headset Playback
[    6.468618] omap-abe-twl6040 sound: ASoC: no source widget found for PDM_DL2
[    6.478553] omap-abe-twl6040 sound: ASoC: Failed to add route PDM_DL2 -> direct -> Handsfree Playback
[    6.489886] omap-abe-twl6040 sound: ASoC: no source widget found for Analog Capture
[    6.498671] omap-abe-twl6040 sound: ASoC: Failed to add route Analog Capture -> direct -> PDM_UL1
[    6.508532] omap-abe-twl6040 sound: ASoC: no source widget found for BT_VX_DL
[    6.516144] omap-abe-twl6040 sound: ASoC: Failed to add route BT_VX_DL -> direct -> omap-mcbsp.1 Playback
[    6.526888] omap-abe-twl6040 sound: ASoC: no source widget found for omap-mcbsp.1 Capture
[    6.535613] omap-abe-twl6040 sound: ASoC: Failed to add route omap-mcbsp.1 Capture -> direct -> BT_VX_UL
[    6.545772] omap-abe-twl6040 sound: ASoC: no source widget found for MM_EXT_DL
[    6.553486] omap-abe-twl6040 sound: ASoC: Failed to add route MM_EXT_DL -> direct -> omap-mcbsp.2 Playback
[    6.563942] omap-abe-twl6040 sound: ASoC: no source widget found for omap-mcbsp.2 Capture
[    6.572656] omap-abe-twl6040 sound: ASoC: Failed to add route omap-mcbsp.2 Capture -> direct -> MM_EXT_UL
[    6.586839] input: Letux Cortex 15 Headset Jack as /devices/platform/sound/sound/card3/input11
[    6.600866] omap-hdmi-audio omap-hdmi-audio.1.auto: snd-soc-dummy-dai <-> 58040000.encoder mapping ok
[    6.613332] random: crng init done
[    6.671301] dmm 4e000000.dmm: initialized all PAT entries
[    6.677865] omapdss_dsi 58004000.encoder: Linked as a consumer to regulator.20
[    6.685727] omapdss_hdmi5 58040000.encoder: Linked as a consumer to regulator.20

...

root at letux:~# aplay -l 
**** List of PLAYBACK Hardware Devices ****
card 0: Bluetooth [Bluetooth], device 0: 40122000.mcbsp-W2CBW003 W2CBW003-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Tuner [Tuner], device 0: 40124000.mcbsp-W2CBW003 W2CBW003-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Telephony [Telephony], device 0: 40126000.mcbsp-gtm601 gtm601-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 0: TWL6040 twl6040-legacy-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 1: (null) (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 3: (null) (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 4: (null) (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 5: (null) (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 6: (null) (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 3: L15 [Letux Cortex 15], device 7: (null) twl6040-legacy-7 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 4: H58040000encode [HDMI 58040000.encoder], device 0: HDMI 58040000.encoder snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
root at letux:~# ./twl somefile.mp3 

...

This is not nice and should certainly be fixed, but does not stop Audio from initially working.


The same setup on the new Pyra 4G devices give scratchy and no loud music.

So in summary I'd say: the audio system almost works for the letux-4.19.44 kernel.
It is not fundamentally broken. But it needs fixes for the scratchy sound (which might
be idle states or PCM clock issues) and fixes for real AESS/ABE integration (and not the
indeterminate state).

The open question is who will commit to work on these fixes.

BR,
Nikolaus






More information about the Letux-kernel mailing list