[Letux-kernel] RFC sdio expander driver thoughts (TXS02612)

[H. Nikolaus Schaller]

Hi Ulf,

> Am 05.07.2018 um 13:55 schrieb Ulf Hansson <ulf.hansson at linaro.org>:
> 
> On 4 July 2018 at 14:58, H. Nikolaus Schaller <hns at goldelico.com> wrote:
>> Hi Ulf,
>> 
>>> Am 04.07.2018 um 13:06 schrieb Ulf Hansson <ulf.hansson at linaro.org>:
>>> 
>>> On 3 July 2018 at 20:45, Belisko Marek <marek.belisko at gmail.com> wrote:
>>>> Hi Ulf,
>>>> 
>>>> CC'ed pyra kernel guys
>>>> 
>>>> On Tue, Jul 3, 2018 at 11:23 AM Ulf Hansson <ulf.hansson at linaro.org> wrote:
>>>>> 
>>>>> On 3 July 2018 at 08:47, Belisko Marek <marek at goldelico.com> wrote:
>>>>>> Hello,
>>>>>> 
>>>>>> on pyra device [0] we're using txs02612 sdio expander to have
>>>>>> possibility to connect SD card and also eMMC. We have some version of
>>>>>> driver (gpio for switching bus is controlled via sysfs) which probably
>>>>>> won't accepted. We want to do it correct way so we was thinking about
>>>>>> same approach as i2c-mux is using (i2c device connected on mux buses
>>>>>> are transparent to user so switching is done completely in driver). I
>>>>>> would like to maybe get some initial thoughts if this is good idea to
>>>>>> go this way and don't be a problem for future upstreaming. Thanks for
>>>>>> any comments.
>>>>> 
>>>>> Sorry, but I doubt we ever get to add proper upstream support for mmc
>>>>> hosts managing multiple slots. Simply because all the efforts it takes
>>>>> isn't worth it.
>>>>> 
>>>>> Today there are some host trying to support this, but those are hacks,
>>>>> which is working very poorly. I don't want to maintain more of those,
>>>>> then it's better those are managed by vendors out of tree.
>>>> 
>>>> Thanks for quick reply.
>>>> 
>>>> (just copy part of internal discussion):
>>>> My approach would be that the driver registers two new host
>>>> controllers, e.g. like mmc1 and mmc2 on the omap5.
>>>> 
>>>> I.e. calls in its probe function
>>>> 
>>>>       mmc[i] = mmc_alloc_host(sizeof(struct omap_hsmmc_host), &pdev->dev);
>>>>       mmc[i]->ops = txs02612_ops;
>>>>       // other setup
>>>>       mmc_add_host(mmc[i]);
>>>> 
>>>> twice for each of the two txs channels. This should create two new
>>>> user-space mmc ports that receive requests and other callbacks.
>>>> 
>>>> It then should takes control of the real host controller to sit in
>>>> the middle.
>>>> 
>>>> So basically we should do something like:
>>>> 
>>>>       host_mmc = sarch_by_of_handle(...)
>>>> 
>>>> And then make the txs02612_ops like:
>>>> 
>>>>       txs02612_request(mmc, ...) {
>>>>               find out which port
>>>>               if(different from current)
>>>>                       {
>>>>                       try to get mutex
>>>>                       throw the switch
>>>>                       }
>>>>               host_mmc->request(host_mmc, ...)
>>>>               if(was different)
>>>>                       release the mutex
>>>>       }
>>>> 
>>>> in this case we don't need to touch mmc core and also bus locking will
>>>> be done in driver.
>>>> Do you think we can get some success with upstreaming by this approach?
>>> 
>>> No.
>>> 
>>>> Thanks.
>>>>> 
>>>>> There have been several discussions on this topic, this is just one
>>>>> that I found while searching.
>>>>> https://lkml.org/lkml/2017/10/6/710
>>> 
>>> Did you read this?
>> 
>> Yes, we did study it and understand the concerns. But concerns don't help
>> to solve problems.
>> 
>> We have a hardware with this sdio switch chip is connected to some omap5 mmc port
>> and multiplexes an internal eMMC and an external µSD slot.
>> 
>> Currently we use a user-space script that ejects the current drive, toggles
>> the control gpio and partprobes the other sd device:
>> 
>> http://git.goldelico.com/?p=letux-kernel.git;a=blob;f=Letux/root/txs;h=a40a28c8e8bbae5c04569aced3de48571d00ef92;hb=e7a136a13aba2373b07637680d2998e7ea19c756
>> 
>> This is slow. And is not transparent for user-space software (e.g. file managers).
>> And you can't copy data from one device to the other with standard cp commands
>> or without temporary storage.
>> 
>> So we would be happy with any improvement over this situation and are not
>> immediately looking at perfection. This is why we want to find a kernel driver
>> solution for the txa02612 chip.
>> 
>>> 
>>>>> 
>>>>> If you need any further explainations, please just ask.
>>>>> 
>>> 
>>> Let me elaborate a bit on the problems.
>>> 
>>> 1. Serialization of requests (several request sent after each other,
>>> with delays, changing clocks etc), is managed by the core, because it
>>> implements the SD/SDIO/(e)MMC specs. This does not belong in a host
>>> driver.
>>> -> A kind of mmc bus slot lock is needed in the core. Not a big deal,
>>> we can probably hide it below mmc_claim|release_host().
>> 
>> Our goal would be to encapsulate this completely in the txs02612 driver
>> so that we do not have to touch the mmc core at all. If that is possible
>> with existing API.
> 
> Sorry, but the core needs to be involved. However, I think adding a
> mmc bus lock, somehow, can be done.
> 
>> 
>>> 
>>> 2. What if there is an SD card that can cope with high-speed mode in
>>> one slot with a clock freq at 50 MHz. Then in the other slot, there is
>>> an eMMC using HS200 mode, running at 200 MHz. Can the HW guarantee all
>>> kind of different combinations, with clock glitches etc?
>> 
>> We would be happy in a first step if it would work at all. Even with certain
>> constraints like same clock freq for everyone so that no clock switching
>> is involved. And yes, the txs02612 chip introduces its own speed limits.
> 
> This is too hand-wavy. I need to know how you plan to solve this,
> because to me, it's really a tricky problem.
> 
> Of course, it depends also on how much the HW offers to help.
> 
>> 
>>> 
>>> 3. The biggest problem: You can *not* provide any service guarantees
>>> in regards to latency or bandwidth for any of the storage media in the
>>> slots, because of the bus slot lock. That because the upper layers
>>> runs I/O scheduling separately for each storage device, thus request
>>> for one slot can starve requests on another.
>> 
>> Yes, we are aware of this limitation. It is not possible to achieve the
>> same speed in multiplexed mode as with a dedicated mmc port for each client.
>> But I expect the problem is not much different from a hard disk which may
>> have significant latency while spinning up the motor.
> 
> This is not only about poor performance. It's about things being
> entirely unpredictable.
> 
> Maybe we can figure something out to release the bus lock in between
> every request (that's not what we do today for the host lock), but it
> may be messy.

Yes, that is basically my view: if each request is locked but released
after being served, the other channel gets a chance (if there is a pending
request). With such a scheme, two processes accessing the two channels
concurrently would alternate the switch and share bandwidth. Like two
USB clients sitting on a remote hub which is connected to a single USB
host port. Well, this analogy isn't perfect as the USB protocol and hub
already takes care of this situation.

> You simply will have to post patches for me to look at,
> before I can decide what to do with them.

Ok, that is fine. Thanks!

> 
>> 
>> And we have the same (even worse) with our current eject/partprobe approach
>> which also does sort of serialization.
>> 
>> So we just want to know about the hurdles towards upstreaming before
>> we have to turn the driver code upside down. And how to setup the driver
>> architecture to get it basically working.
> 
> Fair enough. I hope my comments have given some answers to you.

Yes they do. We will start experimenting a little with our ideas. And then
share them. Or come up with specific questions...

Best regards and thanks,
Nikolaus