[2/4] dt-bindings: touchscreen: add virtual-touchscreen and virtual-buttons properties

  The virtual-touchscreen object defines an area within the touchscreen
where touch events are reported and their coordinates get converted to
the virtual origin. This object avoids getting events from areas that
are physically hidden by overlay frames.

For touchscreens where overlay buttons on the touchscreen surface are
provided, the virtual-buttons object contains a node for every button
and the key event that should be reported when pressed.

Signed-off-by: Javier Carrasco <javier.carrasco@wolfvision.net>
---
 .../bindings/input/touchscreen/touchscreen.yaml    | 54 ++++++++++++++++++++++
 1 file changed, 54 insertions(+)
  

On 10/05/2023 15:50, Javier Carrasco wrote:
> The virtual-touchscreen object defines an area within the touchscreen
> where touch events are reported and their coordinates get converted to
> the virtual origin. This object avoids getting events from areas that
> are physically hidden by overlay frames.
> 
> For touchscreens where overlay buttons on the touchscreen surface are
> provided, the virtual-buttons object contains a node for every button
> and the key event that should be reported when pressed.

Hm, I don't understand - are these separate physical buttons? If so, why
would they be part of touchscreen? Where do the wires go?

> 
> Signed-off-by: Javier Carrasco <javier.carrasco@wolfvision.net>
> ---
>  .../bindings/input/touchscreen/touchscreen.yaml    | 54 ++++++++++++++++++++++
>  1 file changed, 54 insertions(+)
> 

Best regards,
Krzysztof
  

On 11.05.23 11:45, Krzysztof Kozlowski wrote:
> On 10/05/2023 15:50, Javier Carrasco wrote:
>> The virtual-touchscreen object defines an area within the touchscreen
>> where touch events are reported and their coordinates get converted to
>> the virtual origin. This object avoids getting events from areas that
>> are physically hidden by overlay frames.
>>
>> For touchscreens where overlay buttons on the touchscreen surface are
>> provided, the virtual-buttons object contains a node for every button
>> and the key event that should be reported when pressed.
> 
> Hm, I don't understand - are these separate physical buttons? If so, why
> would they be part of touchscreen? Where do the wires go?
Those buttons are actually printed on some overlays which are mounted on
top of the touchscreen and  they are used to provide a predefined
functionality to that touchscreen region. Any touchscreen with such a
physical overlay with buttons printed on them or clipped touchscreen
areas might use this functionality.

These buttons are actually physical i.e. printed and with a given
functionality, but still part of the touchscreen as the physical device
is not aware of them. Therefore they only make sense in the touchscreen
context.
> 
>>
>> Signed-off-by: Javier Carrasco <javier.carrasco@wolfvision.net>
>> ---
>>  .../bindings/input/touchscreen/touchscreen.yaml    | 54 ++++++++++++++++++++++
>>  1 file changed, 54 insertions(+)
>>
> 
> Best regards,
> Krzysztof
>
  

On 11/05/2023 12:28, Javier Carrasco wrote:
> On 11.05.23 11:45, Krzysztof Kozlowski wrote:
>> On 10/05/2023 15:50, Javier Carrasco wrote:
>>> The virtual-touchscreen object defines an area within the touchscreen
>>> where touch events are reported and their coordinates get converted to
>>> the virtual origin. This object avoids getting events from areas that
>>> are physically hidden by overlay frames.
>>>
>>> For touchscreens where overlay buttons on the touchscreen surface are
>>> provided, the virtual-buttons object contains a node for every button
>>> and the key event that should be reported when pressed.
>>
>> Hm, I don't understand - are these separate physical buttons? If so, why
>> would they be part of touchscreen? Where do the wires go?
> Those buttons are actually printed on some overlays which are mounted on
> top of the touchscreen and  they are used to provide a predefined
> functionality to that touchscreen region. Any touchscreen with such a
> physical overlay with buttons printed on them or clipped touchscreen
> areas might use this functionality.
> 
> These buttons are actually physical i.e. printed and with a given
> functionality, but still part of the touchscreen as the physical device
> is not aware of them. Therefore they only make sense in the touchscreen
> context.

So basically you still have the same touchscreen under the buttons and
these are not separate devices. Whether someone put a sticker on
touchscreen, does not really change hardware behavior and it's up to
system to handle this. What you are trying to do here is to create
virtual buttons through Devicetree and DT is not for that.

Best regards,
Krzysztof
  

Hi Krzysztof,

On 5/12/23 08:25, Krzysztof Kozlowski wrote:
> On 11/05/2023 12:28, Javier Carrasco wrote:
>> On 11.05.23 11:45, Krzysztof Kozlowski wrote:
>>> On 10/05/2023 15:50, Javier Carrasco wrote:
>>>> The virtual-touchscreen object defines an area within the touchscreen
>>>> where touch events are reported and their coordinates get converted to
>>>> the virtual origin. This object avoids getting events from areas that
>>>> are physically hidden by overlay frames.
>>>>
>>>> For touchscreens where overlay buttons on the touchscreen surface are
>>>> provided, the virtual-buttons object contains a node for every button
>>>> and the key event that should be reported when pressed.
>>>
>>> Hm, I don't understand - are these separate physical buttons? If so, why
>>> would they be part of touchscreen? Where do the wires go?
>> Those buttons are actually printed on some overlays which are mounted on
>> top of the touchscreen and  they are used to provide a predefined
>> functionality to that touchscreen region. Any touchscreen with such a
>> physical overlay with buttons printed on them or clipped touchscreen
>> areas might use this functionality.
>>
>> These buttons are actually physical i.e. printed and with a given
>> functionality, but still part of the touchscreen as the physical device
>> is not aware of them. Therefore they only make sense in the touchscreen
>> context.
> 
> So basically you still have the same touchscreen under the buttons and
> these are not separate devices. Whether someone put a sticker on
> touchscreen, does not really change hardware behavior and it's up to
> system to handle this. What you are trying to do here is to create
> virtual buttons through Devicetree and DT is not for that.

I have already addressed a similar statement here:
https://lore.kernel.org/lkml/20230504042927.GA1129520@quokka/t/#m1a93595c36535312df0061459a1da5e062de6c44
but let me extend this comment a bit.

The notion of "someone putting a sticker on touchscreen" does not really
reflect the use case we have in mind. We are talking about devices that
are shipped from the factory in a particular configuration, e.g.,

+-----------------------+---------+
|                       |  power  |
|                       |  button |
|   touchscreen         +---------+
|   (behind: display)   |  return |
|                       |  button |
+-----------------------+---------+

Here, the real touchscreen is larger than the display. The display is
behind the "touchscreen" part. Behind the buttons there are symbols
engraved in metal or printed foils or what not. I just would like to
make it clear that these symbols are not going to change.

We believe that the engraved or printed symbols actually define the
(expected) hardware behavior. Of course there is a virtual notion to
that, and of course it would be possible to let the power button work as
return button and vice versa in software. However, the user sees the
engraved or printed symbols (which are not going to change) and expects
the device to react appropriately.

Now if you suggest that the system (that is user space, right?) should
handle this, then the question would be which component is supposed to
handle the touchscreen events and react accordingly. I don't have an
answer to that and hope I don't need to find one. But independent of
that, a configuration file is required that defines the area of the
virtual buttons etc. Wouldn't this be similar to the (mostly) beloved
xorg.conf containing the definitions of input devices?

Best regards,
Michael
  

On 12/05/2023 09:08, Michael Riesch wrote:
> Hi Krzysztof,
> 
>>> These buttons are actually physical i.e. printed and with a given
>>> functionality, but still part of the touchscreen as the physical device
>>> is not aware of them. Therefore they only make sense in the touchscreen
>>> context.
>>
>> So basically you still have the same touchscreen under the buttons and
>> these are not separate devices. Whether someone put a sticker on
>> touchscreen, does not really change hardware behavior and it's up to
>> system to handle this. What you are trying to do here is to create
>> virtual buttons through Devicetree and DT is not for that.
> 
> I have already addressed a similar statement here:
> https://lore.kernel.org/lkml/20230504042927.GA1129520@quokka/t/#m1a93595c36535312df0061459a1da5e062de6c44
> but let me extend this comment a bit.
> 
> The notion of "someone putting a sticker on touchscreen" does not really
> reflect the use case we have in mind. We are talking about devices that
> are shipped from the factory in a particular configuration, e.g.,
> 
> +-----------------------+---------+
> |                       |  power  |
> |                       |  button |
> |   touchscreen         +---------+
> |   (behind: display)   |  return |
> |                       |  button |
> +-----------------------+---------+
> 
> Here, the real touchscreen is larger than the display. The display is
> behind the "touchscreen" part. Behind the buttons there are symbols
> engraved in metal or printed foils or what not. I just would like to
> make it clear that these symbols are not going to change.
> 
> We believe that the engraved or printed symbols actually define the
> (expected) hardware behavior. Of course there is a virtual notion to
> that, and of course it would be possible to let the power button work as
> return button and vice versa in software. However, the user sees the
> engraved or printed symbols (which are not going to change) and expects
> the device to react appropriately.

OK, I actually missed the concept that display is not equal to the
touchscreen ("screen" actually suggests display :) ). In your case here
it sounds good, but please put some parts of this description into this
common binding. The sketch above is nice, especially if you can point
where the virtual origin x/y are. Picture is thousands words.

> 
> Now if you suggest that the system (that is user space, right?) should
> handle this, then the question would be which component is supposed to
> handle the touchscreen events and react accordingly. I don't have an
> answer to that and hope I don't need to find one. But independent of
> that, a configuration file is required that defines the area of the
> virtual buttons etc. Wouldn't this be similar to the (mostly) beloved
> xorg.conf containing the definitions of input devices?

If the case was a bit different - e.g. display is everywhere - I could
easily imagine that on the device rotation you want to move
(re-position) the buttons. Or if user has some accessibility option
enabled, you want bigger buttons. Then it would be a prove that it must
be configured and managed from user-space. How, I don't know.

Best regards,
Krzysztof
  

Hi Krzysztof,

On 5/12/23 09:20, Krzysztof Kozlowski wrote:
> On 12/05/2023 09:08, Michael Riesch wrote:
>> Hi Krzysztof,
>>
>>>> These buttons are actually physical i.e. printed and with a given
>>>> functionality, but still part of the touchscreen as the physical device
>>>> is not aware of them. Therefore they only make sense in the touchscreen
>>>> context.
>>>
>>> So basically you still have the same touchscreen under the buttons and
>>> these are not separate devices. Whether someone put a sticker on
>>> touchscreen, does not really change hardware behavior and it's up to
>>> system to handle this. What you are trying to do here is to create
>>> virtual buttons through Devicetree and DT is not for that.
>>
>> I have already addressed a similar statement here:
>> https://lore.kernel.org/lkml/20230504042927.GA1129520@quokka/t/#m1a93595c36535312df0061459a1da5e062de6c44
>> but let me extend this comment a bit.
>>
>> The notion of "someone putting a sticker on touchscreen" does not really
>> reflect the use case we have in mind. We are talking about devices that
>> are shipped from the factory in a particular configuration, e.g.,
>>
>> +-----------------------+---------+
>> |                       |  power  |
>> |                       |  button |
>> |   touchscreen         +---------+
>> |   (behind: display)   |  return |
>> |                       |  button |
>> +-----------------------+---------+
>>
>> Here, the real touchscreen is larger than the display. The display is
>> behind the "touchscreen" part. Behind the buttons there are symbols
>> engraved in metal or printed foils or what not. I just would like to
>> make it clear that these symbols are not going to change.
>>
>> We believe that the engraved or printed symbols actually define the
>> (expected) hardware behavior. Of course there is a virtual notion to
>> that, and of course it would be possible to let the power button work as
>> return button and vice versa in software. However, the user sees the
>> engraved or printed symbols (which are not going to change) and expects
>> the device to react appropriately.
> 
> OK, I actually missed the concept that display is not equal to the
> touchscreen ("screen" actually suggests display :) ). In your case here
> it sounds good, but please put some parts of this description into this
> common binding. The sketch above is nice, especially if you can point
> where the virtual origin x/y are. Picture is thousands words.

I think this can be arranged :-)

>> Now if you suggest that the system (that is user space, right?) should
>> handle this, then the question would be which component is supposed to
>> handle the touchscreen events and react accordingly. I don't have an
>> answer to that and hope I don't need to find one. But independent of
>> that, a configuration file is required that defines the area of the
>> virtual buttons etc. Wouldn't this be similar to the (mostly) beloved
>> xorg.conf containing the definitions of input devices?
> 
> If the case was a bit different - e.g. display is everywhere - I could
> easily imagine that on the device rotation you want to move
> (re-position) the buttons. Or if user has some accessibility option
> enabled, you want bigger buttons. Then it would be a prove that it must
> be configured and managed from user-space. How, I don't know.

I fully agree here. If user space is able to draw the symbols, then
naturally it is also responsible for handling the events appropriately.
This could happen in an application with a GUI or on display
manager/compositor level (e.g., weston controller or similar).

But I take it that for the situation sketched above the device tree
approach is the correct one. Documentation should be improved, though.

Best regards,
Michael