EP4457588A1

EP4457588A1 - Haptic and thermal feedback touchpad device, system and method thereof for an automotive setting

Info

Publication number: EP4457588A1
Application number: EP21856982.0A
Authority: EP
Inventors: Pedro Miguel DE SOUSA BERNARDO; Vera Patricia CASTRO VILAS BOAS; Nélson Bruno MARTINS MARQUES COSTA; Manuel Sebastião MARQUES COUTINHO; Adriano Dídimo MACHADO CARVALHO; Rosane SCHONBLUM SAMPAIO; Marina MACHADO; Lisandra Maria ARAÚJO TEIXEIRA; Emanuel TEIXEIRA SILVA; José Pedro CORREIA DIAS
Original assignee: Universidade do Minho; Bosch Car Multimedia Portugal SA
Current assignee: Universidade do Minho; Bosch Car Multimedia Portugal SA
Priority date: 2021-12-29
Filing date: 2021-12-30
Publication date: 2024-11-06
Also published as: WO2023126660A1

Abstract

A touchpad device comprising an interface with haptic and thermal feedback for a user in an automotive setting, comprising: a haptic-feedback layer; a capacitive touch sensitive layer; a thermal-feedback layer assembly; and an electronic data processor configured to: drive the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface. A method of operation thereof comprises driving the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface. System thereof further comprising a display.

Description

HAPTIC AN D THERMAL FEEDBACK TOUCHPAD DEVICE, SYSTEM

AND METHOD THEREOF FOR AN AUTOMOTIVE SETTI NG

TECH NICAL FIELD

[0001] The present disclosure relates to an interface device and method for providing haptic and thermal feedback.

BACKGROUND

[0002] Document US9827811B1 discloses a vehicular haptic feedback system including a haptic feedback controller configured to communicate with a vehicle. The haptic feedback controller including a plurality of haptic feedback actuators and processing circuitry configured to detect quality of operation information of the vehicle, a direction and an intensity of threat information corresponding to the vehicle, and a mode of operation of the vehicle. The haptic feedback controller is also configured to determine a desired direction of travel of the vehicle based on the quality of operation information, the direction and the intensity of threat information, and the mode of operation. The haptic feedback controller is further configured to provide haptic feedback corresponding to the quality of operation information, the direction and the intensity of threat information, the mode of operation and the desired direction of travel of the vehicle, via the plurality of haptic feedback actuators.

[0003] Document US2016238040 discloses a multimodal haptic device operating as a closed-loop system, the device including a pipeline configured to allow a closed-loop flow of a fluid medium, a manifold operatively connected to the pipeline, the manifold having a pump and a valve to control and regulate a flow of the fluid medium along the pipeline, and a display unit operatively connected to the pipeline, the display unit having a tactile display and a valve operatively connected to the tactile display for regulating an efflux of the fluid medium from the tactile display into the pipeline. [0004] Document US20200264772A1 discloses a method for providing haptic feedback to an operator of a touch-sensitive display device. The method includes providing an operator interface to be represented in the display device as image data; the operator interface having at least one button assigned to a function to be controlled, graphically displayed in the operator interface, and graphically delimited from the rest of the operator interface; analysing the image data for a presence and a position of shapes for representing a button; determining an intended display region of a shape, which is to be represented and is identified as a button; representing the image data of the operator interface in the touch-sensitive display device; outputting haptic feedback in response to detection of contact with the touch-sensitive display device in the region of a surface of the touch-sensitive display device which is assigned to a display region of a shape identified as a button.

[0005] These facts are disclosed in order to illustrate the technical problem addressed by the present disclosure.

GENERAL DESCRIPTION

[0006] The present disclosure relates to an interface device and method for providing capacitive, haptic and thermal feedback, herein called smart surface.

[0007] According to Hirai & Miki (2019), when one touches objects, various thermal sensations are felt, which correspond to the characteristics of said objects. Therefore, this thermal sensation is very important in material recognition tasks and plays a very important role in tactile perception.

[0008] Metal and wood, for example, are perceived with different thermal sensations in a room with equivalent room temperature. Hirai & Miki (2019) proposed a method of varying the thermal conductivity of a display and present various thermal sensations to users.

[0009] Haptic feedback in devices and surfaces has commonly been achieved through vibrotactile stimuli, generated from mechanical actuators (e.g., EP3582074A1). More recently, new haptic technologies have emerged that make use of other actuation types, such as electro vibration, to generate other types of tactile feedback beyond just vibrations (e.g., US10768749B2).

[0010] Thermal sensations play a big role in tactile perception, and, therefore, thermal display functions must be developed to increase the perception of realism on displays, making the interactions with them more satisfactory for users.

[0011] As display surface technology develops, more focus is being given to display devices and surfaces that can provide new and innovative ways of enriching the interactions users have with them, such as generating richer haptic and/or thermal feedback sensations.

[0012] The present disclosure is employable either inside a level 5 autonomous vehicle, or in passenger compartments that have seats with armrests, with the goal of increasing the user's experience while going about their commutes on these vehicles.

[0013] Recent studies suggest that haptic feedback will improve in the following years, as the goal is to create the most realistic perceptions in VR or displays inside a vehicle. As seen in the literature thermal sensations increase tactile perception as well as the realism of experience (Chouvardas et al., 2008; Hirai & Miki, 2019; Sato, 2018).

[0014] The smart surface comprises, at least, 5 layers. Where in a first layer is an insulative layer, fitted atop a transparent and conductive electrode sheet layer, which is in turn fitted atop a capacitive-based touch sensitive layer, which, when charged with an electric current and when in contact with the human skin, produces a frictional stimulus by controlling the electric current. This structure is fitted atop of a layer composed of either liquid metal (e.g., Galinstan, composed by gallium, indium, and tin, as seen in Hirai & Miki, 2019)), in contact with a titanium plate, with the amount of liquid which is in contact with this plate determining the thermal conductivity (the thermal conductivity of the device's surface layer can be determined by the amount of liquid metal which is in contact with this layer) of the device's surface layer, or a layer of solid- state Peltier elements (cold side facing the layer on top, hot side and heat sink facing the layer below), which can vary the temperature of the top surface layer. This layer is, in turn, fitted atop a layer of vibrotactile actuators. [0015] In an embodiment, the insulative layer should be electrically insulative, so that electro adhesion can occur between the finger and the conductive electrode sheet, but thermally conductive, so that thermal changes can be perceived by the finger.

[0016] This smart surface is to be employed as the interactive surface interface of a device to which it is not physically connected (e.g., touch surface interface communicating wirelessly with a display device in close proximity).

[0017] In the present disclosure, the smart surface can generate both frictional and vibrotactile haptic sensations, as well as different thermal feedback sensations.

[0018] Frictional haptic sensations, also described as electro vibration, are generated thanks to the insulative and conductive layers, which allow for friction between the smart surface and the user's skin to be modulated through electrostatic actuation, while the skin is in motion.

[0019] Vibrotactile haptic sensations, in turn, are generated thanks to the vibrotactile actuators, which can generate vibrations strong enough (e.g., 65Hz to 500Hz) to be felt through the user's skin, either stationary or in motion, when in direct contact with the smart surface.

[0020] Different thermal feedback stimuli sensations are generated by regulating the temperate of the top-most surface of the structure. Temperature regulation is controlled via software, which determines eitherthe amount of liquid metal that should be in contact with the surface layer to obtain the intended temperature, or controls how much power should be applied to the layer of Peltier element, so that the surface layer reaches the intended temperature.

[0021] When the smart surface is used as the interactive interface for an external device, capacitive, haptic, and thermal feedback sensations, programmed/controlled by the external device's software, can be generated on it, as long as the haptic and thermal actuation technology that is present on the smart surface is both capable of interpreting the software's information and of generating such sensations. When the external device's software sends a command to the smart surface, requesting that it generates haptic and/or thermal feedback sensations on the surface layer, the hardware and/or software found on the smart layer must be capable of differentiating between those types of sensations, delegating actuation to the elements that control the actuators capable of generating the sensations requested.

[0022] In one embodiment, the capacitive, haptic and/or thermal feedback sensations of the smart surface are controlled by an external device which can be the same device that the smart surface is controlling (e.g., central stack) or other.

[0023] In an embodiment, the in-vehicle centre stack display and armrest surface has both haptic and thermal feedback capabilities. Both the centre stack display and the armrest surface are connected to any of the vehicle's power supplies supplies or one provided by the user (e.g., batteries), deriving power from it. An armrest equipped with this smart surface is attached to both the right and left side of the seat, with users being able to indicate, through a setting found on the vehicle's centre stack display's software, if they wish to receive haptic and thermal feedback sensations to be actuated on either armrest, on both armrests, or neither armrest. This setting is configurable for each seat found on the vehicle, as long as they have at least one armrest equipped with a smart surface as presented on this document.

[0024] In an embodiment, the armrest would be found on the right side of the seat, it is possible to have an armrest on both sides of each seat, has it been more natural, and leads to a more inclusive solution for the application.

[0025] Haptic feedback sensations are generated though electrovibration and/or vibrotactile actuation, while thermal feedback sensations are generated through a layer that can manipulate the temperature of the outer most layer (the electrically insulative but thermally conductive layer) of the structure. Haptic feedback sensations include the sensation of different textures and edges (generated through electrovibration) or vibrations with different frequencies and intensities (generated through vibrotactile actuation). These can be used to let users know about the placement of User Interface elements thought the sense of touch while sliding one or more fingers, or the entire hand, on top of the armrest's smart surface (for example, when the location of a button is reached, the feeling of hitting an edge is felt, and a different texture than before is perceive while inside the button's area), or to transmit information to users regarding certain warnings and information (for example, the user has their hand resting on the armrest's smart surface, and vibrations generated by vibrotactile actuators are transmitted to the hand to let them know they have arrived at their destination). Thermal sensations, in turn, include the increase or decrease of the temperature of the outer most layer of the structure, which can be achieved by either increasing or decreasing the amount of liquid metal which is in contact with the titanium plate, or the amount of power applied to the layer of Peltier element. These temperature variations can be implemented as part of the vehicle's User Interface elements, providing users with information regarding the state of different functions during interactions with interface elements (for example, when changing the temperature of the vehicle, the smart surface's temperature can increase or decrease according to the applied changes, confirming to users that their inputs are being registered), or to transmit information regarding warnings or information (for example, during a trip to a pre-determined location, the temperature of the armrest's smart surface can go from cold to warm as the vehicle gets closer to its destination).

[0026] In an embodiment, the in-vehicle centre stack display and armrest surface have both haptic and thermal feedback capabilities.

[0027] Haptic feedback sensations are generated though electro vibration and/or vibrotactile actuation, while thermal feedback sensations are generated through a layer that can manipulate the temperature of the outer most layer of the structure.

[0028] When interacting with this surface/display, users will be able to sense different pre-programmed haptic and/or thermal feedback sensations during said interaction. For example, while interacting with a User Interface slider to adjust the temperature inside the vehicle, users feel a haptic sensation, such as hitting different "bumps" with increasing intensities (that is, size and/or vibration amplitude) on the surface while dragging the finger along the slider area towards otter or colder values, with each bump signifying a new milestone, such as "20°C", "25°C" and "30°C". Additionally, to accompany this information, the surface of the armrest's smart surface will also increase or decrease its temperature, at the point of contact, depending on the changes that are being made by the user. To help users know in which areas of the armrest's smart surface different User Interface elements are located and can be interacted with, their location can be visually adjusted on the settings menu of the vehicle's centre stack display. Additionally, areas of the smart surface in which different User Interface control elements are found can be differentiated from each other by providing different and distinct texture sensations where the skin slides over them, informing users that they have moved from one area to another, or, when users have memorized which texture corresponds to which interface element, that they have come into contact with the intended area. However, it is expected that the perception of these sensations might be affected by conditions related to the user's hands. For example, the use of gloves can completely negate electrovibration actuation, as the skin of the finger is required to complete the circuit with the insulating layer and the conductive layer of the structure, as well as severely attenuate or even completely mask the vibrations generated by the electrovibration actuators. High finger humidity, in turn, can also negatively impair or outright negate the sensations generated by both electrovibration and vibrotactile actuators.

[0029] In an embodiment, a device could be a 7" display, 7 mm thick, powered through a socket provided by the vehicle specifically for the device, communication with the vehicle via ethernet, and with user devices via Wi-Fi.

[0030] It is disclosed a touchpad device comprising an interface with haptic and thermal feedback for a user in an automotive setting, comprising: a haptic-feedback layer; a capacitive touch sensitive layer; a thermal-feedback layer assembly; and an electronic data processor configured to: drive the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface.

[0031] In an embodiment, the touchpad device can comprise a frictional haptic- feedback layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations. [0032] In an embodiment, the touchpad device can comprise a vibrotactile haptic layer for providing vibrotactile haptic sensations.

[0033] In an embodiment, the thermal-feedback layer assembly comprises a liquid metal layer and a titanium plate for providing a predetermined thermal conductivity of the thermal feedback layer assembly.

[0034] In an embodiment, the electronic data processor is further configured to drive the haptic-feedback layer to exhibit a tactile texture when the user touches the touchpad device interface.

[0035] In an embodiment, the thermal-feedback layer assembly comprises a Peltier element for providing a predetermined heat flux to the thermal-feedback layer assembly.

[0036] In an embodiment, the electronic data processor is further configured to select a frictional or vibrotactile haptic feedback and to select a thermal conductivity or a heat flux, as a function of an image of a pattern or of an object to be displayed.

[0037] It is also disclosed a system for providing haptic and thermal feedback for a user in an automotive setting, the system comprising a display and a touchpad device according to any of the previous embodiments.

[0038] In an embodiment, the electronic data processor is further configured to display of an image of a pattern or of an object and select a frictional or vibrotactile haptic feedback and to select a thermal conductivity or a heat flux, as a function of the image of the pattern or the object displayed.

[0039] It is also disclosed a method of operating a touchpad device according to any of the embodiments, comprising using said electronic data processor for: driving the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The following figures provide preferred embodiments for illustrating the disclosure and should not be seen as limiting the scope of invention.

[0041] Figure 1: Schematic representation of an embodiment of an interface with a capacitive, haptic, and thermal feedback on armrest and central display.

[0042] Figure 2A shows a scheme of the layers, including when a liquid metal layer, in contact with a titanium plate, is used.

[0043] Figure 2B shows a scheme of the layers, including when a layer of Peltier element, is used.

[0044] Figure 3 shows a schematic representation of an embodiment of a touchpad device.

DETAILED DESCRIPTION

[0045] The present disclosure relates to a touchpad device comprising an interface with haptic and thermal feedback for a user in an automotive setting, comprising: a haptic- feedback layer; a capacitive touch sensitive layer; a thermal-feedback layer assembly; and an electronic data processor configured to: drive the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface. The present disclosure also relates to a method of operation thereof comprising driving the haptic-feedback layer and the thermalfeedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface and to a system thereof further comprising a display.

[0046] Figure 1 shows a schematic representation of an embodiment of a touchpad device comprising an interface with haptic and thermal feedback for a user in an automotive setting, where 12 represents the seat and 14 represents the center stack display that the capacitive, haptic surface of the armrest is in contact with.

[0047] As illustrated in Figure 1, a touchpad device can comprise an interface with haptic and thermal feedback for a user in an automotive setting, comprising: a haptic- feedback layer; a capacitive touch sensitive layer; a thermal-feedback layer assembly; and an electronic data processor configured to: drive the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface.

[0048] Figure 2A shows a scheme of the layer's superposition, wherein A represents an insulative layer; B represents a capacitive-based touch sensitive layer; C represents a vibrotactile actuators layer; D represents a transparent electrode sheet layer and F represents a titanium plate and G represents a liquid metal layer.

[0049] Figure 2B shows a scheme of the layer's superposition, wherein A represents an insulative layer; B represents a capacitive-based touch sensitive layer; C represents a vibrotactile actuators layer; D represents a transparent electrode sheet layer and E represents a Peltier element layer.

[0050] As illustrated in Figure 3, a touchpad device comprising an interface with haptic and thermal feedback for a user 11 in an automotive setting, where 12 represents the seat, 13 represents the armrest with the capacitive, haptic surface, and 14 represents the centre stack display that the capacitive, haptic surface of the armrest is in contact with.

[0051] In an embodiment, the touchpad device can comprise a frictional haptic- feedback layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations and also the touchpad device can comprise a vibrotactile haptic layer for providing vibrotactile haptic sensations.

[0052] The term "comprising" whenever used in this document is intended to indicate the presence of stated features, integers, steps, components, but not to preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0053] The disclosure should not be seen in any way restricted to the embodiments described and a person with ordinary skill in the art will foresee many possibilities to modifications thereof. The above-described embodiments are combinable. The following claims further set out particular embodiments of the disclosure.

[0054] References

1. Chouvardas, V. G., Miliou, A. N., & Hatalis, M. K. (2008). Tactile displays: Overview and recent advances. Displays, 29(3), 185-194. https://doi.Org/10.1016/j.displa.2007.07.003

2. Hirai, S., & Miki, N. (2019). Thermal Sensation Display with Controllable Thermal Conductivity. 2019 20th International Conference on Solid-State Sensors, Actuators and Microsystems and Eurosensors XXXIII, TRANSDUCERS 2019 and EUROSENSORS XXXIII, 1659-1661. https://doi.org/10.1109/TRANSDUCERS.2019.8808369

3.KHOSHKAVA, V., CRUZ-HERNANDEZ, J. M., & SHAH, K. (2019). HAPTIC ACTUATOR ASSEMBLY WITH A SPRING PRE-LOAD DEVICE (Patent No. EP3582074A1).

4.0lley, M. F. D., Peshkin, M. A., & Colgate, J. E. (2020). Electronic controller haptic display with simultaneous sensing and actuation (Patent No. US10768749B2).

5. Sato, K. (2018). Simulating Texture Sensation of Textiles Using Thermal and Vibro- Tactile Stimulations. In Lecture Notes in Electrical Engineering (Vol. 432, pp. 115-120). Springer, Singapore, https://doi.org/10.1007/978-981-10-4157-0_20

Claims

C L A I M S A touchpad device comprising an interface with haptic and thermal feedback for a user in an automotive setting, comprising: a haptic-feedback layer; a capacitive touch sensitive layer; a thermal-feedback layer assembly; and an electronic data processor configured to: drive the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface. The touchpad device according to the previous claim comprising a frictional haptic- feedback layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations. The touchpad device according to any of the previous claims comprising a vibrotactile haptic layer for providing vibrotactile haptic sensations. The touchpad device according to any of the previous claims wherein the thermalfeedback layer assembly comprises a liquid metal layer and a titanium plate for providing a predetermined thermal conductivity of the thermal feedback layer assembly. The touchpad device according to the previous claim wherein the electronic data processor is further configured to drive the haptic-feedback layer to exhibit a tactile texture when the user touches the touchpad device interface. The touchpad device according to any of the claims 1-3 wherein the thermalfeedback layer assembly comprises a Peltier element for providing a predetermined heat flux to the thermal-feedback layer assembly. The touchpad device according to any of the previous claims wherein the electronic data processor is further configured to select a frictional or vibrotactile haptic feedback and to select a thermal conductivity or a heat flux, as a function of an image of a pattern or of an object to be displayed. A system for providing haptic and thermal feedback for a user in an automotive setting, the system comprising a display and a touchpad device according to any of the previous claims. The system according to the previous claim wherein the electronic data processor is further configured to display of an image of a pattern or of an object and select a frictional or vibrotactile haptic feedback and to select a thermal conductivity or a heat flux, as a function of the image of the pattern or the object displayed. Method of operating a touchpad device according to any of the claims 1-7, comprising using said electronic data processor for: driving the haptic-feedback layer and the thermal-feedback layer assembly to provide a haptic and thermal feedback as the user touches the touchpad device interface.