EP4457585A1

EP4457585A1 - Interface and alert device with haptic and thermal feedback for autonomous vehicle

Info

Publication number: EP4457585A1
Application number: EP21854691.9A
Authority: EP
Inventors: Pedro Miguel DE SOUSA BERNARDO; Vera Patricia CASTRO VILAS BOAS; Nélson Bruno MARTINS MARQUES COSTA; Manuel Sebastião MARQUES COUTINHO; Adriano Dídimo MACHADO CARVALHO; Rosane SCHONBLUM SAMPAIO; Marina MACHADO; Lisandra Maria ARAÚJO TEIXEIRA; Emanuel TEIXEIRA SILVA; José Pedro CORREIA DIAS
Original assignee: Universidade do Minho; Bosch Car Multimedia Portugal SA
Current assignee: Universidade do Minho; Bosch Car Multimedia Portugal SA
Priority date: 2021-12-27
Filing date: 2021-12-28
Publication date: 2024-11-06
Also published as: WO2023126645A1

Abstract

A device for providing a touchpad surface with haptic feedback for a user in an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: drive the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback as the user touches the touchpad surface. System thereof further comprising an armrest and method operation thereof.

Description

D E S C R I P T I O N

INTERFACE AND ALERT DEVICE WITH HAPTIC AND THERMAL FEEDBACK FOR AUTONOMOUS VEH ICLE

TECHNICAL FIELD

[0001] The present disclosure relates to an interface and alert device with haptic and thermal feedback for autonomous vehicle.

BACKGROUND

[0002] Document US10899258 discloses an armrest arm having an upper support surface forming a support for the forearm of an occupant of the vehicle seat, and a user interface having at least one display portion of flattened shape, having a front face comprising a screen and a rear face formed by a shell. The user interface is mounted on the armrest arm so as to be movable between a retracted position where the screen is inaccessible and an extended position where the screen is accessible. In the retracted position of the user interface, the shell is integrated with the upper support surface of the armrest arm, and in the extended position of the user interface, the screen is at least partially projecting above the upper support surface of the armrest arm.

[0003] Document US20170308224 discloses a touch control armrest sleeve which includes a body and a connecting portion. The body includes a flexible pad, a flexible touch panel and a protective cover. The connecting portion is connected to a first end and a second end of the body. The flexible touch panel includes a flexible substrate and a carbon nanotube touch function layer. A touch control seat incorporating such touch control armrest sleeve is also provided.

[0004] Document US8659554 discloses a moveable touch pad that opens out of a centre console armrest and is operable to control a navigation unit or other vehicle functions. Advantageously, locating a control touch pad unit in the centre console armrest provides an ergonomically desirable solution for the vehicle's driver. The touch pad can be stored in the vehicle's centre console armrest, and the touch pad is configured to move fore and aft for comfortable driver access. [0005] Document US20200264772 discloses a method for providing haptic feedback to an operator of a touch-sensitive display device. The method includes providing an operator interface to be represented in the display device as image data; the operator interface having at least one button assigned to a function to be controlled, graphically displayed in the operator interface, and graphically delimited from the rest of the operator interface; analysing the image data for a presence and a position of shapes for representing a button; determining an intended display region of a shape, which is to be represented and is identified as a button; representing the image data of the operator interface in the touch-sensitive display device; outputting haptic feedback in response to detection of contact with the touch-sensitive display device in the region of a surface of the touch-sensitive display device which is assigned to a display region of a shape identified as a button.

[0006] These facts are disclosed in order to illustrate the technical problem addressed by the present disclosure.

GENERAL DESCRIPTION

[0007] The present disclosure relates to an interface and alert device with haptic feedback for autonomous vehicle, comprising a smart surface positioned on the seat's armrest.

[0008] According to Basdogan et all. (2020), interactive surfaces with incorporated haptic feedback are a new area of interest in the field of haptic feedback.

[0009] The goal of integrating haptics into touch surfaces is to generate this tactile feedback sensation on other functional surfaces making it more realistic and turning them into new applications in user interface design such as online shopping, entertainment, gaming, education and more.

[0010] Currently haptic feedback is only being used in touchscreens such as mobile phones, tablets, and other electronics. According to the literature (Basdogan et al., 2020), it is extremely challenging to design this type of technology and to implement it successfully on a surface that it is not a touchscreen whilst maintaining protection against dust or water. [0011] On the current applications of haptic feedback (mobile phones, etc.) there is still a long way to improve and to get the attention to the topic as visual and auditory feedback are still prominent in the market although it is also known that tactile feedback improves task performance and its realism.

[0012] Recent studies have shown many different types of actuators for successfully generating tactile stimuli on touch surfaces. The most known approaches are the following:

Electromagnetic actuators, which generate force which is proportional to a current; Vibration motors, e.g., a DC motor with a rotor with eccentric mass distribution which creates rotational movement perceived as vibration by user;

Voice coil actuators, which produce any waveform with fast response;

Piezoelectric actuators, which are suitable for actuation of touch surface at high frequencies and generates high forces with a low displacement;

Electroactive polymers, which can change their shape in volume when electrically stimulated;

Electrostatic actuators, which produce a force that is proportional to the square of the voltage and can be generated on the finger's skin.

[0013] These tactile sensations are considered to be a significant factor in the preference and positive attitude towards consumers in consequence of perceiving their surface texture.

[0014] The smart surface comprises an insulative layer, fitted atop a transparent and conductive electrode sheet layer, which, when charged with an electric current and when in contact with the human skin, produces a frictional stimulus by controlling the electric current. This layer, in turn, is fitted atop a capacitive-based touch sensitive layer, fitted atop a layer of LED lights, which in turn are fitted atop a layer of vibrotactile actuators. The layer of LEDs is optional and when not present the touch-sensitive layer is fitted atop a layer of vibrotactile actuators.

[0015] Wherein the vehicle seat's armrest possesses a portion of its surface functions as a capacitive touchpad surface, which can provide users, using their fingers or hands, with haptic feedback sensations, through electrovibration and/or vibrotactile actuation. This smart surface is connected to the vehicle's infotainment system, and receives commands from it regarding when to engage and disengage the different actuation layers and LED lights of the LED layer. However, the perception of haptic sensations generated by the haptic actuators might be affected by conditions related to the user's hands. For example, the use of gloves can completely negate electrovibration actuation, as the skin of the finger is required to complete the circuit with the insulating layer and the conductive layer of the structure, as well as severely attenuate or even completely mask the vibrations generated by the electrovibration actuators. High finger humidity, in turn, can also negatively impair or outright negate the sensations generated by both electrovibration and vibrotactile actuators.

[0016] This armrest smart surface is in communication (wired or wireless) with the vehicle's infotainment system that is visually displayed on the centre stack display's user interface, functioning as an extension of it, allowing users to interact with the display interface through this surface portion.

[0017] This armrest smart surface is also connected with the vehicle's own electronic system, by wire (e.g., ethernet) or wireless (e.g., Wi-Fi), allowing users to control several of the vehicle's functions, such as window, door, and media volume control through it. [0018] In an embodiment, it is capable of delivering warning information to the users while they are in direct contact with the surface, e.g., grabbing the user's attention through vibration warnings of alerts/messages/events. These warnings can also be accompanied by visual and/or auditory cues, to increase the probability that users that have access to either or both of these cues will receive the intended information, as one cannot ensure that users will always have their hands placed on top of the smart surface area of the armrest.

[0019] In an embodiment, users can also select an option to enable the media content being played in the vehicle to be accompanied alongside vibrations, e.g., vibrotactile stimuli delivered in tune with the music currently being played (for example, tune's spectrogram is acquired through a software included on the vehicle's infotainment system, and vibrations are generated based on the said spectrogram data); or delivered during certain pre-selected moments during a movie clip.

[0020] In an embodiment, the smart surface is to be employed as the interactive surface interface of a device to which it is not physically connected (e.g., touch surface interface communicating wirelessly with a display device in close proximity, namely, the vehicle's centre stack display). This smart surface is embedded on a portion of the armrest's upper side and is connected through the internal wiring of the vehicle, deriving power from it, while also being connected to the vehicle's infotainment system.

[0021] The capacitive-based touch sensitive layer can be made of a variety of materials, depending on the intended use case and aesthetic appeal.

[0022] In an embodiment, the display device's capacitive-based touch sensitive layer is made of a transparent material, such as transparent glass.

[0023] In an embodiment, the interactive surface interface is made of translucent plastic with illumination provided by LED lights placed on the LED lights layer, located between the capacitive-based touch sensitive layer and the vibrotactile actuators layer. [0024] In the present disclosure, the smart surface can generate both frictional and vibrotactile haptic sensations. These can be used, for example, to let users know about the placement of User Interface elements thought the sense of touch while sliding one or more fingers, or the entire hand, on top of the armrest's smart surface (for example, when the location of a toggle button or switch is reached, the texture sensation of said button/switch can be different depending on its state, such as transmitting a rougher texture feeling when toggled off, and a smoother texture feeling when toggled on), or to transmit information to users regarding certain warnings and information (for example, the user has their hand resting on the armrest's smart surface, and vibrations generated by vibrotactile actuators are transmitted to the hand to let them know they have arrived at their destination). Both these haptic sensations can also be used in tandem to provide the same information to users regarding certain warnings and interactions, such as, for example, increasing the amount of vibration provided, and the roughness of the texture feel that users perceive, as they slide a slider of the user interface.

[0025] Frictional haptic sensations, also described as electrovibration, are generated thanks to the insulative and conductive layers, which allow for friction between the smart surface and the user's skin to be modulated through electrostatic actuation, while the skin is in motion. [0026] Vibrotactile haptic sensations, in turn, are generated thanks to the vibrotactile actuators, which can generate vibrations strong enough to be felt through the user's skin, either stationary or in motion, when in direct contact with the smart surface.

[0027] When the smart surface is used as the interactive interface for an external device, haptic feedback sensations, caused by the external device's software, can be generated on it.

[0028] The haptic actuation technology that is present on the smart surface is both capable of interpreting the software's information and of generating such sensations. When the external device's software, for example, the in-vehicle's infotainment system software, sends a command to the smart surface, requesting that it generates haptic and/or thermal feedback sensations on the surface layer, the command technology/software found on the smart layer must be capable of differentiating between these types of actuations, delegating the request to the actuation technology that controls the actuators capable of generating said feedback.

[0029] In an embodiment, the smart surface is equipped on a portion of the surface of the vehicle seat's armrest, where it acts as an external interactive surface interface for the centre stack display device, as well as to control several of the vehicle functions, such as window, door, and media volume control. A specific example includes a 7" display, 7 mm thick, powered through a socket provided by the vehicle specifically for the device, in communication with the vehicle via ethernet, and with user devices via Wi-Fi.

[0030] It is disclosed a device for providing a touchpad surface with haptic feedback for a user in an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: drive the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback as the user touches the touchpad surface.

[0031] In an embodiment, the device further comprises a display for displaying images of patterns or objects, wherein the haptic surface is superimposed on the display.

[0032] In an embodiment, the automotive setting is an autonomous vehicle and the display is a centre stack display of the autonomous vehicle.

[0033] In an embodiment, the electronic data processor is further configured to drive the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback of a vehicular warning.

[0034] In an embodiment, the device comprises a display for displaying images of patterns or objects wherein the electronic data processor is further configured to show in said display a visual alert to provide visual feedback of a vehicular warning.

[0035] It is also disclosed a system for providing a touchpad surface with haptic feedback for a user in an automotive setting comprising a vehicular armrest and a device according to any of the previous claims, wherein the device is arranged in said armrest. [0036] In an embodiment, the touchpad surface is configured to act as an external input device of a centre stack display of an autonomous vehicle.

[0037] In an embodiment, the touchpad surface is configured to act as both an external input device and output device of the centre stack display of an autonomous vehicle.

[0038] It is also disclosed a method of operating a device according to any of the embodiments wherein the method comprises driving the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback as the user touches the touchpad surface.

BRI EF DESCRI PTION OF THE DRAWI NGS

[0039] The following figures provide preferred embodiments for illustrating the disclosure and should not be seen as limiting the scope of invention.

[0040] Figure 1: Schematic representation of an embodiment of a capacitive touch and haptic surface on armrest.

[0041] Figure 2: Schematic representation of layers superposition an embodiment of a capacitive haptic surface on armrest. [0042] Figure 3: Schematic representation of an embodiment of a capacitive touch and haptic surface on armrest.

DETAILED DESCRI PTION

[0043] Figure 1 shows a schematic representation of an embodiment of a capacitive haptic surface on armrest passenger, 12 represents the seat, 13 represents the armrest with the capacitive, haptic surface, and 14 represents the centre stack display that the capacitive, haptic surface of the armrest is in contact with.

[0044] Figure 2 shows a scheme of the layer's superposition, wherein A represents an insulative layer; B represents a capacitive-based touch sensitive layer; C represents a vibrotactile actuators layer; D represents a transparent electrode sheet layer and E represents a LED lights layer.

[0045] As illustrated in Figure 3, a device for providing a touchpad surface with haptic feedback for a user in an automotive setting, where 12 represents the seat, 13 represents the armrest with the capacitive, haptic surface, and 14 represents the centre stack display that the capacitive, haptic surface of the armrest is in contact with, comprising according to an embodiment: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer, a frictional haptic layer comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer for providing vibrotactile haptic sensations; an electronic data processor configured to: drive the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback as the user touches the touchpad surface.

[0046] According to the technical field literature, examples of textures that participants subjectively report feeling during interaction with electrode devices include: wood, leather, paper, painted wall, rubber, and viscous liquid (Bau, Olivier, and Ivan Poupyrev. "REVEL: tactile feedback technology for augmented reality." ACM Transactions on Graphics (TOG) 31.4 (2012): 1-11). [0047] Through the use of an electrode, it is possible to modify the friction that exists between the finger and an actuated surface, when there is movement of one or both parts. Through friction, we can manipulate, for example, the feeling of how smooth or rough a surface is. This, however, is normally limited in two respects, namely how smooth the surface on which the finger slides is (because electrode only creates friction, does not remove it, so the least possible friction to create is the natural friction between the surface and the finger); and, also, how much electrical charge can be given to the surface actuated (the higher the electrical charge, the greater the friction generated between the finger and the surface), up to a certain limit, which is depending on the structure of the device itself and its electric system.

[0048] The term "comprising" whenever used in this document is intended to indicate the presence of stated features, integers, steps, components, but not to preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

[0049] The disclosure should not be seen in any way restricted to the embodiments described and a person with ordinary skill in the art will foresee many possibilities to modifications thereof. The above-described embodiments are combinable. The following claims further set out particular embodiments of the disclosure.

[0050] References

Basdogan, C., Giraud, F., Levesque, V., & Choi, S. (2020). A Review of Surface Haptics: Enabling Tactile Effects on Touch Surfaces. IEEE transactions on haptics, 13(3), 450-470. DOI: 10.1109/TOH.2020.2990712

Claims

C L A I M S A device for providing a touchpad surface with haptic feedback for a user in an automotive setting, comprising: a haptic surface comprising actuators for exhibiting tactile textures, comprising: a capacitive-based touch sensitive layer (B), a frictional haptic layer (D) comprising an insulative sheet for user touch and a transparent conductive electrode film for providing frictional haptic sensations, and a vibrotactile haptic layer (C) for providing vibrotactile haptic sensations; an electronic data processor configured to: drive the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback as the user touches the touchpad surface. The device according to the previous claim further comprising a display for displaying images of patterns or objects, wherein the haptic surface is superimposed on the display. The device according to the previous claim wherein the automotive setting is an autonomous vehicle and the display is a centre stack display (14) of the autonomous vehicle. The device according to any of the previous claims wherein the electronic data processor is further configured to drive the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback of a vehicular warning. The device according to any of the previous claims comprising a display for displaying images of patterns or objects wherein the electronic data processor is further configured to show in said display a visual alert to provide visual feedback of a vehicular warning. The device according to any of the previous claims wherein said textures comprise a texture of wood, leather, paper, painted wall, rubber, viscous liquid, or combinations thereof. A system for providing a touchpad surface with haptic feedback for a user in an automotive setting comprising a vehicular armrest (13) and a device according to any of the previous claims, wherein the device is arranged in said armrest (13). A system according to the previous claim wherein the touchpad surface is configured to act as an external input device of a centre stack display (14) of an autonomous vehicle. A system according to the previous claim wherein the touchpad surface is configured to act as both an external input device and output device of the centre stack display (14) of an autonomous vehicle. Method of operating a device according to any of the claims 1-6 wherein the method comprises driving the frictional haptic layer and vibrotactile haptic layer to provide haptic feedback as the user touches the touchpad surface.