WO2023139417A1

WO2023139417A1 - Intelligent door system for autonomous vehicles

Info

Publication number: WO2023139417A1
Application number: PCT/IB2022/050712
Authority: WO
Inventors: Vera Patrícia CASTRO VILAS BOAS; Nélson Bruno MARTINS MARQUES COSTA; Eduarda Manuel DE BARROS PEREIRA; Helena Isabel LOPES MACEDO
Original assignee: Bosch Car Multimedia Portugal, S.A.; Universidade Do Minho
Priority date: 2022-01-19
Filing date: 2022-01-27
Publication date: 2023-07-27

Abstract

The present invention describes a haptic feedback system developed to provide equal use possibilities for users who may have comprise impairments or difficulties in hearing, sight, or other sensorial fields. The inclusive design based on multimodal interfaces aims to obtain solutions particularly for each of the identified impairment groups, but that can be used by everyone.

Description

DESCRIPTION " Intelligent Door System for Autonomous Vehicles"

Technical Field

The present invention describes a haptic feedback system developed to provide equal use possibilities for users who may have comprise impairments or di f ficulties in hearing, sight , or other sensorial fields .

Background art

Document KR20190123368 , entitled "Smart Car Control And Display System" , discloses a smart car control and display system . According to an embodiment , the smart car control and display system improves display ef ficiency in a smart car, and more conveniently uses entertainment information search and information transmission . Moreover, in a normal driving mode , a trend of a smart display and a user behavior analysis result are reflected on the system to improve smart car display system performance and ef ficiency .

Document US2018354411 , entitled "System And Method For Providing Enhanced Passenger Use Of An Autonomous Vehicle" , disclosed methods for managing a passenger ' s interaction with an autonomous vehicle . The method identi fies a passenger approaching an autonomous vehicle and activates a recognition signal that indicates pairing the passenger with the vehicle . As the passenger approaches , and entry point for the vehicle is identi fied . A recognition signal for the entry point is activated and the passenger is noti fied . As the exit point for the bride approaches , it is identi fied by the vehicle and in exit indicator is activated for the passenger . Any exit obstacles are identified by the vehicle and the passenger is noti fied . And exit indicator is activated for the passenger to provide noti fication of a safe to exit status from the autonomous vehicle .

Summary

The present invention describes a door System for Shared Autonomous Vehicles comprising a haptic feedback system connected to a navigation system comprised in the shared autonomous vehicle ; and an obstacle detection system; the door system being fitted in at least one door of a shared autonomous vehicle , located in an accessible positioning to a user ; wherein the haptic feedback system comprises a haptic surface configured to provide warnings ; and wherein the obstacle detection system comprises at least one of a static or dynamic obstacle detection sensor .

In a proposed embodiment of present invention, the warnings comprise at least one of a sti f fening and/or vibration of a door structure or part , and/or an audio message and/or a visual message displayed in a door structure or part .

Yet in another proposed embodiment of present invention, the door part comprises at least one of a handle .

Yet in another proposed embodiment of present invention, the warnings are triggered i f a user obj ect is left inside of the shared autonomous vehicle .

Yet in another proposed embodiment of present invention, the obstacle detection system is configured to prevent the door, in an opening and/or closing procedure , from hitting obstacles located in the surroundings of the shared autonomous vehicle .

Yet in another proposed embodiment of present invention, the door system is configured to open the door only when the shared autonomous vehicle is stopped, and no obstacles are detected .

Yet in another proposed embodiment of present invention, the door system is configured to automatically open the door of the shared autonomous vehicle i f an emergency occurs .

Yet in another proposed embodiment of present invention, the door system is configured to exchange data and warnings with a smartphone and/or an adequate device .

Yet in another proposed embodiment of present invention, the door system is configured to interact with users , particularly with visually impaired users .

General Description

Autonomous vehicles (AV ' s ) represent an increasingly expected reality, and within this vision of the future , the commuters stuck in soul-crushing traf fic will be freed from the drudgery of driving . One of the great promises of autonomous vehicles is to give back time by freeing the driver from driving . There is also the expectation that passengers will be able to make the most of their travel time to increase their productivity by carrying out work tasks like checking the e-mail , having meetings , or even enj oying their trip with maximum comfort and taking advantage of it for reading the morning paper or a book, use their cellphone or tablet , watch a movie , relax or to perform any other leisure activity like appreciating the view outside of the window .

Multimodal interfaces that combine visual , auditory, and haptic interaction are motivated by recent research showing that Human-Machine Interface (HMI ) concepts that issue nonvisual alerts in addition to the visual ones are the most ef fective for the proposed vehicles . Some operators exhibited a primary task reversal and prioriti zed the completion of a non-driving task over a Take-over Request ( TOR) . In these cases , the HMI concepts that included haptic and/or audio cues were more success ful in catching the attention of the operators when they were engaged in a visually demanding task . These concepts also proved to be more ef fective in obtaining a safe response when compared to those that only provide visual feedback .

In this sense , haptic technology is promising for the development of a Universal product , capable of including the needs of di f ferent populations and simultaneously promoting a good experience for users in the context of 3rd living space .

Haptic technologies are very powerful , providing a multitude of functionalities , from spatial , warning, communication and information to even interaction and aesthetical . These functionalities help enhance their use and the experience of their products in dif ferent contexts . Common examples can be seen in gaming, where they improve the sensation of immersion, and in mobile applications , where they reinforce communication . Moreover, these technologies are particularly valuable for the visually impaired, as it compensates their damaged or weakened sense by strengthening their touch . When it comes to those who use public transportation and have to rely on the driver to get orientations when leaving the vehicle or at the moment of its arrival , these technologies may have a greater impact on self-driving cars since they will bridge the communication between the passenger and the vehicle .

The inclusion of haptic technologies in our lives has j ust begun, having a wide application in many areas ( education, entertainment , art , medical , etc . ) bringing with it several benefits accompanied by a drastic change in the communication field . It is expected that , in a near future , this kind of technology would help enhance the shopping experience in e- commerce , providing more li felike interaction with virtual obj ects and virtual environments , also become part of the personal normal computer interaction . Haptic feedback is also advantageous for multisensory experiences , increasing and improving di f ferent domains such as accessibility, unobtrusive feedback, motor skill training, support for safety, and social communication . Recently, high- fidelity haptic technology has expanded the available range of user experience , improving support for emotional therapy, education, and entertainment .

Considering the above , the goal of the present disclosed invention is to create a solution to provide a great experience in the context of 3rd living space in the particular case of shared autonomous vehicles . The solution is based on an intelligent door system that will enhance the safety of the passengers , their trust and user experience , said door being fitted in the vehicle door of SAV .

The door should also include static or dynamic obstacle detection sensors , that prevent the door from hitting unseen obstacles or bicyclists for example . It should be connected to the vehicle navigation system, thus the door only opens when the car is stopped, unless an emergency occurs , then the door opens immediately . Another innovative concept is the activation of a haptic warning through a smart surface in the door, that will be sti f fening when the passenger tries to leave the car but forgot some obj ect inside . Bes ides the haptic warning, an audio message and a visual icon will be communicated to the passenger . This feature implies the development and the integration of an algorithm for left behind obj ects . Beyond the left behind obj ect detection system, the developed door system resorts to the use of a set of innovative components among which it is possible to provide an intelligent lock/unlock system with the additional integration of a warning sign of the vehicle arrival , of fering convenience , comfort and secure user experience to the vehicle passenger .

Brief description of the drawings

For better understanding of the present application, figures representing preferred embodiments are herein attached which, however, are not intended to limit the technique disclosed herein . Figure 1 - illustrates a possible embodiment of the proposed invention, wherein the shared autonomous vehicle is comprised of a least one door which, that , on the other hand, comprises a door system .

Figure 2 - illustrates a possible embodiment of the proposed invention, where the shared autonomous vehicle ( from an inside perspective ) comprises a door with the haptic feedback system providing visual and sound noti fications .

Figure 3 - illustrates a possible embodiment of the proposed invention, where the shared autonomous vehicle ( from an outside perspective ) comprises the haptic feedback system providing visual and sound noti fications .

Description of Embodiments

With reference to the figures , some embodiments are now described in more detail , which are however not intended to limit the scope of the present application .

The present invention describes a universal intelligent door system with multimodal solutions ( haptic, visual and audio ) conceptuali zed to guarantee good user experience , for all users including the visually impaired, that currently face many challenges during a simple vehicle ride .

The developed system is meant to be fitted in the vehicle door of Shared Automated Vehicles ( SAV) . The system comprises at least a set of static or dynamic obstacle detection sensors , that prevent the door from hitting unseen obstacles or two wheel or plus vehicles .

The system is developed to be connected to the vehicle navigation system, thus the door will only open when the SAV is completely stopped, unless an emergency occurs , that will promote the opening of the door in an immediate action .

Another present feature in the developed system is the inclusion of the activation of a haptic warning through an insertion on a smart surface in the door, that will provide sti f fening of the material when the passenger tries to leave the vehicle but forgot some sort of obj ect inside . The sti f fening of a door part , in particular, of a haptic surface comprised in the haptic feedback system of the door system, can comprise one at least one of an electromagnet apparatus , providing the haptic surface or handle with resi stance to the movement for a few seconds ; a fast deploy air pocket , providing the haptic surface or handle with an increasing awkward diameter feel and di f ficult to operate ; and/or a combination of both approaches . Besides the haptic warning, an audio message and a visual icon will be communicated to the passenger . This feature implies the development and the integration of an algorithm for le ft behind obj ects . In one of the proposed embodiments for this approach, the algorithm comprises the use of computer vision that compares the inside view image of the vehicle before and after the passenger usage in a "top down" dome perspective . The developed algorithm is able to identi fy items in the vehicle , seats or floor included, and establish a catalogue/ l ist for comparison, identi fying i f an obj ect was previously deposited on the site before the passenger usage . Additionally to the door system integration with the navigation system of the autonomous vehicle, the integration with smartphone APPs, or other adequate device, is also to be ensured. For example, and for the visually impaired it is mandatory to conceptualize a method that allows the blind population to easily recognize an autonomous vehicle's arrival. Thus, when a passenger calls for a SAV through, for example, a smartphone application, after the ride confirmation, the user will receive an audible notification that will be replicated by the intelligent door system, at the moment of the vehicle's arrival.

It is assumed that the intelligent door system is equipped with adequate technology required to respond and perform the above-mentioned features' needs. Since the vehicle is autonomous, it is also assumed that the door system is connected to the vehicle navigation system, or other adequate apparatus or device, which is capable to detect emergency occurrences, and allowing the system to respond accordingly.

The door was developed to fulfil the passengers' expectations with regard to safety, user experience, trust and aesthetic. The door system is configured to ensure the safety nor only of the SAV and remaining surrounding objects or vehicles, but also of the occupants of said vehicle, identifying static and dynamic moving obstacles through a detection system incorporated therein. The disclosed door system is also provided with an intelligent lock/unlock system and is configured to be emergency responsive. The obstacle detection system analyses the external environment of the car, such as static or moving obstacles. If an obstacle is detected by the system (obstacle detection sensors + obstacle detection algorithm) , the door will be stiffening, and the passenger will be blocked inside the car temporarily until to be safe to leave the AV. The AV should inform the passenger whenever he/she tries to open the door, informing that there's an obstacle (sidewalk, object...) or that a collision with another road user approaching is very likely (detection or prediction) . The AV should inform the passenger by stiffening the door or the door handle. The passenger should be able to override the AV behaviour at any time. An audio or visual icon should complement the haptic feedback to make it clear.

To ensure the user experience, and provide an adequate feeling to the visually impaired users, the door system comprises a smart surface technology, providing physical or remote connection with a smartphone app or other device properly configured and suitable for the purpose, and ensures also the triggering of multimodal warnings.

The functional surface will provide a programmable haptic effect to generate expected interactions which will consider the interrelationship of four factors: (1) human tactile/haptic perception; (2) type of actuators available for use; (3) type of surface for the embedded actuators; (4) the context of use.

The use of the door as a functional surface for tactile interaction which is particularly suitable for critical safety events. 'Dooring' is an increasingly important event in the interaction between vehicles and Vulnerable Road Users (VRUs) . Therefore, this surface, precisely the interior door handle, will be the basis for haptic stimulation. On the handles, one can use tactile actuator to transmit vibration to the vehicle occupant's fingers or palms and force feedback strategies to indicate danger when opening the door, such as sti f fening the handle . The strategy is to indicate unsafe events through vibrating actuators , paired with visual and audio stimuli to reinforce the information context .

The perception of the vibration frequency should be possible even with the use of gloves but should not be high enough to become annoying or af fright the passenger . The peak frequency for discriminating vibration is 50Hz .

Claims

1 . Door System for Shared Autonomous Vehicles comprising a haptic feedback system connected to a navigation system comprised in the shared autonomous vehicle ; and an obstacle detection system; the door system being fitted in at least one door of a shared autonomous vehicle , located in an accessible positioning to a user ; wherein the haptic feedback system comprises a haptic surface configured to provide warnings ; and wherein the obstacle detection system comprises at least one of a static or dynamic obstacle detection sensor .

2 . Door system according to the previous claim, wherein the warnings comprise at least one of a sti f fening and/or vibration of a door structure or part , and/or an audio message and/or a visual message displayed in a door structure or part .

3 . Door system according to any of the previous claims , wherein the door part comprises at least one of a handle .

4 . Door system according to any of the previous claims , wherein the warnings are triggered i f a user obj ect is left inside of the shared autonomous vehicle .

5 . Door system according to any of the previous claims , wherein the obstacle detection system is configured to prevent the door , in an opening and/or closing procedure , from hitting obstacles located in the surroundings of the shared autonomous vehicle .

6. Door system according to any of the previous claims, characterized by being configured to open the door only when the shared autonomous vehicle is stopped, and no obstacles are detected.

7. Door system according to any of the previous claims, characterized by being configured to automatically open the door of the shared autonomous vehicle if an emergency occurs.

8. Door system according to any of the previous claims, characterized by being configured to exchange data and warnings with a smartphone and/or an adequate device.

9. Door system according to any of the previous claims, characterized by being configured to interact with users, particularly with visually impaired users.