KR20170044731A - Integrated wearable article for interactive vehicle control system - Google Patents

Abstract

A method of operating a vehicle includes receiving from a user a first input indicative of a vehicle function to be performed in the vehicle in a wearable article, receiving a second input indicative of a gesture by a user for authentication, And generating a control signal for performing the vehicle function in the vehicle based on the successful authentication.

Description

[0001] INTEGRATED WEARABLE ARTICLE FOR INTERACTIVE VEHICLE CONTROL SYSTEM [0002]

The present invention relates to vehicles, and more particularly to systems and methods for this. An electronic key (key fob) allows the driver to perform remote control functions such as locking or starting the vehicle. However, the driver must carry the electronic key everywhere, for example in a key chain, pocket or purse. Recent developments in wearable technology allow people to interact with the vehicle through wearable articles such as smart watches or wristbands.

The present invention relates to an integrated wearable article for an interactive vehicle control system. In one embodiment, the system may include a user input subsystem and a user recognition and authentication subsystem in communication with the user input subsystem. The user input subsystem includes a wearable article and is configured to receive input from a user. The user recognition and authentication subsystem is configured to detect and authenticate a user based on inputs received from the wearable article or vehicle, or both. The inputs received from the wearable article may include, for example, a user input representing a vehicle function to be performed, a gesture input by a user for authentication, or both. The inputs received from the vehicle may include, for example, a gesture input by the user for authentication. The wearable article may comprise a wearable computing device configured to perform at least one vehicle function in the vehicle. The wearable article may be, for example, a smart watch, a smart clothing article, a transdermal chip or a wearable sensor. A driver score may be generated for a driver action associated with vehicle functions performed by the user in the vehicle. The driver's score may be sent to the wearable article, for example uploaded to the home computer or an external database via the cloud.

In yet another embodiment, a method of operating a vehicle includes receiving from a user a first input indicative of a vehicle function to be performed in the vehicle, in a wearable article, receiving a second input indicative of a gesture by a user for authentication And generating a control signal for performing a vehicle function in the vehicle based on successful authentication of the user in the wearable article. The wearable article may be, for example, a smart watch, a smart clothing article, a percutaneous chip or a wearable sensor. Driver scores may be generated for driver behavior associated with vehicle functions performed by the user in the vehicle. The driver's score may be sent to the wearable article, for example uploaded to the home computer or an external database via the cloud.

In yet another embodiment, the wearable article may comprise at least one processor and a memory. The memory stores data and program instructions that can be executed by the processors. The wearable article may be a wearable computing device such as a smart watch or wearable sensor. Processors are configured to execute instructions stored in memory. The instructions include receiving from a user a first input indicative of a vehicle function to be performed in the vehicle, receiving a second input indicative of a gesture by the user for authentication, and performing a vehicle function based on the user's successful authentication And generating a control signal for the control signal. The wearable article may be, for example, a smart watch, a smart clothing article, a percutaneous chip or a wearable sensor. Driver scores may be generated for driver behavior associated with vehicle functions performed by the user in the vehicle. The driver's score may be sent to the wearable article, for example uploaded to the home computer or an external database via the cloud.

The foregoing features and elements may be combined in various combinations without limitation, unless expressly stated otherwise. These features and elements and their operation will become more apparent in light of the following description and accompanying drawings. It is to be understood, however, that the following description and drawings are intended to be illustrative in nature and not restrictive.

Those skilled in the art will recognize various features from the following detailed description of the disclosed non-limiting embodiments. The drawings accompanying the detailed description can be explained as follows:
1 is a schematic view of an exemplary vehicle for use with an interactive vehicle window display system;
2 is a schematic block diagram of an interactive vehicle window display system according to one non-limiting embodiment;
Figure 3 is a partial interior view of a vehicle having an interactive vehicle window display system;
4 is a top view of a vehicle illustrating an external user identification subsystem of an interactive vehicle window display system;
5 is a schematic view of a vehicle illustrating user identification through a wearable article, a skeletal joint relationship, an electronic key and / or a user gesture;
6 is a schematic block diagram of an algorithm for operation of a system according to one non-limiting embodiment;
Figure 7 is a schematic illustration of an exemplary osteoarticular relationship recognizable by the system;
FIG. 8 illustrates an exemplary user gesture and exemplary wearable article recognizable by the system according to one non-limiting embodiment; FIG.
9 is an exemplary landing page displayed by an interactive vehicle window display system;
10 shows an exemplary route page displayed by an interactive vehicle window display system;
11 shows an exemplary calendar page displayed by an interactive vehicle window display system;
12 illustrates an exemplary weather page displayed by an interactive vehicle window display system;
13 is an exemplary vehicle status page displayed by an interactive vehicle window display system;
Figure 14 is an exemplary to-do page displayed by an interactive vehicle window display system;
Figure 15 is a partial interior view of an interior space of a vehicle illustrating an interactive environment in which the driver and / or occupant utilizes the functions of the vehicle head unit;
Figure 16 is a partial interior view of a vehicle interior space illustrating the discrimination of the driver and / or occupant to selectively allow use of the function of the vehicle head unit during vehicle operation;
17 is a schematic diagram of a vehicle occupant facial map used with a system to track an occupant location;
18 is an overhead internal view of a vehicle illustrating a sensor arrangement for tracking the position of a passenger in a vehicle interior space;
19 is a schematic block diagram of a wearable article according to one non-limiting embodiment;
20A illustrates an exemplary wearable article;
Figure 20B illustrates another exemplary wearable article;
Figure 20C illustrates another exemplary wearable article;
21 is a flow diagram of a process performed by a wearable article according to one non-limiting embodiment;
Figures 22A-22C illustrate an exemplary screen display of an exemplary wearable article according to one embodiment;
Figures 23A-23F illustrate an exemplary screen of the remote control mode of the exemplary wearable article of Figures 22A-22C;
24A is an exemplary screen for displaying a notification of the vehicle function performed in the vehicle;
Figure 24B is an exemplary driver score notification screen for the exemplary wearable article of Figures 22A-22C; And
25A-25C are exemplary screens of panic mode for the exemplary wearable article of FIGS. 22A-22C.

Wearable technology has evolved over the years to allow people to interact with smart appliances or vehicles. The wearable article can be integrated with the electronic key for operating the vehicle, so that the user will no longer need to carry the electronic key separately. For example, with the digital electronic key features integrated into the SmartWatch, users can wear a smart watch without having to worry about finding a key each morning. The user will no longer need to carry the electronic key separately. In addition to being used as a watch or wristband, the smart watch can be worn by a user to perform vehicle functions in the vehicle.

In some cases, the wearable article may be implemented to receive a user indication of vehicle functions performed in the vehicle, and a gesture input for user authentication to enable such vehicle functions to be performed. A control signal for performing vehicle functions may be generated based on successful authentication. In some implementations (e.g., multi-factor authentication), the gesture input may be used for multi-factor authentication in conjunction with user input received from the wearable article to authenticate the user. A second gesture input may be detected by the sensor, which may be coupled to a vehicle or wearable article. The sensor can be used to detect if the wearer is wearing a wearable article. If the user is not wearing a wearable article, access to some or all of the vehicle functions may not be allowed.

A wearable article can be used to sense information from the user, such as biometrics and driving information. In some implementations, the driver's score may be generated for driver behavior performed in the vehicle. The driver's score helps the driver improve his driving skills and can notify the driver if there is a performance degradation. For example, the driver's score may be a selectable option from the menu screen of the wearable article and may be calculated based on sensed information of driver behavior. When the driver removes the key from the vehicle, the driver's score may be sent to the wearable article for display, storage or further processing. Driver scores received on the wearable article can be uploaded to an external database, for example, via a personal computer, smart phone or cloud. Data analysis can be used to improve safe driving techniques or for other application-specific purposes. For example, the driver's score can be placed in a social ranking system, and the user can review and evaluate his driving performance against others in the social ranking system.

1 schematically illustrates a vehicle 20 having a window 22 and an interactive vehicle window display system 30. It should be appreciated that while window 22 is shown here as a driver side occupant window of a minivan type vehicle in the disclosed non-limiting embodiment, various vehicle types and windows would also benefit from it.

Referring to FIG. 2, selected portions of the system 30 are schematically illustrated. The system 30 generally includes an interactive display subsystem 32, a control subsystem 34, a user input subsystem 36, a user identification subsystem 38, and a user location subsystem 39. In some embodiments, the user identification subsystem 38 may be implemented as a user identification and authentication subsystem. It should be understood that although certain subsystems are separately defined, each or any one of the subsystems may be combined or separated through hardware and / or software of the system 30. [ Additionally, each or any one of the subsystems may be implemented using one or more computing devices, including conventional central processing units or other devices capable of handling or processing information.

The interactive display subsystem 32 may include any device or devices capable of displaying an image on the vehicle window 22 under the control of the system 30 and may be external to the vehicle, And can be configured to be visually displayed on all of them. In a non-limiting example, the interactive display subsystem 32 may include a display device integral with the window 22, such as an LCD. Such a display may be illuminated by one or more light sources under control of the system 30 or by ambient light. Such light sources may be mounted at any of the operating positions that allow light emission from the interior or exterior of the vehicle onto the window, depending on whether the user is visible outside or inside the vehicle. Examples of such mounting locations may be included on the floor, on the vehicle headliner, in the vehicle door structure, or on the exterior door panel.

In another non-limiting example, the interactive display subsystem 32 may include a coating 40 and a projector 42. The coating 40 can be, for example, a polymer dispersed liquid crystal (PDLC) film applied to the window 22 to provide transparency when inactive and partially or fully opaque if active. Thereby, the window 22 treated with the coating 40 is operable to display the content as a projection page visually displayed outside and / or inside the vehicle 20 (FIG. 1). The projector 42 may be mounted at locations on the floor (FIG. 3), or other locations on the vehicle 20, such as a vehicle headliner, or in a vehicle door structure, and external to the vehicle, such as an exterior door panel. An exemplary shaded area extending from the projector 42 toward the window 22 schematically represents the projection of the output in the form of a content page provided by the projector 42. In response to the recognized user's access, the coating 40 may change from transparent to opaque so that the projector 42 projects the output onto the window 22. [

As will be further described, the displayed content may be personalized information or entertainment content, such as video, games, maps, maps, etc., whether the information is generated onboard and / Navigation, vehicle diagnostics, calendar information, weather information, vehicle climate control, vehicle entertainment control, email, Internet browsing, or other interactive applications associated with a recognized user.

The control subsystem 34 generally includes a control module 50 having a processor 52, a memory 54 and an interface 56. The processor 52 may be any type of microprocessor having the intended performance characteristics. The memory 54 may comprise any type of computer readable medium that stores the data and control algorithms described herein, such as a user assistance system algorithm 58. [ The functions of the algorithm 58 are described in relation to a functional block diagram (Figure 6) and a representative page (Figures 9-14), and those skilled in the art will appreciate that these functions may be implemented in a dedicated hardware circuit, It should be understood that the present invention may be implemented as a programmed software routine executable in a processor-based electronic control embodiment.

2, the control module 50 may be part of another system such as a central vehicle control, a stand-alone unit, or a cloud-based system. Other operating software for processor 52 may also be stored in memory 54. The interface 56 enables communication with other subsystems such as the interactive display subsystem 32, the user input subsystem 36, the user identification subsystem 38, and the user location subsystem 39. It should also be appreciated that interface 56 may communicate with other on-board vehicle systems and off-board vehicle systems. The onboard systems include, but are not limited to, a vehicle head unit 300 that communicates with vehicle sensors that provide, for example, vehicle tire pressure, fuel volume, and other vehicle diagnostic information. Off-board vehicle systems may provide information including, but not limited to, weather forecasts, traffic, and other information that may be provided via the cloud 70.

The user input subsystem 36 may include one or more input sensors including an onboard input sensor 60, an offboard input device, or both. The onboard input sensors 60 may include one or more motion cameras or other optical sensors configured to detect gesture commands, one or more touch sensors configured to detect touch commands, one or more touch sensors configured to detect voice commands, A microphone, or other onboard device configured to detect user input. The user input subsystem may also include an off board input device such as a wearable article 61, an electronic key 62 and / or a user's personal electronic device 63, such as a tablet, smart phone or other mobile device . The wearable article 61 may be a wearable computing device such as a smart watch or a wearable sensor.

In some cases, the wearable article 61 may be integrated with the electronic key 62 so that the user will no longer need to carry the electronic key 62 separately. As will be described in more detail below, the wearable article 61 may be configured to receive user input to indicate vehicle functions to be performed on the vehicle 20. [ In addition, the wearable article 61 may be configured to receive gesture input from a user for authentication before these vehicle functions can be performed.

In some embodiments, the system 30 uses multi-factor authentication for security and authorization. Authentication may be implemented in the user identification subsystem 38, for example. Exemplary multi-factor authentication may include receiving input from a wearable article 61, an electronic key 62, an osteoarticular relationship recognition (Fig. 5) and / or a gesture cipher (Fig. 8). The user may be provisionally identified with one of these factors, but may require at least a total of two arguments to authenticate the user prior to display of the particular content. That is, access to all features of the user mode 104 will not be allowed to the user until multi-factor authentication is passed and the user is within a predetermined range of the vehicle 20. This authentication process ensures personal information embedded in the system 30 and security of the vehicle. In one disclosed non-limiting embodiment, the first authentication factor may be a wearable article 61 incorporating the function of a digital electronic key, and the second authentication factor may be the user's bone articulation relationship (FIG. 7). If the user does not have a wearable article 61 or electronic key 62, then the bone articulation relationship may be the first authentication factor, and a gesture password such as a wave or a specific arm movement (FIG. 8) 2 Authentication factor. In yet another example, the first authentication factor may be a wearable article having an integrated electronic key function, and the second authentication factor may be a gesture input from a user, such as a gestural password or an osteoarthritic relationship. Further, different combinations of authentication factors are possible, and the second factor may be optional. For example, the second factor may be required if security needs to be enhanced, such as when the vehicle is parked in a public place or in a high crime zone. In another example, the user can be authenticated only if it is detected that the wearer is wearing the wearable article 61 or has the electronic key 62.

The wearable article 61 in the disclosed non-limiting embodiment may be encrypted to uniquely identify each user to the system 30. [ Additional security protocols such as a rolling time key may additionally be used to ensure that the encrypted key can not be intercepted and reused by unauthorized devices.

Once the wearable article 61 is recognized, it will welcome the user and pre-authenticate to allow limited access to the selected content in use mode 104. [ This will provide the user with enough time to cycle through a number of content features during an access experience and to secure other content features, e.g., destinations. If the user is fully authenticated, all content features created during the pre-authentication state, e.g., the destination, are validated to be displayed. If the authentication fails, the user will not be allowed access to the vehicle 20 or any sensitive information. The system 30 in the disclosed non-limiting embodiment allows pre-authenticated access at about 30 to 40 feet from the vehicle and allows full access at about 15 to 25 feet.

Referring to Fig. 7, in order to provide additional authentication, the system 30 is operable to recognize a user by a bone articulation relationship. The bone joint relationship in the disclosed non-limiting embodiment enables pre-authentication, but does not enable full authentication that allows full access to vehicle 20. However, if the user is pre-authenticated via wearable article 61 or electronic key 62, the matched bone joint relationship will fully authenticate the user. That is, the user identification subsystem 38 may use the bone joint relationship as the second identification point.

FIG. 19 is a block diagram of a computing device 1000 for implementing a wearable article 61. FIG. The wearable article 61 may include some or all of the functions of the digital electronic key, such as the electronic key 62. [ For example, with a digital electronic key integrated into the SmartWatch, users can wear a smart watch without having to worry about finding a key every morning. The computing device 1000 may be any type of wearable, compact, or other type of single computing device, or it may include multiple computing devices. For example, the computing device 1000 may include a smart watch 2002 (Fig. 20A), a personal portable device, a smart apparel article 2004 (Fig. 20B), a percutaneous chip (not shown), a wearable sensor , Or a smartglasses article (2006) (Figure 20c).

The processing unit of computing device 1000 may be a conventional central processing unit (CPU) 1102 or any other type of device or devices capable of handling or processing information. The memory 1104 of the computing device 1000 may be a random access memory device (RAM) or any other suitable type of storage device. Memory 1104 may include data 1106 accessed by the CPU using bus 1108. [ The memory 1104 may also include an operating system 1110 and installed applications 1112 and the installed applications 1112 may be used by the CPU 1102 to perform the functions of the vehicle in the vehicle as described And a program for causing the computer to implement instructions to generate control signals. In addition, the instructions may include performing functions that are not associated with the vehicle, such as tracking the user's biometrics or displaying time. In addition, computing device 1100 may include secondary, additional, or external storage 1114, e.g., a memory card, flash drive, or other form of computer readable medium. In one embodiment, the installed applications 1112 can be stored in the external storage 1114 in whole or in part and loaded into the memory 1104 if necessary for processing.

Computing device 1000 may include one or more input devices such as a display 1116 and one or more input devices such as a keypad, a touch sensitive device, a sensor, or a gesture sensitive input device capable of receiving user input have. The computing device 1000 may communicate with one or more subsystems via a transponder / transceiver device or a communication device (not shown) such as a Wi-Fi, infrared or Bluetooth device. For example, the computing device 1000 may communicate with the control subsystem 34 via the interface 56.

The computing device 1000 may be coupled to one or more vehicle devices configured to receive input from a user and to provide feedback to a driver of the vehicle 20. [ Also, as will be described later, the computing device 1000 may include a sensor (not shown) that takes sensed information, such as voice commands, ultrasound, gestures, or other input, from a user.

In some embodiments, the computing device 1000 may be a wearable computing device configured to perform vehicle functions in the vehicle 20. Vehicle functions can be implemented with the installed applications described above. As described in more detail below, vehicle functions include, among other things, various remote control functions (Figs. 23A-23F), a driver point function (Fig. 24B), a panic mode (Figs. 25A- A video function, a climate control function, an internet access function, and a remote control function for controlling the vehicle. The remote control functions include, for example, unlocking, locking (2308 in FIG. 23A), light switch on (2310 in FIG. 23A), light switch off, horn honk (2312 in FIG. 23A) 2302), stop (2306 in FIG. 23C), vehicle power on or vehicle power off.

In the computing device 1000 illustrated in FIG. 19, applications 1112 stored in memory 1104 may include a vehicle application, such as application 2204 shown in FIG. 22B. The applications 1112 may also identify autonomous driving applications such as a data analyzer, a path planner, a target value generator, an error detector, an adaptive module, or a driver, And other applications configured to implement an autonomous drive system by performing actions such as improving the position accuracy of the vehicle.

20A-20C illustrate some non-limiting examples of a wearable article 61. FIG. Referring to FIG. 20A, the wearable article 61 may be implemented as a smart watch 200. For example, with a digital electronic key integrated into SmartWatch (2002), users can wear a smart watch without having to worry about finding a key each morning. The user will no longer need to carry the electronic key 62 separately. In addition to being used as a watch or a wrist band, the smart watch 2002 can be worn by a user to perform a vehicle function in the vehicle 20. [ Referring to FIG. 20B, the integrated wearable article 61 may be implemented as a smart apparel article 2004. The user can activate the smart apparel article 2004 by controlling certain vehicle functions integrated into the smart apparel article 2004. Further, referring to Fig. 20C, the integrated wearable article 61 may be implemented as a smart glass 2006. Fig. Other implementations of the wearable article 61 are also possible. For example, the wearable article 61 may be implemented as a smartphone device, a percutaneous chip, a wearable sensor, or a remote access electronic key.

FIG. 21 is a flow diagram of an exemplary process 2100 performed by the wearable computing device 1000 of FIG. 19, described in more detail below.

22A-C are exemplary screen displays 2202 of a wearable article 2200 according to one embodiment. Figure 22A illustrates a screen displaying the date and time. 22B illustrates a main menu from which a vehicle application icon 2204 can be selected. 22C illustrates a menu screen having three icons: remote control 2206, driver score 2208, and panic mode 2210. FIG.

23A-23F are exemplary screen displays with remote control 2206 selected. 23A illustrates a remote control screen showing a list of remote control functions to be selected by the user. The user can scroll up or down the list using the up and down buttons. For example, when the user clicks the "vehicle start" icon 2302, a control signal is sent to the vehicle to start the vehicle. As illustrated in Figure 23B, a status message 2304 may be displayed on the control screen. When the start function is performed, the icon may be switched or toggled to indicate a "stop vehicle" icon 2306 as illustrated in Fig. 23C. In another example illustrated in Figures 23d-23f, a "Hear Ring" icon 2312 may be selected and a status message 2314 may be generated for display on the screen. When a beeping is performed, the user can repeat this operation by clicking on the "Beeping" icon 2316 again.

24A illustrates a screen displaying an exemplary alert indication 2402 ("vehicle started"). The user can return to the previous screen by clicking on the notification display 2402. [ FIG. 24B illustrates a driver score notification screen 2404, which will be described in more detail below.

25A-25C illustrate an exemplary screen display of the panic mode of the vehicle application 2204 of the wearable article 2200. FIG. 25A illustrates a main menu where a panic mode icon 2210 can be selected. 25B illustrates a screen 2502 illustrating that the user presses and holds the select button to activate the panic mode. 25C shows a message 2504 indicating that the panic mode is activated.

As illustrated in Figures 22A-25C, the vehicle functions to be performed in the vehicle 20 include various remote control functions (Figures 23A-23F), a driver point function (Figure 24B), and a panic mode (Figures 25A- . ≪ / RTI > The remote control functions include, for example, unlocking, locking (2308 of FIG. 23A), light switch on (2310A of FIG. 23A), light switch off, horn ringing (2312A of FIG. 23A) Stop (2306 in FIG. 23C), vehicle power on or vehicle power off. Other vehicle functions may include, for example, a navigation function, an audio / video function, a climate control function, or an Internet access function.

The wearable article can be used by the user to sense information such as biometrics and driving information. For example, the wearable article 61 may be used to store and forward driver scores. When the driver removes the key from the vehicle 20, a driver's score can be sent from the vehicle 20 to the wearable article 61. [ After a period of time, the wearable article 61 may upload the driver's score to the remote server or cloud 70, which may perform additional analysis of the driver's score to help the driver improve driving skills and become safer driver . As previously described, data analysis based on driver scores may be used to improve safe driving skills or for other purposes. For example, the user can review and evaluate his driving performance against others in the social ranking system based on the driver's score.

In some embodiments, a driver score may be generated for driver behavior performed in the vehicle 20. The driver's score helps the driver improve his driving skills and can notify the driver if there is a performance degradation. The driver behavior may be associated with or triggered by vehicle functions performed in the vehicle 20. [ The driver's score may be calculated based on information from a sensor, such as a motion camera or optical sensor, onboard device and / or wearable article 61 that detects gesture commands, for example. For example, when the vehicle is started, information about driver behavior may be collected to calculate the driver's score. If the vehicle is stopped and locked, the driver behavior information collected during this run can be used to calculate the driver's score. If the driver's score is calculated by a device other than the wearable article 61, it can be transmitted to the wearable article 61 for display and / or storage.

In one implementation, the driver's score may be a selectable option from the menu screen of the wearable article. As shown in FIG. 22C, the driver's score option 2208 may be selected on the menu screen 2202 of the wearable article 2200. Referring to FIG. 24B, when the driver score option 2208 is selected, an exemplary driver score "87" is generated and displayed on the driver score notification screen 2404. As previously described, driver scores can be uploaded and further processed for various application-specific purposes, such as driving skill enhancement.

The wearable article 61 may also be used to control multiple vehicles or to allow multiple users to share control of the vehicle. As will be described later, encryption techniques may be implemented in wearable items and / or in some of the vehicle subsystems for security purposes.

In some cases, at least one on-board input sensor 60 or off-board input device may be integrated into, or operate in conjunction with, the interactive display subsystem 32. In one non-limiting example, the interactive display subsystem 32 includes an LCD display integrated into the window 22 and can operate in conjunction with one or more touch sensors integrated into the window 22, To function as a touch screen. In another non-limiting example, the interactive display subsystem 32 includes a projector 40 and a coating 40 on the window 22, and operates in conjunction with one or more motion detectors configured to detect user gesture commands And allows the window to act as a gesture-based interactive display. The combination of subsystems that interact with the interactive display subsystem 32 and user input subsystem and enable display and user interaction on the vehicle window 22 will be referred to herein as an interactive window display.

The user identification subsystem 38 may also be configured to provide a user interface to the vehicle 20 that is mounted to the vehicle 20 to provide a desired field of view to the vehicle 20, A closed-circuit television (CCTV) camera, infrared, thermal or other sensors. An exemplary user identification subsystem 38 may include image data captured by the identification sensors 64, such as, for example, a wireless device such as a wearable article 61 associated with a particular driver and / And / or the occupant based on the bone joint relationship 66 and / or other user shape data (Fig. 5). In addition, the wearable article 61 may include a sensor (not shown) that takes information sensed by the user, such as a pulse or heart rate. The sensor of the wearable article 61 may be a wrist-mounted sensor that recognizes the user based on, for example, voice commands, ultrasound, gestures or other input. Based at least in part on this identification, the system 30 provides access to an interactive interface on an interactive display subsystem 32 associated with a particular driver and / or occupant.

FIG. 21 is a flow diagram of an exemplary process 2100 performed by the wearable computing device 1000 of FIG. Process 2100 may be implemented as a software program executed by computing device 1000. The software program may be stored in a memory such as memory 1104 and may be stored in a machine readable storage medium such as a CPU 1102 that may enable the wearable computing device 1000 to perform process 2100. [ ≪ / RTI > Also, the process 2100 may be implemented using special hardware or firmware.

In step 2102, a user input representing the vehicle function to be performed in the vehicle may be received in the wearable article, such as the wearable computing device 1000. In one example, the user can press the touch button on the display of the smart watch 200 to activate the vehicle function to unlock the front door of the vehicle 20. [ In another example, the user can select a vehicle function by pressing the icon of the smart apparel article 2004 worn by the user. The user input may also include, for example, an indication to activate a window display of the inactive display subsystem. Other types of inputs are also possible. For example, the user can activate vehicle functions using voice commands.

In step 2104, the gesture input by the user may be received in a wearable article, such as the wearable computing device 1000. The gesture input can be used for user authentication. In some embodiments (e.g., multi-factor authentication), the gesture input may be used for multi-factor authentication in conjunction with user input received from the wearable article in step 2102 to authenticate the user. The user may be authenticated based on a first input received from a wearable article indicating a vehicle function to be performed and a second gesture input detected by the sensor. The sensor can be coupled to, for example, the vehicle 20 or the wearable article 61. In addition, the sensor may be integrated with the vehicle 20 or the wearable article 61. For example, the sensor may be an onboard input sensor, such as a camera or optical sensor, configured to detect gesture commands, or a microphone configured to detect voice commands. In addition, the sensor may be an off-board input device coupled with the wearable article 61 or other device, such as an electronic key 62 or personal electronic device 63.

In some embodiments, when the user is within a predetermined distance of the vehicle 20, the second input may include a gesture input by the user detected by the sensor. The sensor can be used to detect if the wearer is wearing a wearable article. If the user is not wearing a wearable article, access to some or all of the vehicle functions may not be allowed.

In step 2106, a vehicle function may be performed in the vehicle 20 based on successful authentication of the user. The vehicle function may be as shown in step 2102, such as an input indication that unlocks or switches the vehicle 20, for example. As shown in the examples of FIGS. 22A-C, the vehicle functions may include various remote control functions, a driver point function, and a panic mode. The remote control functions may include, for example, unlocking, locking, light switch on, light switch off, horn ring, start, stop, vehicle power on or vehicle power off. In addition, the vehicle functions may include a navigation function, an audio / video function, a climate control function, or an Internet access function.

The information used for user authentication may include either the user input received in step 2102, the gesture input received in step 2104, or any of the multi-factors described above. Exemplary multi-factor authentication may include receiving input from a wearable article 61, an electronic key 62, an osteoarticular relationship recognition (Fig. 5) and / or a gesture cipher (Fig. 8). The user may be provisionally identified with one of these factors, but may require at least a total of two factors to perform some or all of the vehicle functions.

After the user is successfully authenticated, a control signal for performing the vehicle function shown in step 2102 can be generated and sent to the vehicle 20. Once successfully authenticated, the user can interact with the display subsystem via the integrated wearable article 61 and an output can be generated to display on the vehicle window.

The system 30 can store user profiles of known users, and the user profiles include identification information associated with individual users. For example, the user profile may include facial recognition data or bone joint relationship data usable by the user identification subsystem 38 to identify or authenticate a user. The user profile may further include personal interest information such as personal calendar and event information, driving / destination history, web browsing history, entertainment preferences, climate preferences, and the like. In some variations, any or all of the information contained in the user profile may be stored on or shared with the wearable article 61, personal electronic device 63, remote server, or other cloud 70 based system. Such off-board storage or sharing of user profile data may enable the use of user profile data in other vehicles such as any additional cars, lending vehicles, etc. owned by the user. Such user profile data may be secured by making it accessible via biometric authentication or other effective means through a password protection application running in the cloud 70 based system.

In some cases, the user profile further includes user access information; And may include data as to whether the user is allowed to control a given vehicle function. For example, a user profile associated with a user may indicate full user access or capability control rights for that user. This may be similar to the control authority of the administrator of the personal computer. The user profile may alternatively represent limited user access. For example, a user profile associated with a child may be configured to block a user from accessing specific audio or video controls, navigation systems, user profile changes, and the like.

The registration of the various user profiles to the system 30 can be accomplished in any way, for example over the Internet or with a direct vehicle interface. The user profile may be based on the identity of individual users known or registered in the user category or in the system, such as "unknown user" or "valet. &Quot; In different variants, default user categories such as "unknown user" or "surrogate parking" may be associated with limited default access or may be associated with access inaccessibility,

The user position subsystem 39 operable to determine the position of one or more users inside or outside the vehicle includes one or more position sensors 66, such as pressure sensors, temperature sensors, cameras, located inside or outside the vehicle . In some cases, one device may serve as both the identification sensor 64 and the position sensor 66. For example, a camera mounted within a vehicle may provide information regarding the user's specific identity, such as the seat of the driver or the seat of the passenger in front of the user, by means previously described. Also, in some cases, elements of the interactive display subsystem 32 may operate as position sensors 66 in the user location subsystem 39. For example, pressure sensors in a smart screen or motion detector that operate as part of an interactive display can be used to obtain user location information.

In some cases, the user access may be based on a user location as determined by the user location subsystem 39. [ For example, passengers in the second or third row may not be allowed or allowed access to various vehicle functions, such as navigation systems. Optionally, a user with a user profile associated with unlimited access for each access information associated with the user profile can specify these settings. In some cases, the user access may be based on a combination of a user profile as applied by the user identification system 38 and a user location as detected by the user location subsystem 39. Access to certain vehicle functions may be blocked, for example, if the occupant seats the driver of the moving vehicle, despite being a user with unrestricted access as specified by the applied user profile.

6, operation of the system 30 in accordance with one disclosed non-limiting embodiment generally includes a sleep mode 100, a watcher mode 102, and a usage mode 104, do. It should be understood that other modes may additionally or alternatively be provided.

If the system 30 is active but has not yet detected a user, the system 30 will be in the sleep mode 100 until the user identification subsystem 38 wakes up. Before detection by system 30, but after detection, the monitoring mode 102 may be used to interact with authenticated and unauthorized persons. For example, when a person approaches the vehicle 20, the system 30 recognizes the direction the person is approaching and then activates the interactive display subsystem 32 to display the avatar, Display. These graphics can be oriented specifically toward the direction that people approach, for example, the graphic eye "looks" at the approaching person. Alternatively, the audio capability allows the system 30 to respond to commands and initiate interaction from the blind side of the vehicle 20, i. E. Without the interactive display subsystem 32 . The monitoring mode 102 uses the user identification subsystem 38 to distinguish between authenticated and unauthenticated persons.

The use mode 104 allows a user with a user profile of a known worker and / or occupant in the system 30 to determine access to the vehicle 20 so that the particular vehicle interaction awaits entry into the vehicle 20 Do not need to. The use mode 104 reduces peripherals around the perimeter through the recognition of the driver within the vehicle 20, and the reduction of movement-related decisions from the visual and manual workload streams. To facilitate this, an overview of information is presented to the user including, for example, weather, traffic, calendar events and vehicle status. As will be described in more detail, the prediction function of the system 30 identifies likely actions and provides an optimal path for completion, such as planning an efficient path.

The maximum extent of content provision by the interactive display subsystem 32 may be associated with the maximum distance at which the content can be efficiently interacted with the user. In one disclosed, non-limiting embodiment, the maximum extent of each content feature is prioritized over the legibility range of the content displayed by the interactive display subsystem 32. This range metric enables the determination of the order in which content appears in the walkup experience. Access to the preferred content with a larger maximum extent allows the access experience to start at a further distance from the vehicle 20, thereby providing the user with more total time to interact with the system 30.

If successfully authenticated, the interactive display subsystem allows the user to interact with the display subsystem via the integrated wearable article 61 and generate output for display on the vehicle window.

Referring to FIG. 9, once authenticated, the "landing" or "home" page 200 provides the user with a summary of the alerts and important information. The landing page 200 provides the user with an overview of the vehicle condition that is easily viewable and provides an indication of how this can affect the user's schedule and activities. In this example, the content includes time information, vehicle diagnostic information, and personal calendar information. As shown here, in addition to a traffic-based route update used by the vehicle navigation system and a calendar event reminder for "pick up children within 20 minutes ", a fuel shortage warning is provided. In another example, the system 30 will include a gas station as a stop during route guidance if the destination is at a greater distance than the available fuel range. In particular, preferred gas stations or other waypoints can be predefined in the user profile.

The landing page 200 also displays a plurality of icons to indicate additional content pages that are visible to the authorized user. The landing page 200 itself may be accessed on each content page as an icon, such as a vehicle manufacturer mark icon, on each content page. The landing page 200 may allow an authorized user to understand what vehicle systems or individual user profile items may require another attention and to provide such items in the form of navigable icons that lead to additional content pages Provides access to additional content feature details about the content features. The landing page 200 may additionally or alternatively incorporate an interactive display, for example a smart page or a video game. Other interactive vehicle display page configurations are also possible.

The selection of the content is achieved, for example, by a wearable article 61, an electronic key 62, a user gesture, a voice command, a touch input, or the like. In one example, the user views the various pages displayed by the interactive display subsystem 32 using the input of the wearable article 61 in a circular fashion. In one example, the wearable article 61 may include four button directional pads and two auxiliary buttons. Alternatively, a hand gesture can be used to "swipe" between the pages. In another example, a user may cycle through the pages using the electronic key 62. It should be understood that although specific pages have been illustrated in the disclosed non-limiting embodiment, various alternative or additional pages may be provided.

Referring to FIG. 10, the path page 202 defaults to a predicted optimal path for the user for the next explicit or estimated destination. Any alternative destinations or paths that may be reliably estimated or explicit from a user's personal electronic device, such as, for example, a wearable article 61, are presented to allow user selection by scrolling through the options. The proposed route screen is shown here as being accessed using a folded-map icon, but other icons may be used.

Referring to Fig. 11, the calendar page 204 displays the user's calendar. In this example, this screen is near the end, showing only the next few or two upcoming appointments. Also, if the event includes location information, the user is given the option to use the event for destination selection. As shown here, the calendar page 204 provides the user with highlighted content for the next appointment and provides a reminder for "child pick up ". Here, although the calendar screen is shown as being accessed using the flip calendar icon, other icons may be used.

Referring to FIG. 12, the weather page 206 leases information about the path to provide relevant weather information - which is particularly effective when the user moves away from home. For example, system 30 determines whether it is more valuable to present local weather information, destination weather information, or both to the user, depending on the type of weather information available or the set values selected by the user. As shown here, the weather forecast is chronological. The weather page 206 may be accessed with a sun icon, but other icons may be used. Also, a weather condition may be used to create a reminder for the display on the landing screen 200, which suggests that an umbrella be provided to the vehicle, for example, when rain is predicted.

Referring to FIG. 13, vehicle status page 208 provides the user with a screen of impending vehicle maintenance necessity requiring attention. The alert indication may include the source detail of the alert indication, severity, and options for resolving potential problems. For example, given a notification of "low fuel ", the system 30 may suggest a route to nearby gas stations within the vehicle range. The vehicle status page 208 is accessed here by the vehicle icon, but other icons may be used.

14, the to-do list page 210 presents information from any associated to-do list available to the user's personal wearable device 61 to an authorized user, for example. As shown here, the recognized user must perform the tasks of "send courier", "pay taxes" and "update the car registration" among other items. Alternatively, if location information is included in a given listing item in the todo list of the personal electronic device, the todo list page 210 may be incorporated into the path selection page. An example of such an integration would include providing a detailed path to a dry cleaning business if the dry cleaning pickup is on the to-do list and the current path is close to the location of the location of the dry cleaning business. The to-do list page is shown here as being accessed using the check-mark icon, but other icons may be used.

As noted above, this property of information that may be included in the user profile may be stored in the wearable device 61, another personal electronic device 63, a remote server, or some other cloud 70 based system in some variations Or may be shared with, and enable the use of more than one vehicle. Any such information can be secured by making it accessible via biometric authentication or other effective means through a password protection application running in the cloud 70 based system. In some of these variants, partial or total access to the profile of the second user may be granted to the first user, for example by password sharing. This access sharing allows a first user to record a reminder or job from a remote location to a user profile of a second user, e.g., a family member, so that the second user can access the user profile of the second user, A reminder or job recorded by the first user will be displayed on the window when the user approaches or enters any vehicle equipped with the enabled system 30. [

Referring to FIG. 15, a user access to various vehicle functions may include direct or remote access to utilize the functions of the vehicle head unit 300.

With the interaction between the vehicle head unit 300 and the system 30, and more particularly between the vehicle head unit 300 and the various interactive window displays, the occupant can make a selection for the vehicle system that is typically performed by the driver, In some cases it is only possible when the vehicle is stopped. Having only the occupant to interact with particular vehicle systems while the vehicle is moving increases safety by minimizing driver inattention. In addition, occupant interaction enables greater functionality for the system 30. For example, the front seat occupant may be provided with more menu choices than the driver, while the occupants of the second and third rows may be provided with far more menu selections than the front seat occupant. In these embodiments, occupants can take over some of the driver's workload.

The vehicle occupants may be contacted by the system 30 and the associated vehicle (s) via, for example, an interactive window display, or via a wearable article 61 communicating therewith, or via other wireless technology standards for Bluetooth, RFID, And can interact with the head unit 300. In addition, the system 30 may allow the vehicle occupants to form a personal area network (PAN) to share information. For example, the occupant's wearable article 61 may include a map application-feature is not locked that is operable to communicate with the vehicle navigation system on the vehicle head unit 300, allowing the occupant to search for a destination , And selectively transmit to the vehicle navigation system through the vehicle head unit 300.

In addition, the interaction of the system 30 with the vehicle head unit 300 allows the driver and / or occupants to select content for other vehicle occupants and / or drivers. For example, one of the passengers can select a destination for the driver while the vehicle is moving and display it on the navigation system. In another example, a driver may select entertainment content for display to a child passenger. In another example, an occupant may control infotainment or climate control features that are controlled by the vehicle head unit 300.

16, in a non-limiting example of the operation of the user location subsystem 39, in order to further increase safety through driver careless minimization, the system 30 includes a user location subsystem 39 (FIG. 18) via the bone position (FIG. 16), face map data (FIG. 17), pressure sensors, interactive window display input sensors, It is possible. For example, for a vehicle with three rows, three distinct areas - the front row, the middle row and the back row are traced. Typically, at least two sensors 402 per column are required to track the condition of each passenger in the vehicle 20. In some cases, each individual seat within the vehicle 20 may be tracked. Alternatively, in addition, data from all the sensors 402 may be combined to produce one central map (2D or 3D) used by the system 30. It should be appreciated that the sensors 402 may communicate with, or be a part of, the user identification subsystem 38, the user location subsystem 39, or both.

When vehicle riders typically sit in a seat and wear a belt, the multi-point bone joint relationship and facial recognition map data can track each passenger's condition to a desired level of accuracy with a specific snapshot in time Provides a relatively accurate location of each rider captured on the XYZ axis map. The status of each rider enables another tailored operation for various vehicle functions. For example, the user location subsystem 39 detects and distinguishes the hands of the vehicle front passenger and the driver's hands to selectively unlock various head unit functions, such as navigation path selection (Fig. 16). For example, the content menu items of the vehicle head unit 300 may be selectively displayed depending on whether the user (driver or passenger) attempts to access the system 30 and depending on whether the vehicle is moving. For example, certain content, such as route selection, may be color coded only for passenger access, while other content such as zooming and scrolling may be available at all times, regardless of the user.

When approaching the vehicle, the system 30 advantageously recognizes the user with the first and second identification points and displays information for that particular authorized user. This authentication process ensures the security of the vehicle, and the personal information embedded in the system 30 permits vehicle interaction before the user enters the vehicle interior space. The system 30 also advantageously distinguishes occupants from the driver and optionally allows access to personalized content or specific vehicle system interfaces.

The use of the similar references to the terms "a "," an ", and "the" Should be construed as encompassing all of them. The qualifier "about" used in connection with an amount includes the stated value and has the meaning referred to in the text (e.g., it includes the degree of error associated with a particular amount of measurement). All ranges disclosed herein include endpoints. It is to be understood that relative terms such as "front," "rear," "top," "bottom," "above," "below," and the like refer to the normal operational attitude of the vehicle, do.

While the various non-limiting embodiments have specific elements illustrated, embodiments of the invention are not limited to these specific combinations. In combination with features or elements from any of the non-limiting embodiments, some of the elements and features from any of the other non-limiting embodiments may be utilized.

It is to be understood that the same reference numerals identify corresponding or similar elements throughout the several views. It should also be understood that although certain component arrangements are disclosed in the illustrated embodiment, other arrangements therefrom would be beneficial.

It should be understood that although certain step sequences have been shown, described and claimed, it will be appreciated that the steps may be performed, separated or combined in any order, and still be advantageous from the present invention, unless otherwise indicated.

The above description is illustrative rather than limiting. Various non-limiting embodiments are disclosed herein, but one of ordinary skill in the art will appreciate that various modifications and changes may be made within the scope of the appended claims in light of the above teachings. For example, in the embodiments described above, the vehicle 20 is generally described as an automobile. However, the vehicle 20 is not limited to an automobile, as the integrated wearable article may be implemented with other vehicles generally controlled by the driver or operator such as an airplane, a boat, or the like. In addition, the vehicle 20 need not be limited to being controlled by a driver or an operator, but may be one or more robots or robot tools that perform operations under the control of an application equivalent to a path planner application. It is, therefore, to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described. For that reason, the appended claims should be examined to determine their actual scope and content. Accordingly, the scope of the claims should be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures as are permitted by law.

Claims (20)

A system for operating a vehicle,
A user input subsystem comprising a wearable article, the user input subsystem being configured to receive input from a user; And
A user recognition and authentication subsystem in communication with the user input subsystem, the user recognition and authentication subsystem being configured to detect and authenticate a user associated with the received input. The method according to claim 1,
Wherein the wearable article comprises at least one of a smart watch, a personal portable device, a smart clothing article, a transdermal chip, a wearable sensor, or a smartglass article. The method according to claim 1,
Wherein the wearable article comprises a wearable computing device configured to perform at least one vehicle function in the vehicle. The method of claim 3,
The at least one vehicle function may comprise at least one of a remote control function, a driver score function, a panic mode function, a navigation function, an audio / video function, a climate control function, Of the vehicle. 5. The method of claim 4,
Wherein the remote control function is one of an unlock, a lock, a light switch on, a light switch off, a beep sound, a start, a stop, a vehicle power on, or a vehicle power off. The method according to claim 1,
Wherein the user recognition and authentication subsystem includes a sensor configured to detect at least one gesture input by a user. The method according to claim 6,
The user is authenticated based on a first input received from the wearable article indicative of a vehicle function to be performed and a second input detected by the sensor associated with the vehicle or the wearable article. The method according to claim 6,
Wherein the sensor is further configured to detect whether the wearer is wearing the wearable article and wherein the user is authenticated based on the at least one gesture input by the detected user within a predetermined range of the vehicle. The method of claim 3,
Further comprising a control subsystem configured to allow the user to control the at least one vehicle function from the wearable article. 10. The method of claim 9,
The control subsystem comprising:
Generating driver scores for at least one driver behavior associated with one or more vehicle functions performed by a user in the vehicle,
And transmit the driver's score to the wearable article. In a wearable article,
One or more processors; And
A memory for storing data and program instructions executed by the one or more processors, the one or more processors comprising:
Receiving from a user a first input indicative of a vehicle function to be performed in the vehicle;
Receive a second input indicative of a gesture by the user for authentication;
To generate a control signal for performing the vehicle function based on a successful authentication of the user,
And to execute instructions stored in the memory. 12. The method of claim 11,
The vehicle function is one of a remote control function, a driver point function, a panic mode function, a navigation function, an audio / video function, a climate control function, or an internet access function. 13. The method of claim 12,
Wherein the remote control function is one of an unlock, a lock, a light switch on, a light switch off, a beep sound, a start, a stop, a vehicle power on, or a vehicle power off. 12. The method of claim 11,
And a sensor configured to detect a gesture by a user and to generate a second input based on the gesture. 12. The method of claim 11,
Wherein the one or more processors comprise:
To receive driver scores generated for at least one driver behavior associated with one or more vehicle functions performed by a user of the vehicle,
And to execute instructions stored in the memory. A method of operating a vehicle,
In a wearable article, receiving from a user a first input indicative of a vehicle function to be performed in the vehicle;
Receiving a second input indicative of a gesture by the user for authentication; And
And in the wearable article, generating a control signal for performing the vehicle function in the vehicle based on successful authentication of the user. 17. The method of claim 16,
Further comprising detecting whether the user is wearing the wearable article, wherein the user's successful authentication is based on the user wearing the wearable article. 17. The method of claim 16,
Wherein the vehicle function is at least one of a remote control function, a driver point function, a panic mode function, a navigation function, an audio / video function, a climate control function, or an internet access function. 19. The method of claim 18,
Wherein the remote control function is one of unlocking, locking, light switch on, light switch off, horn ringing, start, stop, vehicle power on or vehicle power off. 17. The method of claim 16,
Wherein the user's successful authentication is based on a first input received from the wearable article indicative of a vehicle function to be performed and a second input detected by a sensor associated with the vehicle or the wearable article.

Images