TWI817823B - Vehicle control system, head-mounted display device, and vehicle control method - Google Patents

Abstract

A vehicle control system is provided. The vehicle control system includes: a vehicle and a head-mounted display device. The vehicle is configured to provide first coordinate information. The head-mounted display device is configured to provide and display the second coordinate information. In response to the head-mounted display device receiving the first coordinate information, the head-mounted display device matches the first coordinate information with the second coordinate information to provide third coordinate information and display an augmented image based on the third coordinate information. In response to the head-mounted display device receiving a control instruction input by a user based on the augmented image, the head-mounted display device outputs a control command to the vehicle, so that the vehicle performs a control action.

Description

Vehicle control system, head-mounted display device and vehicle control method

The present invention relates to a vehicle control system, and in particular, to a vehicle control system, a head-mounted display device and a vehicle control method.

In the field of vehicle assisted driving, vehicles can be equipped with a variety of assistance systems to increase convenience and safety. For example, Adaptive Cruise Control System (ACC), Anti-lock Braking System (ABS), Autonomous Emergency Braking (AEB), Electronic Stability Program (ESP) ), Lane Keeping Assistance (LKA), Automatic Parking System (APS)...etc. The above systems are usually integrated into an Advanced Driver Assistance System (ADAS) for users to operate.

However, information from advanced driver assistance systems is usually provided via the dashboard display. A Head-Up Display (HUD) on the screen or windshield is presented to the user in a two-dimensional way. Moreover, when users perform operations, they also issue instructions in a two-dimensional manner. Therefore, the information presented by advanced driving assistance systems is usually relatively complex, and users need a certain amount of learning time to operate it.

The present invention provides a vehicle control system, a head-mounted display device and a vehicle control method, which present vehicle information to users in an intuitive and convenient manner.

The vehicle control system of the present invention includes: a vehicle for providing first coordinate information; and a head-mounted display device for providing second coordinate information, wherein in response to the head-mounted display device receiving the first coordinate information, the head-mounted display device The head-mounted display device matches the first coordinate information with the second coordinate information to provide third coordinate information and displays an augmented image based on the third coordinate information. In response to the head-mounted display device receiving the user's input based on the augmented image According to the control instructions, the head-mounted display device outputs control commands to the vehicle to enable the vehicle to perform control actions.

The head-mounted display device of the present invention includes: a device sensor for sensing device sensing information around the user; and a processor for receiving the first coordinate information from the vehicle and receiving the device sensing information to Generate second coordinate information, wherein the processor matches the first coordinate information with the second coordinate information to provide third coordinate information and display an augmented image based on the third coordinate information, in response to the head-mounted display device receiving the user's Based on the control instructions input by the augmented image, the head-mounted display device outputs control commands to the vehicle to cause the vehicle to perform control actions.

The vehicle control method of the present invention includes: receiving first coordinate information from the vehicle; receiving second coordinate information from the head-mounted display device; matching the first coordinate information with the second coordinate information to generate third coordinate information ; Display the augmented image based on the third coordinate information; receive control instructions input by the user based on the augmented image; and output control commands to the vehicle according to the control instructions to cause the vehicle to perform control actions.

Based on the above, users can clearly understand the current driving status and give instructions in an intuitive and convenient way.

In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, embodiments are given below and described in detail with reference to the accompanying drawings.

100, 200: Vehicle control system

110:Vehicle

112:Advanced driving assistance system

114: First sensor

120:Head mounted display device

122: Processor

124: Second sensor

301, 302, 303: Object

400: Vehicle control method

AI: augmented imaging

C1: First coordinate information

C2: Second coordinate information

C3: Third coordinate information

CC: control command

CI: control instruction

R: indicator ray

S1: First sensing information

S2: Second sensing information

S410, S420, S430, S440, S450, S460: steps

U:User

Figure 1 is a schematic diagram of a vehicle control system according to an embodiment of the present invention.

Figure 2 is a schematic diagram of a vehicle control system according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of a usage scenario according to an embodiment of the present invention.

Figure 4 is a flow chart of a vehicle control method according to an embodiment of the present invention.

In order to make the content of the present invention easier to understand, the following embodiments are given as examples according to which the present invention can be implemented. In addition, wherever possible, the same reference numbers are used in the drawings and embodiments to represent the same or similar parts.

And, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be construed to have meanings consistent with their meanings in the context of the relevant technology and the present invention, and are not to be construed as idealistic or excessive Formal meaning, unless expressly defined as such herein.

The present invention may be understood by reference to the following detailed description taken in conjunction with the accompanying drawings. It should be noted that, in order to facilitate readers' understanding and to keep the drawings concise, many of the drawings in the present invention only depict a part of the electronic device, and specific components in the drawings are not drawn according to actual scale. In addition, the number and size of components in the figures are only for illustration and are not intended to limit the scope of the present invention.

It should be noted that in the following embodiments, the technical features in several different embodiments can be replaced, reorganized, and mixed to complete other embodiments without departing from the spirit of the present invention. Furthermore, in the description below and the scope of the patent application, the words "including" and "include" are open-ended words, so they should be interpreted to mean "including but not limited to...".

In the field of vehicle assisted driving, vehicles can be equipped with advanced driving assistance systems to increase convenience and safety. However, advanced driver assistance systems Information is usually presented to the user in a two-dimensional manner through a screen on the dashboard or a head-up display on the windshield. Moreover, when users perform operations, they also give instructions in a two-dimensional manner.

Compared with the two-dimensional method, the three-dimensional method can present information more clearly or operate more intuitively. This disclosure introduces the application of the head-mounted display device, so that the information of the advanced driving assistance system can be presented in a three-dimensional manner, and the user can give instructions in a three-dimensional manner. In this way, the user's convenience and safety when driving the vehicle can be effectively improved.

Figure 1 is a schematic diagram of a vehicle control system according to an embodiment of the present invention. Referring to FIG. 1 , the vehicle control system 100 includes a vehicle 110 and a head-mounted display device 120 . The vehicle 110 may be used to provide the first coordinate information C1. The head-mounted display device 120 may be used to provide second coordinate information C2 (not shown). And, in response to the head-mounted display device 120 receiving the first coordinate information C1, the head-mounted display device 120 matches the first coordinate information C1 with the second coordinate information C2 to provide the third coordinate information C3 (Fig. (not shown) and displays the augmented image AI based on the third coordinate information C3. Furthermore, in response to the head-mounted display device 120 receiving the control instruction CI input by the user U based on the augmented image AI, the head-mounted display device 120 outputs the control command CC to the vehicle 110 so that the vehicle 110 performs the control behavior.

In one embodiment, the vehicle 110 may be, for example, a car, a truck, a bus, a train, a subway, a locomotive, etc., suitable for carrying the user U, but the disclosure is not limited thereto. The vehicle 110 may include advanced driver assistance systems to provide information about the vehicle 110 itself and the environment.

In one embodiment, the head-mounted display device 120 may be, for example, head-mounted display (Head-Mounted Display) glasses or other wearable display devices adapted to be worn on the head of the user U, but this disclosure does not This is the limit. The head mounted display device 120 may include one or more screens to display various images to the user U.

In one embodiment, the first coordinate information C1 and the second coordinate information C2 are generated through Simultaneous Localization And Mapping (SLAM) technology. In other words, the first coordinate information C1 and the second coordinate information C2 may be, for example, SLAM maps.

In one embodiment, the head-mounted display device 120 can match the SLAM maps respectively corresponding to the first coordinate information C1 and the second coordinate information C2 to provide the third coordinate information C3. For example, the environment where the vehicle 110 and the user U are located may include multiple objects, such as other vehicles, road signs, pedestrians, and buildings. In the first coordinate information C1, a specific object can be presented in the form of a first characteristic totem (Pattern). In the second coordinate information C2, the same specific object can be presented in the form of a second characteristic totem. The head-mounted display device 120 can compare the first characteristic totem and the second characteristic totem to determine whether they are the same object.

In one embodiment, the vehicle 110 may be preset with a specific characteristic totem for identification by the vehicle control system 100 . For example, specific feature totems may include University of Cordoba augmented reality (Augmented Reality University of Cordoba, ArUco) markers, trackers, reference objects, other elements with similar functions, or The combination of these components is not limited by this disclosure. These specific feature totems may be provided to the vehicle 110 as well as the head mounted display The display device 120 serves as the first characteristic totem and the second characteristic totem respectively, thereby achieving a correction effect. In this way, the vehicle control system 100 does not need to additionally search for other objects for comparison, thereby increasing the efficiency of comparing the first characteristic totem and the second characteristic totem.

In one embodiment, the three-dimensional model information of the vehicle 110 may be pre-stored in the head-mounted display device 120 . Furthermore, the head-mounted display device 120 can match the first coordinate information C1 and the second coordinate information C2 according to the three-dimensional model information of the vehicle 110 . For example, the three-dimensional model information of the vehicle 110 may include information on the shape and size of various internal and external components of the vehicle 110 , but the disclosure is not limited thereto. That is to say, when performing comparison, the head-mounted display device 120 can prioritize the characteristics of the vehicle 110 itself, thereby increasing the efficiency of comparing the first characteristic totem and the second characteristic totem.

Then, when the head-mounted display device 120 determines that the two are the same object (that is, the first characteristic totem and the second characteristic totem match each other), the head-mounted display device 120 can obtain the first coordinate information C1 and the second coordinate information The matching relationship between information C2. Then, the head-mounted display device 120 can project or superimpose the content in the second coordinate information C2 into the first coordinate information C1 according to the matching relationship to generate the third coordinate information C3. That is to say, the third coordinate information C3 aligns the coordinates of the first coordinate information C1 and the second coordinate information C2 to integrate the contents of the first coordinate information C1 and the second coordinate information C2. Then, the head-mounted display device 120 can generate and display the augmented image AI based on the third coordinate information C3. In other words, the user U can obtain the first coordinate information obtained by the vehicle 110 by viewing the augmented image AI. Contents of message C1.

Furthermore, after viewing the augmented image AI, the user U can input the control instruction CI based on the augmented image AI. In one embodiment, the control instruction CI may be, for example, a gesture or voice of the user U, but the present invention is not limited thereto. After the head-mounted display device 120 detects and receives the control instruction CI, the head-mounted display device 120 can output the control command CC to the vehicle 110 to cause the vehicle 110 to perform control actions. In one embodiment, the control behavior of the vehicle 110 may be, for example, planning a driving route (such as going to a specific destination, automatically following a car, automatically parking) or setting a driving state (such as switching to automatic driving or manual driving), but this disclosure does not Not limited to this.

In one embodiment, while viewing the augmented image AI, the user U finds a destination around the vehicle 110 that he wants to go to, and the user U can issue a control instruction CI through gestures or voice. The head-mounted display device 120 can output the control command CC according to the control instruction CI, so that the vehicle 110 plans a driving route to the destination or automatically moves to the destination. For example, the above-mentioned destination may be the entrance of a building, a parking space, or other scenery seen by the user U in the augmented image AI, but the present disclosure is not limited thereto.

In one embodiment, when viewing the augmented image AI, the user U finds that there is a target vehicle in front of the vehicle 110 heading for the same route, and the user U can issue a control instruction CI through gestures or voice. The head-mounted display device 120 may output the control command CC according to the control instruction CI, so that the vehicle 110 automatically follows the target vehicle. For example, the target vehicle may be a vehicle driving on the same expressway or a vehicle driven by a person known to the user U, but the disclosure is not limited thereto.

In one embodiment, when the user U is watching the augmented image AI, he finds that there is information he wants to know around the vehicle 110 (such as the license plate of the vehicle in front or the road sign on the road). The user U can use gestures or voice to Issue control instructions CI. The head-mounted display device 120 can amplify the information that the user U wants to know in the augmented image AI according to the control instruction CI. In addition, the head-mounted display device 120 can output the control command CC according to the control instruction CI to control the driving state of the vehicle 110 (for example, switching to automatic driving) while maintaining safety conditions, so that the user U has enough time to clearly see what he is thinking. Information to know.

In one embodiment, while viewing the augmented image AI, the user U discovers that the road conditions around the vehicle 110 are complicated or a sudden and noteworthy situation occurs. The user U can issue a control instruction CI through gestures or voice. The head-mounted display device 120 can switch the vehicle 110 from automatic driving to manual driving according to the control instruction CI, so as to facilitate the user U to intervene in the planning of the driving route.

It can be seen from the above that by integrating the relevant information of the vehicle 110 into the head-mounted display device 120, the user U can clearly understand the current driving status and give instructions in an intuitive and convenient manner. In this way, the convenience and safety of the user U when driving the vehicle 110 can be effectively improved.

Figure 2 is a schematic diagram of a vehicle control system according to an embodiment of the present invention. Please refer to FIG. 2 . The vehicle control system 200 of FIG. 2 is an implementation of the vehicle control system 100 of FIG. 1 , but the present disclosure is not limited thereto. Vehicle control system 200 may include vehicle 110 and head mounted display device 120 . The vehicle 110 may include an advanced driver assistance system 112 and a first sensor 114 (also referred to as a vehicle sensor). wear on head The display device 120 may include a processor 122 and a second sensor 124 (also called a device sensor).

In one embodiment, the first sensor 114 may be used to sense the first sensing information S1 (also called vehicle sensing information) around the vehicle 110 and provide the first sensing information S1 to the advanced driving assistance system 112 , so that the advanced driving assistance system 112 generates the first coordinate information C1. The first sensor 114 may be, for example, an infrared sensor, an ultrasonic sensor, a radar, a light radar (LIDAR), a camera or other similar devices or a combination of these devices, which is not limited by this disclosure. The advanced driving assistance system 112 may, for example, include at least one or more functions such as adaptive cruise system, anti-lock braking system, automatic emergency braking, electronic stability program, lane keeping assist, automatic parking system, etc. Driving assistance systems, but this disclosure is not limited to this.

In one embodiment, the second sensor 124 can be used to sense the second sensing information S2 (also known as device sensing information) around the user U, and provide the second sensing information S2 to the head-mounted display. The processor 122 of the device 120 generates the second coordinate information C2. The second sensor 124 may be, for example, a camera or other similar device, which is not limited by this disclosure. The processor 122 is, for example, a controller (Microcontroller Unit, MCU), a central processing unit (Central Processing Unit, CPU), a microprocessor (Microprocessor), a digital signal processor (Digital Signal Processor, DSP), a programmable controller, The present invention is not limited to programmable logic devices (PLD) or other similar devices or combinations of these devices. Furthermore, in one embodiment, various functions of the processor 122 may be implemented as multiple program codes. These codes will be stored in a memory and executed by the processor 122 these codes. Alternatively, in one embodiment, the functions of processor 122 may be implemented as one or more circuits. The present invention is not limited to using software or hardware to implement each function of the processor 122 .

It should be noted that in this embodiment, the processor 122 and the second sensor 124 are both provided in the wearable display device 120, but the disclosure is not limited thereto. In another embodiment, one or both of the processor 122 and the second sensor 124 may be disposed outside the wearable display device 120 and transmit data to each other in a wired or wireless manner. That is to say, the present disclosure does not limit the processor 122 or the second sensor 124 to be disposed inside or outside the wearable display device 120 .

In one embodiment, through simultaneous positioning and mapping technology, the data detected by the first sensor 114 of the vehicle 110 and the second sensor 124 of the head-mounted display device 120 can be provided to the advanced driver respectively. The auxiliary system 112 and the processor 122 generate the first coordinate information C1 and the second coordinate information C2. At this time, since the first coordinate information C1 and the second coordinate information C2 are generated independently of each other, they may each have different reference coordinates. In other words, the coordinate values of the same object may be offset in the first coordinate information C1 and the second coordinate information C2. In order to correct this offset, the vehicle 110 provides the first coordinate information C1 to the head-mounted display device 120 and matches the first coordinate information C1 and the second coordinate information C2 via the head-mounted display device 120 .

For example, the head-mounted display device 120 can search for the first characteristic totem and the second characteristic totem corresponding to the same object in the first coordinate information C1 and the second coordinate information C2 respectively, and through the alignment of the two characteristic totems, to correct the offset. like As shown in Figure 2, after the first coordinate information C1 and the second coordinate information C2 undergo "feature matching", the corrected third coordinate information C3 can be generated. It should be noted that, for convenience of presentation, the "feature matching" in Figure 2 is shown outside the processor 122, but the "feature matching" can be performed by the processor 122 or other components that provide the second coordinate information C2. This disclosure and are not restricted.

Also, the first sensor 114 may have more diverse data than the second sensor 124 . In one embodiment, the first sensor 114 is a light radar, and the second sensor 124 is a camera, but the disclosure is not limited thereto. In addition to providing the first coordinate information C1, the first sensor 114 can also provide three-dimensional detection information. In one embodiment, the three-dimensional detection information may be, for example, point cloud image information, but the disclosure is not limited thereto. The vehicle 110 can provide the three-dimensional detection information of the first sensor 114 to the head-mounted display device 120, and the head-mounted display device 120 can integrate the three-dimensional detection information into the third coordinate information C3. That is to say, the third coordinate information C3 may simultaneously include image information detected by the camera and point cloud image information detected by the optical radar. In other words, compared with the original first coordinate information C1, the accuracy of the third coordinate information C3 can be effectively improved.

Therefore, the head-mounted display device 120 can obtain detection values from the optical radar of the vehicle 110 without the need for optical radar, thereby improving the restoration degree and details of the augmented image AI. In this case, the user U can clearly know the relevant information of the vehicle 110 through the augmented image AI provided by the head-mounted display device 120 . Moreover, the user U can accurately issue precise control instructions CI through the augmented image AI provided by the head-mounted display device 120 . As a result, user U is in The convenience and safety when driving the vehicle 110 can be effectively improved.

FIG. 3 is a schematic diagram of a usage scenario according to an embodiment of the present invention. Referring to FIGS. 1 to 3 , the usage scenario 300 may include a vehicle 110 , an object 301 , an object 302 and an object 303 . Furthermore, the user U is located in the vehicle 110 and is wearing the head-mounted display device 120 .

As shown in FIG. 3 , the head-mounted display device 120 can detect the direction of the gesture of the user U. In one embodiment, the head-mounted display device 120 can detect the gesture of the user U through the second sensor 124 that provides the second sensing information S2. In another embodiment, the head mounted display device 120 may detect the gesture of the user U through another sensor different from the second sensor 124 . Furthermore, the indication direction of the gesture of the user U may include, for example: the extension direction of the index finger of the user U or the axial direction of the center point of the wrist of the user U, but the present disclosure is not limited thereto.

Then, the head-mounted display device 120 may display the indication ray R in the augmented image AI according to the indication direction of the user U. In one embodiment, the indication ray R can be a straight line or a curve, which is not limited by this disclosure. In addition, the head-mounted display device 120 can determine the pointing object based on the third coordinate information C3 and the pointing ray R in the augmented image AI. In one embodiment, the head-mounted display device 120 can determine, based on the third coordinate information C3, that the indicator ray R extending from the user U hits an object around the vehicle 110, and determine that the object is the indicator object. . For example, the pointing ray R in FIG. 3 hits the object 301, so the object 301 is determined to be the pointing object. Then, the head-mounted display device 120 can generate the control command CC according to the pointing object and provide it to the vehicle 110 so that the vehicle 110 performs the control behavior.

In one embodiment, the control command CC may be, for example, setting a driving route based on the pointing object. For example, the pointing object is the vehicle in front of the vehicle 110, and the control command CC is to set the driving route of the vehicle 110 so that the vehicle 110 follows the vehicle in front. Alternatively, the instruction object is a parking space near the vehicle 110, and the control command CC is to set the driving route of the vehicle 110 so that the vehicle 110 stops at the parking space. Alternatively, the instruction object is a building near the vehicle 110, and the control command CC is to set the driving route of the vehicle 110 so that the vehicle 110 goes to the nearby building.

In one embodiment, the control command CC may be, for example, setting the driving state (autonomous driving/manual driving) of the vehicle 110 based on the pointing object. For example, the instruction object is the license plate of the vehicle in front of the vehicle 110, and the control command CC is to set the driving state of the vehicle 110 so that the vehicle 110 switches from manual driving to automatic driving, so that the user U can clearly see the license plate of the vehicle in front.

It should be noted that the augmented image AI may further include two-dimensional map information generated based on the third coordinate information C3. In other words, user U can see the two-dimensional map information of the selected area in the three-dimensional augmented image AI. Furthermore, the user U can send an indication ray R to the two-dimensional map information, and then select the indication object.

It is worth mentioning that the head-mounted display device 120 can detect the augmented image AI at any time to determine whether there is a noteworthy situation (warning situation). For example, the distance between the vehicle 110 and surrounding objects is lower than a safe distance, there is a vehicle or foreign object on the side that is about to cut into the driving route of the vehicle 110 , the road ahead is abnormal, etc. In response to determining the warning situation, the head-mounted display device 120 may display a warning message (for example, in the form of a bounding box, a mask, or other visual means) in the augmented image AI to allow the user to Understand the situation in real time to avoid danger.

Figure 4 is a flow chart of a vehicle control method according to an embodiment of the present invention. Referring to FIG. 1 and FIG. 4 , the vehicle control method 400 schematically represents the operation process of the vehicle control system 100 , but the present disclosure is not limited thereto. In this embodiment, the vehicle control method 400 includes steps S410 to S460.

In step S410 , the head-mounted display device 120 receives the first coordinate information C1 from the vehicle 110 . In step S420, the head-mounted display device 120 receives the second coordinate information C2 generated from the head-mounted display device 120 itself. In step S430, the head-mounted display device 120 matches the first coordinate information C1 and the second coordinate information C2 to generate third coordinate information C3. In step S440, the head-mounted display device 120 displays the augmented image AI based on the third coordinate information C3. In step S450, the head-mounted display device 120 receives the control instruction CI input by the user U based on the augmented image AI. In step S460, the head-mounted display device 120 outputs the control command CC to the vehicle 110 according to the control instruction CI, so that the vehicle 110 performs the control behavior. Relevant implementation details may refer to the descriptions in Figures 1 to 3 and will not be described again here.

In this way, the vehicle control method 400 can integrate relevant information of the vehicle 110 into the head-mounted display device 120 . Therefore, the user U can clearly understand the current driving status and give instructions in an intuitive and convenient way. In this way, the convenience and safety of the user U when driving the vehicle 110 can be effectively improved.

To sum up, the vehicle control system, head-mounted display device and vehicle control method of the present invention realize the integration of vehicle-related information into the head-mounted display device. Therefore, the user can clearly understand the image through the augmented image provided by the head-mounted display device. Chudi knows the relevant information about the vehicle. Moreover, users can accurately issue precise control instructions through the augmented images provided by the head-mounted display device. In this way, the user's convenience and safety when driving the vehicle can be effectively improved.

Although the present invention has been disclosed above through embodiments, they are not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make some modifications and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the appended patent application scope.

100:Vehicle control system

110:Vehicle

120:Head mounted display device

AI: augmented imaging

C1: First coordinate information

CC: control command

CI: control instruction

U:User

Claims (11)

A vehicle control system includes: a vehicle for providing first coordinate information; and a head-mounted display device for providing second coordinate information, wherein in response to the head-mounted display device receiving the first coordinate information, The head-mounted display device matches the first coordinate information with the second coordinate information to provide third coordinate information and displays an augmented image based on the third coordinate information, in response to the head-mounted display device The head-mounted display device receives a control instruction input by the user based on the augmented image, and the head-mounted display device outputs a control command to the vehicle to cause the vehicle to perform a control behavior. The vehicle control system of claim 1, wherein the first coordinate information and the second coordinate information are generated through simultaneous positioning and map construction technology. The vehicle control system of claim 1, wherein the head-mounted display device compares the first characteristic totem of the first coordinate information with the second characteristic totem of the second coordinate information, so as to compare the first characteristic totem of the second coordinate information. The first coordinate information is matched with the second coordinate information. The vehicle control system of claim 1, wherein the head-mounted display device is further used to match the first coordinate information with the second coordinate information according to the three-dimensional model information of the vehicle. The vehicle control system of claim 1, wherein the vehicle includes: an advanced driving assistance system; and a first sensor, wherein the first sensor is used to sense first sensing around the vehicle. The information is provided to the advanced driving assistance system, so that the advanced driving assistance system generates the first coordinate information. The vehicle control system of claim 1, further comprising: a second sensor for sensing second sensing information around the user, wherein the head-mounted display device is configured according to the second sensing information. Sensing information generates the second coordinate information. The vehicle control system of claim 1, wherein the vehicle further provides three-dimensional detection information to the head-mounted display device, and the head-mounted display device integrates the three-dimensional detection information into the third Three coordinate information. The vehicle control system of claim 1, wherein the head-mounted display device is further used to: detect the pointing direction of the user's gesture; and in the augmented image, display the indication according to the pointing direction. Ray; in the augmented image, determine the pointing object according to the third coordinate information and the pointing ray; and generate the control command according to the pointing object. The vehicle control system of claim 8, wherein the control command includes setting a driving route based on the indicating object or setting a driving state of the vehicle based on the indicating object. A head-mounted display device includes: a device sensor for sensing device sensing information around a user; and a processor for receiving first coordinate information from a vehicle and receiving the device sensing information to Generate second coordinate information, wherein the processor matches the first coordinate information with the second coordinate information to provide third coordinate information and display an augmented image based on the third coordinate information, in response to The head-mounted display device receives a control instruction from the user based on the augmented image input, and the head-mounted display device outputs a control command to the vehicle to cause the vehicle to perform a control behavior. A vehicle control method, including: receiving first coordinate information from a vehicle; receiving second coordinate information from a head-mounted display device; matching the first coordinate information with the second coordinate information to generate a third coordinate information; display an augmented image based on the third coordinate information; receive a control instruction input by a user based on the augmented image; and output a control command to the vehicle according to the control instruction, so that the vehicle Engage in controlling behavior.

Images