JP7100100B2 - Virtual image display device, control method, program, and storage medium - Google Patents

Description

The present invention relates to a technique for displaying a virtual image.

Conventionally, in a head-up display, a technique of changing the display content according to a user's gesture has been known. For example, Patent Document 1 includes a viewpoint detection camera that captures the driver's eyes and a gesture detection camera that captures the driver's hand, and when the driver points to a display virtual image with his or her fingertips, the driver A head-up display is disclosed that identifies a virtual image in the driver's line of sight by detecting the eye position and the fingertip position of the driver, and erases the specified virtual image when a predetermined motion is detected. Further, Patent Document 2 discloses a head-up display capable of displaying at least two virtual images on the front window at the same time and making the display size and the focal length of the driver with respect to the virtual image different for each virtual image. ing.

Japanese Unexamined Patent Publication No. 2005-138755 Japanese Unexamined Patent Publication No. 2006-106254

In a virtual image display device such as a head-up display capable of displaying virtual images having different virtual image distances, if the virtual image distances of the displayed virtual images are different, but the display directions are close, the method described in Patent Document 1 is used. , It is difficult to identify the virtual image pointed to by the user. Therefore, the present invention provides a virtual image display device capable of accurately identifying the virtual image pointed to by the user even when the direction of the displayed virtual image is close when the user makes a gesture with respect to the virtual image. The main purpose is to do.

The invention described in the claims is a virtual image display device mounted on a vehicle .
A display control means for simultaneously displaying a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as a virtual image on a display means.
A detection means for detecting the direction in which the observer who sits on the seat of the vehicle and visually recognizes the virtual image reaches out, and the gesture length corresponding to the distance reached out.
If the direction in which the first object is displayed and the direction in which the second object is displayed are not close to each other, the first object or the second object is based on the direction in which the hand is reached. Is specified as the object selected by the observer, and
When the direction in which the first object is displayed and the direction in which the second object is displayed are close to each other , the first object is based on the direction in which the hand is reached and the gesture length. Alternatively, a specific means for identifying any of the second objects as the object selected by the observer, and
It is characterized by having.

Further, the invention described in the claims is a control method executed by a virtual image display device mounted on a vehicle .
A display control step of simultaneously displaying a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as a virtual image on a display means.
A detection step of detecting the direction in which the observer sitting on the seat of the vehicle and visually recognizing the virtual image reaches out, and the gesture length corresponding to the distance reached out.
If the direction in which the first object is displayed and the direction in which the second object is displayed are not close to each other, the first object or the second object is based on the direction in which the hand is reached. Is specified as the object selected by the observer, and
When the direction in which the first object is displayed and the direction in which the second object is displayed are close to each other , the first object is based on the direction in which the hand is reached and the gesture length. Alternatively, a specific step of specifying any of the second objects as an object selected by the observer, and
It is characterized by having.

Further, the invention described in the claims is a program executed by a computer mounted on a vehicle .
A display control means for simultaneously displaying a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as a virtual image on a display means.
A detection means for detecting the direction in which the observer who sits on the seat of the vehicle and visually recognizes the virtual image reaches out, and the gesture length corresponding to the distance reached out.
If the direction in which the first object is displayed and the direction in which the second object is displayed are not close to each other, the first object or the second object is based on the direction in which the hand is reached. Is specified as the object selected by the observer, and
When the direction in which the first object is displayed and the direction in which the second object is displayed are close to each other , the first object is based on the direction in which the hand is reached and the gesture length. Alternatively, the computer is made to function as a specific means for identifying any of the second objects as an object selected by the observer.

A configuration example of a head-up display system is shown. It is a block diagram of the control part of a head-up display system. It is a flowchart which shows the processing procedure which concerns on Example. An example of displaying the front window before and after accepting the operation of selecting the mark image and displaying the detailed information of the facility is shown. A display example on the front window when the head-up display system has three displays is shown. It is a figure which shows the relationship between the position of the extended hand and the virtual image selected by the position of each hand. The relationship between each range and the position of the extended hand when the non-reaction range is provided is shown. The relationship between the position of the virtual image and the reaction range and the non-reaction range is shown.

In one preferred embodiment of the present invention, the imaginary image display device simultaneously displays a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as imaginary images. Based on the display control means to be displayed on the screen, the detection means for detecting the distance reached by the observer who visually recognizes the imaginary image, and the distance reached by the observer, the first object or the second object. It is provided with a specific means for identifying any of the above as an object selected by the observer.

The virtual image display device includes display control means, detection means, and specific means. The display control means simultaneously displays the first object to be visually recognized at the first distance and the second object to be visually recognized at the second distance farther than the first distance as virtual images on the display unit. The detecting means detects the distance reached by the observer who visually recognizes the virtual image. The identifying means identifies either the first object or the second object as the object selected by the observer, based on the distance reached by the observer. According to this configuration, the virtual image display device is suitable based on the distance reached by the observer to display the object selected by the observer when two objects having different virtual image distances are displayed as virtual images by the display means. Can be specified in.

In one aspect of the imaginary image display device, the detection means further detects the direction in which the observer reaches out, and the specific means is the first within a predetermined range from the direction in which the observer reaches out. When the display position of the virtual image corresponding to the object and the second object is included, the observer can see either the first object or the second object based on the distance reached by the observer. Identifies as the selected object. According to this aspect, the virtual image display device is based on the distance at which the observer reaches for the object selected by the observer, even if a plurality of objects are displayed in the direction in which the observer reaches out. Can be suitably specified.

In another aspect of the imaginary image display device, the identifying means identifies the first object as an object selected by the observer when the distance reached by the observer is shorter than the reference distance. When the distance reached by the observer is longer than the reference distance, the second object is specified as the object selected by the observer. In this aspect, the virtual image display device sets a longer distance to reach when the observer selects a second object displayed far away than when selecting a first object displayed closer. .. As a result, the operation of selecting an object can be suitably performed by the observer in accordance with the sense of distance of the apparent virtual image.

In another aspect of the virtual image display device, the specific means changes the reference distance according to the first distance and the second distance. According to this aspect, the virtual image display device can make the observer preferably perform the operation of selecting an object in accordance with the sense of distance of the apparent virtual image.

In another aspect of the virtual image display device, the specific means of the first object and the second object when the predetermined distance range including the reference distance includes the distance reached by the observer. Neither is specified as an object selected by the observer. According to this aspect, the virtual image display device can suitably suppress the erroneous recognition of the object selected by the observer.

In another aspect of the virtual image display device, the virtual image display device is determined by the recognition means after the recognition means for recognizing the gesture of the observer and the object selected by the observer are specified by the specific means. Further includes an execution means for executing a process related to the selected object when the gesture of the above is recognized. According to this aspect, the virtual image display device can suitably perform the processing related to the object selected by the user according to the gesture of the user.

In another preferred embodiment of the present invention, the control method executed by the imaginary image display device includes a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance. Is displayed on the display means at the same time as an imaginary image, a detection step of detecting the distance reached by the observer who visually recognizes the imaginary image, and the first step based on the distance reached by the observer. It has a specific step of identifying either one object or the second object as an object selected by the observer. By executing this control method, the virtual image display device causes the display means to display two objects having different virtual image distances as virtual images, and the object selected by the observer is set to the distance reached by the observer. It can be suitably specified based on this.

In still another embodiment of the present invention, a program executed by a computer, in which a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance are regarded as imaginary images. The first object or the said The computer is made to function as a specific means for identifying any of the second objects as the object selected by the observer. When the computer executes this program to display two objects with different virtual image distances as virtual images on the display means, the object selected by the observer is suitable based on the distance reached by the observer. Can be specified in. Preferably, the program is stored in a storage medium.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[System configuration]
FIG. 1 is a configuration example of the head-up display system 2 according to the embodiment. As shown in FIG. 1, the head-up display system 2 according to the present embodiment displays two virtual images at the same time and can be operated by gestures, and is mainly operated by the first and second displays 41 and 42. (When not distinguished, they are collectively referred to as "display 4"), a vehicle interior photographing camera 6, a concave mirror 8, and a control unit (not shown) described later. The head-up display system 2 is attached to a vehicle including a front window 25, a ceiling portion 27, a bonnet 28, and a dashboard 29.

The first and second displays 41 and 42 are provided in the dashboard 29 and display images showing facility information, route guidance information, traveling speed information, and other information that assists driving. The first display 41 is installed at a position that does not block the optical path from the second display 42 to the concave mirror 8. The first and second displays 41 and 42 are directed to the concave mirror 8 also provided in the dashboard 29, and the light emitted from the first and second displays 41 and 42 (also referred to as "display light"). ) Is incident on the concave mirror 8. In this case, the light reflected by the concave mirror 8 reaches the front window 25 through the opening 89 provided in the dashboard 29, and further reflected by the front window 25 to reach the position of the driver's eyes. In FIG. 1, the optical path of the light emitted from the first display 41 is shown by a broken line, and the optical path of the light emitted from the second display 42 is shown by a dashed line.

In this way, the first and second displays 41 and 42 bring the display light to the position of the driver's eyes so that the driver can visually recognize the virtual images Iv1 and Iv2, respectively. Hereinafter, the image displayed as the virtual image Iv1 is referred to as "first object Ob1", the image displayed as virtual image Iv2 is referred to as "second object Ob2", and when these are not distinguished, they are also simply referred to as "object".

Here, the distance from the position of the driver's eyes to the virtual image (also simply referred to as “virtual image distance”) varies depending on the optical path length from the display 4 to the front window 25. In the example of FIG. 1, since the second display 42 is installed at a position farther from the concave mirror 8 than the first display 41, the virtual image distance of the virtual image Iv2 is longer than the virtual image distance of the virtual image Iv1. There is. The virtual image distance of the virtual image Iv1 is an example of the "first distance" in the present invention, and the virtual image distance of the virtual image Iv2 is an example of the "second distance" in the present invention.

In addition to the liquid crystal display, the first and second displays 41 and 42 may be screens on which the light constituting the image is projected and an intermediate image is displayed. The display 4, the concave mirror 8, and the front window 25 are examples of the "display means" in the present invention. Further, at least one of the first display 41 and the second display 42 may be movably configured so that the distance from the concave mirror 8 can be adjusted by the user.

The vehicle interior photographing camera 6 is a camera that photographs the vehicle interior, and is installed so that the face and hands of the driver sitting in the driver's seat are within the photographing range. The image taken by the vehicle interior photographing camera 6 (also referred to as “photographed image Im”) is used to measure the three-dimensional position of the driver's hand. The vehicle interior photographing camera 6 is preferably a stereo camera.

The concave mirror 8 reflects the display light emitted from the first display 41 and the second display 42 toward the opening 89 provided in the dashboard 29 and reaches the front window 25. In this case, the concave mirror 8 magnifies and reflects the image indicated by the display light. The concave mirror 8 may be provided for each of the first display 41 and the second display 42, respectively. Here, since the optical path length from the first display 41 to the concave mirror 8 is shorter than the optical path length from the second display 42 to the concave mirror 8, the virtual image Iv1 has a shorter virtual image distance than the virtual image Iv2.

In the following, a case where the driver is the operator will be described, but another occupant may be the operator. Even in this case, the vehicle interior photographing camera 6 is directed so that the operator's face and hands are within the photographing range.

[Functional configuration]
The head-up display system 2 has a control unit that controls the head-up display system 2 at an arbitrary position in the vehicle interior. FIG. 2 is a block diagram showing a functional configuration of the control unit 10 of the head-up display system 2. As shown in FIG. 2, the control unit 10 is electrically connected to the display 4 and the vehicle interior photographing camera 6, and has a display control unit 11, a gesture detection unit 12, and a target identification unit 13. The display control unit 11, the gesture detection unit 12, and the target identification unit 13 may be integrally configured or may be installed at separate locations while being electrically connected.

The display control unit 11 generates an image signal representing an object to be displayed on each of the first display 41 and the second display 42. For example, when the display control unit 11 displays a mark image indicating the position of the facility as an object, the display control unit 11 is located at a position visible to the driver via the front window 25 based on the output of a GPS receiver (not shown), map data, or the like. It recognizes an existing facility and generates an image signal representing a mark image representing the facility. At this time, the display control unit 11 stores each virtual image distance of the virtual images Iv1 and Iv2 in advance, and depending on each virtual image distance and the distance from the vehicle to the facility for displaying the mark image, the first display 41 or It is determined which of the second displays 42 the mark image is to be displayed.

Further, the display control unit 11 supplies information about an object to be displayed as a virtual image (also referred to as “object information Io”) to the target identification unit 13. Here, the object information Io includes, for example, information on the position and shape on the front window 25, information on the virtual image distance when displayed as a virtual image, information indicating whether or not gesture operation is possible, and the like. When the object information Io is generated, the display control unit 11 may determine whether or not each object can be gesture-operated based on the type of the object. For example, when the object is a mark image, the display control unit 11 determines that the gesture operation is possible because the detailed information of the facility indicated by the mark image can be extracted from the map data or the like.

In addition to the mark image, the display control unit 11 may display various images such as an image showing a distance and an image of an arrow for route guidance as an object. In this case, the display control unit 11 may determine, for example, whether to display on the first display 41 or the second display 42 for each type of information to be displayed.

The gesture detection unit 12 recognizes the driver's gesture based on the captured image Im supplied from the vehicle interior photographing camera 6. In this embodiment, when the gesture detection unit 12 detects a pointing motion based on a known image recognition process, the length of the extended hand (also referred to as “gesture length Ltag”) and the pointing direction. (Also referred to as "pointing direction Dtag") is detected. For example, the gesture detection unit 12 recognizes the position of the driver's face (eyes) and the position of the hand (fingers), and calculates the distance between the detected face and hand as the gesture length Ltag. Further, the gesture detection unit 12 calculates the direction of the position of the hand with respect to the position of the face as the pointing direction Dtag. When the vehicle interior shooting camera 6 is a monocular camera that is not a stereo camera, the gesture detection unit 12 is based on the measured values of a plurality of distance sensors (not shown) provided on the dashboard 29 and the shot image Im. The gesture length Ltag may be calculated. The gesture detection unit 12 supplies information on the gesture length Ltag and the pointing direction Dtag to the target identification unit 13.

The target identification unit 13 is pointed out by the driver among the objects that can be operated by the driver based on the object information Io received from the display control unit 11 and the gesture length Ltag and the pointing direction Dtag notified from the gesture detection unit 12. Identify the object. Specifically, the target specifying unit 13 first identifies an object displayed around the direction in which the driver's face and hand are tied, based on the object information Io and the pointing direction Dtag. At this time, the target specifying unit 13 refers to the object information Io and excludes the object that cannot be gestured from the specific target. Further, when the first object Ob1 displayed as the virtual image Iv1 and the second object Ob2 displayed as the virtual image Iv2 exist around the pointing direction Dtag, the target specifying unit 13 determines which of the two objects is based on the gesture length Ltag. Determine if the object was the target of a gesture operation. In this case, the target specifying unit 13 stores in advance a threshold value (also referred to as “threshold value Lth”) of the gesture length Ltag. Then, the target specifying unit 13 determines that the second object Ob2 having a long virtual image distance is the operation target when the gesture length Ltag is equal to or greater than the threshold value Lth, and when the gesture length Ltag is shorter than the threshold value Lth, the virtual image It is determined that the first object Ob1 having a short distance is the operation target. Then, when the target specifying unit 13 recognizes the object to be operated, the target specifying unit 13 transmits information for identifying the object to the display control unit 11. The threshold Lth is an example of the "reference distance" in the present invention.

After the target identification unit 13 recognizes the object to be operated, the display control unit 11 highlights the selected object by changing the color of the object specified by the target identification unit 13 to the driver. Present. When the gesture recognition unit 12 further recognizes that a predetermined gesture such as the driver waving his / her hand from that state is further recognized, the display control unit 11 changes to the gesture recognized by the gesture detection unit 12. In response, perform the processing related to that object. Specifically, the display control unit 11 erases the display of the object and displays additional information about the object.

The display control unit 11 is an example of the "display control means" and the "execution means" in the present invention, and the gesture detection unit 12 is an example of the "detection means" and the "recognition means" in the present invention. The specific unit 13 is an example of the "specific means" in the present invention.

[Processing flow]
FIG. 3 is a flowchart showing a processing procedure executed by the control unit 10 in this embodiment. The control unit 10 repeatedly executes the process of the flowchart shown in FIG. In FIG. 3, as a premise, the display control unit 11 appropriately displays an object such as a mark image on the first display 41 and / and the second display 42, and targets the object information Io corresponding to the displayed object. It is assumed that the image is transmitted to the specific unit 13.

First, the gesture detection unit 12 has a gesture length Ltag corresponding to the distance reached when the driver recognizes a pointing motion based on a captured image Im or the like received from the vehicle interior photographing camera 6, and a hand. The pointing direction Dtag, which is the extended direction, is detected (step S101). Then, the gesture detection unit 12 supplies the information of the detected gesture length Ltag and the pointing direction Dtag to the target identification unit 13.

Next, the target specifying unit 13 determines whether or not an operable object is displayed around the pointing direction Dtag (step S102). Specifically, the target specifying unit 13 first recognizes the display position on the front window 25 for each of the objects being displayed that can be gesture-operated based on the object information Io received from the display control unit 11. Then, the display control unit 11 determines whether or not there is an object in which the position on the front window 25 pointed to by the pointing direction Dtag notified from the gesture detection unit 12 and the above-mentioned display position are close to each other.

Then, when the target specifying unit 13 determines that an operable object is displayed around the pointing direction Dtag (step S102; Yes), the process proceeds to step S103. On the other hand, when the target specifying unit 13 determines that the operable object is not displayed around the pointing direction Dtag (step S102; No), it determines that the operation for the object has not been performed, and proceeds to step S101. Return the process.

Next, in step S103, the target specifying unit 13 determines whether or not an object having a different virtual image distance is detected in step S102 (step S103). That is, the target specifying unit 13 determines whether or not there are a plurality of operable objects around the pointing direction Dtag and the virtual image distances thereof are different. Then, when the object detected in step S102 is an object having a different virtual image distance (step S103; Yes), the target specifying unit 13 proceeds to step S104. On the other hand, when the target specifying unit 13 does not detect objects having different virtual image distances (step S103; No), that is, when objects having different virtual image distances are not overlapped and displayed around the pointing direction Dtag. Recognizes that an operable object around the pointing direction Dtag has been selected (step S108). When a plurality of objects having the same virtual image distance are detected in step S102, the target specifying unit 13 recognizes that, for example, the object whose center position is closest to the pointing direction Dtag is selected.

Next, in step S104, the target specifying unit 13 sets the threshold value Lth according to the virtual image distances of the objects having different virtual image distances (that is, the first object Ob1 and the second object Ob2) (step S104). In this case, the target specifying unit 13 may calculate the threshold Lth by referring to a predetermined formula or the like based on the virtual image distances of the first object Ob1 and the second object Ob2, and the pre-calculated threshold Lth is calculated from the memory. It may be extracted and used. Then, the target specifying unit 13 determines whether or not the gesture length Ltag detected in step S101 is equal to or greater than the threshold value Lth (step S105). Then, when the gesture length Ltag is equal to or greater than the threshold value Lth (step S105; Yes), the target specifying unit 13 recognizes that the second object Ob2, which is the object having the farther virtual image distance, is selected (step S106). On the other hand, when the gesture length Ltag is less than the threshold value Lth (step S105; No), the target specifying unit 13 recognizes that the first object Ob1 which is the object having the shorter virtual image distance is selected (step S107).

Then, the display control unit 11 highlights the object recognized by the target identification unit 13 as being selected by the pointing motion in steps S106 to S108 (step S109). After that, when the gesture detection unit 12 further detects a predetermined gesture operation for the selected object, the gesture detection unit 12 displays additional information of the selected object or erases the display according to the detected gesture operation. do.

[Display example]
FIG. 4 shows a display example of the front window 25 before and after the execution of the gesture when the driver selects the mark image displaying the facility name by the gesture and displays the detailed information of the facility.

As shown in FIG. 4A, the display control unit 11 displays a mark image 51 representing the facility name “A store” of the building in front of the intersection, and also displays the facility name “A store” of the building opposite the road of the A store. The mark image 52 representing "B mart" is displayed as a virtual image. In this case, since the building having the facility name "A store" is within a predetermined distance from the virtual image distance (for example, 10 m) of the virtual image Iv1, the display control unit 11 uses the first display 41 to display the mark image 51 as the first object Ob1 ( That is, it is displayed as a virtual image Iv1). Further, since the building having the facility name "B store" is within a predetermined distance from the virtual image distance (for example, 30 m) of the virtual image Iv2, the display control unit 11 displays the mark image 52 on the second display 42 by the second object Ob2 (that is, that is). It is displayed as a virtual image Iv2).

Here, the display positions of the mark image 51 and the mark image 52 on the front window 25 are close to each other. Therefore, in the state of FIG. 4A, when the driver points to either the mark image 51 or 52, the target identification unit 13 has two of the mark images 51 and 52 around the pointing direction Dtag. Recognize it as an object. In this case, the target specifying unit 13 compares the threshold value Lth set according to the virtual image distances of the virtual images Iv1 and Iv2 with the gesture length Ltag detected by the gesture detection unit 12, and the mark image 51 or 52 is used. Is selected.

Then, in the example of FIG. 4B, since the gesture length Ltag is equal to or greater than the threshold value Lth, the target identification unit 13 recognizes that the mark image 52 has been selected by the pointing motion, and the selected mark image 52 is used. On the other hand, it is highlighted by adding a edging effect. Further, after the gesture detection unit 12 further detects a predetermined gesture after recognizing the pointing motion, the display control unit 11 shows detailed information such as the business hours of the facility “B Mart” corresponding to the mark image 52. The image 53 is displayed by referring to map data or the like. At this time, it is preferable that the display control unit 11 displays the image 53 at the same virtual image distance as the selected mark image 52, that is, by the second display 42.

As described above, the display control unit 11 of the head-up display system 2 according to the present embodiment has the first object Ob1 and the second object Ob2 visually recognized by the virtual image distance farther than the virtual image distance of the first object Ob2. , Are displayed on the front window 25 at the same time as virtual images Iv1 and Iv2, respectively. The gesture detection unit 12 detects the gesture length Ltag, which is the distance reached by the observer who visually recognizes the virtual image. The target specifying unit 13 identifies either the first object Ob1 or the second object Ob2 as an object selected by the observer based on the gesture length Ltag. According to this configuration, the head-up display system 2 observes the object selected by the observer when the two first and second objects Ob1 and Ob2 having different virtual image distances are displayed as virtual images Iv1 and Iv2, respectively. It can be suitably identified based on the distance at which the person reaches out.

[Modification example]
Hereinafter, a modification suitable for the above-mentioned embodiment will be described. The following modifications may be applied to the above-described embodiment in any combination.

(Modification 1)
In the example of FIG. 1, the head-up display system 2 includes two displays 4, a first display 41 and a second display 42. Instead, the head-up display system 2 may have three or more displays 4. In this case, each display 4 is installed so that the distance from the concave mirror 8 is different, and the objects displayed on each display 4 are displayed as virtual images having different virtual image distances.

FIG. 5 shows a display example on the front window 25 when the head-up display system 2 includes three displays 4. In the example of FIG. 5, each display 4 is installed in the dashboard 29 by adjusting the distance from the concave mirror 8 so that the virtual image distances of the objects to be displayed are 10 m, 20 m, and 30 m, respectively.

In FIG. 5, the display control unit 11 displays the mark image 51 of the facility “A store” which is approximately the same distance ahead of the virtual image distance by the display 4 which displays the virtual image having the shortest virtual image distance (10 m). The display 4 displaying the virtual image having the longest virtual image distance (30 m) displays the mark image 52 of the facility "B store" which is visually recognized forward by substantially the same distance as the virtual image distance. Further, the display control unit 11 superimposes the display 4 displaying a virtual image having an intermediate virtual image distance (20 m) on an intersection visually recognized forward by substantially the same distance as the virtual image, and indicates an arrow image 54 indicating the traveling direction. Is displayed.

In the example of FIG. 5, when the display control unit 11 generates the object information Io of the arrow image 54, the display control unit 11 includes the information that the gesture operation is impossible in the object information Io and transmits the information to the target identification unit 13. As a result, the target specifying unit 13 does not recognize the arrow image 54 as the selection target by the pointing operation even when it is determined that the arrow image 54 exists around the pointing direction Dtag. On the other hand, when the target specifying unit 13 determines that the mark images 51 and 52 exist around the pointing direction Dtag, it determines which of the detected gesture length Ltag is selected based on the two threshold values Lth described later.

Next, a method of setting the threshold value Lth when three displays 4 are present will be described.

The target specifying unit 13 sets two threshold values Lth with respect to the gesture length Ltag, so that the range in which the object having the longest virtual image distance is determined to be selected, the range in which the object having the second longest virtual image distance is determined to be selected, and the range in which the object having the longest virtual image distance is selected are selected. And a range for determining that the object having the shortest virtual image distance is selected, respectively. Hereinafter, the smaller threshold value Lth of the two threshold values Lth is referred to as “first threshold value Lth1”, and the larger threshold value Lth is also referred to as “second threshold value Lth2”.

FIG. 6 is a diagram showing the relationship between the position of the extended hand and the virtual image selected at the position of each hand. In the example of FIG. 6, the head-up display system 2 displays virtual images Iv1 to Iv3 having different virtual image distances around the pointing direction Dtag. Further, the reaction ranges R1 to R3 indicate the range of the gesture length Ltag divided by the first threshold value Lth1 and the second threshold value Lth2.

In the example of FIG. 6, when the gesture detection unit 12 detects a gesture pointing in the direction of the virtual image Iv1 to Iv3, the target identification unit 13 determines that the detected gesture length Ltag belongs to the reaction range R1. Recognizes that the object corresponding to is selected. Further, when the detected gesture length Ltag belongs to the reaction range R2, the target identification unit 13 recognizes that the object corresponding to the virtual image Iv2 is selected. Similarly, when the detected gesture length Ltag belongs to the reaction range R3, the target identification unit 13 recognizes that the object corresponding to the virtual image Iv3 is selected.

In this way, even when three virtual images having different virtual image distances are displayed, the target specifying unit 13 can accurately recognize the object selected by the user by the gesture based on the gesture length Ltag. Even when there are four or more displays 4, by setting the threshold value Lth according to the number of displays 4, the user can accurately recognize the object selected by the gesture.

Further, preferably, when the gesture length Ltag belongs to the neighborhood range including the set threshold value Lth, the target specifying unit 13 can perform an operation of selecting an object regardless of the magnitude relationship between the gesture length Lth and the threshold value Lth. It may be considered that it has not been done.

FIG. 7 shows the relationship between each range and the gesture length Ltag when the non-reaction ranges NR1 and NR2, which are considered not to have been operated to select an object, are provided between the reaction ranges R1 to R3 shown in FIG. show.

In the example of FIG. 7, the target identification unit 13 sets the non-reaction range NR1 to a neighborhood range within a predetermined distance before and after the first threshold value Lth1, and sets the non-reaction range NR2 to a predetermined distance before and after the second threshold value Lth2. It is set within the neighborhood range. Then, when the gesture length Ltag belongs to any of the non-reaction ranges NR1 and NR2, the target identification unit 13 considers that the pointing motion is invalid.

As described above, the gesture detection unit 12 can suitably suppress erroneous recognition of the operation of selecting an object by providing the non-reaction ranges NR1 and NR2. In the example of FIG. 7, the case where three displays 4 are present has been described, but even when there are two displays 4 or when there are four or more displays 4, a non-reaction range may be provided in the same manner. good. Even in this case, the target specifying unit 13 can suitably suppress erroneous recognition of the object selection operation by setting the neighborhood range of each threshold value Lth to a range that does not accept the object selection operation.

Further, the target specifying unit 13 may set the first threshold value Lth1 and the second threshold value Lth2 according to the virtual image distances of the virtual images Iv1 to Iv3, or may set them regardless of the virtual image distances of the virtual images Iv1 to Iv3. good.

FIG. 8 shows the relationship between the positions of the virtual images Iv1 to Iv3 in the line-of-sight direction of the driver and the reaction ranges R1 to R3 and the non-reaction ranges NR1 and NR2. Specifically, FIG. 8A shows the positional relationship of the virtual images Iv1 to Iv3 in the line-of-sight direction of the driver, and FIG. 8B shows the first threshold value Lth1 and the virtual image distances of the virtual images Iv1 to Iv3. The reaction ranges R1 to R3 and the non-reaction ranges NR1 and NR2 when the second threshold value Lth2 is set are shown, and FIG. 8C shows the first threshold value Lth1 and the second threshold value regardless of the virtual image distances of the virtual images Iv1 to Iv3. The reaction ranges R1 to R3 and the non-reaction ranges NR1 and NR2 when Lth2 is set are shown. In the examples of FIGS. 8B and 8C, the reaction ranges R1 to R3 are set to have the same length.

In the example of FIG. 8B, the difference between the median value of the reaction range R1 and the median value of the reaction range R2 (see arrow B1) and the difference between the median value of the reaction range R2 and the median value of the reaction range R3 (arrow). The ratio to (see B2) is equal to the ratio of the difference in the virtual image distance between the virtual image Iv1 and the virtual image Iv2 (see arrow A1) and the difference in the virtual image distance between the virtual image Iv2 and the virtual image Iv3 (see arrow A2). In this case, the first threshold Lth1 is set to the average value of the median value of the reaction range R1 and the median value of the reaction range R2, and the second threshold Lth2 is the median value of the reaction range R2 and the median value of the reaction range R3. Is set to the average value of. Further, the non-reaction range NR1 is set in a range centered on the first threshold value Lth1, and the non-reaction range NR2 is set in a range centered on the second threshold value Lth2.

On the other hand, in the example of FIG. 8C, the difference between the median value of the reaction range R1 and the median value of the reaction range R2 (see arrow C1) and the difference between the median value of the reaction range R2 and the median value of the reaction range R3. (See arrow C2). That is, in this case, the reaction ranges R1 to R3 are set at equal intervals regardless of the virtual image distances of the virtual images Iv1 to Iv3. Further, as in the example of FIG. 8B, the first threshold Lth1 is set to the average value of the median value of the reaction range R1 and the median value of the reaction range R2, and the second threshold Lth2 is the center of the reaction range R2. It is set to the average value of the value and the median value of the reaction range R3. Further, the non-reaction range NR1 is set in a range centered on the first threshold value Lth1, and the non-reaction range NR2 is set in a range centered on the second threshold value Lth2.

(Modification 2)
The configuration of the head-up display system to which the virtual image display device according to the present invention can be applied is not limited to the configuration shown in FIG. 1 as long as each of the plurality of virtual images is simultaneously displayed by a plurality of virtual image distances. For example, instead of the front window 25, the head-up display may allow the driver to visually recognize a virtual image by reflecting the light of the display 4 on a combiner provided between the driver and the landscape in front of the vehicle. ..

Further, the virtual image display device according to the present invention is also suitably applied to a head-mounted display that simultaneously displays each of a plurality of virtual images at a plurality of virtual image distances. In this case, for example, a vehicle interior photographing camera 6 having a user's gesture in the photographing range is provided on the temple or the bridge portion of the head-mounted display.

2 Head-up display system 4 Display 6 Vehicle interior shooting camera 8 Concave mirror 10 Control unit 11 Display control unit 12 Gesture detection unit 13 Target identification unit 25 Front window 28 Bonnet 29 Dashboard

Claims (6)

It is a virtual image display device mounted on a vehicle.
A display control means for simultaneously displaying a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as a virtual image on a display means.
A detection means for detecting the direction in which the observer who sits on the seat of the vehicle and visually recognizes the virtual image reaches out, and the gesture length corresponding to the distance reached out.
If the direction in which the first object is displayed and the direction in which the second object is displayed are not close to each other, the first object or the second object is based on the direction in which the hand is reached. Is specified as the object selected by the observer, and
When the direction in which the first object is displayed and the direction in which the second object is displayed are close to each other , the first object is based on the direction in which the hand is reached and the gesture length. Alternatively, a specific means for identifying any of the second objects as the object selected by the observer, and
A virtual image display device characterized by comprising. The display control means visually recognizes the first object corresponding to the first object around the vehicle to the first distance set according to the distance between the vehicle and the first object. Moreover, the display means so that the second object corresponding to the second object around the vehicle is visually recognized at the second distance set according to the distance between the vehicle and the second object. The virtual image display device according to claim 1, wherein the object is displayed on the screen. When the observer recognizes the gesture of paying the hand in a direction different from the direction in which the hand is reached after the object selected by the observer is specified by the specific means, the object is related to the selected object. The virtual image display device according to claim 1 or 2, further comprising an execution means for executing the processing. It is a control method executed by the virtual image display device mounted on the vehicle .
A display control step of simultaneously displaying a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as a virtual image on a display means.
A detection step of detecting the direction in which the observer sitting on the seat of the vehicle and visually recognizing the virtual image reaches out, and the gesture length corresponding to the distance reached out.
If the direction in which the first object is displayed and the direction in which the second object is displayed are not close to each other, the first object or the second object is based on the direction in which the hand is reached. Is specified as the object selected by the observer, and
When the direction in which the first object is displayed and the direction in which the second object is displayed are close to each other , the first object is based on the direction in which the hand is reached and the gesture length. Alternatively, a specific step of specifying any of the second objects as an object selected by the observer, and
A control method characterized by having. A program executed by a computer installed in a vehicle .
A display control means for simultaneously displaying a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as a virtual image on a display means.
A detection means for detecting the direction in which the observer who sits on the seat of the vehicle and visually recognizes the virtual image reaches out, and the gesture length corresponding to the distance reached out.
If the direction in which the first object is displayed and the direction in which the second object is displayed are not close to each other, the first object or the second object is based on the direction in which the hand is reached. Is specified as the object selected by the observer, and
When the direction in which the first object is displayed and the direction in which the second object is displayed are close to each other , the first object is based on the direction in which the hand is reached and the gesture length. Alternatively, a program that causes the computer to function as a specific means for identifying any of the second objects as an object selected by the observer. A storage medium that stores the program according to claim 5.

Images