JP2022121625A - Virtual image display device, control method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a virtual image display device that is able to accurately specify a virtual image indicated by a user even when directions of displayed virtual images are close when the user performs a gesture on the virtual image.

SOLUTION: A display control unit 11 of a head-up display system 2 simultaneously displays a first object Ob1 and a second object Ob2 visually recognized at a virtual image distance farther than a virtual image distance of the first object Ob2 on a front window 25 as virtual images Iv1 and Iv2, respectively. A gesture detection unit 12 detects a gesture length Ltag that is a distance to which an observer who visually recognizes the virtual images extends his/her hand. Based on the gesture length Ltag, an object specifying unit 13 specifies either the first object Ob1 or the second object Ob2 as an object selected by the observer.

SELECTED DRAWING: Figure 3

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to technology for displaying virtual images.

2. Description of the Related Art Conventionally, technology for changing display contents in a head-up display in accordance with user's gestures is known. For example, in Patent Document 1, a viewpoint detection camera that captures the driver's eyes and a gesture detection camera that captures the driver's hand are provided. A head-up display is disclosed that identifies a virtual image on the driver's line of sight by detecting the eye position and fingertip position of the driver, and erases the identified virtual image when a predetermined action is detected. Further, Patent Document 2 discloses a head-up display capable of simultaneously displaying at least two virtual images on the front window and having different display sizes and different focal lengths with respect to the driver's virtual images for each virtual image. ing.

JP 2005-138755 A JP 2006-106254 A

In a virtual image display device such as a head-up display capable of displaying virtual images with different virtual image distances, the method described in Patent Document 1 is used when the virtual image distances of the virtual images to be displayed are long but the displayed directions are close. , it is difficult to identify the virtual image pointed by the user. SUMMARY OF THE INVENTION Accordingly, the present invention provides a virtual image display device capable of accurately specifying a virtual image pointed by a user when the user makes a gesture with respect to the virtual image, even if the directions of the displayed virtual images are close to each other. The main purpose is to

The claimed invention is a virtual image display device,
display control means for causing a display means to display, as virtual images, a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance;
a detecting means for detecting a direction in which an observer who sits on a seat and visually recognizes the virtual image extends a hand and a gesture length corresponding to the distance of the extended hand;
identifying means for identifying either the first object or the second object as the object selected by the observer based on the direction in which the hand is extended and the length of the gesture;
characterized by comprising

Further, the claimed invention is a control method executed by a virtual image display device,
a display control step of displaying a first object visually recognized at a first distance and a second object visually recognized at a second distance farther than the first distance on a display means as virtual images;
a detection step of detecting a direction in which an observer who sits on a seat and visually recognizes the virtual image stretches out a hand and a gesture length corresponding to a distance of the stretched hand;
an identifying step of identifying either the first object or the second object as the object selected by the observer based on the direction in which the hand is extended and the length of the gesture;
characterized by having

Further, the claimed invention is a program executed by a computer,
display control means for causing a display means to display, as virtual images, a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance;
a detecting means for detecting a direction in which an observer who sits on a seat and visually recognizes the virtual image extends a hand and a gesture length corresponding to the distance of the extended hand;
The computer is caused to function as identification means for identifying either the first object or the second object as the object selected by the observer based on the direction in which the hand is extended and the length of the gesture. and

1 shows a configuration example of a head-up display system; It is a block diagram of the control part of a head up display system. 4 is a flow chart showing a processing procedure according to an embodiment; 4A and 4B show display examples of the front window before and after receiving an operation of selecting a mark image and displaying detailed information of a facility. A display example on the front window when the head-up display system has three displays is shown. FIG. 10 is a diagram showing the relationship between the position of the outstretched hand and the virtual image selected at the position of each hand; The relationship between each non-reactive range and the position of the outstretched hand is shown. 4 shows the relationship between the position of the virtual image and the responsive range and the non-responsive range.

In one preferred embodiment of the present invention, the virtual image display device simultaneously displays a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance as virtual images. display control means for displaying the first object or the second object based on the distance to which the observer who visually recognizes the virtual image extends the hand, and the distance to which the observer extends the hand as an object selected by the observer.

The virtual image display device includes display control means, detection means, and identification means. The display control means simultaneously displays a first object visually recognized at a first distance and a second object visually recognized at a second distance farther than the first distance as virtual images on the display unit. The detection means detects the distance that an observer viewing the virtual image extends his/her hand. The identifying means identifies either the first object or the second object as the object selected by the observer based on the distance at which the observer extends his/her hand. According to this configuration, when two objects having different virtual image distances are displayed as virtual images on the display means, the virtual image display device preferably selects the object selected by the observer based on the distance at which the observer extends his or her hand. can be specified.

In one aspect of the virtual image display device, the detecting means further detects a direction in which the hand of the observer extends, and the identifying means detects the first virtual image within a predetermined range from the direction in which the hand of the observer extends. When the display positions of the virtual images corresponding to the object and the second object are included, the observer can select either the first object or the second object based on the distance at which the observer extends his or her hand. is identified as the selected object. According to this aspect, even when a plurality of objects are displayed in the direction in which the observer extends his hand, the virtual image display device can display the object selected by the observer based on the distance in which the observer extends his hand. can be suitably specified.

In another aspect of the virtual image display device, the identifying means identifies the first object as the object selected by the observer when the distance at which the observer stretches out the hand is shorter than a reference distance, The second object is identified as the object selected by the observer when the distance over which the observer extends his/her hand is longer than the reference distance. In this aspect, the virtual image display device sets a longer distance for the observer to extend the hand when selecting the second object displayed farther than when selecting the first object displayed closer. . As a result, the operation of selecting an object can be made to match the visual sense of distance of the virtual image, and the observer can be suitably executed.

In another aspect of the virtual image display device, the identifying means changes the reference distance according to the first distance and the second distance. According to this aspect, the virtual image display device can cause the observer to suitably perform the operation of selecting an object by making it match the visual sense of distance of the virtual image.

In another aspect of the virtual image display device, the specifying means determines whether the first object and the second object are different from each other when the distance to which the observer reaches out is included in the predetermined distance range including the reference distance. Neither is identified as the object selected by the observer. According to this aspect, the virtual image display device can suitably suppress erroneous recognition of the object selected by the observer.

In another aspect of the virtual image display device, the virtual image display device includes recognition means for recognizing a gesture of the observer, and after the object selected by the observer is specified by the specifying means, a predetermined and execution means for executing processing associated with the selected object when a gesture of is recognized. According to this aspect, the virtual image display device can suitably execute processing related to the object selected by the user according to the user's gesture.

In another preferred embodiment of the present invention, there is provided a control method executed by a virtual image display device, wherein a first object is visually recognized at a first distance, and a second object is visually recognized at a second distance farther than the first distance. are simultaneously displayed as a virtual image on a display means, a detection step of detecting a distance to which an observer who visually recognizes the virtual image extends his/her hand, and based on the distance to which the observer extends his/her hand, the first and a specifying step of specifying either the first object or the second object as the object selected by the observer. By executing this control method, the virtual image display device displays two objects with different virtual image distances as virtual images on the display means, and displays the object selected by the observer at a distance that the observer extends his/her hand. It can be preferably specified based on.

In yet another embodiment of the present invention, there is provided a program executed by a computer, wherein a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance are virtual images. display control means for simultaneously displaying on a display means; detection means for detecting a distance to which an observer who visually recognizes the virtual image extends his/her hand; The computer is caused to function as specifying means for specifying one of the second objects as the object selected by the observer. By executing this program, the computer, when two objects having different virtual image distances are displayed as virtual images on the display means, preferably selects the object selected by the observer based on the distance at which the observer extends his/her hand. can be specified. Preferably, the program is stored in a storage medium.

Preferred embodiments of the present invention will be described below with reference to the drawings.

[System configuration]
FIG. 1 is a configuration example of a head-up display system 2 according to an embodiment. As shown in FIG. 1, the head-up display system 2 according to this embodiment simultaneously displays two virtual images and can be operated by gestures. (When not distinguished, it is collectively referred to as a "display 4".), a vehicle interior photographing camera 6, a concave mirror 8, and a later-described control unit (not shown). The head-up display system 2 is attached to a vehicle having a front window 25, a ceiling portion 27, a bonnet 28, and a dashboard 29.

The first and second displays 41 and 42 are provided in the dashboard 29 and display images showing facility information, route guidance information, traveling speed information, and other information that assists driving. The first display 41 is installed at a position that does not block the optical path from the second display 42 to the concave mirror 8 . The first and second displays 41, 42 are also directed toward a concave mirror 8 provided in the dashboard 29, and the light emitted from the first and second displays 41, 42 (also called "display light"). ) is incident on the concave mirror 8 . In this case, the light reflected by the concave mirror 8 reaches the front window 25 through the opening 89 provided in the dashboard 29, and is reflected by the front window 25 to reach the eye position of the driver. In FIG. 1, the optical path of light emitted from the first display 41 is indicated by a dashed line, and the optical path of light emitted from the second display 42 is indicated by a dashed-dotted line.

In this way, the first and second displays 41 and 42 cause the display light to reach the positions of the driver's eyes and allow the driver to visually recognize the virtual images Iv1 and Iv2, respectively. Hereinafter, the image displayed as the virtual image Iv1 will be referred to as the "first object Ob1", the image displayed as the virtual image Iv2 will be referred to as the "second object Ob2", and when they are not distinguished, they will simply be referred to as the "object".

Here, the distance from the position of the driver's eyes to the virtual image (also referred to simply as "virtual image distance") varies depending on the optical path length from the display 4 to the front window 25. FIG. In the example of FIG. 1, the second display 42 is installed at a position farther from the concave mirror 8 than the first display 41, so the virtual image distance of the virtual image Iv2 is longer than the virtual image distance of the virtual image Iv1. there is The virtual image distance of the virtual image Iv1 is an example of the "first distance" in the present invention, and the virtual image distance of the virtual image Iv2 is an example of the "second distance" in the present invention.

The first and second displays 41 and 42 may be liquid crystal displays, or may be screens on which light forming an image is projected to display an intermediate image. The display 4, the concave mirror 8, and the front window 25 are examples of "display means" in the present invention. At least one of the first display 41 and the second display 42 may be configured to be movable so that the user can adjust the distance to the concave mirror 8 .

The vehicle interior photographing camera 6 is a camera for photographing the interior of the vehicle, and is installed so that the face and hands of the driver sitting in the driver's seat are within the photographing range. An image captured by the vehicle interior camera 6 (also referred to as a “captured image Im”) is used to measure the three-dimensional position of the driver's hand. The vehicle interior photographing camera 6 is preferably a stereo camera.

The concave mirror 8 reflects the display light emitted from the first display 41 and the second display 42 toward the opening 89 provided in the dashboard 29 to reach the front window 25 . In this case, the concave mirror 8 magnifies and reflects the image indicated by the display light. In addition, the concave mirror 8 may be provided for each of the first display 41 and the second display 42 . Since the optical path length from the first display 41 to the concave mirror 8 is shorter than the optical path length from the second display 42 to the concave mirror 8, the virtual image Iv1 has a shorter virtual image distance than the virtual image Iv2.

In addition, although the case where a driver|operator is made into an operator henceforth is demonstrated, another passenger|crew may be an operator. Even in this case, the vehicle interior photographing camera 6 is directed so that the operator's face and hands are within the photographing range.

[Function configuration]
The head-up display system 2 has a control unit that controls the head-up display system 2 at an arbitrary position inside the vehicle. FIG. 2 is a block diagram showing the functional configuration of the control section 10 of the head-up display system 2. As shown in FIG. As shown in FIG. 2 , the control unit 10 is electrically connected to the display 4 and the vehicle interior photography camera 6 and has a display control unit 11 , a gesture detection unit 12 and an object identification unit 13 . Note that the display control unit 11, the gesture detection unit 12, and the target identification unit 13 may be configured integrally, or may be installed at separate locations while being electrically connected.

The display control unit 11 generates an image signal representing an object to be displayed on each of the first display 41 and the second display 42 . For example, when displaying a mark image indicating the position of a facility as an object, the display control unit 11 places the mark image in a position visible to the driver through the front window 25 based on the output of a GPS receiver (not shown), map data, or the like. An existing facility is recognized and an image signal representing a mark image representing the facility is generated. At this time, the display control unit 11 stores the respective virtual image distances of the virtual images Iv1 and Iv2 in advance, and displays the first display 41 or It is determined on which of the second displays 42 the mark image is to be displayed.

In addition, the display control unit 11 supplies information about an object to be displayed as a virtual image (also referred to as “object information Io”) to the target identification unit 13 . Here, the object information Io includes, for example, information on the position and shape on the front window 25, information on the virtual image distance when displayed as a virtual image, information indicating whether gesture operation is possible, and the like. Note that when generating the object information Io, the display control unit 11 may determine whether or not each object can be gesture-operated based on the type of the object. For example, when the object is a mark image, the display control unit 11 determines that the gesture operation is possible because detailed information of the facility indicated by the mark image can be extracted from map data or the like.

In addition to the mark image, the display control unit 11 may display various images such as an image indicating a distance and an image of an arrow for route guidance as an object. In this case, the display control unit 11 may determine, for example, which one of the first display 41 and the second display 42 should be used for each type of information to be displayed.

The gesture detection unit 12 recognizes the driver's gesture based on the photographed image Im supplied from the vehicle interior photographing camera 6 . In this embodiment, when the gesture detection unit 12 detects a pointing motion based on known image recognition processing, the gesture detection unit 12 detects the length of the extended hand (also referred to as “gesture length Ltag”) and the pointing direction. (also called “pointing direction Dtag”) is detected. For example, the gesture detection unit 12 recognizes the positions of the driver's face (eyes) and hands (fingers), and calculates the distance between the detected face and hands as the gesture length Ltag. The gesture detection unit 12 also calculates the direction of the hand position with respect to the face position as the pointing direction Dtag. If the vehicle interior shooting camera 6 is a monocular camera that is not a stereo camera, the gesture detection unit 12 is based on the measurement values of a plurality of distance sensors (not shown) provided on the dashboard 29 and the captured image Im. A gesture length Ltag may be calculated. The gesture detection unit 12 supplies the information of the gesture length Ltag and the pointing direction Dtag to the target identification unit 13 .

Based on the object information Io received from the display control unit 11 and the gesture length Ltag and pointing direction Dtag notified from the gesture detection unit 12, the object identification unit 13 selects an object that the driver can operate and which the driver points to. identify the object Specifically, the object identification unit 13 first identifies an object displayed around the direction in which the driver's face and hands are joined based on the object information Io and the pointing direction Dtag. At this time, the target identification unit 13 refers to the object information Io and excludes objects that cannot be gesture-operated from the identification targets. Further, when there are a first object Ob1 displayed as a virtual image Iv1 and a second object Ob2 displayed as a virtual image Iv2 around the pointing direction Dtag, the target identifying unit 13 determines which one of them based on the gesture length Ltag. Determines whether the object is the target of the gesture operation. In this case, the target identifying unit 13 stores in advance a threshold value of the gesture length Ltag (also referred to as "threshold value Lth"). Then, when the gesture length Ltag is equal to or greater than the threshold Lth, the target identifying unit 13 determines that the second object Ob2 having a long virtual image distance is the operation target. It is determined that the first object Ob1 with a short distance is the operation target. Then, when the target identification unit 13 recognizes the object that is the operation target, the target identification unit 13 transmits information for identifying the object to the display control unit 11 . The threshold Lth is an example of the "reference distance" in the present invention.

After the target specifying unit 13 recognizes the object to be operated, the display control unit 11 highlights the object specified by the target specifying unit 13 by changing the color of the object, thereby displaying the selected mark image to the driver. Present. From this state, when the gesture recognition unit 12 further recognizes that the driver has made a predetermined gesture such as waving his hand to the side, the display control unit 11 responds to the gesture recognized by the gesture detection unit 12. If so, perform the processing associated with that object. Specifically, the display control unit 11 erases the display of the object or displays additional information about the object.

The display control unit 11 is an example of the “display control unit” and the “execution unit” in the present invention, and the gesture detection unit 12 is an example of the “detection unit” and the “recognition unit” in the present invention. The identifying unit 13 is an example of "identifying means" in the present invention.

[Processing flow]
FIG. 3 is a flow chart showing the processing procedure executed by the control unit 10 in this embodiment. The control unit 10 repeatedly executes the processing of the flowchart shown in FIG. Note that FIG. 3 assumes that the display control unit 11 appropriately displays an object such as a mark image on the first display 41 and/or the second display 42, and also displays object information Io corresponding to the object being displayed. It is assumed that the data is transmitted to the specifying unit 13 .

First, when the gesture detection unit 12 recognizes the pointing action of the driver based on the photographed image Im or the like received from the vehicle interior photographing camera 6, the gesture length Ltag corresponding to the distance the hand is stretched and the hand A pointing direction Dtag, which is the extended direction, is detected (step S101). Then, the gesture detection unit 12 supplies information on the detected gesture length Ltag and pointing direction Dtag to the target identification unit 13 .

Next, the target identifying unit 13 determines whether or not an operable object is displayed around the pointing direction Dtag (step S102). Specifically, the object identifying unit 13 first recognizes the display position on the front window 25 of each of the displayed objects that can be gesture-operated, based on the object information Io received from the display control unit 11 . Then, the display control unit 11 determines whether or not there is an object whose display position is close to the position on the front window 25 indicated by the pointing direction Dtag notified from the gesture detection unit 12 .

Then, when the object specifying unit 13 determines that an operable object is displayed around the pointing direction Dtag (step S102; Yes), the process proceeds to step S103. On the other hand, when the target specifying unit 13 determines that an operable object is not displayed around the pointing direction Dtag (step S102; No), it determines that the object is not being operated, and proceeds to step S101. Return processing.

Next, in step S103, the object specifying unit 13 determines whether or not objects with different virtual image distances are detected in step S102 (step S103). That is, the object identifying unit 13 determines whether or not there are a plurality of operable objects around the pointing direction Dtag and whether or not the virtual image distances of these objects are different. If the object detected in step S102 is an object with a different virtual image distance (step S103; Yes), the target identification unit 13 advances the process to step S104. On the other hand, if the object identifying unit 13 does not detect objects with different virtual image distances (step S103; No), that is, if objects with different virtual image distances are not superimposed and displayed around the pointing direction Dtag, recognizes that an operable object around the pointing direction Dtag has been selected (step S108). When a plurality of objects having the same virtual image distance are detected in step S102, the target identifying unit 13 recognizes that the object whose center position is closest to the pointing direction Dtag has been selected, for example.

Next, in step S104, the object identifying unit 13 sets a threshold Lth corresponding to the virtual image distance of each object having a different virtual image distance (that is, the first object Ob1 and the second object Ob2) (step S104). In this case, the object identifying unit 13 may calculate the threshold Lth by referring to a predetermined formula based on the respective virtual image distances of the first object Ob1 and the second object Ob2. It can be extracted and used. Then, the target identifying unit 13 determines whether or not the gesture length Ltag detected in step S101 is equal to or greater than the threshold value Lth (step S105). Then, when the gesture length Ltag is equal to or greater than the threshold Lth (step S105; Yes), the target identifying unit 13 recognizes that the second object Ob2, which is the object with the longer virtual image distance, has been selected (step S106). On the other hand, when the gesture length Ltag is less than the threshold value Lth (step S105; No), the target identifying unit 13 recognizes that the first object Ob1, which is the object with the shorter virtual image distance, has been selected (step S107).

Then, the display control unit 11 highlights the object that the object identifying unit 13 recognizes as having been selected by pointing at steps S106 to S108 (step S109). After that, when the gesture detection unit 12 further detects a predetermined gesture operation on the selected object, the gesture detection unit 12 displays additional information of the selected object or clears the display according to the detected gesture operation. do.

[Display example]
FIG. 4 shows a display example of the front window 25 before and after execution of the gesture when the driver selects the mark image displaying the facility name by gesture to display the detailed information of the facility.

As shown in FIG. 4A, the display control unit 11 displays a mark image 51 representing the facility name "A store" of the building located in front of the intersection, and also displays the facility name "A store" of the building across the road from the A store. A mark image 52 representing "B Mart" is displayed as a virtual image. In this case, the display control unit 11 displays the mark image 51 on the first display 41 as the first object Ob1 ( That is, it is displayed as a virtual image Iv1). Further, the display control unit 11 displays the mark image 52 on the second display 42 as the second object Ob2 (that is, It is displayed as a virtual image Iv2).

Here, the display positions of the mark image 51 and the mark image 52 on the front window 25 are close to each other. Therefore, when the driver points to either the mark images 51 or 52 in the state of FIG. recognized as an object. In this case, the target identifying unit 13 compares the threshold Lth set according to the virtual image distances of the virtual images Iv1 and Iv2 with the gesture length Ltag detected by the gesture detecting unit 12 to determine which of the mark images 51 and 52 is selected.

In the example of FIG. 4B, since the gesture length Ltag is equal to or greater than the threshold value Lth, the target identifying unit 13 recognizes that the mark image 52 has been selected by the pointing motion, and the selected mark image 52 It is highlighted by adding a border effect to it. Further, after recognizing the pointing motion, the gesture detection unit 12 further detects a predetermined gesture, so that the display control unit 11 displays detailed information such as business hours of the facility "B Mart" corresponding to the mark image 52. The image 53 is displayed by referring to map data or the like. At this time, preferably, the display control unit 11 displays the image 53 at the same virtual image distance as the selected mark image 52 , that is, with the second display 42 .

As described above, the display control unit 11 of the head-up display system 2 according to the present embodiment distinguishes between the first object Ob1 and the second object Ob2 visually recognized at a virtual image distance farther than the virtual image distance of the first object Ob2. , are simultaneously displayed on the front window 25 as virtual images Iv1 and Iv2, respectively. The gesture detection unit 12 detects the gesture length Ltag, which is the distance that the observer who visually recognizes the virtual image extends his/her hand. The target identifying unit 13 identifies either the first object Ob1 or the second object Ob2 as the object selected by the observer based on the gesture length Ltag. According to this configuration, when the two first and second objects Ob1 and Ob2 having different virtual image distances are displayed as virtual images Iv1 and Iv2, respectively, the head-up display system 2 can observe the object selected by the observer. It can be preferably specified based on the distance that the person extended his or her hand.

[Modification]
Modifications suitable for the above embodiment will be described below. The following modifications may be applied to the above embodiments in any combination.

(Modification 1)
In the example of FIG. 1 , the head-up display system 2 includes two displays 4, namely a first display 41 and a second display 42. In the example of FIG. Alternatively, head-up display system 2 may have three or more displays 4 . In this case, the displays 4 are installed at different distances from the concave mirror 8, and the objects displayed on the displays 4 are displayed as virtual images at different virtual image distances.

FIG. 5 shows a display example on the front window 25 when the head-up display system 2 has three displays 4 . In the example of FIG. 5, each display 4 is installed in the dashboard 29 with its distance from the concave mirror 8 adjusted so that the virtual image distances of the displayed objects are 10 m, 20 m, and 30 m.

In FIG. 5, the display control unit 11 causes the display 4 that displays the virtual image with the shortest virtual image distance (10 m) to display the mark image 51 of the facility "A store" located ahead by approximately the same distance as the virtual image distance. , the display 4 that displays the virtual image with the longest virtual image distance (30 m) displays the mark image 52 of the facility "B store" that is visually recognized in front by approximately the same distance as the virtual image distance. Furthermore, the display control unit 11 superimposes an arrow image 54 indicating the direction of travel on the intersection that is visually recognized in front by approximately the same distance as the virtual image on the display 4 that displays the virtual image at the intermediate virtual image distance (20 m). is displayed.

In the example of FIG. 5 , when generating the object information Io of the arrow image 54 , the display control unit 11 includes information indicating that the gesture operation is disabled in the object information Io and transmits the object information Io to the target identification unit 13 . As a result, even when the target identification unit 13 determines that the arrow image 54 exists around the pointing direction Dtag, the target specifying unit 13 does not recognize the arrow image 54 as a target to be selected by the pointing motion. On the other hand, when the target identifying unit 13 determines that the mark images 51 and 52 exist around the pointing direction Dtag, it determines which of the detected gesture lengths Ltag is selected based on two thresholds Lth described later.

Next, a method of setting the threshold Lth when there are three displays 4 will be described.

By setting two thresholds Lth for the gesture length Ltag, the target identifying unit 13 determines a range in which an object with the longest virtual image distance is selected, a range in which it is determined that an object with the second longest virtual image distance is selected, and a range for determining that the object with the shortest virtual image distance is selected. Hereinafter, the smaller threshold Lth of the two thresholds Lth will be referred to as "first threshold Lth1", and the larger threshold Lth will also be referred to as "second threshold Lth2".

FIG. 6 is a diagram showing the relationship between the position of the outstretched hand and the virtual image selected at each hand position. In the example of FIG. 6, the head-up display system 2 displays virtual images Iv1 to Iv3 having different virtual image distances around the pointing direction Dtag. Also, the reaction range R1 to R3 indicates the range of the gesture length Ltag divided by the first threshold Lth1 and the second threshold Lth2.

In the example of FIG. 6, when the gesture detection unit 12 detects a gesture pointing in the direction of the virtual images Iv1 to Iv3, the target identification unit 13 detects the virtual image Iv1 if the detected gesture length Ltag belongs to the reaction range R1. It recognizes that the object corresponding to is selected. Further, when the detected gesture length Ltag belongs to the reaction range R2, the target identifying unit 13 recognizes that the object corresponding to the virtual image Iv2 has been selected. Similarly, when the detected gesture length Ltag belongs to the reaction range R3, the target identification unit 13 recognizes that the object corresponding to the virtual image Iv3 has been selected.

In this way, even when three virtual images with different virtual image distances are displayed, the target identifying unit 13 can accurately recognize the object selected by the user with a gesture based on the gesture length Ltag. Note that even when there are four or more displays 4, by setting the threshold Lth according to the number of displays 4, the object selected by the user's gesture can be accurately recognized.

Preferably, when the gesture length Ltag belongs to the vicinity range including the set threshold value Lth, the target identifying unit 13 can perform the operation of selecting the object regardless of the magnitude relationship between the gesture length Ltag and the threshold value Lth. may be considered not to have been done.

FIG. 7 shows the relationship between each range and the gesture length Ltag when non-reaction ranges NR1 and NR2 are provided between the response ranges R1 to R3 shown in FIG. show.

In the example of FIG. 7, the target identifying unit 13 sets the non-reacting range NR1 to a neighborhood range within a predetermined distance before and after the first threshold Lth1, and sets the non-reacting range NR2 to a predetermined distance before and after the second threshold Lth2. It is set to a neighborhood range within Then, when the gesture length Ltag belongs to either of the non-reaction ranges NR1 and NR2, the target identifying unit 13 regards the pointing motion as invalid.

Thus, by providing the non-reaction ranges NR1 and NR2, the gesture detection unit 12 can suitably suppress erroneous recognition of the operation of selecting an object. In the example of FIG. 7, the case where there are three displays 4 has been described. good. Even in this case, the target identification unit 13 can preferably suppress erroneous recognition of the object selection operation by setting the vicinity range of each threshold value Lth to a range in which the object selection operation is not accepted.

Further, the target specifying unit 13 may set the first threshold Lth1 and the second threshold Lth2 according to the virtual image distances of the virtual images Iv1 to Iv3, or may set them regardless of the virtual image distances of the virtual images Iv1 to Iv3. good.

FIG. 8 shows the relationship between the positions of the virtual images Iv1-Iv3 in the line-of-sight direction of the driver, the reaction ranges R1-R3 and the non-reaction ranges NR1 and NR2. Specifically, FIG. 8A shows the positional relationship of the virtual images Iv1 to Iv3 in the line-of-sight direction of the driver, and FIG. 8B shows the first threshold Lth1 and Shows the reaction range R1 ~ R3 and the non-reaction range NR1, NR2 when the second threshold Lth2 is set, FIG. Reaction ranges R1 to R3 and non-reaction ranges NR1 and NR2 are shown when Lth2 is set. In both examples of FIGS. 8B and 8C, the reaction ranges R1 to R3 are set to have the same length.

In the example of FIG. 8(B), the difference between the median value of the reaction range R1 and the median value of the reaction range R2 (see arrow B1) and the difference between the median value of the reaction range R2 and the median value of the reaction range R3 (arrow B2) is equal to the ratio of the difference in virtual image distance between the virtual images Iv1 and Iv2 (see arrow A1) and the difference in virtual image distance between the virtual images Iv2 and Iv3 (see arrow A2). In this case, the first threshold Lth1 is set to the average value of the median of the reaction ranges R1 and R2, and the second threshold Lth2 is set to the median of the reaction ranges R2 and R3. is set to the average value of Further, the non-reaction range NR1 is set to a range centered on the first threshold Lth1, and the non-reaction range NR2 is set to a range centered on the second threshold Lth2.

On the other hand, in the example of FIG. 8(C), the difference between the median value of the reaction range R1 and the median value of the reaction range R2 (see arrow C1) and the difference between the median value of the reaction range R2 and the median value of the reaction range R3 (See arrow C2). That is, in this case, the reaction ranges R1 to R3 are set at regular intervals regardless of the virtual image distances of the virtual images Iv1 to Iv3. Further, as in the example of FIG. 8B, the first threshold Lth1 is set to the average value of the median value of the reaction range R1 and the median value of the reaction range R2, and the second threshold Lth2 is set to the center of the reaction range R2. and the median of the response range R3. Further, the non-reaction range NR1 is set to a range centered on the first threshold Lth1, and the non-reaction range NR2 is set to a range centered on the second threshold Lth2.

(Modification 2)
The configuration of a head-up display system to which the virtual image display device according to the present invention can be applied is not limited to the configuration shown in FIG. For example, the head-up display may allow the driver to visually recognize a virtual image by reflecting light from the display 4 on a combiner provided between the driver and the scenery ahead of the vehicle instead of the front window 25. .

Also, the virtual image display device according to the present invention is preferably applied to a head-mounted display that simultaneously displays a plurality of virtual images at a plurality of virtual image distances. In this case, for example, the temple or bridge portion of the head-mounted display is provided with an in-vehicle camera 6 that captures the user's gestures.

2 head-up display system 4 display 6 in-vehicle camera 8 concave mirror 10 control unit 11 display control unit 12 gesture detection unit 13 object identification unit 25 front window 28 bonnet 29 dashboard

Claims (8)

display control means for causing a display means to display, as virtual images, a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance;
a detecting means for detecting a direction in which an observer who sits on a seat and visually recognizes the virtual image extends a hand and a gesture length corresponding to the distance of the extended hand;
identifying means for identifying either the first object or the second object as the object selected by the observer based on the direction in which the hand is extended and the length of the gesture;
A virtual image display device comprising: The identification means determines that the observer reaches out his or her hand when the display positions of the virtual images corresponding to the first object and the second object are within a predetermined range from a direction in which the observer extends the hand. 2. The virtual image display device according to claim 1, wherein either the first object or the second object is specified as the object selected by the observer based on a gesture length corresponding to a distance. The identifying means identifies the first object as an object selected by the observer when a gesture length corresponding to a distance over which the observer extends the hand is shorter than a reference distance. 3. The virtual image display device according to claim 1, wherein the second object is specified as the object selected by the observer when the gesture length corresponding to the stretched distance is longer than the reference distance. The display control means displays the first object and the second object simultaneously on the display means,
4. The virtual image display device according to claim 3, wherein said specifying means changes said reference distance according to said first distance and said second distance. The identifying means determines that, when the predetermined distance range including the reference distance includes a gesture length corresponding to the distance that the observer extended his or her hand, both the first object and the second object are determined by the observer. 4. The virtual image display device according to claim 3, wherein the selected object is not specified. A control method executed by a virtual image display device,
a display control step of displaying a first object visually recognized at a first distance and a second object visually recognized at a second distance farther than the first distance on a display means as virtual images;
a detection step of detecting a direction in which an observer who sits on a seat and visually recognizes the virtual image stretches out a hand and a gesture length corresponding to a distance of the stretched hand;
an identifying step of identifying either the first object or the second object as the object selected by the observer based on the direction in which the hand is extended and the length of the gesture;
A control method characterized by having A program executed by a computer,
display control means for causing a display means to display, as virtual images, a first object to be visually recognized at a first distance and a second object to be visually recognized at a second distance farther than the first distance;
a detecting means for detecting a direction in which an observer who sits on a seat and visually recognizes the virtual image extends a hand and a gesture length corresponding to the distance of the extended hand;
A program that causes the computer to function as identifying means for identifying either the first object or the second object as the object selected by the observer based on the direction in which the hand is stretched and the length of the gesture. A storage medium storing the program according to claim 7 .

Images