JP2021046171A - Display control device and display control method - Google Patents

Abstract

To provide a display control device and so on which can shorten a time until a user is allowed to visually recognize a virtual image.SOLUTION: A display control device 50 includes: an acquisition unit 101 for acquiring the detection results of objects in the periphery of a vehicle 300, distance information indicating distances from the vehicle 300 to the objects in the periphery, and a user's line-of-sight direction; a first calculation unit 102 for determining, on the basis of the detection results, a first object on which a virtual image is to be superposed, and calculating a first distance from the vehicle to the first object; a second calculation unit 103 for calculating a second distance from the vehicle to a second object in the line-of-sight direction by using the distance information and the line-of-sight direction; a determination unit 104 for determining a depth position at which the user is allowed to recognize the virtual image, as a third position closer to a second position at the second distance from the vehicle than a first position at the first distance from the vehicle; and a control unit 105 for controlling a display device so that the virtual image is visually recognized by the user at the determined depth position and a position which overlaps the first object.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a display control device and a display control method.

Patent Document 1 discloses a vehicle head-up display capable of stepwise switching the perspective of the image formation position of the projected image.

Patent Document 2 discloses an in-vehicle display device that sets a display distance from the driver's eyes to a virtual image and controls the display of the virtual image at the set display distance.

Japanese Unexamined Patent Publication No. 9-185012 JP-A-2017-226292

The present disclosure provides a display control device and the like that can shorten the time until the user visually recognizes a virtual image.

The display control device according to one aspect of the present disclosure is a display control device provided on a moving body and controls a display device that allows a user to visually recognize an image projected on a screen as a virtual image, and is a display control device around the moving body. Among the surrounding objects, based on the detection result of the object, the acquisition unit that acquires the distance information indicating the distance from the moving body to the surrounding object, and the line-of-sight direction of the user, and the detection result. The first calculation unit that determines the first object on which the virtual image is superimposed and calculates the first distance from the moving body to the first object using the distance information, and the distance information and the line-of-sight direction are used. The second calculation unit that calculates the second distance from the moving body to the second object in the line-of-sight direction among the surrounding objects, and the depth position in the depth direction that makes the user visually recognize the virtual image. A determination unit that determines a third position between the first position separated from the moving body by the first distance and the second position separated by the second distance, the determined depth position, and the first object. A control unit that controls the display device so that the virtual image can be visually recognized by the user at a position overlapping the above.

The above-mentioned comprehensive or specific embodiment may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM, and the system, a method, an integrated circuit, or a computer program. And any combination of recording media may be realized.

According to the display control device of the present disclosure, it is possible to shorten the time until the user visually recognizes the virtual image.

FIG. 1 is a diagram showing a usage example of the video display system according to the present embodiment. FIG. 2 is a diagram showing an example of D1 which is a region where light is projected by the display device according to the present embodiment. FIG. 3 is a block diagram showing an example of the functional configuration of the display control device according to the present embodiment. FIG. 4 is a view showing a scene of a moving vehicle, and is a plan view seen from above. FIG. 5A is a graph for explaining a first example of the determination process for determining the depth position of the determination unit. FIG. 5B is a graph for explaining a second example of the determination process for determining the depth position of the determination unit. FIG. 5C is a graph for explaining a third example of the determination process for determining the depth position of the determination unit. FIG. 6 is a diagram for explaining how the virtual image I1 is viewed from the driver. FIG. 7 is a flowchart for explaining the operation of the display control device according to the present embodiment. FIG. 8 is a flowchart showing another example of the determination process by the determination unit.

(Knowledge on which this disclosure was based)
As described above, in Patent Documents 1 and 2, a display device called a three-dimensional augmented reality head-up display (3D-AR-HUD) that can change the position at which the user visually recognizes a virtual image in the direction of the user's line of sight Is disclosed. Such a 3D-AR-HUD can be mounted on a vehicle, for example, and can notify the user (driver) of the vehicle by displaying a virtual image that there is a pedestrian crossing the front of the vehicle. As described above, in the 3D-AR-HUD, it is possible to support the user in driving the vehicle by displaying a virtual image for notifying the user of the information.

However, in such a 3D-AR-HUD, a virtual image is displayed at a position separated by a predetermined distance or more from the position where the user focuses his eyes and gazes (hereinafter, also referred to as a focal position) in the direction of the user's line of sight. In the case of display, the user takes a lot of time to visually recognize the displayed virtual image because it takes time to move the focus for focusing the eyes. For this reason, it is difficult for the user to receive support for notification by a virtual image in real time.

In order to solve such a problem, the display control device according to one aspect of the present disclosure is a display control device provided on a moving body and controlling a display device that allows a user to visually recognize an image projected on a screen as a virtual image. The acquisition unit that acquires the detection result of the object around the moving body, the distance information indicating the distance from the moving body to the surrounding object, and the line-of-sight direction of the user, and the detection result. Based on this, the first calculation unit that determines the first object on which the virtual image is superimposed among the surrounding objects and calculates the first distance from the moving body to the first object using the distance information. Using the distance information and the line-of-sight direction, a second calculation unit that calculates the second distance from the moving body to the second object in the line-of-sight direction among the surrounding objects, and the virtual image to the user. It was determined as a determination unit that determines the depth position in the depth direction to be visually recognized as a third position between the first position separated from the moving body by the first distance and the second position separated by the second distance. It includes a control unit that controls the display device so that the virtual image can be visually recognized by the user at the depth position and the position overlapping the first object.

According to this, in the depth direction, between the first position based on the first distance to the first object on which the virtual image is superimposed and the second position based on the second distance to the second object in the user's line-of-sight direction. At the position of, the depth position at which the virtual image is visually recognized by the user is determined. That is, the depth position can be determined from the first position to the position closer to the second position. Therefore, the virtual image can be displayed at a position close to the focal position in the user's line of sight, and the user can shorten the time required for the focus movement. Therefore, it is possible to shorten the time until the user visually recognizes the virtual image.

Further, when the difference between the first distance and the second distance is equal to or greater than the first threshold value, the determination unit sets a predetermined fixed position between the first position and the second position as the depth position. May be determined as.

Therefore, when the first position is separated from the second position by the first threshold value or more, the virtual image can be displayed at a position closer to the second position than the first position.

Further, when the difference between the first distance and the second distance is equal to or greater than the first threshold value, the determination unit determines a position closer to the second position than the first position by a fixed distance as the depth position. You may.

Therefore, when the first position is separated from the second position by the first threshold value or more, the virtual image can be displayed at the position with reference to the first position.

Further, when the difference between the first distance and the second distance is equal to or greater than the first threshold value, the determination unit sets a position closer to the second position than the first position by a longer distance as the difference increases. It may be determined as the depth position.

Therefore, when the first position is separated from the second position by the first threshold value or more, the virtual image can be displayed at a position where the distance closer to the second position increases as the first position moves away from the second position.

Further, when the difference between the first distance and the second distance is less than the first threshold value, the determination unit may determine the first position as the depth position.

Therefore, when the first position is less than the first threshold value from the second position, a virtual image can be displayed at the first position.

Further, the first threshold value may be a larger value as the second distance is larger.

Therefore, the virtual image can be displayed at a position based on the second position.

Further, the acquisition unit sequentially acquires the detection result, the distance information, and the line-of-sight direction, and the first calculation unit sequentially acquires the detection result and the distance information, and the first calculation unit obtains the detection result and the distance information. One distance is sequentially calculated, the second calculation unit sequentially calculates the second distance based on the sequentially acquired distance information and the line-of-sight direction, and the determination unit sequentially calculates the first. The depth position is sequentially determined based on the distance and the second distance, and the control unit visually recognizes the virtual image at the sequentially determined depth position and the position overlapping the first object. The display device may be controlled as described above.

Therefore, it is possible to control the depth position for displaying the virtual image in real time.

Further, when the difference between the first depth position determined at the first timing and the second depth position determined at the second timing prior to the first timing is equal to or greater than the second threshold value, the determination unit determines the first depth position. The position between the one depth position and the second depth position may be determined as the depth position of the first timing.

Therefore, the virtual image can be displayed so that the change in the depth position with time does not become large.

A comprehensive or specific embodiment of the display device may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM, and the system, the method, the integrated circuit, or the computer. It may be realized by any combination of programs or recording media.

Hereinafter, the display control device and the display control method according to the embodiment will be described with reference to the drawings. The display device according to the embodiment described below shows a comprehensive or specific example. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the components in the following embodiments, the components not described in the independent claims indicating the highest level concept are described as arbitrary components.

(Embodiment)
[1. Constitution]
FIG. 1 is a diagram showing a usage example of the video display system 100 according to the present embodiment.

The image display system 100 according to the present embodiment includes a display device 10, a first camera 20, a second camera 30, a distance sensor 40, and a display control device 50.

The display device 10 is configured as an in-vehicle head-up display (HUD) and is attached near the upper surface of the dashboard 301 of the vehicle 300. The vehicle 300 is an example of a moving body.

The display device 10 projects light onto the display area R1 of the windshield (windshield) 201, which is a display medium. The projected light is reflected by the windshield 201. This reflected light is directed to the eyes of the driver sitting in the driver's seat, who is the user of the display device 10. The driver perceives the reflected light that has entered his eyes as a virtual image I1 that can be seen on the opposite side (outside of the vehicle) of the windshield 201 against the background of an actual object that can be seen through the windshield 201. That is, the display device 10 displays the virtual image I1 using the windshield 201.

The first camera 20 is a camera that captures an image of the driver's face. For example, the first camera 20 is arranged above the windshield 201 in a posture toward the rear of the vehicle 300, and the area where the driver's face in the driver's seat is located is targeted for imaging. The first camera 20 captures an image including the driver's face at a plurality of different timings. Specifically, the first camera 20 captures an image including the driver's face at a predetermined cycle (for example, 1/20 [second], 1/30 [second], 1/60 [second], etc.). The first camera 20 need only capture an image including the driver's eyes, and is not limited to being arranged above the windshield 201.

The second camera 30 is a camera that captures an image of the surroundings of the vehicle 300. Specifically, the second camera 30 is arranged in a forward posture toward the front side of the vehicle 300, and captures an image of the front of the vehicle 300. The second camera 30 captures an image of the front of the vehicle 300 at a plurality of different timings. Specifically, the second camera 30 captures an image in front of the vehicle 300 at a predetermined cycle (for example, 1/20 [second], 1/30 [second], 1/60 [second], etc.). The second camera 30 may capture an image of the surroundings of the vehicle 300, and may capture an image of the right side of the vehicle 300, not limited to the front of the vehicle 300, or an image of the left side of the vehicle 300. The image may be taken, the image behind the vehicle 300 may be taken, or these images may be combined.

The first camera 20 and the second camera 30 may capture a still image or a moving image as an image.

The distance sensor 40 is a sensor that detects the distance to an object around the vehicle 300. The distance sensor 40 is, for example, a LiDAR (Light Detection and Ranger) that detects the distance to an object around 360 degrees of the vehicle 300. The distance sensor 40 is not limited to LiDAR, and may be a millimeter wave sensor, a stereo camera, or a TOF (Time Of Flight) camera.

The display control device 50 causes the display device 10 to display the detection result obtained by the first camera 20, the second camera 30, and the distance sensor 40, or the processing result by the display control device 50 using the detection result. An image is generated, and the generated image is displayed on the display device 10 as a virtual image. The display control device 50 may be composed of a processor and a memory for storing a program, or may be composed of a dedicated circuit.

The display device 10 displays the image output by the display control device 50 as a virtual image.

FIG. 2 is a diagram showing an example of D1 which is a region where light is projected by the display device 10 in the present embodiment.

As shown in FIG. 2, the display device 10 attached to the dashboard 301 projects, for example, light onto the display area R1 (the area surrounded by the broken line in the figure) located below the driver's seat side of the windshield 201. .. As a result, the virtual image I1 that can be seen on the opposite side (outside of the vehicle) of the windshield 201 when viewed from the driver sitting in the driver's seat is displayed.

FIG. 3 is a block diagram showing an example of the functional configuration of the display control device according to the present embodiment.

The display control device 50 includes an acquisition unit 101, a first calculation unit 102, a second calculation unit 103, a determination unit 104, and a control unit 105 as functional configurations.

The acquisition unit 101 acquires the line-of-sight direction of the user. Specifically, the acquisition unit 101 acquires a plurality of images captured by the first camera 20, and the plurality of images captured by the driver's eyes (eyeballs) are acquired, and the acquired plurality of images are obtained. By performing a predetermined image analysis (eye tracking), the line-of-sight direction of the driver is detected. In this way, the acquisition unit 101 acquires the line-of-sight direction of the driver using the image captured by the first camera 20.

The acquisition unit 101 acquires the detection result of the objects around the vehicle 300. Specifically, the acquisition unit 101 acquires an image captured by the second camera 30 and performs a predetermined image analysis using the acquired image to detect an object around the vehicle 300. More specifically, the acquisition unit 101 performs predetermined video analysis on a plurality of captured videos in front (side or rear) of the vehicle 300 to perform pedestrians, other vehicles (bicycles, motorcycles, etc.). Including), the relative position of other objects with respect to the vehicle (ie, the distance and direction from the vehicle 300) may be detected. Alternatively, the acquisition unit 101 may detect a motion distribution that is a relative position and a relative movement of a pedestrian, another vehicle (including a bicycle, a motorcycle, etc.), and other objects with respect to the vehicle 300.

The acquisition unit 101 may be narrowed down to a person or an object on the traveling path of the vehicle 300, or a person or an object presumed to cross the traveling path of the vehicle 300 in front of the vehicle 300. Further, the acquisition unit 101 may recognize all people or objects (including buildings, structures, etc.) presumed to be included in the driver's field of view as peripheral objects.

As described above, the acquisition unit 101 acquires the detection result of the objects around the vehicle 300 by using the image captured by the second camera 30.

Further, the acquisition unit 101 acquires distance information indicating the distance from the vehicle 300 to the surrounding objects from the distance sensor 40. The distance information is information indicating the distance to the surrounding objects in each of the plurality of directions around the vehicle 300. That is, the distance information is information indicating a plurality of distances corresponding to each of the plurality of directions.

The acquisition unit 101 is supposed to perform a process of detecting the line-of-sight direction of the driver using the image captured by the first camera 20, but the present invention is not limited to this. The process of detecting the line-of-sight direction of the driver may be performed by a device other than the display control device 50. In this case, the acquisition unit 101 acquires the line-of-sight direction of the driver detected by another device.

Further, the acquisition unit 101 is supposed to perform a process of detecting objects around the vehicle 300 using the image captured by the second camera 30, but the process of detecting the surrounding objects is not limited to this. , A device other than the display control device 50 may be used. In this case, the acquisition unit 101 acquires the detection result of surrounding objects by another device.

The first calculation unit 102 determines the first object on which the virtual image is superimposed among the surrounding objects based on the detection result of the surrounding objects acquired by the acquisition unit 101. For example, the first calculation unit 102 may determine a person or an object that is presumed to cross the traveling path of the vehicle 300 in front of the vehicle 300 as the first object on which the virtual image is superimposed. Further, the first calculation unit 102 may determine, for example, another vehicle traveling in front of the vehicle 300 as a first object on which a virtual image is superimposed. Further, the first calculation unit 102 may determine, for example, a building at the destination of movement of the vehicle 300, which is set by the user in a car navigation system or the like, as a first object on which a virtual image is superimposed.

Then, the first calculation unit 102 calculates the first distance from the vehicle 300 to the first object by using the distance information acquired by the acquisition unit 101. Specifically, the first calculation unit 102 determines the direction of the first object with respect to the vehicle 300 by determining the first object. Next, the first calculation unit 102 calculates the first distance by specifying the distance in the direction of the specified first object from the plurality of distances in the plurality of directions indicated by the distance information.

The second calculation unit 103 calculates the second distance from the vehicle 300 to the second object in the line-of-sight direction among the surrounding objects by using the distance information and the line-of-sight direction. That is, the second calculation unit 103 estimates the focal position of the driver's eye by calculating the second distance to the second object, which is presumed that the driver is focusing and gazing. This is because it is considered that a person is often gazing at an object in the line-of-sight direction, and is considered to be focusing the eye on the object being gazed at.

Here, a specific description will be given with reference to FIG. FIG. 4 is a view showing a scene of a moving vehicle, and is a plan view seen from above. FIG. 4 shows a scene in which another vehicle 320 is traveling on the front right side while the vehicle 300 is traveling, and a pedestrian 310 is trying to cross the front of the vehicle 300 on the front left side.

In the display control device 50, the first calculation unit 102 determines, for example, a pedestrian 310 who is about to cross the front of the vehicle 300 as the first object, and walks from the vehicle 300 within a plurality of distances indicated by the distance information. The distance in the first direction D1 which is the direction to the person 310 is calculated as the first distance d1 from the vehicle 300 to the pedestrian 310.

By the way, the driver of the vehicle 300 is looking at the vehicle 320 and is gazing at the vehicle 320. Therefore, in the display control device 50, the second calculation unit 103 recognizes the vehicle 320 as the second object, and among the plurality of distances indicated by the distance information, the distance from the vehicle 300 in the second direction D2 is determined by the vehicle 300. It is calculated as the second distance d2 from to the vehicle 320.

The first object and the second object are located at different positions when viewed from the line-of-sight direction. That is, the first direction D1 when the user sees the first object and the second direction D2 when the user sees the second object are different directions.

The determination unit 104 sets the depth position in the depth direction so that the user can visually recognize the virtual image I1 between the first position P1 which is the first distance d1 away from the vehicle 300 and the second position P2 which is the second distance d2 away. Determined at position P3. Specifically, when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1, the determination unit 104 determines the depth position to the third position P3. On the other hand, when the difference between the first distance d1 and the second distance d2 is less than the first threshold value Δd1, the determination unit 104 determines the first position P1 as the depth position.

The second position P2 is determined at a position where the circle c1 having a radius centered on the driver position P11 and the first direction D1 intersect. Since the virtual image I1 is superimposed on the first object and displayed, the third position P3 is arranged at a position on the first direction D1.

FIG. 5A is a graph for explaining a first example of the determination process for determining the depth position of the determination unit. The graph of FIG. 5A shows the relationship between the first distance and the virtual image distance from the vehicle to the depth position when the second distance is fixed to one value.

In the first example, the determination unit 104 determines a predetermined fixation between the first position P1 and the second position P2 when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1. Determine the position as the depth position. For example, the determination unit 104 determines the virtual image I1 when the first distance d1 is shorter than the second distance d2 by the first threshold value Δd1 or more, that is, when the first object is located between the vehicle 300 and the distance d11. The position in the depth direction in which is displayed is determined to be a position separated from the vehicle 300 by a distance d11. The distance d11 is, for example, a distance shorter than the second distance d2 by the first threshold value Δd1.

FIG. 5B is a graph for explaining a second example of the determination process for determining the depth position of the determination unit. Similar to the graph of FIG. 5A, the graph of FIG. 5B shows the relationship between the first distance and the virtual image distance from the vehicle to the depth position when the second distance is fixed to one value.

In the second example, when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1, the determination unit 104 is positioned closer to the second position P2 than the first position P1 by the fixed distance Δd2. Is determined as the depth position. For example, the determination unit 104 determines the virtual image I1 when the first distance d1 is shorter than the second distance d2 by the first threshold value Δd1 or more, that is, when the first object is located between the vehicle 300 and the distance d11. Is determined to be a position separated from the vehicle 300 by the distance obtained by adding the fixed distance Δd2 to the first distance d1. When the distance obtained by adding the fixed distance Δd2 to the first distance d1 is the distance d11 shorter than the second distance d2 by the first threshold value Δd1, the determination unit 104 determines the position separated from the vehicle 300 by the distance d11. It is determined as a position in the depth direction in which the virtual image I1 is displayed.

FIG. 5C is a graph for explaining a third example of the determination process for determining the depth position of the determination unit. Similar to the graphs of FIGS. 5A and 5B, the graph of FIG. 5C shows the relationship between the first distance and the virtual image distance from the vehicle to the depth position when the second distance is fixed to one value.

In the third example, when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1, the determination unit 104 is positioned at the second position from the first position P1 by a longer distance as the difference increases. The position close to P2 is determined as the depth position. For example, the determination unit 104 determines the virtual image I1 when the first distance d1 is shorter than the second distance d2 by the first threshold value Δd1 or more, that is, when the first object is located between the vehicle 300 and the distance d11. Is determined to be a position separated from the vehicle 300 by the distance obtained by adding the distance Δd3, which increases as the first distance d1 becomes shorter, to the first distance d1.

For example, the relationship information indicating the relationship between the first distance and the depth position is different for each second distance, and a memory (not shown) may store a plurality of different relationship information for each second distance. In this case, the determination unit 104 reads the relationship information corresponding to the calculated second distance d2 from the memory according to the second distance d2 calculated by the second calculation unit 103, and uses the read relationship information. Determine the depth position. For example, in a plurality of relational information, the first threshold value Δd1 is set to a larger value as the second distance is larger.

The control unit 105 controls the display device 10 so that the virtual image I1 can be visually recognized by the user at a depth position determined by the determination unit 104 and at a position overlapping the first object. Specifically, the control unit 105 outputs an image for displaying the virtual image I1 and position information indicating a position for displaying the virtual image I1 to the display device 10, and displays the virtual image I1 at the position indicated by the position information. The display device 10 is controlled so as to cause the display device 10. The position where the virtual image I1 is displayed includes a position where it overlaps with the first object and a depth position. The virtual image I1 is displayed superimposed on the pedestrian 310, for example, as shown in FIG. Note that FIG. 6 is a diagram for explaining how the virtual image I1 is viewed from the driver.

The display device 10 displays the virtual image I1 based on the image at the position indicated by the position information based on the image and the position information output by the control unit 105. As a result, the display device 10 allows the user to visually recognize the virtual image I1 at the position indicated by the position information.

[2. motion]
Next, the operation of the display control device 50 will be described.

FIG. 7 is a flowchart for explaining the operation of the display control device according to the present embodiment.

As shown in FIG. 7, in the display control device 50, first, the acquisition unit 101 detects the object around the vehicle 300, the distance information indicating the distance from the vehicle 300 to the surrounding object, and the line of sight of the driver. Acquire the direction (S11). The acquisition unit 101 does not have to acquire the detection result of surrounding objects, the distance information, and the line-of-sight direction at the same time, and may acquire each at different timings.

Next, the first calculation unit 102 determines the first object on which the virtual image I1 is superimposed among the surrounding objects based on the detection result of the surrounding objects, and uses the distance information from the vehicle 300 to the first object. The first distance of is calculated (S12).

Next, the second calculation unit 103 calculates the second distance from the vehicle 300 to the second object in the line-of-sight direction among the surrounding objects by using the distance information and the line-of-sight direction (S13).

Next, the determination unit 104 determines the depth position in the depth direction in which the virtual image I1 is visually recognized by the user according to the calculated first distance d1 and second distance d2 (S14). Specifically, the determination unit 104 reads the relational information corresponding to the second distance d2 from the memory, identifies the virtual image distance associated with the relational information read by the first distance d1, and sets the virtual image distance from the vehicle 300. A position separated by the specified virtual image distance is determined as the depth position.

Next, the control unit 105 displays the virtual image I1 by controlling the display device 10 so that the virtual image I1 can be visually recognized by the user at the depth position determined by the determination unit 104 and the position overlapping the first object. Displayed on the device 10 (S15).

When step S15 is completed, the process returns to step S11, and the processes of steps S11 to S15 are repeated.

That is, the acquisition unit 101 sequentially acquires the detection result of the objects around the vehicle 300, the distance information indicating the distance from the vehicle 300 to the surrounding objects, and the line-of-sight direction of the driver. The first calculation unit 102 sequentially calculates the first distance d1 based on the detection results and the distance information sequentially acquired. The second calculation unit 103 sequentially calculates the second distance d2 based on the sequentially acquired distance information and the line-of-sight direction. The determination unit 104 sequentially determines the depth position based on the sequentially calculated first distance d1 and second distance d2. The control unit 105 controls the display device 10 so that the virtual image I1 can be visually recognized by the user at the sequentially determined depth position and the position overlapping the first object. Thereby, the depth position for displaying the virtual image I1 can be controlled in real time.

In this case, the determination unit 104 may perform the process shown in FIG. 8 instead of step S14.

FIG. 8 is a flowchart showing another example of the determination process by the determination unit.

Similar to the process described in step S14, the determination unit 104 determines the depth position in the depth direction in which the virtual image I1 is visually recognized by the user according to the calculated first distance d1 and second distance d2 (S21).

Next, the determination unit 104 reads the previously determined depth position from the memory, and determines whether or not the difference between the previously determined depth position and the depth position determined in step S21 is the second threshold value Δd4 or more. (S22).

When the difference is equal to or greater than the second threshold value Δd4 (Yes in S22), the determination unit 104 corrects the position between the previously determined depth position and the depth position determined in step S21 as the depth position. Do (S23).

When step S23 is performed, the determination unit 104 outputs the corrected depth position to the control unit 105 and stores it in the memory. Further, the determination unit 104 outputs the depth position determined in step S21, which is not corrected when step S23 is not performed, to the control unit 105 and stores it in the memory (S24).

As described above, when the difference between the first depth position determined at the first timing and the second depth position determined at the second timing before the first timing is greater than or equal to the second threshold value Δd4, the determination unit 104 determines. The position between the first depth position and the second depth position is determined as the depth position of the first timing. Therefore, the virtual image I1 can be displayed so that the change in the depth position with time does not become large.

[3. Effect etc.]
The display control device 50 according to the present embodiment controls the display device 10 provided on the vehicle 300 and causing the user to recognize the image projected on the screen as a virtual image I1. The display control device 50 includes an acquisition unit 101, a first calculation unit 102, a second calculation unit 103, a determination unit 104, and a control unit 105. The acquisition unit 101 acquires the detection result of the objects around the vehicle 300, the distance information indicating the distance from the vehicle to the surrounding objects, and the line-of-sight direction of the user. Based on the detection result, the first calculation unit 102 determines the first object on which the virtual image I1 is superimposed among the surrounding objects, and calculates the first distance d1 from the vehicle 300 to the first object using the distance information. To do. The second calculation unit 103 calculates the second distance d2 from the vehicle 300 to the second object in the line-of-sight direction among the surrounding objects by using the distance information and the line-of-sight direction. The determination unit 104 sets the depth position in the depth direction so that the user can visually recognize the virtual image I1 between the first position P1 which is the first distance d1 away from the vehicle 300 and the second position P2 which is the second distance d2 away. Determined at position P3. The control unit 105 controls the display device 10 so that the virtual image I1 can be visually recognized by the user at a determined depth position and a position overlapping the first object.

According to this, in the depth direction, the first position P1 based on the first distance d1 to the first object on which the virtual image I1 is superimposed and the second based on the second distance d2 to the second object in the user's line-of-sight direction. At a position between the position P2 and the position P2, a depth position for making the virtual image I1 visible to the user is determined. That is, the depth position can be determined at a position closer to the second position P2 from the first position P1. Therefore, the virtual image I1 can be displayed at a position close to the focal position in the line of sight of the user, and the user can shorten the time required for the focus movement. Therefore, it is possible to shorten the time until the user visually recognizes the virtual image I1.

Further, in the display control device 50 according to the present embodiment, when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1, the determination unit 104 determines the first position P1 and the second position P2. A predetermined fixed position between the two is determined as the depth position.

Therefore, when the first position P1 is separated from the second position P2 by the first threshold value Δd1 or more, the virtual image I1 can be displayed at a position closer to the second position P2 than the first position P1.

Further, in the display control device 50 according to the present embodiment, when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1, the determination unit 104 sets the fixed distance Δd2 from the first position P1. Also determines the position close to the second position P2 as the depth position.

Therefore, when the first position P1 is separated from the second position P2 by the first threshold value Δd1 or more, the virtual image I1 can be displayed at a position with reference to the first position P1.

Further, in the display control device 50 according to the present embodiment, when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1, the determination unit 104 determines that the greater the difference, the longer the distance. A position closer to the second position P2 than the first position P1 is determined as the depth position.

Therefore, when the first position P1 is separated from the second position P2 by the first threshold value Δd1 or more, the virtual image I1 is located at a position where the distance closer to the second position P2 increases as the first position P1 moves away from the second position P2. Can be displayed.

Further, in the display control device 50 according to the present embodiment, when the difference between the first distance d1 and the second distance d2 is less than the first threshold value Δd1, the determination unit 104 determines the first position P1 as the depth position. To do.

Therefore, when the first position P1 is separated from the second position P2 by less than the first threshold value Δd1, the virtual image I1 can be displayed at the first position P1.

Further, in the display control device 50 according to the present embodiment, the first threshold value Δd1 is a larger value as the second distance d2 is larger. Therefore, the virtual image I1 can be displayed at a position based on the second position P2.

[4. Modification example]
In the display control device 50 according to the above embodiment, when the first distance d1 is shorter than the second distance d2 by the first threshold value Δd1 or more, the determination unit 104 sets the depth position to the first position P1 and the second position. Although it is determined that the third position P3 is between P2 and P2, it is not limited to the case where the first distance d1 is shorter than the second distance d2 by the first threshold value Δd1 or more. Further, when the first distance d1 is longer than the second distance d2 by the first threshold value Δd1 or more, the determination unit 104 sets the depth position to the third position P3 between the first position P1 and the second position P2. You may decide. In this way, even when the first object is located farther from the vehicle 300 than the second object, the position where the virtual image I1 is displayed is brought closer to the second position P2 with respect to the position of the second object. You may adjust. Further, the determination unit 104 sets the depth position between the first position P1 and the second position P2 only when the first distance d1 is longer than the second distance d2 by the first threshold value Δd1 or more. It may be decided to P3.

In the display control device 50 according to the above embodiment, the determination unit 104 determines the depth position of the virtual image I1 superimposed on the first object when the difference between the first distance d1 and the second distance d2 is equal to or greater than the first threshold value Δd1. Is determined to be the third position P3 between the first position P1 and the second position P2 from the vehicle 300, but the present invention is not limited to this. The determination unit 104 may perform the above determination process only when the first object is an object that threatens the vehicle 300. As described above, the determination unit 104 may perform the above determination process only when the type of the first object is a specific type. The object that poses a threat to the vehicle 300 is, for example, a person or an object that is presumed to cross the traveling path of the vehicle 300 in front of the vehicle 300. Further, the determination unit 104 may perform the above determination process only when the position P1 of the first object is on a dangerous place. Dangerous places are, for example, some areas on the road where an accident may occur, areas in the direction of travel of the vehicle 300, and the like.

Although the display device 10 according to the above embodiment is provided in the vehicle 300 as an example of the moving body, the display device 10 is not limited to this and may be provided in another vehicle. Moreover, the moving body may be a person. When the moving body is a person, the display device 10 may be, for example, an HMD (Head Mounted Display). In this case, the HMD includes a video display system 100. The first camera 20 captures an image of the face of a person wearing the HMD. The second camera 30 captures an image of the surroundings of the person wearing the HMD. The distance sensor 40 detects the distance to an object around the person wearing the HMD.

In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the display control device 50 and the like according to each of the above embodiments is the following program.

That is, this program is a display control method for controlling a display device provided on a moving body on a computer to allow a user to visually recognize an image projected on the screen as a virtual image, and detects objects around the moving body. The result is acquired, the distance information indicating the distance from the moving object to the surrounding object is acquired, the line-of-sight direction of the user is acquired, and the virtual image of the surrounding objects is obtained based on the detection result. The first object to be superposed is determined, the first distance from the moving body to the first object is calculated using the distance information, and the distance information and the line-of-sight direction are used from the moving body to the surroundings. The second distance to the second object in the line-of-sight direction of the object is calculated, and the depth position in the depth direction at which the virtual image is visually recognized by the user is set to the first position separated from the moving body by the first distance. The virtual image is visually recognized by the user at the determined depth position and the position overlapping the first object, which is determined at the third position between the second position and the second position separated by the second distance. Execute the display control method that controls the display device.

Although the display control device and the display control method according to one or more aspects of the present invention have been described above based on the embodiments, the present invention is not limited to the embodiments. As long as it does not deviate from the gist of the present invention, one or more of the present embodiments may be modified by those skilled in the art, or may be constructed by combining components in different embodiments. It may be included within the scope of the embodiment.

The present disclosure is useful as a display control device or the like that can shorten the time until the user visually recognizes a virtual image.

10 Display device 20 First camera 30 Second camera 40 Distance sensor 50 Display control device 100 Video display system 101 Acquisition unit 102 First calculation unit 103 Second calculation unit 104 Decision unit 105 Control unit 201 Windshield 301 Dashboard

Claims (9)

A display control device provided on a moving body that controls a display device that allows a user to visually recognize an image projected on a screen as a virtual image.
An acquisition unit that acquires a detection result of an object around the moving object, distance information indicating a distance from the moving object to the surrounding object, and a line-of-sight direction of the user.
Based on the detection result, the first object on which the virtual image is superimposed is determined among the surrounding objects, and the first distance from the moving object to the first object is calculated using the distance information. Calculation unit and
A second calculation unit that calculates the second distance from the moving object to the second object in the line-of-sight direction among the surrounding objects by using the distance information and the line-of-sight direction.
A determination to determine the depth position in the depth direction for visualizing the virtual image to the user at a third position between the first position separated by the first distance from the moving body and the second position separated by the second distance. Department and
A display control device including a control unit that controls the display device so that the virtual image can be visually recognized by the user at a determined depth position and a position overlapping the first object. When the difference between the first distance and the second distance is equal to or greater than the first threshold value, the determination unit determines a predetermined fixed position between the first position and the second position as the depth position. The display control device according to claim 1. When the difference between the first distance and the second distance is equal to or greater than the first threshold value, the determination unit determines a position closer to the second position than the first position by a fixed distance as the depth position. Item 1. The display control device according to item 1. When the difference between the first distance and the second distance is equal to or greater than the first threshold value, the determination unit sets a position closer to the second position than the first position by a longer distance as the difference increases. The display control device according to claim 1, wherein the position is determined. The determination unit according to any one of claims 2 to 4, wherein when the difference between the first distance and the second distance is less than the first threshold value, the first position is determined as the depth position. Display control device. The display control device according to any one of claims 2 to 5, wherein the first threshold value is a value larger as the second distance is larger. The acquisition unit sequentially acquires the detection result, the distance information, and the line-of-sight direction.
The first calculation unit sequentially calculates the first distance based on the detection result and the distance information acquired sequentially.
The second calculation unit sequentially calculates the second distance based on the distance information and the line-of-sight direction that are sequentially acquired.
The determination unit sequentially determines the depth position based on the first distance and the second distance calculated sequentially.
Any one of claims 1 to 6, wherein the control unit controls the display device so that the virtual image can be visually recognized by the user at the depth position determined sequentially and at the position overlapping the first object. The display control device described in. When the difference between the first depth position determined at the first timing and the second depth position determined at the second timing prior to the first timing is equal to or greater than the second threshold value, the determination unit determines the first depth. The display control device according to claim 7, wherein a position between the position and the second depth position is determined as the depth position of the first timing. It is a display control method for controlling a display device provided on a moving body and allowing a user to visually recognize an image projected on a screen as a virtual image.
The detection result of the object around the moving body is acquired, and the detection result is obtained.
Obtaining distance information indicating the distance from the moving body to the surrounding object,
Acquire the line-of-sight direction of the user and
Based on the detection result, the first object on which the virtual image is superimposed is determined from the surrounding objects, and the first distance from the moving object to the first object is calculated using the distance information.
Using the distance information and the line-of-sight direction, the second distance from the moving object to the second object in the line-of-sight direction among the surrounding objects is calculated.
The depth position in the depth direction at which the virtual image is visually recognized by the user is determined as a third position between the first position separated from the moving body by the first distance and the second position separated by the second distance.
A display control method for controlling the display device so that the virtual image is visually recognized by the user at a determined depth position and a position overlapping the first object.

Images