JP6610376B2 - Display device - Google Patents

Description

  The present invention relates to a display device.

  Patent Document 1 discloses a display device that displays an image so as to be superimposed on a foreground by projecting display light onto a windshield of a vehicle or the like. The display device disclosed in Patent Document 1 displays a frame-shaped captured image (enhanced image in the same document) so as to surround an object (front object) in front of the vehicle, thereby indicating the presence of a preceding vehicle to the user. Inform. The captured image is displayed in a manner in which an interval is maintained between the outer shape of the target object in consideration of the positional deviation of the captured image with respect to the target object. Specifically, the interval is more horizontal than the vertical direction. Is also set larger.

JP 2010-120617 A

  In the technique described in Patent Literature 1, the captured image may be too large for the target object, and the target object may not be properly notified. Specifically, in the technology, as shown in FIG. 8, when the captured image V1 is superimposed and displayed on the preceding vehicle W1 viewed from the own vehicle, the head of the user (mainly the driver of the own vehicle). There is a possibility that the display position is moved due to part movement or the like and looks like the captured image V2, and the other preceding vehicle W2 or the like is also located within the frame of the image and looks like a notification target.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a display device that can appropriately capture a front object with a captured image.

In order to achieve the above object, a display device according to the present invention includes:
A display device that is mounted on a vehicle and displays a virtual image of the image in front of the translucent member by projecting display light representing the image displayed by the display unit onto the translucent member,
Identifying means for identifying a forward object in front of the vehicle;
Predicting means for predicting the traveling direction of each of the forward object and the vehicle;
Display control means for controlling the display means to display, in the image, a captured image that is visually recognized and superimposed on at least a part of the front object specified by the specifying means,
The display control means expands and displays the captured image on the matched traveling direction side when the traveling direction of the front object predicted by the predicting means matches the traveling direction of the vehicle.
It is characterized by that.

  According to the present invention, it is possible to appropriately capture a front object with a captured image.

It is a block diagram of the display system for vehicles concerning one embodiment of the present invention. It is a figure which shows the mounting aspect to the vehicle of the display apparatus which concerns on one Embodiment of this invention. It is a schematic block diagram of the display apparatus which concerns on one Embodiment of this invention. It is a flowchart of the captured image deformation | transformation process which concerns on one Embodiment of this invention. (A) is a figure which shows the structural example of a preceding vehicle estimated travel direction determination table, (b) is a figure which shows the structural example of the own vehicle predicted travel direction determination table. (A) And (b) is a figure for demonstrating how the deformation amount of a capture image is determined. (A) And (b) is a figure which shows an example of the deformation | transformation aspect of the captured image according to the advance angle of the preceding vehicle. It is a figure for demonstrating the conventional problem.

  A display device according to an embodiment of the present invention will be described with reference to the drawings.

  As shown in FIG. 1, the vehicle display system 1 according to this embodiment includes a display device 100 and an information acquisition unit 200. As shown in FIG. 2, the vehicle display system 1 displays various information related to the vehicle 2 (hereinafter referred to as vehicle information) to a user 4 (mainly a driver) who has boarded the vehicle 2 by communicating with each unit. . The vehicle information includes not only information on the vehicle 2 itself but also external information on the vehicle 2.

(Display device 100)
As shown in FIG. 2, the display device 100 is mounted on the vehicle 2 (for example, installed in an instrument panel of the vehicle 2). The display device 100 projects a display light L onto the windshield 3 (windshield) of the vehicle 2 to display an image represented by the display light L as a virtual image V in front of the windshield 3 (HUD: Head). -Up Display). The image visually recognized as the virtual image V is for reporting vehicle information. Thereby, the user 4 can visually recognize the vehicle information displayed so as to overlap with the scenery seen through the windshield 3.

  As shown in FIGS. 1 to 3, the display device 100 includes a display unit 10, a reflection unit 20, a housing 30, a control unit 40, and an adjustment unit 50.

  The display means 10 displays an image for informing vehicle information under the control of the control means 40. The display means 10 includes a liquid crystal display (LCD), a backlight that illuminates the LCD from behind, and the like. The display means 10 displays an image toward a plane mirror 21 described later. Thereby, the display light L representing the image is emitted toward the plane mirror 21. The display means 10 may be configured by an organic EL (Electro-Luminescence) display, DMD (Digital Micromirror Device), reflection type and transmission type LCOS (registered trademark: Liquid Crystal On Silicon), or the like.

  The reflecting means 20 includes a plane mirror 21 and a concave mirror 22. The plane mirror 21 is formed of, for example, a cold mirror, and reflects the display light L from the display unit 10 to the concave mirror 22. The concave mirror 22 reflects the display light L emitted from the display means 10 and reflected by the plane mirror 21 to the windshield 3 while expanding it. Thereby, the virtual image V visually recognized by the user 4 is an enlarged image displayed on the display unit 10. In the present embodiment, the example in which the reflecting unit 20 is configured by two mirrors is shown, but the reflecting unit 20 may be configured by one or three or more mirrors.

  The housing 30 accommodates the display means 10, the reflection means 20, and the adjustment means 50 at appropriate positions where each of the functions described above can be realized. Note that at least a part of the control unit 40 may be outside the housing 30. The housing 30 has a light shielding property and is formed in a box shape from synthetic resin or metal. The housing 30 is provided with an opening 30 a that secures the optical path of the display light L. A translucent cover 31 is attached to the housing 30 so as to close the opening 30a. The translucent cover 31 is made of a translucent resin such as acrylic.

  The display light L reflected by the concave mirror 22 passes through the translucent cover 31 and travels toward the windshield 3. In this way, the display light L is emitted from the display device 100 toward the windshield 3. The display light L is reflected toward the user 4 by the windshield 3 so that the virtual image V is displayed in front of the windshield 3 when viewed from the user 4. The concave mirror 22 is provided so as to be rotatable by the adjusting means 50.

  The adjusting means 50 includes a motor (such as a stepping motor), a support component that rotatably supports the concave mirror 22, a gear mechanism that transmits the rotational power of the motor to the concave mirror 22, and the like. The adjusting means 50 changes the optical path of the display light L by rotating the concave mirror 22 clockwise or counterclockwise in FIG. 3 based on the control signal from the control means 40. The adjusting means 50 is provided for adjusting the display position of the virtual image V and for preventing external light from being reflected toward the display means 10 when the display device 100 is not used. The adjusting unit 50 may change the optical path of the display light L by translating the concave mirror 22.

  The control means 40 controls the entire operation of the display device 100. As shown in FIG. 1, the control means 40 is a CPU (Central Processing Unit) 41, a storage unit 42, a video memory 43, and an I / F (InterFace). 44.

  Note that a power supply (not shown) is connected to the display device 100 and, for example, operating power is supplied to the control means 40 when the ignition of the vehicle 2 is turned on. Further, at least a part of the control means 40 may be configured by a dedicated circuit such as an ASIC (Application Specific Integrated Circuit).

  The storage unit 42 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The ROM stores in advance an operation program (such as a program for executing captured image deformation processing described later) and fixed data (such as a preceding vehicle predicted traveling direction determination table TA1 and a host vehicle predicted traveling direction determination table TA2). The RAM temporarily stores various calculation results and the like. For example, the RAM stores data indicating a determination result or a determination result in a captured image deformation process described later.

  The video memory 43 includes an HDD (Hard Disk Drive), a flash memory, and the like, and stores image data (image data) to be displayed on the display means 10.

  The I / F 44 enables various signals to be input / output from / to the outside of the control means 40. The control means 40 communicates with the information acquisition unit 200 via a CAN (Controller Area Network) via the I / F 44.

  The CPU 41 executes processing for controlling the overall operation of the display device 100 using an operation program and fixed data read from the ROM of the storage unit 42. At this time, the CPU 41 reads fixed data from the ROM, the CPU 41 writes various data to the RAM of the storage unit 42 and temporarily stores the data, and the CPU 41 stores various data temporarily stored in the RAM. Data reading operation for reading, receiving operation for receiving various signals from the outside of the control means 40 through the I / F 44, and sending operation for outputting various signals to the outside of the control means 40 through the I / F 44 Etc. are also performed. In addition, the CPU 41 appropriately measures time with a built-in timer.

  Further, the CPU 41 performs display control of the display unit 10 based on the image data stored in the video memory 43 via a display control unit (graphic controller) (not shown). The display control unit determines the control content of the display operation in the display unit 10 based on a display control command from the CPU 41 and the like. For example, the display control unit performs control for executing various displays by determining the switching timing of an image to be displayed on the display screen of the display unit 10. In the control means 40, a layer is assigned to each display content (a captured image or other notification image described later included in an image viewed as a virtual image V), and individual display control is possible. Yes.

(Information acquisition unit 200)
As shown in FIG. 1, the information acquisition unit 200 includes a front object detection unit 210, a user detection unit 220, a navigation system 230, a GPS (Global Positioning System) controller 240, an ECU (Electronic Control Unit) 250, Is provided.

  The front object detection unit 210 is a stereo camera or the like that images the front of the vehicle 2, and is provided in a room of the vehicle 2, for example, as illustrated in FIG. 2. In addition, as long as the front object detection means 210 can image the front of the vehicle 2, the arrangement | positioning location is arbitrary and may be provided outside the passenger compartment.

  The front object detection means 210 detects various objects in front of the vehicle 2 (hereinafter also referred to as the host vehicle 2) by analyzing the captured front image data (front image data) by a pattern matching method or the like. Then, the front object detection unit 210 outputs information (hereinafter referred to as front information) relating to the detected various objects to the control unit 40. Note that at least a part of the image analysis may be executed by the control unit 40, and the front object may be detected on the control unit 40 side (forward information is generated).

  Forward information includes information on objects on the road (forward vehicles and obstacles) and road shape information (lanes, white lines, stop lines, pedestrian crossings, road width, number of lanes, intersections, curves, branch roads, etc.) is there. The front object detection unit 210 is configured to be able to calculate the distance between the imaged object and the vehicle 2. The calculated distance is also included in the forward information.

  The main forward information in this embodiment is the position of the preceding vehicle W (see FIGS. 6A and 7A), the blinker information of the preceding vehicle W, the road shape information, and the movement amount of the preceding vehicle W with respect to the lane. is there. The blinker information of the preceding vehicle W is whether or not the blinker (direction indicator light) of the preceding vehicle W is lit, and if it is lit, which of the left and right blinkers is lit.

  The user detection means 220 is a monaural camera or the like that captures an image of the user 4, and is provided in the room of the vehicle 2, for example, as shown in FIG. In addition, if the user detection means 220 can image the head of the user 4, the arrangement | positioning location is arbitrary.

  The user detection unit 220 performs image analysis on the captured image data of the user 4 using a pattern matching method or the like, so that the viewpoint position (including the line-of-sight position and the head position) of the user 4 and the face direction (head position). And a visual recognition state including at least one of them. Then, the user detection unit 220 outputs information indicating the viewing state (hereinafter, viewing state information) to the control unit 40.

  Based on the information, the control unit 40 that has acquired the viewing state information controls the adjustment unit 50 to adjust the angle of the concave mirror 22. Further, the position, size, and shape (distortion correction) of the image displayed on the display unit 10 may be changed based on the viewing state information. In this way, the control unit 40 can adjust the display position of the virtual image V in accordance with the viewpoint position and face orientation of the user 4.

  The navigation system 230 includes a storage unit that stores map data. Based on the position information from the GPS controller 240, the navigation system 230 reads map data in the vicinity of the current position from the storage unit and determines a guide route. Then, the navigation system 230 outputs information regarding the determined guide route to the control means 40. The navigation system 230 outputs information indicating the name and type of the facility in front of the vehicle 2 and the distance between the facility and the vehicle 2 to the control means 40 by referring to the map data. Map data includes road shape information (road width, number of lanes, intersections, curves, branch roads, etc.), restriction information on road signs such as speed limits, information about each lane when there are multiple lanes (which lanes are Various information such as where the lane is heading) is associated with the position data. The navigation system 230 is configured to be able to output these various types of information (navigation information) to the control means 40.

  The GPS controller 240 receives a GPS signal from an artificial satellite or the like, and calculates the position of the vehicle 2 based on the received GPS signal. When the position of the vehicle 2 is calculated, the GPS controller 240 outputs the position data to the navigation system 230.

  The ECU 250 controls each part of the vehicle 2 and is output from various sensors (such as a steering sensor, a vehicle speed sensor, and a winker sensor that detects an operating state of the winker by a signal from the winker lever). The detected signal is acquired, and the data (detection information) indicating the detected content is output to the control means 40. The state information (information related to the host vehicle 2) acquired by the control means 40 via the ECU 250 is a steering angle, turn signal information, vehicle speed, and the like. Note that the control means 40 may obtain detection signals from various sensors without using the ECU 250 and obtain the state information by calculation or the like.

  In the vehicular display system 1 having the above configuration, the virtual image V is displayed in a display area defined as a range (so-called eyebox) in which the user 4 can visually recognize the virtual image V. In the vehicle display system 1, the image formation distance of the virtual image V is set to be large (for example, 15 m or more) in order to display information superimposed on the real scene.

  The image visually recognized by the user 4 as the virtual image V is displayed so as to be superimposed on a front object (an object outside the vehicle appearing in a real scene) such as a preceding vehicle W (front vehicle) or a pedestrian, and the presence of the object is determined by the user. 4 includes a captured image that calls attention.

  The captured image is an image that is visually recognized by being superimposed on at least a part of the object (front object). For example, the captured image is a frame image indicating a frame, and is displayed with at least a part of the object positioned in the frame. Note that “the captured image is superimposed on at least a part of the object” means that at least a part of the object is positioned within the outline of the frame. Therefore, the state in which the target object is within the frame area indicated by the captured image is also a state in which the target object is superimposed on the captured image.

  Although the display mode of the captured image and the forward object captured by the captured image are not limited, in this embodiment, the captured image Va indicating the frame (see FIG. 7B) is the preceding vehicle W (FIG. 6) as the forward object. An example of capturing (a) and FIG. 7 (a) will be described. The display mode of the captured image Va is the same as the mode illustrated in FIG. 8 in that it indicates a frame. However, since the display mode can be changed depending on the case as described later, the captured image Va is displayed more than necessary. The difference is that you do not need to make it larger.

  The captured image Va is displayed superimposed on the specified preceding vehicle W when using, for example, adaptive cruise control (ACC) that controls the speed of the vehicle 2 so as to follow the preceding vehicle W. The user 4 is made to recognize the preceding vehicle W. The control unit 40 controls the display unit 10 to display a captured image Va that captures the preceding vehicle W specified by the front target detection unit 210.

  The display device 100 having the above configuration can display a notification image other than the captured image Va. Hereinafter, mainly with reference to FIG. 4, captured image deformation processing unique to the present embodiment will be described. explain. For example, when the supply of operating power to the display device 100 is started as the ignition of the vehicle 2 is turned on, the CPU 41 is activated. The activated CPU 41 executes a predetermined initialization process, and then executes a captured image deformation process.

(Captured image deformation processing)
First, the CPU 41 specifies the position of the preceding vehicle W based on the forward information from the forward target detection unit 210, and causes the display unit 10 to display the captured image Va that is visually recognized as a frame image that captures the preceding vehicle W (step S1). .

  Subsequently, the CPU 41 acquires the predicted traveling direction of the preceding vehicle W (step S2). Specifically, the CPU 41 predicts the preceding vehicle W based on parameters such as the blinker information of the preceding vehicle W, road shape information, and the amount of movement of the preceding vehicle W included in the forward information from the forward target detection unit 210. Determine / obtain direction of travel. Note that the predicted traveling direction here is an assumption that the preceding vehicle W is “straight forward”, “right”, or “left”, and is different from a traveling angle described later. The same applies to the predicted traveling direction of the vehicle 2 to be described next. For example, the CPU 41 determines the predicted traveling direction of the preceding vehicle W with reference to the preceding vehicle predicted traveling direction determination table TA1 stored in advance in the ROM of the storage unit 42.

  FIG. 5A is a diagram illustrating a configuration example of the preceding vehicle predicted traveling direction determination table TA1. The preceding vehicle predicted traveling direction determination table TA1 is configured by associating a combination of each parameter of the blinker information, road shape information, and moving amount of the preceding vehicle W with the predicted traveling direction of the preceding vehicle W. Yes. The CPU 41 refers to such a preceding vehicle predicted travel direction determination table TA1, and predicts the travel direction of the preceding vehicle W corresponding to the combination of parameters included in the forward information acquired from the forward target detection unit 210 (illustrated). “Right”, “Left”, “Straight”, etc.). The “movement amount” column in the figure shows, for example, the continuous movement amount (movement amount within a predetermined period) of the preceding vehicle W with respect to the traveling lane, and in the illustrated example, “D” is the width of the preceding vehicle W with respect to the traveling lane. The amount of movement in the direction is indicated, and “C” indicates a threshold value. In this example, the moving amount D is set to increase when the preceding vehicle W moves to the right with respect to the travel lane, and the moving amount D decreases when the preceding vehicle W moves to the left. If the movement amount D is equal to or smaller than the absolute value of the threshold value C, it is regarded as “straight forward”, and if the movement amount D is larger than the threshold value C that takes a positive value, it is regarded as “right”. If it is smaller than the threshold value C which takes a negative value, it is regarded as “left”.

  In the preceding vehicle predicted traveling direction determination table TA1, it is not necessary to use all of the parameters. For example, if the blinker information indicates “right on” (the right blinker of the preceding vehicle W is lit), other parameters Regardless of the value, the predicted traveling direction of the preceding vehicle W may be determined as “right”. That is, a priority may be provided for each parameter. Moreover, how to determine the amount of movement of the preceding vehicle W is not limited, and is arbitrary depending on the design.

  Returning to FIG. 4, following step S <b> 2, the CPU 41 acquires the predicted traveling direction of the host vehicle 2 (step S <b> 3). Specifically, the CPU 41 is based on each parameter such as turn signal information, steering angle information, and road shape information (included in the navigation information) acquired from the navigation system 230 obtained through the ECU 250 and the like. Then, the predicted traveling direction of the own vehicle 2 is determined and acquired. For example, the CPU 41 determines the predicted traveling direction of the host vehicle 2 with reference to the host vehicle predicted traveling direction determination table TA2 stored in advance in the ROM of the storage unit 42.

  FIG. 5B is a diagram illustrating a configuration example of the own vehicle predicted traveling direction determination table TA2. The own vehicle predicted traveling direction determination table TA2 is configured by associating a combination of each parameter of the turn signal information, road shape information, and steering angle of the own vehicle 2 with the predicted traveling direction of the own vehicle 2. The CPU 41 refers to the host vehicle predicted travel direction determination table TA2, and predicts the travel direction of the host vehicle 2 corresponding to the combination of parameters included in various information acquired from the information acquisition unit 200 (as illustrated). "Right", "Left", "Straight", etc.). In the column of “Steering angle” in the figure, “α” indicates a steering angle, and “c” indicates a threshold value. In this example, the angle α is increased when the steering is turned to the right, and the angle α is decreased when the steering is turned to the left. If the angle α is equal to or smaller than the absolute value of the threshold c, it is regarded as “straight forward”, and if the angle α is larger than the threshold c taking a positive value, it is regarded as “right”, and the angle α is a negative value. If it is smaller than the threshold value c, the “left” is assumed.

  Note that it is not necessary to use all the parameters in the host vehicle predicted travel direction determination table TA2. For example, if the turn signal information indicates “right on” (the right turn signal of the host vehicle 2 is lit), other parameters Regardless of the value, the predicted traveling direction of the vehicle 2 may be determined as “right”. That is, a priority may be provided for each parameter. Further, the method of determining the steering angle is not limited and is arbitrary depending on the design. Further, as with the preceding vehicle predicted travel direction determination table TA1, the movement amount of the host vehicle 2 may be used as a parameter.

  Returning to FIG. 4, following step S3, the CPU 41 determines whether or not the predicted traveling direction of the preceding vehicle W acquired in step S2 matches the predicted traveling direction of the host vehicle 2 acquired in step S3 (step S3). S4). When it is determined that the predicted traveling directions of both the vehicles do not match (step S4; No), the CPU 41 deletes the captured image Va (step S8), and returns to step S1 to execute the process.

  When it is determined that the predicted traveling directions of the two vehicles coincide with each other (step S4; Yes), the CPU 41 obtains the preceding vehicle based on the difference between the forward image of a predetermined time and the current forward image acquired from the forward target detection unit 210. The traveling angle of W is calculated (step S5). At the same time, the CPU 41 also calculates the traveling angle of the host vehicle 2 based on the steering angle acquired via the ECU 250 or the like. These traveling angles are expressed as, for example, an angle θ with respect to the reference direction B shown in FIG.

  Subsequently, the CPU 41 determines a deformation amount of the captured image Va based on the advance angle of the preceding vehicle W calculated in step S5 and the advance angle of the host vehicle 2 (step S6), and according to the determined deformation amount. The captured image Va is deformed (step S7).

  Here, with reference to FIGS. 6A and 6B and FIGS. 7A and 7B, a method of determining the deformation amount of the captured image Va and the deformation of the captured image Va will be described.

  FIG. 6A is a diagram showing an example in which the own vehicle 2 turns right at the intersection following the preceding vehicle W. FIG. In order to facilitate understanding of the situation, the trajectory of both vehicles is assumed to be an angle θ when both vehicles follow the same route (lane R1, intersection R2, right turn lane R3) and move at a constant speed. FIG. 6B shows a graph represented by time t. The straight line TR1 represents the locus of the preceding vehicle W, and the straight line TR2 represents the locus of the host vehicle 2. It is assumed that the portions having inclinations in both loci follow an arcuate route in the intersection R2 when making a right turn. In this case, the relationship between θ and t can be regarded as a linear function (linear relationship).

  As shown in FIG. 6B, the period T1 corresponds to a state where both the preceding vehicle W and the host vehicle 2 are traveling straight in the lane R1. The following period T2 corresponds to a state in which the preceding vehicle W is turning right (running in the intersection R2), but the own vehicle 2 is traveling straight in the lane R1. In the period T2, it is assumed that the predicted traveling directions of both vehicles coincide with each other on the “right” side. In the period T2, the advance angle of the preceding vehicle W with respect to the host vehicle 2 continues to increase gradually.

  Here, a schematic diagram of the relationship between the two cars in the period T2 is shown in FIG. 7 (a), and a schematic diagram of the captured image corresponding to FIG. 7 (a) is shown in FIG. 7 (b). Reference sign Vn represents a captured image (referred to as a captured image Vn when traveling straight) just surrounding the preceding vehicle W when both the vehicles are traveling straight ahead. When the preceding vehicle W turns right, if the straight ahead captured image Vn remains as it is, the preceding vehicle W looks oblique when viewed from the driver of the host vehicle 2, so the preceding vehicle W protrudes from the frame indicated by the straight traveling captured image Vn, The preceding vehicle W cannot be properly captured. Further, if the frame image is displayed in a large size in the horizontal direction as in the conventional example described with reference to FIG. 8, the preceding vehicle W cannot be properly captured (as described above, it can be captured other than the target to be captured). There is a risk of losing.)

  Therefore, in this embodiment, as shown in FIG. 7B, the captured image Va is deformed and displayed. Specifically, the CPU 41 determines the deformation amount of the captured image Va according to the difference between the advance angle of the preceding vehicle W and the advance angle of the host vehicle 2. In the period T2, since the advance angle of the preceding vehicle W is set to be positive when the advance angle of the host vehicle 2 is decreased and the angle θ is set to increase clockwise, the captured image Va is set as shown in FIG. Expand to the right as shown in For example, there is a one-to-one relationship between the advance angle of the preceding vehicle W, the advance angle of the own vehicle 2, and the difference value, and the deformation amount of the captured image Va (including the expansion amount and the reduction amount in the case of reduction as described later). A table set in correspondence is stored in advance in the ROM, and the CPU 41 may determine the deformation amount with reference to the table. Note that the calculation may be performed regardless of the table. The deformation amount of the captured image Va may be determined based on the ratio of the traveling angle of the host vehicle 2 to the traveling angle of the preceding vehicle W.

  Returning to FIG. 6B, the period T3 corresponds to a state in which both vehicles are turning right (traveling in the intersection R2). The period T4 corresponds to a state in which the preceding vehicle W is traveling straight in the lane R3 after the right turn, but the own vehicle 2 is turning right (running in the intersection R2). In the period T4, it is assumed that the predicted traveling directions of both vehicles coincide with each other on the “right” side. The period T5 corresponds to a state in which both of them are traveling straight in the lane R3 after turning right.

As can be seen from the graph of FIG. 6B, in the period T4, the traveling angle of the preceding vehicle W with respect to the host vehicle 2 continues to gradually decrease.
Based on the above, the captured image Va gradually expands to the right in the period T2, and the expansion amount becomes maximum at the end of the period T2 (the start of the period T3). On the other hand, in the period T4, the expansion amount gradually decreases, and the expansion amount becomes zero at the end of the period T4 (the start of the period T5). In the period T3, since the difference in the traveling angle between the two vehicles is constant, the expansion amount is kept at the maximum. That is, the period T2 to T4 is an extended display period T in which the captured image Va is extended and displayed. Note that the concept of expansion on the right side of the captured image Va may be expressed differently depending on how to capture the reference position of the captured image Va. The captured image Va is not limited to the one recognized as viewed from the user 4 and actually expanded to the right.

  If the deformation amount of the captured image Va is determined as described above, and the captured image Va is deformed according to the determined deformation amount, the captured image Va can appropriately capture the preceding vehicle W, The captured image Va need not be displayed larger than necessary. For this reason, it can suppress that the situation which shows forward objects (other cars of an adjacent lane, etc.) other than the preceding vehicle W accidentally arises.

  The CPU 41 repeatedly executes the above captured image deformation processing until the power of the display device 100 is turned off, for example, when the ignition of the vehicle 2 is turned off.

  In order to facilitate understanding of the situation, it is assumed as shown in the graph of FIG. 6B, but the concept is the same even if the trajectories of both vehicles cannot be expressed linearly. Even when the trajectories of both vehicles are expressed as a high-order function, it is only necessary to consider a difference in travel angle within a predetermined timing or a predetermined period.

  Further, in the captured image deformation process, even if the preceding vehicle W and the host vehicle 2 are both traveling straight, a determination of Yes is made in step S4, and the processes after step S5 are executed. In this case, since the difference between the two traveling angles is zero or minute, the captured image Va is not deformed, or even if it is deformed, it is deformed by a minute amount. In addition, when both vehicles are going straight ahead, you may comprise a captured image deformation | transformation process so that step S1-S3 may be performed repeatedly. Moreover, the advance angle calculation process of step S5 may be performed before the determination process of step S4, and the estimated advance direction of the preceding vehicle W or the own vehicle 2 may be estimated based on the calculated advance angle. By doing in this way, the predicted traveling direction of the preceding vehicle W and the host vehicle 2 may be predicted in detail instead of the rough directions such as “right” and “left”.

(1) The display device 100 described above is mounted on the vehicle 2 (an example of a vehicle), and projects display light L representing an image displayed by the display unit 10 onto the windshield 3 (an example of a translucent member). Thus, the virtual image V is displayed in front of the windshield 3. The display device 100 controls the specifying means for specifying the preceding vehicle W (an example of a forward object), the predicting means for predicting the traveling directions of the preceding vehicle W and the vehicle (own vehicle) 2, and the display means 10. In the image, display control means (for example, control means 40 including a graphic controller) that displays the captured image Va that is visually recognized superimposed on at least a part of the preceding vehicle W specified by the specifying means is provided. The specifying unit and the predicting unit are realized by the cooperation of the control unit 40 and at least a part of the information acquisition unit 200. When the traveling direction of the preceding vehicle W predicted by the predicting unit coincides with the traveling direction of the host vehicle 2, the display control unit expands and displays the captured image Va on the matching traveling direction side.
Since it did in this way, as mentioned above, the preceding vehicle W (an example of front object) can be appropriately caught with a capture image. Moreover, by having the following configurations (2) to (6) as appropriate, the preceding vehicle W can be better captured in the captured image.

(2) Moreover, it has an imaging means (for example, the front object detection means 210) that images the front of the vehicle 2, the specifying means specifies the preceding vehicle W based on the captured image captured by the imaging means, and the predicting means The traveling direction of the preceding vehicle W is predicted based on at least the captured image captured by the imaging unit.

(3) Also, the difference between the forward image before a predetermined time and the current forward image (note that the difference between the captured image captured at the first timing by the imaging means and the captured image captured at the second timing) Based on the progress angle calculation means for calculating the advance angle of the preceding vehicle W, the advance angle of the preceding vehicle W calculated by the advance angle calculation means, and the advance angle of the host vehicle 2 And an expansion amount determining means for determining an expansion amount when expanding the image Va. When the traveling direction of the preceding vehicle W predicted by the predicting unit and the traveling direction of the host vehicle 2 match, the display control unit determines the captured image Va on the matching traveling direction side by the expansion amount determining unit. Expand and display with expansion amount. The advance angle calculation means and the expansion amount determination means are realized as functional units of the control means 40, for example.

(4) Further, the predicting unit identifies the lighting state of the turn signal of the preceding vehicle W based on the captured image captured by the imaging unit, and predicts the traveling direction of the preceding vehicle W using at least the lighting state.

(5) Moreover, a prediction means predicts the advancing direction of the own vehicle 2 using at least one of the steering angle of the own vehicle 2 and the lighting state of the turn signal of the own vehicle 2.

(6) Further, the display control unit includes a visual state detection unit (such as the user detection unit 220) that detects a visual state including at least one of a viewpoint position and a face direction of the user 4 viewing the image. The display position of the captured image Va is adjusted according to the visual recognition state. Since it did in this way, even if the viewpoint shift of the user 4 arises, the preceding vehicle W can be appropriately caught with a capture image.

  In addition, this invention is not limited by the above embodiment and drawing. Changes (including deletion of components) can be made as appropriate without departing from the scope of the present invention.

  The projection target of the display light L is not limited to the windshield 3 but may be a combiner configured by a plate-shaped half mirror, a hologram element, or the like.

  Note that the forward target detection unit 210 is not limited to a method using a visible light camera such as a stereo camera. The forward target detection means 210 may detect the forward situation using a short-range detection radar such as an infrared camera or a millimeter wave laser, a sonar using an ultrasonic wave, or the like. Further, since the forward target detection unit 210 only needs to be able to detect the situation in front of the vehicle 2, the forward target detection unit 210 may be configured by the navigation system 230 that can identify the position of the vehicle 2 and obtain information on the road on which the vehicle 2 travels. Good. Further, the forward target detection means 210 is constituted by a communication means capable of communicating with an external communication device such as a road traffic information communication system, and the situation in front of the vehicle 2 is estimated based on information received from the external communication device. May be.

  Moreover, the user detection means 220 may be comprised with a stereo camera, and may be comprised not only with a visible light camera but with an infrared camera.

  The front object is not limited to the preceding vehicle W, and may be another object such as a pedestrian. Further, the display device 100 is not limited to expanding (deforming) the captured image Va to the right or left when turning right or left (including when turning right or left). For example, the captured image Va may be expanded (deformed) upward or downward when climbing or downhill. In addition, expansion (deformation) of the captured image Va may be continuously expanded (deformation) so as to reach a gradually determined expansion amount. Thereby, a display without a sense of incongruity is possible without imposing a mental burden on the driver.

  Further, the captured image Va does not have to be a frame image indicating a frame. The image may be solid or gradation and can be viewed through the front object. The shape of the captured image Va is not limited to a rectangle, and may be a circle (including an ellipse), a polygon, a mosaic, or an icon imitating a predetermined object.

  The virtual image V (image displayed on the display unit 10) displayed by the display device 100 may include a guidance route image, a white line recognition image, an operation state image, a restriction image, and the like. The guide route image is an image for guiding the route of the vehicle 2 and is displayed based on information on the guide route from the navigation system 230. The white line recognition image is an image for recognizing the presence of a lane, and is displayed based on information from the front target detection unit 210. The operation state image is an image showing the operation state of the vehicle 2 such as the vehicle speed and the engine speed, and is displayed based on information from the ECU 250 or the like. The restriction image is an image indicating restriction information such as a speed limit, and is displayed based on the restriction information from the navigation system 230.

  Further, when calculating the movement amount of the host vehicle 2, various physical quantities such as a vehicle speed from the vehicle speed sensor, an acceleration from the acceleration sensor, and an angular velocity from the gyro sensor may be used as appropriate.

  Further, if wireless communication is possible between the preceding vehicle W and the host vehicle 2, the predicted traveling direction and the traveling angle of the preceding vehicle W are determined and calculated based on information received from the preceding vehicle W. Also good.

  Moreover, the display apparatus 100 is not restricted to what is mounted in the vehicle 2 as an illustrated automobile. The display device 100 may be mounted on a vehicle other than a vehicle such as a motorcycle or a vehicle other than a vehicle such as a water motorcycle, a snow motorcycle, a ship, or an aircraft.

  The operation program is stored in advance in the ROM of the storage unit 42, but may be distributed and provided by a removable recording medium. In addition, the operation program may be downloaded from another device connected to the display device 100. Further, the display device 100 may execute each process according to the operation program by exchanging various data with other devices via a telecommunication network or the like.

  In the above description, in order to facilitate understanding of the present invention, descriptions of known technical matters are omitted as appropriate.

DESCRIPTION OF SYMBOLS 1 ... Vehicle display system 2 ... Vehicle, 3 ... Windshield, 4 ... User 100 ... Display apparatus 10 ... Display means, 20 ... Reflection means, 30 ... Housing, 50 ... Adjustment means 40 ... Control means 41 ... CPU, 42 ... Storage unit, 43 ... Video memory, 44 ... I / F
200: Information acquisition unit 210: Front target detection means (an example of imaging means)
220... User detecting means (an example of visual state detecting means)
230 ... Navigation system 240 ... GPS controller 250 ... ECU
L ... Display light V ... Virtual image Va ... Captured image W ... Preceding vehicle

Claims (7)

A display device that is mounted on a vehicle and displays a virtual image of the image in front of the translucent member by projecting display light representing the image displayed by the display unit onto the translucent member,
Identifying means for identifying a forward object in front of the vehicle;
Predicting means for predicting the traveling direction of each of the forward object and the vehicle;
Display control means for controlling the display means to display, in the image, a captured image that is visually recognized and superimposed on at least a part of the front object specified by the specifying means,
The display control means expands and displays the captured image on the matched traveling direction side when the traveling direction of the front object predicted by the predicting means matches the traveling direction of the vehicle.
A display device characterized by that. Comprising imaging means for imaging the front of the vehicle;
The specifying unit specifies the front object based on a captured image captured by the imaging unit,
The predicting means predicts a traveling direction of the front object based on at least a captured image captured by the imaging means;
The display device according to claim 1. Imaging means for imaging the front of the vehicle;
A travel angle calculating means for calculating a travel angle of the front object based on a difference between a captured image captured at a first timing and a captured image captured at a second timing by the imaging means;
Expansion amount determining means for determining an expansion amount in the case of expanding the captured image, based on the advance angle of the front object calculated by the advance angle calculating means and the advance angle of the vehicle,
The display control means, when the traveling direction of the front object predicted by the prediction means matches the traveling direction of the vehicle, determines the captured image on the matching traveling direction side by the expansion amount determining means. Expanded and displayed with the specified expansion amount,
The display device according to claim 1. The vehicle is a vehicle, and the front object is a preceding vehicle of the vehicle;
The predicting unit specifies a lighting state of the blinker of the preceding vehicle based on a captured image captured by the imaging unit, and predicts a traveling direction of the preceding vehicle using at least the lighting state.
The display device according to claim 2, wherein the display device is a display device. The vehicle is a vehicle;
The predicting means predicts a traveling direction of the vehicle using at least one of a steering angle of the vehicle and a lighting state of a blinker of the vehicle;
The display device according to claim 1, wherein the display device is a display device. A visual state detection means for detecting a visual state including at least one of a viewpoint position and a face direction of a user who visually recognizes the image;
The display control means adjusts the display position of the captured image according to the visual recognition state.
The display device according to claim 1, wherein the display device is a display device. The display control means continuously expands and displays the captured image;
The display device according to claim 1, wherein the display device is a display device.

Images