JP6984341B2 - Superimposed image display device and computer program - Google Patents

Description

The present invention relates to a superimposed image display device and a computer program that support the traveling of a vehicle.

Conventionally, various means have been used as information providing means for providing various information for providing vehicle running support such as route guidance and obstacle warning to vehicle occupants. For example, a display on a liquid crystal display installed in a vehicle, a sound output from a speaker, or the like. In recent years, as one of such information providing means, an image superimposed on the driver's surrounding environment (landscape) such as a head-up display (hereinafter referred to as HUD) or a window shield display (hereinafter referred to as WSD) has been added. There is a device that provides information by displaying.

For example, in Japanese Patent Application Laid-Open No. 2005-69800, when guiding a route by a navigation system, a landmark (convenience store, etc.) that serves as a mark of the route is announced by a voice guidance device, and a real view in front of the vehicle is used by using a HUD. It is disclosed to display a virtual image superimposed on the position of a landmark in. Based on the position of the virtual image seen through the front window, the occupant can grasp the position of the landmark that marks the right / left turn (that is, the position of the intersection where the right / left turn is made).

JP-A-2005-69800 (Fig. 4)

Here, in the technique described in Patent Document 1, the virtual image is displayed so as to be visually recognized by the occupant by superimposing it on the position of the landmark that serves as a mark for turning left or right in the actual scene. However, when there is a moving object around the landmark that should be visible to the occupants of a vehicle such as a pedestrian or another vehicle, if the virtual image overlaps with the moving object, there is a problem that it becomes difficult for the occupant to see the moving object. there were.

The present invention has been made to solve the above-mentioned conventional problems, and while it is possible to accurately guide the vehicle occupant on the path on which the vehicle should travel by using a guide image, the moving body should be visually recognized by the vehicle occupant. It is an object of the present invention to provide a superimposed image display device and a computer program that do not reduce the visibility of an object.

In order to achieve the above object, the superimposed image display device according to the present invention is mounted on a vehicle, and a guide image for guiding the traveling direction of the traveling path on which the vehicle travels after leaving the guidance branch point is displayed as a landscape in front of the vehicle. A superposed image display device that superimposes and visually recognizes the image, the captured image acquisition means for acquiring the captured image of the landscape in front of the vehicle, the map information acquisition means for acquiring the three-dimensional map information, and the captured image. An image arrangement for arranging the guide image on the traveling path of the vehicle after leaving the guide branch point in the three-dimensional map information based on the collation means for collating the three-dimensional map information and the collation result of the collation means. Based on the collation result of the means and the collation means, the virtual body arrangement of the moving object is arranged at the position where the moving object exists in the three-dimensional map information for the moving object existing around the guide branch point. Acquired by means, a shape acquisition means for acquiring a portion of the shape of the guide image when the guide image is visually recognized from the position of the vehicle in the three-dimensional map information, and a portion that does not overlap with the virtual body, and the shape acquisition means. by displaying the guidance image of the shape, having a guide image display means for displaying the guide image in a manner that does not overlap with the body thereof.

Further, the computer program according to the present invention is a program for supporting the running of a vehicle. Specifically, a superposed image display device mounted on the vehicle and superimposing a guide image that guides the traveling direction of the traveling path on which the vehicle travels after leaving the guidance branch point on the scenery in front of the vehicle is visualized. , The captured image acquisition means for acquiring the captured image of the landscape in front of the vehicle, the map information acquisition means for acquiring the three-dimensional map information, the collation means for collating the captured image with the three-dimensional map information, and the collation. Based on the collation result of the collation means, the image arranging means for arranging the guide image on the traveling path of the vehicle after leaving the guide branch point in the three-dimensional map information and the collation result of the collation means. From the virtual body placement means for arranging the virtual body of the moving object at the position where the moving object exists in the 3D map information for the moving object existing around the guide branch point, and the position of the vehicle in the 3D map information. By displaying the shape acquisition means for acquiring the portion of the shape of the guide image that does not overlap with the virtual body when the guide image is visually recognized, and the guide image of the shape acquired by the shape acquisition means. a guidance image display means for displaying the guide image in a manner that it does not overlap with the body thereof, is to function.

According to the superimposed image display device and the computer program according to the present invention having the above configuration, the guide image for guiding the traveling direction after leaving the guide branch point is displayed so as not to overlap with the moving object. While it is possible to accurately guide the vehicle occupant on the path on which the vehicle should travel, it does not reduce the visibility of moving objects that the vehicle occupant should see. In addition, by using the captured image of the landscape in front of the vehicle, the three-dimensional map information, and the virtual body of the moving object, it is easy to generate a guiding image that does not overlap with the moving object existing around the guidance branch point. It becomes possible to do.

It is a schematic block diagram of the superimposition image display device which concerns on this embodiment. It is a block diagram which showed the navigation device which concerns on this embodiment. It is a flowchart of the driving support processing program which concerns on this embodiment. It is a figure which showed the 3D map generated based on the 3D map information. It is a figure which showed the 3D three-dimensional map after arranging a guide image and a moving object. It is a figure explaining the moving object which influences when the own vehicle travels along a traveling path. It is a figure which showed the example which corrects the guide image so that a virtual image is not displayed in the area which overlaps with an influence moving object. It is a figure which showed the example which corrects the guide image so that a virtual image is not displayed in the area which overlaps with an influence moving object. It is a figure which showed the example of the virtual image of the guide image visually seen from the occupant of a vehicle. It is a flowchart of the sub-processing program of the display range determination processing.

Hereinafter, a detailed description will be given with reference to the drawings based on an embodiment embodying the superimposed image display device according to the present invention. First, a schematic configuration of the superimposed image display device 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a schematic configuration diagram of a superimposed image display device 1 according to the present embodiment.

As shown in FIG. 1, the superimposed image display device 1 includes a navigation device 3 mounted on the vehicle 2 and a head-up display device (hereinafter referred to as HUD) 4 mounted on the vehicle 2 and connected to the navigation device 3. Basically has.

Here, the navigation device 3 is set to search for a recommended route to the destination, to display a map image around the current position of the vehicle 2 based on the map data acquired from the server or stored in the memory. It has a function of providing traveling guidance along the guide route together with the HUD4. It should be noted that it is not necessary for the navigation device 3 to have all of the above functions, and the present invention can be configured as long as it has at least a function of providing traveling guidance along a guide route. The details of the structure of the navigation device 3 will be described later.

On the other hand, the HUD 4 is installed inside the dashboard 5 of the vehicle 2, and has a liquid crystal display 6 which is an image display surface on which an image is displayed. Then, the image projected on the liquid crystal display 6 is reflected on the front window 8 in front of the driver's seat via the concave mirror 7 or the like provided in the HUD 4 as described later, and is configured to be visually recognized by the occupant 9 of the vehicle 2. ing. The images displayed on the liquid crystal display 6 include information on the vehicle 2 and various information used to support the driving of the occupant 9. For example, a warning to an object (another vehicle or a pedestrian) to be warned to the occupant 9, a guidance route set by the navigation device 3, guidance information based on the guidance route (arrow indicating the right / left turn direction, etc.), and a road surface. There are warnings to be displayed (crash caution, speed limit, etc.), current vehicle speed, advertisement image, guide sign, map image, traffic information, news, weather forecast, time, connected smartphone screen, TV program, etc.

Further, in the HUD 4 of the present embodiment, when the front window 8 is reflected and the occupant 9 visually recognizes the image displayed on the liquid crystal display 6, the occupant 9 is not at the position of the front window 8 but at the tip of the front window 8. The image displayed on the liquid crystal display 6 at a distant position is configured to be visually recognized as a virtual image 10. Further, the virtual image 10 is displayed superimposed on the landscape (actual view) in front of the vehicle. For example, the virtual image 10 is superimposed on an arbitrary object (road surface, building, moving object to be warned, etc.) located in front of the vehicle. It is also possible to display it. The virtual image 10 that can be visually recognized by the occupant 9 is an image displayed on the liquid crystal display 6, but the vertical direction and the horizontal direction may be reversed by passing through the concave mirror 7 and other mirrors, and the inversion thereof is taken into consideration. It is necessary to display the image of the liquid crystal display 6. The size is also changed via the concave mirror 7.

Here, regarding the position where the virtual image 10 is generated, more specifically, the distance L from the occupant 9 to the virtual image 10 (hereinafter referred to as an imaging distance), the curvature of the concave mirror 7 provided in the HUD 4, the liquid crystal display 6 and the concave mirror 7 It is possible to set appropriately according to the relative position of. For example, if the curvature of the concave mirror 7 is fixed, the image formation distance L is determined by the distance (optical path length) along the optical path from the position where the image is displayed on the liquid crystal display 6 to the concave mirror 7. For example, the optical path length is set so that the image formation distance L is 2.5 m.

Further, in the present embodiment, HUD4 is used as a means for displaying an image superimposed on the landscape in front of the vehicle, but other means may be used. For example, a window shield display (WSD) that displays an image on the front window 8 may be used. In WSD, the front window 8 may be used as a screen to display an image from the projector, or the front window 8 may be used as a transmissive liquid crystal display. The image displayed on the front window 8 by the WSD is an image superimposed on the landscape in front of the vehicle as in the HUD4.

Further, it is also possible to display the landscape in front of the vehicle captured by the front camera 11 described later on the liquid crystal display in the vehicle and display the image superimposed on the landscape displayed in the same liquid crystal display. Even in that case, the image displayed on the liquid crystal display is an image superimposed on the landscape in front of the vehicle as in the case of HUD4.

Further, the front camera 11 is installed above the front bumper of the vehicle, behind the rearview mirror, and the like. The front camera 11 is an image pickup device having a camera using a solid-state image pickup element such as a CCD, and is installed with the optical axis direction facing forward in the traveling direction of the vehicle. Then, by performing image processing on the captured image captured by the front camera 11, the situation of the front environment (that is, the environment on which the virtual image 10 is superimposed) visually recognized by the occupant 9 through the front window 8 is detected. Will be done. A sensor such as a millimeter wave radar may be used instead of the front camera 11.

In addition, an in-vehicle camera 12 is installed on the upper surface of the instrument panel of the vehicle. The in-vehicle camera 12 is an image pickup device having a camera using a solid-state image pickup element such as a CCD, and is installed with the optical axis direction facing the driver's seat. Then, the face of the occupant 9 sitting in the driver's seat is imaged. Then, image processing is performed on the captured image captured by the in-vehicle camera 12, so that the eye position (line-of-sight start point) and the line-of-sight direction of the occupant 9 are detected.

Next, a schematic configuration of the navigation device 3 constituting the superimposed image display device 1 will be described with reference to FIG. FIG. 2 is a block diagram showing a navigation device 3 according to the present embodiment.

As shown in FIG. 2, the navigation device 3 according to the present embodiment includes a current position detection unit 13 that detects the current position of the vehicle 2 on which the navigation device 3 is mounted, and a data recording unit 14 in which various data are recorded. , A navigation ECU 15 that performs various arithmetic processes based on the input information, an operation unit 16 that accepts operations from the user, and a liquid crystal display 17 that displays a map around the vehicle and facility information related to the facility to the user. Communication between a speaker 18 that outputs voice guidance regarding route guidance, a DVD drive 19 that reads a DVD as a storage medium, and an information center such as a VICS (registered trademark: Vehicle Information and Communication System) center. It has a module 20 and. Further, the navigation device 3 is connected to the above-mentioned HUD4, the front camera 11, the in-vehicle camera 12, and the like via an in-vehicle network such as CAN.

Hereinafter, each component of the navigation device 3 will be described in order.
The current position detection unit 13 includes a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, and the like, and can detect the current position, direction, vehicle running speed, current time, and the like of the vehicle. .. Here, in particular, the vehicle speed sensor 22 is a sensor for detecting the moving distance and the vehicle speed of the vehicle, generates a pulse according to the rotation of the drive wheel of the vehicle, and outputs the pulse signal to the navigation ECU 15. Then, the navigation ECU 15 calculates the rotation speed and the moving distance of the drive wheels by counting the generated pulses. It is not necessary for the navigation device 3 to include all of the above four types of sensors, and the navigation device 3 may include only one or a plurality of of these sensors.

Further, the data recording unit 14 is a driver for reading a hard disk (not shown) as an external storage device and a recording medium, a map information DB 31 recorded on the hard disk, a predetermined program, and the like, and writing predetermined data to the hard disk. It is equipped with a recording head (not shown). The data recording unit 14 may have a flash memory, a memory card, an optical disk such as a CD or a DVD, instead of the hard disk. Further, the map information DB 31 may be stored in an external server and acquired by the navigation device 3 by communication.

Here, the map information DB 31 stores the two-dimensional map information 33 and the three-dimensional map information 34, respectively. The two-dimensional map information 33 is map information used in a general navigation device 3, and is, for example, link data related to a road (link), node data related to a node point, facility data related to a facility, and search data used for route search processing. , Map display data for displaying a map, intersection data for each intersection, search data for searching a point, and the like.

On the other hand, the three-dimensional map information 34 has not only the plane but also the height information, and is the map information for expressing the map in three dimensions. In particular, in the present embodiment, the map information is used to represent the contour of the road, the shape of the building, the lane marking of the road, the traffic light, the road sign, the signboard, etc. in three dimensions. The three-dimensional map information 34 may include information other than the contour of the road, the shape of the building, the lane marking of the road, the traffic light, the road sign, and the signboard. For example, information for expressing roadside trees, road markings, etc. in three dimensions may be included. Further, as the three-dimensional map information 34, the map itself in which each object such as the outline of the road, the shape of the building, the lane marking of the road, the traffic light, the road sign, and the signboard is arranged on the three-dimensional space may be stored. However, information necessary for expressing a map in three dimensions (road contours, building shapes, road lane markings, traffic lights, road signs, three-dimensional coordinate data such as signs, etc.) may be stored. .. When the information necessary for expressing the map in three dimensions is stored, the navigation device 3 uses the information stored as the three-dimensional map information 34 at the required timing to set the target area in three dimensions. Generate a represented map.

Then, the navigation device 3 uses the two-dimensional map information 33 for general functions such as displaying a map image on the liquid crystal display 17 and searching for a guide route. Further, as will be described later, the processing related to the display of the guide image is performed by using the three-dimensional map information 34 in addition to the two-dimensional map information 33.

On the other hand, the navigation ECU (electronic control unit) 15 is an electronic control unit that controls the entire navigation device 3, a CPU 41 as a calculation device and a control device, and a working memory for the CPU 41 to perform various calculation processes. The RAM 42, which stores the route data when the route is searched, the control program, and the ROM 43 and ROM 43, which record the driving support processing program (FIG. 3) described later, are read. It is provided with an internal storage device such as a flash memory 44 for storing a program. The navigation ECU 15 has various means as a processing algorithm. For example, the captured image acquisition means acquires an captured image of a landscape in front of the vehicle. The map information acquisition means acquires three-dimensional map information. The collation means collates the captured image with the three-dimensional map information. The image arranging means arranges the guide image on the traveling path where the vehicle travels after leaving the guide branch point in the three-dimensional map information based on the collation result of the collation means. The virtual body arranging means arranges the virtual body of the moving object at the position where the moving object exists in the three-dimensional map information based on the collation result of the collation means. The shape acquisition means acquires a portion of the shape of the guide image when the guide image is visually recognized from the position of the vehicle in the three-dimensional map information, which does not overlap with the virtual body.

The operation unit 16 is operated when inputting a departure point as a travel start point and a destination as a travel end point, and has a plurality of operation switches (not shown) such as various keys and buttons. Then, the navigation ECU 15 controls to execute various corresponding operations based on the switch signal output by pressing each switch or the like. The operation unit 16 may have a touch panel provided on the front surface of the liquid crystal display 17. Further, it may have a microphone and a voice recognition device.

In addition, the liquid crystal display 17 has a map image including roads, traffic information, operation guidance, operation menus, key guidance, guidance routes from the departure point to the destination, guidance information along the guidance routes, news, weather forecasts, and the like. The time, mail, TV program, etc. are displayed. Since the HUD 4 is provided as an information display means in the present embodiment, the liquid crystal display 17 may be omitted if the HUD 4 is used to display the map image or the like.

Further, the speaker 18 outputs voice guidance for guiding the traveling along the guidance route and traffic information guidance based on the instruction from the navigation ECU 15.

Further, the DVD drive 19 is a drive capable of reading data recorded on a recording medium such as a DVD or a CD. Then, based on the read data, music or video is played back, the map information DB 31 is updated, and the like. A card slot for reading / writing a memory card may be provided instead of the DVD drive 19.

Further, the communication module 20 is a communication device for receiving traffic information including various information such as traffic jam information, regulation information, and traffic accident information transmitted from a traffic information center, for example, a VICS center or a probe center. For example, mobile phones and DCM.

Subsequently, among the superimposed image display devices 1 having the above configuration, a traveling support processing program executed in the navigation device 3 will be described with reference to FIG. FIG. 3 is a flowchart of the driving support processing program according to the present embodiment. Here, the traveling support processing program is executed after the ACC power supply (accessory power supply) of the vehicle is turned on, and is a program that guides the occupant 9 of the vehicle 2 in the traveling direction of the vehicle at the guidance branch point by using the HUD4. .. The program shown in the flowchart in FIG. 3 below is stored in the RAM 42 or ROM 43 included in the navigation device 3 and is executed by the CPU 41.

The "guidance branch point" is a branch point where it is necessary to guide the traveling direction of the vehicle when the vehicle is guided along the guidance route set in the navigation device 3, for example, a crossroads. This applies to intersections such as T-junctions and Y-junctions. However, the guidance branch point is not limited to the intersection where two roads intersect, and may be, for example, a branch portion where one road branches into a plurality of roads.

First, in the travel support processing program, in step 1 (hereinafter abbreviated as S) 1, the CPU 41 determines whether or not the guide branch point exists within a predetermined distance (for example, 700 m) ahead of the vehicle 2 in the traveling direction. Specifically, the CPU 41 is based on the guide route currently set in the navigation device 3, the map information stored in the map information DB 31, and the current position of the vehicle 2 detected based on the current position detection unit 13. To judge. The guide route is determined by a route search process using a known Dijkstra method when the user sets a destination, for example, at the start of traveling.

Further, the means for determining whether or not the guide branch point exists at a predetermined distance ahead of the traveling direction of the vehicle 2 in S1 is not limited to the above method. For example, the CPU 41 identifies the current position of the vehicle in the three-dimensional map information 34 by collating the captured image captured by the front camera 11 with the three-dimensional map information 34, and a guide branch point is set forward in the traveling direction. It may be determined whether or not it exists.

Then, when it is determined that the guide branch point exists at a predetermined distance ahead of the traveling direction of the vehicle 2 (S1: YES), the process proceeds to S2. On the other hand, when it is determined that the guide branch point does not exist at a predetermined distance ahead of the traveling direction of the vehicle 2 (S1: NO), the vehicle waits until it is determined that the guide branch point exists. In this case, the CPU 41 repeatedly executes the process of S1 at intervals of, for example, a preset time.

In S2, the CPU 41 acquires a captured image of the scenery in front of the vehicle by the front camera 11. Further, the CPU 41 reads out the three-dimensional map information 34 around the current position of the vehicle from the map information DB 31. The three-dimensional map information 34 contains three-dimensional coordinate data such as information necessary for expressing a map in three dimensions (road contour, building shape, road lane marking, traffic light, road sign, signboard, etc.). Etc.) are memorized.

Next, in S3, the CPU 41 generates a three-dimensional three-dimensional map (a map representing buildings, roads, etc. in three dimensions) showing the vicinity of the guide branch point based on the three-dimensional map information 34 read in S2. Specifically, a three-dimensional three-dimensional map is generated by executing the following processing.

First, the CPU 41 acquires an object (shape data) that models a road, a building, a road sign, a signboard, etc. in a three-dimensional space from the three-dimensional map information 34. The CPU 41 may be configured to acquire the corresponding object from the objects modeled in advance and stored in the DB, or may be configured to perform modeling in S3 to newly create the object. When modeling is performed, information that identifies the shape and position of structures (buildings, road signs, signboards, etc.) around roads and guide junctions is acquired from the 3D map information 34, and the acquired information is used. Perform modeling processing based on.

Here, modeling is a process of creating the shape of a model (object) in a three-dimensional space, and more specifically, determining the coordinates of each vertex, determining the boundary line, and determining the parameters of the equation expressing the surface. I do. Since modeling is a known technique, details will be omitted. Then, the modeled object (shape data) is expressed in the form of a "wireframe model" that displays only the sides, a "surface model" that displays the faces, and the like, depending on the intended use. Then, the three-dimensional space in which each object is formed is made into a three-dimensional three-dimensional map.

Further, in S3, the CPU 41 also specifies the current position and orientation of the own vehicle in the generated three-dimensional three-dimensional map based on the parameters detected by the current position detection unit 13. In order to facilitate the collation process between the 3D map and the captured image, which will be described later, the current position of the own vehicle is set to the installation position of the front camera 11 installed in the own vehicle, and the orientation of the own vehicle is the front. It is desirable that the direction is the optical axis of the camera 11.

Here, FIG. 4 is a diagram showing an example of the three-dimensional three-dimensional map 101 generated in S3. As shown in FIG. 4, in the three-dimensional three-dimensional map 101, each object 102 showing a road or a structure (building, road sign, signboard, etc.) is arranged in the three-dimensional space. In particular, each object 102 indicating a road or a structure around the guidance branch point 103 in front of the traveling direction of the vehicle is arranged. Further, the own vehicle position mark 104 indicating the current position and direction of the own vehicle is also arranged on the three-dimensional three-dimensional map 101.

Subsequently, in S4, the CPU 41 collates the captured image acquired in S2 with the three-dimensional three-dimensional map generated in S3, and determines whether or not there is a deviation between the two. Specifically, the CPU 41 looks at the current position of the own vehicle set on the three-dimensional map (more specifically, the installation position of the front camera 11 and the height is also taken into consideration), and looks at the direction of the own vehicle. The direction is set, and the image when the three-dimensional three-dimensional map is visually recognized according to the set viewpoint and line-of-sight direction is collated with the captured image. Since the three-dimensional map does not include moving objects such as pedestrians and other vehicles, and some fixed objects such as trees, the deviation caused by them is basically ignored for determination. Further, it is desirable to consider that no deviation has occurred not only when the two are completely matched, but also when there is a deviation within a certain allowable range.

Then, when it is determined that there is no deviation between the captured image acquired in S2 and the three-dimensional three-dimensional map generated in S3 (S4: YES), the process proceeds to S6. On the other hand, when it is determined that there is a discrepancy between the captured image acquired in S2 and the three-dimensional stereoscopic map generated in S3 (S4: NO), the process proceeds to S5.

In S5, the CPU 41 sets the current position and orientation of the own vehicle set in the three-dimensional map so that the deviation between the three-dimensional map generated in S3 and the captured image acquired in S2 becomes small. to correct. The current position and orientation of the own vehicle on the three-dimensional map may be fixed and the object side may be corrected. This makes it possible to accurately specify the current position and orientation of the own vehicle (more specifically, the installation position of the front camera 11 and the direction of the optical axis) in the three-dimensional three-dimensional map. After that, it returns to S4.

On the other hand, in S6, the CPU 41 arranges a guide image for guiding at the guide branch point with respect to the three-dimensional stereoscopic map generated in S3. In the present embodiment, the image of the arrow that guides the traveling direction of the traveling path on which the vehicle travels after leaving the guidance branch point is used as a guiding image. Then, a predetermined distance (for example, 1 m) above the road surface of the traveling path on which the vehicle travels after exiting the guiding branch point (for example, the road connected to the left side of the guiding branch point when turning left at the guiding branch point). The guide image is placed in a vertical position at a position separated from the road.

Next, in S7, the CPU 41 detects a moving object (moving object) located around the guide branch point. The moving object to be detected is, for example, a pedestrian, a bicycle, or a vehicle, and is detected by performing image processing on the captured image captured by the front camera 11. The CPU 41 also specifies the position, type, and shape of the moving object detected based on the captured image, and the subsequent traveling path (traveling direction).

Subsequently, in S8, the CPU 41 arranges the model (virtual body) of the moving object detected around the guide branch point at the position detected with respect to the three-dimensional three-dimensional map generated in S3. When a plurality of moving objects are detected, all the detected moving objects are arranged on the three-dimensional three-dimensional map. The virtual body placed on the three-dimensional map has a shape that imitates the shape of an actual moving object. For example, if it is a pedestrian, it is a virtual body having a human shape, and if it is a vehicle, it is a virtual body having a vehicle shape. The size should also correspond to the size of the actual moving object. However, when the moving object is irregular (the shape and type cannot be specified), the virtual body is exceptionally the smallest rectangular parallelepiped including all of the moving objects.

Here, FIG. 5 is a diagram in which the guide image 105 and the virtual bodies 106 and 107 of the moving object are arranged on the three-dimensional three-dimensional map 101, respectively. In the example shown in FIG. 5, the virtual body 106 indicates a pedestrian, and the virtual body 106 indicates a bicycle. That is, it is shown that pedestrians and bicycles exist as moving objects around the guide branch point 103, respectively.

After that, in S9, the CPU 41 determines whether or not there is a "moving object that has an influence when the own vehicle travels along the traveling path" among the moving objects in which the virtual body is arranged with respect to the three-dimensional map. To judge. Specifically, a moving object in which the traveling path of the vehicle and the traveling path in the future overlap is determined to be "a moving object having an influence when the own vehicle travels along the traveling path".

For example, if the path (direction of travel) of the pedestrian modeled by the virtual body 106 as shown in FIG. 6 is a path that crosses the road connecting to the left side of the guide branch point, the future of the pedestrian The traveling path of the vehicle and the traveling path 108 of the own vehicle overlap. Therefore, the pedestrian modeled by the virtual body 106 is determined to be "a moving object having an influence when the own vehicle travels along the traveling path". On the other hand, the path (direction of travel) of the bicycle modeled by the virtual body 107 is a path that moves away from the guide branch 103 without crossing the road connected to the left side of the guide branch. , The future course of the bicycle and the course 108 of the own vehicle do not overlap. Therefore, it is determined that the bicycle modeled by the virtual body 107 is not "a moving object having an influence when the own vehicle travels along the traveling path".

Then, when it is determined that there is a "moving object that has an influence when the own vehicle travels along the traveling path" among the moving objects in which the virtual body is arranged with respect to the three-dimensional three-dimensional map (S9: YES), the process proceeds to S10. On the other hand, when it is determined that none of the moving objects in which the virtual body is arranged with respect to the three-dimensional three-dimensional map is "a moving object having an influence when the own vehicle travels along the traveling path" ( S9: NO) shifts to S11.

In S10, the CPU 41 displays a guide image in a manner that does not overlap with a moving object (hereinafter referred to as an affected moving object) determined to have an influence when the vehicle travels along the traveling path, so that the CPU 41 displays a three-dimensional three-dimensional map. Correct the guide image placed in. Specifically, first, the CPU 41 acquires an image (hereinafter referred to as a visual image) in which a three-dimensional three-dimensional map on which a moving object or a guide image is arranged is visually recognized in the traveling direction of the vehicle from the viewpoint of the vehicle (occupant). In particular, the viewpoint of the vehicle is the position of the eyes of the occupants of the vehicle. The position of the occupant's eyes can be detected by the in-vehicle camera 12, and the current position and orientation of the vehicle finally identified as a result of collation of the captured image in S4 with the three-dimensional stereoscopic map. , The position of the occupant's eyes on the three-dimensional three-dimensional map is specified by combining the detection results of the in-vehicle camera 12. Then, in the visual image, each object (road, building, road sign, signboard, virtual body of moving object, guide image, etc.) arranged on the three-dimensional three-dimensional map is displayed in the traveling direction of the vehicle from the viewpoint of the vehicle (occupant). It is an image that can be visually recognized when it is visually recognized, and corresponds to the view of a vehicle occupant. It should be noted that the visual image does not necessarily have to be an image visually recognized in the traveling direction of the vehicle as long as it is an image visually recognized from the viewpoint of the vehicle (occupant). However, at least the visual image needs to include the guide image. Then, when the virtual body 106 of the influential moving object and the guide image overlap in the visual image acquired as described above, as shown in FIG. 5, the correction of either (1) or (2) below is performed. I do.

(1) The guide image arranged on the three-dimensional three-dimensional map is corrected to a guide image lacking the area overlapping with the virtual body of the affected moving object so that the virtual image is not displayed in the area overlapping the affected moving object. For example, as shown in FIG. 7, the arrow is a visual image lacking a region where the guide image 105 and the virtual body 106 of the affected moving object overlap. However, especially when the tip of the arrow overlaps with the virtual body 106 of the affected moving object, the tip of the arrow is displayed by extending or contracting the tip of the arrow.
(2) The guide image arranged on the three-dimensional three-dimensional map is corrected to a guide image having a shape that bypasses the virtual body of the affected moving object so that the virtual image is not displayed in the area overlapping the affected moving object. For example, as shown in FIG. 8, in the visual recognition image, the guide image 105 is a curved arrow so as not to overlap with the virtual body 106 of the affected moving object.

In this embodiment, the process of S10 is performed only for the affected object located on the back side of the traveling path of the own vehicle (that is, on the back side of the guide image 105). For the affected object located on the front side of the traveling path of the own vehicle (that is, on the front side of the guide image 105), when the shape of the guide image is determined in the process of S11 described later even if the correction of S10 is not performed. This is because the guide image is obtained by removing the part that overlaps with the affected object.

Next, in S11, the CPU 41 stores the shape of the guide image included in the visual recognition image (the shape after the correction when the correction of S10 is performed) as the shape of the guide image to be displayed by the HUD 4. The shape of the guide image included in the visual image is the shape of each object (road, building, road sign, signboard, virtual body of moving object, guide image, etc.) arranged on the three-dimensional map of the vehicle (occupant). It is the shape of the guide image that can be visually recognized when visually recognized from the viewpoint. Here, the shape of the guide image stored in S11 is visually recognized by overlapping with each object (for example, a road sign, a building, a virtual body of a moving object arranged in S28, etc.) located on the front side of the traveling path. If it is, the shape will be the shape excluding the overlapping part. For example, if the shape of the guide image is the shape of an arrow, the shape of the arrow is excluded from the area that overlaps with each object and is visually recognized. However, especially when the tip of the arrow overlaps with the object, the tip of the arrow is extended or contracted so that the tip of the arrow does not overlap with the object, and at least the tip of the arrow is included in the shape. On the other hand, when the guide image overlaps with each object located on the back side of the traveling path and is visually recognized, the shape includes the overlapping portion.

Further, in S12, the CPU 41 acquires the position of the guide image arranged on the three-dimensional three-dimensional map. Specifically, the position is separated from the road surface of the traveling path on which the vehicle travels by a predetermined distance (for example, 1 m) after exiting the guide branch point.

After that, in S13, the CPU 41 performs a display range determination process described later. In the display range determination process, a range for displaying the guide image on the HUD 4 (a range for projecting the guide image on the front window 8 or a range for displaying the guide image on the liquid crystal display 6 of the HUD 4) is determined.

Subsequently, in S14, the CPU 41 transmits a control signal to the HUD4, and displays the guide image of the shape stored in the S11 on the liquid crystal display 6 of the HUD4 in the display range determined in the S13. As a result, a virtual image of an arrow is particularly visually recognized by the occupant of the vehicle as a guide image for guiding the traveling direction of the traveling path of the vehicle after leaving the guidance branch point, superimposed on the scenery in front of the vehicle.

Here, FIG. 9 shows a virtual image of the guidance image visually recognized by the occupants of the vehicle, especially when the traveling direction of the traveling path on which the vehicle travels after leaving the guidance branch point is the left direction (that is, a left turn). It is a figure.
As shown in FIG. 9, an arrow indicating that the vehicle goes to the left through the front window 8 of the vehicle by superimposing it on a position separated upward from the road surface of the traveling path after turning left at the guidance branch point 103. The guide image 105 is displayed as a virtual image. Further, the virtual image of the guide image 105 is displayed except for the area overlapping with the road sign 112, which is a peripheral object located on the front side of the traveling path. Therefore, the guide image 105 is visually recognized so as to pass through the back side (rear side) of the road sign 112. On the other hand, the road sign 113 standing on the back side of the traveling path is displayed overlapping. Therefore, the guide image 105 is visually recognized so as to pass in front of the road sign 113. As a result, the occupant 9 can recognize the anteroposterior relationship between the positions of the road signs 112 and 113 and the position of the virtual image of the guide image 105, and can accurately grasp the course to be followed. In the example shown in FIG. 9, the peripheral object that overlaps with the virtual image of the guide image 105 is a road sign, but the same applies to a building or a moving object (pedestrian or other vehicle). ..
Further, as described above, when the affected moving object 110 is located around the guide branch point 103, the corresponding area is lacking or the affected moving object 110 is omitted so that the virtual image is not displayed in the area overlapping the affected moving object 110. The guide image 105 has a shape that bypasses the above. As a result, the occupant 9 can reliably see the affected moving object 110 without the guidance image 105 impairing the visibility of the affected moving object 110.

Next, a sub-process of the display range determination process executed in S13 will be described with reference to FIG. FIG. 10 is a flowchart of a sub-processing program for display range determination processing.

First, in S21, the CPU 41 detects the position of the eyes of the occupant of the vehicle based on the captured image captured by the in-vehicle camera 12. The detected eye position is specified by three-dimensional position coordinates.

Next, in S22, the CPU 41 determines whether or not the display of the HUD 4 is ON. It should be noted that the display of the HUD 4 can be switched on or off by the operation of the occupant of the vehicle. Further, ON or OFF may be automatically switched based on the surrounding conditions and the state of the vehicle.

Then, when it is determined that the display of HUD4 is ON (S22: YES), the process proceeds to S23. On the other hand, when it is determined that the display of the HUD 4 is OFF (S22: NO), the process ends without displaying the virtual image of the guide image by the HUD 4.

In S23, the CPU 41 acquires the position coordinates of the front window 8 on which the image is projected by the HUD 4. The position coordinates of the front window 8 are specified by three-dimensional position coordinates.

Next, in S24, the CPU 41 acquires the coordinates for specifying the position of the guide image arranged on the three-dimensional stereoscopic map acquired in S12 as the position coordinates of the guide image. The position coordinates of the guide image are also specified by the three-dimensional position coordinates.

Subsequently, in S25, the CPU 41 determines the projection range of the guide image in the front window 8 based on the position coordinates acquired in S21, S23, and S24. Further, the display range of the guide image on the liquid crystal display 6 inside the HUD 4 is also determined from the determined projection range. After that, it shifts to S14 and displays a virtual image using HUD4 based on the determined projection range and display range. The processes of S11 to S14 are repeated until the HUD display is turned off.

As described in detail above, according to the superimposed image display device 1 and the computer program executed by the superimposed image display device 1 according to the present embodiment, the traveling direction of the traveling path on which the vehicle travels after leaving the guidance branch point. When the guide image for guiding is superimposed on the scenery in front of the vehicle and visually recognized, the guide image is displayed in a manner that does not overlap with the moving object existing around the guide branch point (S10 to S14). While it is possible to accurately guide the vehicle occupant on the path on which the vehicle should travel, it does not reduce the visibility of moving objects that the vehicle occupant should see.

It should be noted that the present invention is not limited to the above embodiment, and it goes without saying that various improvements and modifications can be made without departing from the gist of the present invention.
For example, in the present embodiment, the HUD 4 is configured to generate a virtual image in front of the front window 8 of the vehicle 2, but a virtual image may be generated in front of a window other than the front window 8. Further, the target for reflecting the image by the HUD 4 may be a visor (combiner) installed around the front window 8 instead of the front window 8 itself.

Further, in the present embodiment, the HUD 4 is used as a means for displaying an image superimposed on the surrounding environment, but a window shield display (WSD) for displaying an image on the front window 8 may be used. Alternatively, it may be a display device that superimposes a guide image on the actual scene displayed on the liquid crystal display 17 of the navigation device 3.

Further, in the present embodiment, the processing of the driving support processing program (FIG. 3) is executed by the navigation ECU 15 of the navigation device 3, but the execution subject can be changed as appropriate. For example, it may be configured to be executed by the control unit of the HUD4, the vehicle control ECU, or other in-vehicle device. When the control unit of the HUD4 is executed, the superimposed image display device according to the present invention may be configured only by the HUD4.

Further, although the embodiment in which the superimposed image display device according to the present invention is embodied has been described above, the display device can also have the following configuration, and in that case, the following effects are obtained.

For example, the first configuration is as follows.
A guide image (105) mounted on the vehicle (2) and guiding the traveling direction of the traveling path (108) on which the vehicle travels after leaving the guidance branch point (103) is superimposed on the landscape in front of the vehicle. It is a superimposed image display device (1) for visually recognizing, and has a guide image display means (41) for displaying the guide image in a manner that does not overlap with a moving object existing around the guide branch point.
According to the superimposed image display device having the above configuration, the guide image that guides the traveling direction after leaving the guide branch point is displayed so as not to overlap with the moving object, so that the guide image indicates the course on which the vehicle should travel. While it is possible to accurately guide the vehicle occupants, it does not reduce the visibility of moving objects that should be visually recognized by the vehicle occupants.

The second configuration is as follows.
The guide image display means (41) displays the guide image (105) in a manner that does not overlap with the moving object (110) that has an influence when the vehicle (2) travels along the traveling path (108). ..
According to the superimposed image display device having the above configuration, the visibility of a moving object that needs to be visually recognized by the occupant as the vehicle travels is not deteriorated.

The third configuration is as follows.
The guide image display means (41) has a traveling path predicting means (41) for predicting a future traveling path of the moving object (110), and the guiding image display means (41) has a traveling path (108) of the vehicle (2) and a future traveling path. The guide image (105) is displayed in such a manner that it does not overlap with the overlapping moving object.
According to the superimposed image display device having the above configuration, the visibility of a moving object whose traveling path overlaps with that of the vehicle will not be deteriorated in the future.

The fourth configuration is as follows.
The guide image display means (41) displays a region overlapping the moving object (110) by removing it from the guide image (105).
According to the superimposed image display device having the above configuration, since the guidance image in which the area overlapping with the moving object is hidden is displayed, it is possible to accurately guide the vehicle occupant on the course on which the vehicle should go by the guidance image. Therefore, the visibility of moving objects that should be visually recognized by the occupants of the vehicle is not reduced.

The fifth configuration is as follows.
The guide image display means (41) displays the guide image that bypasses the moving object (110).
According to the superimposed image display device having the above configuration, since the guidance image bypassing the moving object is displayed, the guidance image can accurately guide the vehicle occupant on the course on which the vehicle should go, while the vehicle occupant. It does not reduce the visibility of moving objects that should be visually recognized.

The sixth configuration is as follows.
The captured image acquisition means (41) for acquiring an captured image of the landscape in front of the vehicle (2), the map information acquisition means (41) for acquiring three-dimensional map information (34), and the captured image and three dimensions. The guide image (105) is attached to the collation means (41) for collating the map information and the traveling path on which the vehicle travels after leaving the guidance branch point in the three-dimensional map information based on the collation result of the collation means. Based on the collation result of the image arranging means (41) to be arranged and the collation result, the virtual body (106, 107) of the moving object is arranged at the position where the moving object exists in the three-dimensional map information. The guide image display means has an arrangement means (41) and a shape acquisition means (41) for acquiring the shape of the guide image when the guide image is visually recognized from the position of the vehicle in the three-dimensional map information. Displays the guide image of the shape acquired by the shape acquisition means.
According to the superimposed image display device having the above configuration, by using the captured image of the landscape in front of the vehicle, the three-dimensional map information, and the virtual body of the moving object, the weight of the moving object existing around the guide branch point is increased. It is possible to easily generate a guide image in a mode that does not become.

The seventh configuration is as follows.
When the moving object (110) is amorphous, the virtual body (106, 107) is the smallest rectangular parallelepiped including all of the moving objects.
According to the superimposed image display device having the above configuration, even if the shape of the moving object cannot be specified, the guiding image that guides the traveling direction after leaving the guiding branch point is displayed so as not to overlap with the moving object. It becomes possible.

1 Superimposed image display device 2 Vehicle 3 Navigation device 4 HUD
8 Front window 9 Crew 41 CPU
42 RAM
43 ROM
103 Guidance branch point 105 Guidance image 106, 107 Virtual body of moving object 108 Path 110 Moving object 112, 113 Road sign

Claims (5)

It is a superimposed image display device that is mounted on a vehicle and superimposes and visually recognizes a guidance image that guides the traveling direction of the traveling path on which the vehicle travels after leaving the guidance branch point on the scenery in front of the vehicle.
An image capture image acquisition means for acquiring an image captured by capturing the scenery in front of the vehicle, and
Map information acquisition means for acquiring 3D map information,
A collation means for collating the captured image with the three-dimensional map information,
Based on the collation result of the collation means, the image arranging means for arranging the guide image on the traveling path of the vehicle after leaving the guide branch point in the three-dimensional map information.
Based on the collation result of the collation means, the virtual body arranging means for arranging the virtual body of the moving object at the position where the moving object exists in the three-dimensional map information for the moving object existing around the guide branch point.
A shape acquisition means for acquiring a portion of the shape of the guide image when the guide image is visually recognized from the position of the vehicle in the three-dimensional map information and which does not overlap with the virtual body.
Wherein by displaying the guidance image of the acquired shape by the shape acquiring unit, superimposed image display apparatus having a guide image display means for displaying the guide image in a manner that does not overlap with the body thereof. The superimposed image display device according to claim 1, wherein the moving object is a moving object that has an influence when the vehicle travels along a traveling path. It has a course prediction means for predicting the future course of the moving object, and has
The superimposed image display device according to claim 2, wherein the moving object is a moving object in which the traveling path of the vehicle and the traveling path in the future overlap. The superimposed image display device according to any one of claims 1 to 3 , wherein when the moving object is amorphous, the virtual body is the smallest rectangular parallelepiped including all of the moving objects. A superimposed image display device mounted on the vehicle and superimposing a guidance image that guides the traveling direction of the traveling path of the vehicle after exiting the guidance branch point on the scenery in front of the vehicle for visual recognition.
An image capture image acquisition means for acquiring an image captured by capturing the scenery in front of the vehicle, and
Map information acquisition means for acquiring 3D map information,
A collation means for collating the captured image with the three-dimensional map information,
Based on the collation result of the collation means, the image arranging means for arranging the guide image on the traveling path of the vehicle after leaving the guide branch point in the three-dimensional map information.
Based on the collation result of the collation means, the virtual body arranging means for arranging the virtual body of the moving object at the position where the moving object exists in the three-dimensional map information for the moving object existing around the guide branch point.
A shape acquisition means for acquiring a portion of the shape of the guide image when the guide image is visually recognized from the position of the vehicle in the three-dimensional map information and which does not overlap with the virtual body.
A guide image display means for displaying the guide image in a manner that does not overlap with the moving object by displaying the guide image of the shape acquired by the shape acquisition means .
A computer program to make it work.

Images