JP2002048565A - Display device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device for vehicle allowing a driver to easily recognize a forward obstacle in a traveling environment with poor visual confirmability and a map image for navigation. SOLUTION: A display control device performs a control (S16) of displaying the three-dimensional map image for navigation and the infrared image taken by use of an infrared camera in a superposed display mode, and regulates the display by the superposed display mode, when the ratio of a partial image area corresponding to the obstacle present forward included in the infrared image to the whole area of the map image is, for example, 50% or more during the control S16, except the traveling environment with extremely poor visual confirmability to display only the map image and a course guidance arrow for guiding a route (S14, S15 and S17).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device for a vehicle which informs an occupant of the vehicle of a state of an obstacle or the like existing around the vehicle by displaying the information. And a suitable display device.

[0002]

2. Description of the Related Art Hitherto, there has been proposed an apparatus which takes an image of the surroundings of a vehicle using an infrared imaging apparatus and displays the taken image in front of a driver's seat. According to these devices, even if the headlights are turned on, even in a situation around the vehicle (for example, at night, in dense fog, or the like) where it is difficult for the driver to recognize due to human visual characteristics, there is an obstacle that actually exists. The driver can easily grasp the state of things and the like, and can effectively support the driving operation.

[0003] Japanese Patent Application Laid-Open No. 8-263784 discloses a pedestrian in any part of a road ahead by displaying temperature distribution information imaged using an infrared imager on a map screen for navigation. There has been proposed a technique for notifying a driver whether or not a driver exists.

[0004]

According to the above prior art, the driver can confirm the presence of a pedestrian on a map image even in bad weather, which can contribute to safe driving. If the heat source such as is too close to the own vehicle, it becomes difficult for the driver to recognize the information displayed on the map image for navigation, which is inconvenient.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a vehicle display device which enables a driver to easily recognize a forward obstacle and a map image for navigation in a traveling environment having poor visibility.

[0006]

In order to achieve the above object, a vehicle display device according to the present invention has the following features.

That is, an image pickup device provided in a vehicle and taking an image of the front of the vehicle using infrared light, a three-dimensional map image generated based on map image information, and an infrared image picked up by the image pickup device. And a display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, the display control means comprising: On the other hand, when the ratio occupied by the partial image area corresponding to the obstacle existing ahead included in the infrared image is larger than a predetermined ratio, for example, Prohibit, or, in the display area of the display device, the map image and the infrared image are displayed separately by a method such as
The display according to the display mode is regulated.

[0008] In a preferred embodiment, the display device for a vehicle further includes estimating means for estimating the visibility ahead, and when the estimating means determines that the visibility is extremely poor, the display control is performed. The means may continue the display in the display mode regardless of the ratio of the partial image area to the entire area of the map image.

Further, in the above-described apparatus configuration, the display control means determines a partial image area corresponding to an obstacle included in the infrared image based on a luminance representing a radiant heat amount of the obstacle.・ Alternatively, by further providing obstacle detection means for detecting a relative distance and a position with respect to an obstacle existing in front, the partial image area corresponding to the obstacle included in the infrared image is displayed on the obstacle. The determination may be made based on a result of detection of the obstacle by the detection unit and information about a preset arrangement relationship between the imaging device and the obstacle detection unit on the vehicle.

[0010] Alternatively, in order to achieve the above object, a vehicle display device according to the present invention has the following configuration.

That is, an image pickup device provided in a vehicle for picking up an image of the front of the vehicle using infrared light, a three-dimensional map image generated based on map image information, and an infrared image picked up by the image pickup device. And a display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, the display control means comprising: On the other hand, when the ratio of the partial image area included in the infrared image and corresponding to the obstacle existing ahead occupies is larger than a predetermined ratio, the map image is highlighted. I do.

[0012] Alternatively, an image pickup device provided in a vehicle for picking up an image of the front of the vehicle using infrared rays, an obstacle detecting means for detecting a relative distance and a position of an obstacle present ahead, and a map image A three-dimensional map image generated based on information and an infrared image captured by the imaging device,
A display device for a vehicle, comprising: display control means for displaying on a display provided in the vehicle in a display mode superimposed on a display screen, wherein the display control means comprises: The correspondence between the partial image area corresponding to the obstacle included in the image and the detection result of the obstacle by the obstacle detection unit and the vehicle of the preset imaging device and the obstacle detection unit are set. And based on the information on the above arrangement relationship, when the relative distance to the detected obstacle is shorter than a predetermined value,
The display according to the display mode is regulated.

[0013]

According to the present invention, there is provided a vehicle display device in which a driver can easily recognize a forward obstacle and a map image for navigation in a driving environment having poor visibility. I do.

That is, according to the first, seventh, and eighth aspects of the present invention, when the relative distance to the obstacle in front becomes short, the superimposed display is restricted (for example, when the infrared image is displayed). Since the display is prohibited (claim 2) and the map image and the infrared image are displayed separately in the display area of the display (claim 3), the driver can use the map image for navigation (route guidance such as an arrow). (Including the guidance symbol for the vehicle) can be easily recognized.

According to the fourth aspect of the present invention, even in the case of stormy weather such as heavy fog or heavy rain, which cannot be easily recognized even if an obstacle exists at a short distance (for example, about one meter ahead), the display is superimposed. Is maintained, the driving operation can be efficiently supported by giving priority to ensuring the front view of the driver.

According to the fifth and sixth aspects of the present invention, it is possible to reliably detect an obstacle such as a pedestrian present ahead.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a display device for a vehicle according to the present invention will be described in detail as an embodiment of a display device mounted on an automobile, which is a typical vehicle, with reference to the drawings.

[First Embodiment] FIG. 1 shows a first embodiment of the present invention.
FIG. 3 is a block diagram showing a system configuration of a display device according to the embodiment.

In FIG. 1, reference numeral 2 denotes a light switch capable of turning on and off a headlamp, a small lamp, and a fog lamp. Reference numeral 11 denotes a wiper switch capable of setting an operation speed and an operation pattern of the wiper. In the present embodiment, the display on / off condition for displaying an image captured by the infrared camera 3 in a situation where visibility ahead is not good is determined by, for example, the operating state of the light switch 2 by the driver. A situation in which visibility is extremely poor (for example, a situation in which a field of view of about 10 meters in front of the host vehicle cannot be ensured) is determined based on the operation state of the wiper switch 11. Instead, the display is turned on in the evening or at night by referring to the time information or calendar information, or the display is turned on from 19:00 to 5:00 in summer and from 17:00 to 7:00 in winter. Use weather conditions obtained from outside via communication (for example, display on when it is rainy or dense fog) It may be determined Te.

Reference numeral 3 denotes an infrared camera which takes an image of the front of the vehicle using infrared rays. The infrared camera 3 is arranged in a rightward direction from a central axis extending in the front-rear direction of the vehicle due to the installation space at the head of the vehicle. At a position offset by B (meters). Here, the offset amount B is stored in the memory of the display control device 1 in advance.

Reference numeral 4 denotes a front obstacle sensor for detecting a relative distance from an obstacle present ahead by utilizing a laser radar, a millimeter wave radar, or the like. It is mounted on a central axis extending in the front-rear direction.

Reference numeral 5 denotes a GPS sensor for detecting the current position of the vehicle based on a GPS (global positioning system) signal received from the outside. Reference numeral 6 denotes a navigation main switch capable of turning on / off a navigation function.

Reference numeral 7 denotes a map scroll switch capable of instructing a scroll of a map image displayed on the display device 9A or 9B. Reference numeral 8 denotes a destination setting switch capable of setting a desired destination for which route guidance is desired.

Reference numerals 9A and 9B denote map images based on map information stored in the map database 10 in advance and / or images captured by the infrared camera 3 (hereinafter referred to as infrared images: see the display example of FIG. 8A). ) To display a liquid crystal display or a head-up display.

Here, the display device 9A (first display device)
A position in front of the driver's seat of the host vehicle, where the driver can easily see the display image without making a large line-of-sight movement when the driver stares ahead, and is a high position (dashboard May be located near the center position of the camera). Further, the display device 9B (second display device) is disposed on a center console which is closer to the outer peripheral portion of the line of sight when the driver gazes forward as compared with the disposition position of the display device 9A.

Reference numeral 12 denotes a road-to-vehicle communication device capable of receiving information on obstacles existing in front of the vehicle and information on visibility such as rainy weather and dense fog from the outside.

Reference numeral 1 denotes display control of a map image based on map information stored in the map database 10 in advance, and current position information detected by the GPS sensor 5 and a map corresponding to the current position information. A control function of a general navigation device for guiding a route to a desired destination set by an occupant (driver) using information and the like, and a display control using an infrared image (details will be described later).
And a display control device that performs the following. The navigation control and the display control by the display control device 1 are realized by executing software stored in a memory in advance by a CPU (not shown).

In this embodiment, the display control device 1
As shown in FIG. 8 (b), the display mode of the map image is a type in which the driver can perform a stereoscopic view similar to the front view seen from the driver's seat. Is assumed to be a type in which an arrow symbol to be advanced is displayed. Note that a general method for displaying a three-dimensional map image based on the map information is adopted, and the description in this embodiment is omitted.

Next, in this embodiment, the display control device 1
A specific control process performed by the server will be described.

When displaying a three-dimensional map image by the navigation function, the display control device 1 superimposes and displays an infrared image of a forward obstacle at a position in front of the traveling path of the own vehicle, so that the display control device 1 provides the driver with a display. Even in a traveling environment with poor visibility, the recognition of the type and size of the obstacle present ahead and the location (relationship with the traveling road) of the obstacle is improved. At this time, when the route guidance to the desired destination is performed and the relative distance from the obstacle in front is shortened, the information included in the map image for navigation is displayed when the superimposed display is continued. In such a case, it is not easy for the driver to recognize (guidance display such as an arrow), and in such a case, the superimposed display is restricted (in the present embodiment, the display of the infrared image is prohibited), so that the navigation display is not performed. Ensure the driver's recognition of map images.

However, even in the case of extremely poor visibility such as heavy fog or heavy rain, which cannot be easily recognized even if the obstacle exists at a short distance (for example, about one meter to several meters ahead), Can efficiently support the driver's driving operation by continuing the superimposed display.

Next, the method of adjusting the infrared camera 3 and the display standard of the map image, which must be performed when the map image and the infrared image are superimposed on each other, is described below. The description will be made separately for the case of a lane-only motorway and the case of a multi-lane motorway.

<Case where the traveling road is a one-lane road> First, the road on which the own vehicle travels is basically a one-lane automobile-only road where no human exists on the traveling road. Next, a case will be described in which only a partial image of a vehicle as a front obstacle in an infrared image is superimposed and displayed. As a display mode in this case, an infrared image of a preceding vehicle traveling in the same lane as the host vehicle may be displayed based on the center of the traveling lane of the traveling road.

As a more specific operation, the display control device 1
The display device displays a map image as illustrated in FIG. 8B so as to be positioned substantially at the center of a traveling lane in which the host vehicle is traveling, while gradually scrolling the map image according to a change in the current position. It is displayed on 9A or 9B. At this time, when there is a preceding vehicle ahead, only a partial image of the vehicle among the infrared images of the infrared camera 3 is superimposed so as to be located substantially at the center of the traveling lane, and FIG. )
The image of the display mode as illustrated in FIG.

Here, the shift amount of the infrared image (partial image representing the preceding vehicle) is determined by the relative position of the preceding vehicle to the own vehicle detected by the forward obstacle sensor 4, the offset amount B, the infrared image, and the map image. Set based on each scale of. For example, both images have a scale of 1/2,
When the preceding vehicle is located at a position shifted A (m) to the left from the center axis of the host vehicle and the offset amount of the infrared camera 3 is B (m) to the right as described above, the shift amount of the infrared image is: In the right direction, it is (A−B) × １ ／ (m).

The scale 1 / n refers to the lateral direction (horizontal direction) of the driver's eyes with respect to an obstacle existing in front of the host vehicle.
Of the driver's eyes with respect to the obstacle displayed on the display device 9A in the lateral direction (horizontal direction), and is a value obtained by an experiment in advance.

When displaying the map image at the set scale, the display control device 1 enlarges or reduces the entire infrared image in accordance with the scale (if both images have the same scale, they remain the same). It shifts in the horizontal direction by the shift amount calculated as described above, extracts a partial image of the preceding vehicle from the infrared image, and displays the partial image overlaid on the map image. Thus, when the partial image of the preceding vehicle is superimposed on the map image as it is, the display screen as shown in FIG. 8C is changed according to the driver's view from the driver's seat as shown in FIG. Display screen.

The correspondence between the angle of view of the infrared camera 2 and the detection range of the front obstacle sensor 4 is stored in a memory in advance, so that the partial image corresponding to the vehicle ahead can be extracted by using the front obstacle. Based on the detection result (distance and position with respect to the own vehicle) of the sensor 4 and the corresponding relationship, it can be easily extracted by detecting from the entire infrared image.
Further, the temperature characteristics of a general vehicle as a heat source may be stored in a memory in advance and referred to when extracting a partial image.

<Case where the Traveling Road is a Multiple Lane Road> Next, the road on which the own vehicle travels is basically a multiple lane automobile exclusive road where no human exists on the traveling road, A case where the entire infrared image is superimposed on the image and displayed will be described. As a display mode in this case, by referring to the map information based on the detected current position, the lane in which the host vehicle is traveling is detected from a plurality of lanes, and the preceding vehicle traveling in the same lane as the host vehicle is detected. It is preferable to perform the same display as that in FIG. 8D in which the traveling road is a one-lane road, with the center of the detected lane as a reference, based on the infrared image of the detected lane.

As a more specific operation, the shift amount of the infrared image is set based on the offset amount B and the respective scales of the infrared image and the map image. For example, when the scale of both images is １ ／ and the offset amount of the infrared camera 3 is B (m) to the right as described above, the shift amount of the infrared image is B × １ ／ ( m).

When displaying the map image at the set scale, the display control device 1 enlarges or reduces the entire infrared image in accordance with the scale (if both images have the same scale, they remain the same). The infrared image is shifted in the horizontal direction by the shift amount calculated as described above, and is displayed on the map image in a superimposed manner. May be superimposed on the map image).

In either case of one lane or a plurality of lanes, if the own vehicle is largely deviated from the center of the traveling lane, the infrared image (or a partial image thereof) is shifted as described above. When the vehicle is superimposed and displayed on the position where the host vehicle actually exists, the difference between the display screen and the actual field of view of the driver is large, and it is expected that the driver will feel uncomfortable.

Therefore, the lateral position (deviation amount) of the vehicle with respect to the traveling lane is detected, and when it is determined that the detected lateral position is more than a predetermined amount away from the center of the traveling lane, the deviation amount is determined. However, it is preferable to shift the display position of the obstacle (vehicle ahead) displayed on the map image or to shift the map image. For example, if the detected horizontal position is 1
If it is meters, the display position of the obstacle is shifted to the right by (1 × scale) meters, or almost right in the center of the display screen from the center of the driving lane (1 × scale).
What is necessary is just to display the line deviated by meters.

Incidentally, the method of detecting the lateral position with respect to the traveling lane is currently common, and the description thereof will be omitted.

<Display Control Process> Next, the procedure of the display control process performed by the display control device 1 to realize the display mode in the case of one lane or a plurality of lanes will be described.

FIGS. 2 and 3 are flowcharts showing the display control processing by the display control device 1 according to the first embodiment.

In the figure, step S1: input of reception data of the road-vehicle communication device 12, detection results of various sensors and operation states of various operation switches are input.

Step S2: The operating state of the light switch 2 is detected, and when the switch is turned on, the process proceeds to step S3, and when the switch is turned off, the process proceeds to step S3.

Step S3: As an obstacle (heat source) such as a person to be avoided, a partial image area whose luminance is larger than the threshold value A in the infrared image is detected, and the infrared image of the large partial image area is detected. Is calculated as the ratio of the size (area) to the entire region.

Here, as the threshold value A, a value at which the visibility of the map image for navigation is deteriorated is determined in advance by an experiment or the like, and the value is set.

In calculating the area ratio, the weight of the peripheral portion of the image may be set smaller than that of the central portion. For example, the entire area of the infrared image is divided into a central portion and a peripheral portion, and a value obtained by multiplying the area of the peripheral high luminance portion by a coefficient of about 0.3 and the area of the central high luminance portion are added. What is necessary is just to compare the obtained high-luminance part with the whole area of the infrared image.

Steps S4 and S5: When the calculated area ratio is larger than 70%, it is determined whether or not the driving environment is extremely poor in visibility. The determination in step S5 may be made based on the setting state of the wiper switch 11, whether the fog lamp is set to the lighting state by the light switch 2, or the like, or the information may be directly obtained by the road-vehicle communication device 12. And step S
When the area ratio is equal to or less than 70% in step 4, it can be determined that the distance to the obstacle in front is not short enough to make the map image difficult to see. Also, the infrared image should be displayed at this time, so that the process proceeds to step S7.

Step S6: Although it is not determined that the visibility is extremely poor in the determination in step S5, it is determined in step S2 that the driving environment is not good in visibility. To facilitate recognition,
Only the infrared image is displayed on the display device 9A as the first display.

Step S7: In the preceding step, the driving environment with good visibility, the driving environment where the distance to the obstacle in front is not short enough to make the map image difficult to see even if the visibility is not good, or the visibility is low. Since it is determined that the vehicle is in one of the extremely bad driving environments, in this step, the map image by the navigation function and the infrared image are displayed on the display device 9A, which is the first display, as described above. (Or a partial image thereof) is superimposed and displayed.

Step S8: The operation state of the navigation main switch 6 is detected, and if the switch is turned on in accordance with the operation state of the switch, step S8 is performed.
The routine proceeds to 9 and returns when the switch is operated in the off state.

Step S9: Based on the current position information detected by the GPS sensor 5, map image information near the current position is read from the map database 10, and a map image corresponding to the map image information is displayed on the second display. In addition to displaying on a certain display device 9B, a symbol representing the current position is displayed on the map image.

Step S10: The destination is set by the driver by detecting the operation state of the destination setting switch 8, and it is determined whether or not route guidance to the destination is being performed. If so, the process proceeds to step S11; otherwise, the process returns.

Step S11: In the route guidance to the set destination, it is determined whether or not the distance to a point where a lane change or a right or left turn is to be made is shorter than a predetermined distance and it is time to display guidance. When the guidance display is required, the process proceeds to step S12, and otherwise, returns.

Step S12: It is determined whether the light switch 2 is turned on and the map image and the infrared image are superimposed and displayed on the display device 9A. ), The process proceeds to step S13, and if NO (the switch is off), the process proceeds to step S17.

Steps S13 to S15: Based on whether or not the calculated area ratio of the partial image area is greater than 50%, the judgment is made in the same procedure as the above-described steps S3 to S5. If the area ratio is equal to or less than 50% in step S14, it can be determined that the distance to the obstacle in front is not short enough to make the map image difficult to see, so the process proceeds to step S16 and proceeds to step S1.
When it is determined that the visibility is extremely poor in the judgment of 5,
Since an infrared image should be displayed, step S1
Proceed to 6. Then, in step S14, the area ratio is 50
%, And it is not determined in step S15 that the driving environment is extremely poor visibility, so that the display of the infrared image is restricted to facilitate the recognition of the route guidance road by the driver. Step S1 to display only
Go to 7.

Here, the determination criterion is set to 50% in this step because the route guidance is being performed, and the visibility of the map image on the display is determined in step S4.
Because it is more important than

Step S16: The navigation guidance arrow is further superimposed and displayed on the screen displayed on the display device 9A (the display screen on which the map image and the infrared image are superimposed).

Step S17: A map image representing the road ahead is displayed on the display device 9A, and an arrow for route guidance is displayed on the map image.

According to the above-described embodiment, the infrared image of the front obstacle is superimposed and displayed at a position in front of the traveling road of the host vehicle, so that the driver can move forward even in an environment where visibility is poor for the driver. It is possible to improve the recognizability of the type and size of the existing obstacle and the location (relationship with the traveling road), and it is possible to efficiently support the driving operation of the driver.

During route guidance, when the relative distance from the obstacle in front becomes short, the superimposed display is restricted (in the present embodiment, the display of the infrared image is prohibited). It is possible to ensure the driver's recognizability of the map image (and the arrow for the route guidance in the map image). Even in such a case, even in the case of an obstacle existing at a short distance (for example, about one meter to several meters ahead), it is extremely difficult to visually recognize the obstacle such as heavy fog or heavy rain, which cannot be easily recognized. In the situation where the performance is poor, the superimposed display is continued, so that the driving operation of the driver can be efficiently supported.

Further, the two display devices can be properly used in combination with the display control processing shown in FIGS. 2 and 3, so that the map image and the It is possible to prevent the driver from feeling uncomfortable (complexity) due to the infrared image being displayed on the display device 9A from the center of the field of view.

In the above-described embodiment, step S
If YES in step 15 (the visibility is extremely poor), the map image and the infrared image are displayed in two display areas (for example, left and right) in the display area of the display device 9A instead of performing the superimposed display in step S16. ) May be displayed separately.

[Second Embodiment] Next, a second embodiment based on the above-described vehicle display device according to the first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

In the display control processing of FIGS. 2 and 3, when the display condition of the infrared image (the light switch 2 is turned on) is satisfied, the map image and the infrared image (or a partial image thereof) are always displayed. Although superimposed and displayed, such a display screen, when superimposing the entire infrared image on the map image,
It is also expected that the contours of the objects in the display screen are not clear, such as overlapping, and this gives a burden to the driver's vision.

Therefore, in the present embodiment, even if such display conditions are satisfied, when no obstacle is detected, only a map image corresponding to the current position is displayed.
Only when an obstacle is detected, as in the first embodiment, the map image and the infrared image are displayed in a superimposed manner.

In this embodiment, the method of calculating the shift amount of the infrared image in the case of one lane or a plurality of lanes is the same, and redundant description will be omitted.

FIGS. 4 and 5 are flowcharts showing a display control process by the display control device 1 according to the second embodiment.

In the figure, the flowchart of FIG. 2 and FIG. 3 in the first embodiment is different from the flowchart of FIG. Then, a map image corresponding to the current position is displayed on the display device 9A as the first display. Then, in step S24, it is determined whether or not a heat source representing the vehicle ahead is included in the infrared image of the infrared camera 3 in the infrared image. The difference is that the determination is made based on whether or not the area is included, and only when the preceding vehicle is detected in this determination, the process proceeds to the processing after step S25.

The processing of each step after step S30 is the same as the processing of each step after step S8 in FIG.

According to the present embodiment, as in the first embodiment, it is possible to improve the recognizability of the type and size of the obstacle located ahead and the location of the obstacle (relation with the traveling road). In addition to being able to efficiently assist the driver's driving operation, when there is no obstacle, only a map image is displayed, and a display screen where a map image and an infrared image are superimposed So you can reduce your chances,
The burden on the driver's vision can be reduced.

During route guidance, when the relative distance from the obstacle in front becomes short, the superimposed display is restricted (in the present embodiment, the display of the infrared image is prohibited). It is possible to ensure the driver's recognizability of the map image (and the arrow for the route guidance in the map image). Even in such a case, even in the case of an obstacle existing at a short distance (for example, about one meter to several meters ahead), it is extremely difficult to visually recognize the obstacle such as heavy fog or heavy rain, which cannot be easily recognized. In the situation where the performance is poor, the superimposed display is continued, so that the driving operation of the driver can be efficiently supported.

[Third Embodiment] Next, a third embodiment based on the vehicle display device according to the above-described first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

In the first embodiment described above, the configuration in which the display of the map image is taken as a basis and the infrared image is superimposed on the map image being displayed has been described. When displaying the infrared image and displaying the infrared image, when the user arrives at a point (near the point) where the route should be guided by the navigation function, an arrow symbol for the course guidance is displayed on the infrared image. Are superimposed and displayed.

Here, a method of superposing the infrared image and the symbol of the arrow in this embodiment will be described.

In this embodiment, the infrared camera 3
Is B rightward from the center axis extending in the front-rear direction of the vehicle.
(M) Since it is mounted at an offset position, when the vehicle is running substantially in the center of the driving lane,
The center line of the infrared image is shifted from the center line of the traveling lane by the offset amount B (m) of the infrared camera 3.

The route guidance by the general navigation function is performed by scrolling the map image according to the change of the current position so that the center line of the traveling lane of the own vehicle is arranged on the center line of the display screen. The guidance guidance arrow is displayed such that the starting point of the arrow is located on the center line of the traveling lane of the vehicle.

Therefore, in order to display an arrow for route guidance on the center line of the traveling lane of the own vehicle in the infrared image, it is necessary to shift the symbol of the arrow by the offset amount of the infrared camera 3 and then superimpose it. There is.

That is, when the offset amount of the arrangement position of the infrared camera 3 is B (m) on the right side, a value obtained by multiplying the value by the scale between the infrared image and the map image is used for the route guidance. It becomes the amount of the arrow. For example, when the scale of the infrared image is 1/2 and the scale of the map image is 1/3, the shift amount of the arrow for the route guidance is C × 1.
/ 2 × 3/2 (m).

<Display Control Process> Next, the procedure of the display control process performed by the display control device 1 in the present embodiment to superimpose and display the navigation guidance arrow on the infrared image as described above will be described. I do.

FIG. 6 is a flowchart showing a display control process by the display control device 1 according to the third embodiment.

In the figure, step S41: As in step S1 of FIG. 2, the detection results of various sensors are input.

Steps S42 and S43: The operation state of the light switch 2 is detected (step S42). When the switch is operated to be turned on, in step S43, the display device 9A, which is the first display, is displayed.
Next, an infrared image is displayed. On the other hand, when the switch has been turned off, the process proceeds to step S44.

Step S44: The operation state of the navigation main switch 6 is detected. According to the operation state of the switch, the operation proceeds to step S45 when the switch is turned on, and returns when the switch is turned off.

Step S45: Based on the current position information detected by the GPS sensor 5, map image information near the current position is read from the map database 10, and a map image corresponding to the map image information is displayed on the second display. In addition to displaying on a certain display device 9B, a symbol representing the current position is displayed on the map image.

Step S46: By detecting the operation state of the destination setting switch 8, it is determined whether the driver has set the destination and the route guidance to the destination is being performed. If so, proceed to step S47, otherwise return.

Step S47: In the route guidance to the set destination, it is determined whether the distance to a point where a lane change or a right or left turn is to be made is shorter than a predetermined distance and it is time to display guidance, and this determination is made. When the guidance display is required, the process proceeds to step S48, and otherwise, returns.

Steps S48 to S53: Steps S12 to S in FIG. 3 in the first embodiment
By performing the same processing as in each step of step 17, an arrow for the course guidance whose display position has been adjusted (shifted) as described above is superimposed on the infrared image being displayed on the display device 9A (step). S52) Alternatively, a map image representing the road ahead is displayed on the display device 9A, and an arrow for route guidance is displayed on the map image (step S52).
53).

According to the above-described embodiment, the infrared image of the obstacle in front is superimposed and displayed at the position in front of the traveling path of the host vehicle, so that the driver can stay ahead even in an environment where the driver has low visibility. It is possible to improve the recognizability of the type and size of the obstacle and the location of the obstacle (the relationship with the traveling path), and to efficiently support the driver's driving operation.

In addition, similar to the first and second embodiments, the superposition display during the route guidance is restricted, so that the driver can recognize the map image for navigation (and the arrow for the route guidance in the map image) by the driver. And the visibility in the forward direction under extremely poor visibility such as in stormy weather such as dense fog or heavy rain can be effectively achieved.

[Fourth Embodiment] Next, a fourth embodiment based on the vehicle display device according to the above-described third embodiment will be described. In the following description, the same configuration as that of the third embodiment will not be described repeatedly, and will be described focusing on the characteristic portions of the present embodiment.

In the third embodiment described above, the arrow symbol is displayed when the route guidance arrow display condition is satisfied when the route guidance is performed during the display of the infrared image. In the embodiment, the arrow is superimposed on the infrared image only when an obstacle is detected based on the infrared image.

In the present embodiment, the method of calculating the shift amount of the arrow for the course guidance described above is the same, and redundant description will be omitted.

FIG. 7 is a flowchart showing a display control process by the display control device 1 according to the fourth embodiment.

In the figure, the flowchart of FIG. 6 in the third embodiment is different from the flowchart of FIG. 6 in that when the operation state of the light switch 2 is determined to be ON in step S68, first, in step S69,
It is determined whether the infrared image from the infrared camera 3 includes a radiant heat source representing an obstacle such as a pedestrian or a vehicle ahead,
Only when an obstacle is detected in this determination, the step S
If the infrared image does not include any obstacle, the process proceeds to step S74 to display the display device 9A.
A map image representing the road ahead is displayed on the map, and an arrow for route guidance is displayed on the map image.

The processing of each step before step S68 is the same as the processing of each step before step S48 in FIG.

According to this embodiment, as in the third embodiment, it is possible to improve the recognizability of the type and size of the obstacle existing ahead and the location (relationship with the traveling road) of the obstacle. In addition to being able to efficiently support the driver's driving operation, when there is no obstacle, only an arrow for route guidance is displayed on the map image, and an infrared image and the arrow for route guidance are displayed. Since the chances of displaying the display screen on which the arrow is superimposed can be reduced, the driver's judgment can be facilitated.

In addition, by restricting the superimposed display during the route guidance as in the first to third embodiments, the driver can recognize the map image for navigation (and the arrow for the route guidance in the map image). And the visibility in the forward direction under extremely poor visibility such as in stormy weather such as dense fog or heavy rain can be effectively achieved.

[Fifth Embodiment] The first to fourth embodiments described above
In the embodiment, the positional relationship between the obstacle in front and the host vehicle is determined based on the area ratio in the captured infrared image. However, in this embodiment, the detection result of the front obstacle sensor 4 is used. In the following description, the same configuration as in the first to fourth embodiments will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

FIGS. 9 and 10 are flowcharts showing the display control processing by the display control device 1 according to the fifth embodiment, and are the first to fourth embodiments (FIGS. 2 to 7).
It is the flowchart which extracted the part different from. Specifically, Step S in the first embodiment (FIGS. 2 and 3)
Steps S3 to S6 are replaced with the flowchart shown in FIG. 9, and steps S13 to S15 are replaced with those in FIG.
An embodiment replaced by the flowchart shown in FIG. 7; and Step S in the second embodiment (FIGS. 4 and 5).
An embodiment in which steps 25 to S28 are replaced by the flowchart shown in FIG. 9 and steps S35 to S37 are replaced by the flowchart shown in FIG. 10; and steps S49 to S51 in the third embodiment (FIG. 6). It is assumed that the embodiment is replaced with the flowchart shown in FIG. 10 and that the steps S70 to S72 in the fourth embodiment (FIG. 7) are replaced with the flowchart shown in FIG.

The steps in the flowcharts of FIGS. 9 and 10 will be described below.

Step S1001 in FIG. 9: Based on the detection result of the front obstacle sensor 4, a relative distance to an obstacle present ahead is obtained. At this time, a correspondence relationship between the obstacle detected by the sensor and a partial image region which is likely to be an obstacle included in the infrared image is proposed in Japanese Patent Application No. 11-253309 and the like by the present applicant. It is good to adopt the method. Here, the method will be described.

FIG. 20 is a diagram showing a state in which the detection range of the front obstacle sensor 4 is viewed from above.
An obstacle A (another preceding vehicle) and an obstacle B corresponding to the image captured by the infrared camera 3 illustrated in FIG.
(Pedestrian) is included.

In this embodiment, the front obstacle sensor 4
Of the obstacle A and the obstacle B according to the distance Da of the own vehicle and the obstacle A and the angle θa (≒
0), and the distance Db between the host vehicle and the obstacle B and the angle θb with the center axis.

Therefore, if the positional relationship between the center axis in the detection range of the front obstacle sensor 4 and the coordinate axis of the imaging plane (angle of view) of the infrared camera 3 is made to correspond in advance, the output from the front obstacle sensor 4 is obtained. D to the obstacle and angle θ
On the imaging surface, as shown in FIG.
Is counted as the number of lines (the number of pixels) from the lower side of the imaging surface, and the angle θ is counted as the number of pixels from the left and right center lines of the imaging surface, whereby the angle within the imaging surface specified by the count is determined. It can be determined that the radiant heat source including the position (pixel) corresponds to the entire shape of the obstacle detected by the front obstacle sensor 4. Therefore, if such processing is performed in step S1001, an obstacle included in the image captured by the infrared camera 3 can be associated with the distance D to the obstacle.

Step S1002, Step S100
3: When the distance to the obstacle detected by the front obstacle sensor 4 and associated with the partial image area included in the infrared image is shorter than 1 meter, the visibility is low as in step S5 in FIG. It is determined whether the driving environment is extremely bad. When the distance to the obstacle is 1 m or more in step S1002, it can be determined that the distance to the obstacle in front is not short enough to make the map image difficult to see, so the infrared image and the map image are displayed in a superimposed manner. When the visibility is determined to be extremely poor in step S1003, the infrared image should be displayed, and the process proceeds to the superimposed display processing step.

Step S1004: Step S1003
It is not determined that the visibility is extremely poor in the determination of. Therefore, in order to give priority to the display of the map image for navigation, only the map image by the navigation function is displayed on the display device 9A as the first display.

Next, in each of steps S1101 to S1103 in FIG. 10, 1.5 meters is used as a criterion for determination, and the above-described step S1001 in FIG.
The processing is performed in the same procedure as in steps S1003 to S1003. In this case, the criterion is set to 1.5 meters, and step S1 is performed.
The reason why the value larger than the criterion of 002 (1 meter) is set in this step is that the route guidance is being performed, so that the visibility of the map image on the display is more important than in step S1002. by.

By adopting the processing of FIGS. 9 and 10 described above, even if display control is performed according to the distance to the obstacle detected by the front obstacle sensor 4, the same as in the first to fourth embodiments is performed. Effects can be enjoyed.

[Sixth Embodiment] Next, a sixth embodiment based on the above-described vehicle display device according to the first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

FIGS. 11 and 12 are flowcharts showing a display control process by the display control device 1 according to the sixth embodiment.

In the figure, the flowcharts of FIGS. 2 and 3 in the first embodiment are different from the flowcharts of FIG. 2 and FIG. 3 in that the display step of the map image in step S6 (FIG. 2) is replaced by step S106, and that in step S15 (FIG. Step S
(Corresponding to step 115) is different in that step S116, which is performed when visibility is extremely poor, is added.

That is, in the present embodiment, step S106
In this example, the infrared image and the map image are superimposed and displayed on the display device 9A as the first display. At this time, in order to improve the recognizability of the map image by the driver, the brightness of the map image is reduced by the infrared. The image is displayed with higher brightness (higher contrast) than in a normal case where no image is displayed. In step S116, the infrared image and the map image are superimposed and displayed on the display device 9A serving as the first display device. At this time, the recognizability of the map image and the arrow for the route guidance by the driver is improved. Therefore, the brightness of the arrow is displayed with a higher brightness (higher contrast) as compared with the normal case where the infrared image is not displayed. Other processing procedures are the same as in the first embodiment, and a description thereof will be omitted.

Note that instead of increasing the brightness of the map image,
The two images may be superimposed while the luminance of the infrared image is low.

According to this embodiment, the same effects as those of the first embodiment can be obtained.

[Seventh Embodiment] Next, a seventh embodiment based on the vehicular display device according to the second embodiment will be described. In the following description, the same configuration as that of the second embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

FIGS. 13 and 14 are flowcharts showing the display control processing by the display control device 1 according to the seventh embodiment.

In the figure, the flowcharts of FIGS. 4 and 5 in the second embodiment are different from the flowcharts of FIGS. 4 and 5 in that the display step of the map image in step S28 (FIG. 4) is replaced by step S128, and that in step S37 (FIG. 14). Step S
(Corresponding to 137) is different in that step S138 performed when visibility is extremely poor is added.
The processing performed in step S128 and step S138 is the same as the processing in step S128 in the above-described sixth embodiment.
This is the same as 106 and step S116. Other processing procedures are the same as in the second embodiment, and a description thereof will not be repeated.

According to this embodiment, the same effects as those of the second embodiment can be obtained.

[Eighth Embodiment] Next, an eighth embodiment based on the vehicle display device according to the third embodiment will be described. In the following description, the same configuration as that of the third embodiment will not be described repeatedly, and will be described focusing on the characteristic portions of the present embodiment.

FIG. 15 is a flowchart showing a display control process by the display control device 1 according to the eighth embodiment.

In the figure, the flowchart of FIG. 6 in the third embodiment is different from the flowchart of FIG. 6 in that step S162, which is performed when the visibility is extremely poor in step S51 (corresponding to step S161 in FIG. 15), is added. Are different. The processing performed in step S162 is the same as step S116 in the above-described sixth embodiment. Other processing procedures are the same as in the second embodiment, and a description thereof will not be repeated.

According to this embodiment, the same effects as those of the third embodiment can be obtained.

[Ninth Embodiment] Next, a ninth embodiment based on the above-described vehicle display device according to the fourth embodiment will be described. In the following description, the same configuration as that of the fourth embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

FIG. 16 is a flowchart showing a display control process by the display control device 1 in the ninth embodiment.

In the figure, the flowchart of FIG. 7 of the fourth embodiment is different from the flowchart of FIG. 7 in that a step S183 is performed when the visibility is extremely poor in the determination of the step S72 (corresponding to the step S182 in FIG. 16). Are different. The processing performed in step S183 is the same as step S116 in the above-described sixth embodiment. The other processing procedures are the same as in the fourth embodiment, and a description thereof will not be repeated.

According to this embodiment, the same effects as those of the fourth embodiment can be obtained.

[Tenth Embodiment] In the above-described sixth to ninth embodiments, the positional relationship between the obstacle ahead and the host vehicle is determined based on the area ratio in the captured infrared image. In the present embodiment, the detection result of the front obstacle sensor 4 is used. In the following description, the same components as those described above will not be described repeatedly, and will be described focusing on the characteristic portions of the present embodiment.

FIGS. 17 and 18 are flowcharts showing the display control processing by the display control device 1 according to the tenth embodiment, and the portions different from the sixth to ninth embodiments (FIGS. 11 to 16) are extracted. It is the flowchart which was performed. Specifically, the sixth embodiment (FIGS. 11 and 1)
An embodiment in which steps S103 to S106 in 2) are replaced by the flowchart shown in FIG. 17 and steps S113 to S116 are replaced by the flowchart shown in FIG. 18, and a seventh embodiment (FIGS. 13 and 14) ) Are replaced with the flowchart shown in FIG.
An embodiment in which steps S135 to S138 are replaced by the flowchart shown in FIG. 18; and an embodiment in which steps S159 to S162 in the eighth embodiment (FIG. 15) are replaced by the flowchart shown in FIG. -It is assumed that steps S180 to S183 in the ninth embodiment (Fig. 16) are replaced with the flowchart shown in Fig. 18.

The flowcharts of FIGS. 17 and 18 correspond to FIGS. 9 and 10 described in the fifth embodiment.
The processing procedure is substantially the same as that of the flowchart of FIG. 7, except that the display step of the map image in step S1004 is replaced by step S2004,
3 (corresponding to step S2103 in FIG. 18) is different in that step S2104 performed when visibility is extremely poor is added. Then, step S200
4 and step S2104 are performed in step S106 and step S2 in the above-described sixth embodiment.
Same as 116.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a system configuration of a display device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating a display control process performed by the display control device 1 according to the first embodiment.

FIG. 3 is a flowchart illustrating a display control process performed by the display control device 1 according to the first embodiment.

FIG. 4 is a flowchart illustrating a display control process performed by a display control device 1 according to a second embodiment.

FIG. 5 is a flowchart illustrating a display control process performed by a display control device 1 according to a second embodiment.

FIG. 6 is a flowchart illustrating a display control process performed by a display control device 1 according to a third embodiment.

FIG. 7 is a flowchart illustrating a display control process performed by a display control device 1 according to a fourth embodiment.

FIG. 8 is a diagram illustrating a display example of a map image or the like on a display device.

FIG. 9 is a flowchart in which a portion different from the first to fourth embodiments (FIGS. 2 to 7) is extracted from the flowchart showing the display control processing by the display control device 1 in the fifth embodiment.

FIG. 10 is a flowchart in which a portion different from the first to fourth embodiments (FIGS. 2 to 7) is extracted from the flowchart showing the display control processing by the display control device 1 in the fifth embodiment.

FIG. 11 is a flowchart illustrating a display control process performed by a display control device 1 according to a sixth embodiment.

FIG. 12 is a flowchart illustrating a display control process performed by a display control device 1 according to a sixth embodiment.

FIG. 13 is a flowchart illustrating a display control process performed by a display control device 1 according to a seventh embodiment.

FIG. 14 is a flowchart illustrating a display control process performed by a display control device 1 according to a seventh embodiment.

FIG. 15 is a flowchart illustrating a display control process by a display control device 1 according to an eighth embodiment.

FIG. 16 is a flowchart illustrating a display control process performed by a display control device 1 according to a ninth embodiment.

FIG. 17 is a flowchart in which a portion different from the sixth to ninth embodiments (FIGS. 11 to 16) is extracted from the flowchart showing the display control processing by the display control device 1 in the tenth embodiment.

FIG. 18 is a flowchart in which a portion different from the sixth to ninth embodiments (FIGS. 11 to 16) is extracted from the flowchart showing the display control processing by the display control device 1 according to the tenth embodiment.

FIG. 19 is a diagram illustrating an image captured by an infrared camera (in the case of normal display).

FIG. 20 is a diagram showing a state where the detection range of the front obstacle sensor 4 is viewed from above.

[Explanation of symbols]

 1: display control device, 2: light switch, 3: infrared camera, 4: forward obstacle sensor, 5: GPS sensor, 6: navigation main switch, 7: map scroll switch, 8: destination setting switch, 9A, 9B : Display device, 10: map database, 11: wiper switch, 12: road-to-vehicle communication device,

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G09B 29/10 G09B 29/10 A (72) Inventor Kenichi Okuda 3-1 Shinchi, Fuchu-cho, Aki-gun, Hiroshima Prefecture Mazda Co., Ltd. (72) Inventor Hiroki Uemura 3-1, Fuchu-machi, Shinchi, Aki-gun, Hiroshima Mazda Co., Ltd. (72) Inventor Hidekazu Sasaki 3-1, Fuchu-cho, Shinchi, Aki-gun, Hiroshima Mazda F-term (Ref.)

Claims (8)

[Claims] An image pickup apparatus provided in a vehicle and taking an image of the front of the vehicle using infrared rays, a three-dimensional map image generated based on map image information, and an infrared image taken by the image pickup apparatus. And a display control means for displaying on a display provided in the vehicle in a display mode in which the map image is superimposed on a display screen, the display control means comprising: On the other hand, when the ratio of the partial image area corresponding to the obstacle existing ahead included in the infrared image becomes larger than a predetermined ratio, the display in the display mode is regulated. Vehicle display device. 2. The vehicle display device according to claim 1, wherein the display control unit prohibits display of the infrared image so as to regulate the display mode. 3. The display control means displays the map image and the infrared image separately in a display area of the display device in order to regulate the display mode. 2. The vehicle display device according to 1. 4. An estimating means for estimating the visibility in front of the vehicle, wherein when the estimating means determines that the visibility is extremely poor, the display control means controls the entire area of the map image. 2. The display device for a vehicle according to claim 1, wherein the display in the display mode is continued regardless of a ratio occupied by the partial image area. 5. The display control device according to claim 1, wherein the partial image area corresponding to the obstacle included in the infrared image is determined based on a luminance representing a heat radiation amount of the obstacle. The display device for a vehicle as described in the above. 6. An obstacle detecting means for detecting a relative distance and a position to an obstacle present ahead, wherein the display control means includes a portion corresponding to an obstacle included in the infrared image. The image area is determined based on a result of detection of an obstacle by the obstacle detection unit and information on a preset arrangement relationship between the imaging device and the obstacle detection unit on the vehicle. The vehicle display device according to claim 1. 7. An image pickup device provided in a vehicle and taking an image of the front of the vehicle using infrared light, a three-dimensional map image generated based on map image information, and an infrared image picked up by the image pickup device. And a display control means for displaying on a display provided in the vehicle in a display mode in which the map image is superimposed on a display screen, the display control means comprising: On the other hand, when the ratio of the partial image area included in the infrared image and corresponding to the obstacle existing ahead occupies is larger than a predetermined ratio, the map image is highlighted. Vehicle display device. 8. An image pickup device provided in a vehicle, for picking up an image of the front of the vehicle using infrared rays, an obstacle detecting means for detecting a relative distance and a position of an obstacle present ahead, and a map image. Display control means for displaying a three-dimensional map image generated based on information and an infrared image picked up by the image pickup device on a display provided in the vehicle in a display mode in which the three-dimensional map image is superimposed on a display screen A display device for a vehicle, comprising: a display control unit configured to display a correspondence relationship between the obstacle and a partial image area corresponding to the obstacle included in the infrared image by the obstacle detection unit. Judgment is performed based on the detection result of the obstacle and information about a preset arrangement relationship of the imaging device and the obstacle detection unit on the vehicle, and a relative position between the detected obstacle and the obstacle is determined. Predetermined distance When it becomes shorter, the vehicle display apparatus characterized by regulating the display by the display mode.