JP2007043318A - Vehicle surrounding supervisory apparatus and vehicle surrounding supervisory method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle surrounding supervisory apparatus capable of displaying an object existing at a position higher than ground at a correct position on a bird's-eye image. <P>SOLUTION: The vehicle surrounding supervisory apparatus 1 is characterized in to include: a camera 2 for photographing a state around its own vehicle; a first conversion section 3 for converting a photographed image into a bird's-eye image whose height is a ground height; a sonar 4 for detecting an object existing around its own vehicle; a distance calculation section 5 for calculating a distance up to the detected object; a second conversion section 6 for converting the photographed image into a bird's-eye image at a height equal to that of the sonar 4 on the basis of the distance to the object; a composite section 7 for composing the bird's-eye image at a ground height with the bird's-eye image at a height of the sonar 4; and a monitor 8 for displaying a composed image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a vehicle surrounding monitoring apparatus that monitors the surroundings of a host vehicle, and more particularly to a vehicle surrounding monitoring apparatus and method for displaying an object present at a high position from the ground at a correct position on an overhead image.

  Conventionally, the periphery of a vehicle has been photographed with a camera, and this image has been converted into a bird's-eye view and presented to the driver. However, if an image captured by the camera is converted into a bird's-eye view image, an object that exists in a state of floating from the ground like a bumper of a vehicle is displayed at a position far from the actual position.

  Therefore, conventionally, a sensor is installed to estimate the outer edge of the object, and the outer edge of the object is displayed in a bird's-eye view image so that the actual distance to the object is presented to the driver. As a conventional example of such a vehicle surrounding display device, for example, Japanese Unexamined Patent Application Publication No. 2004-354326 (Patent Document 1) is disclosed.

In this conventional example, the camera image is converted into a bird's-eye view image, and the distance measurement sensor repeatedly detects the distance to the object around the vehicle to estimate the outer edge of the object. It was displayed superimposed on the top.
JP 2004-354326 A

  However, in the conventional vehicle surrounding display device described above, since the information of the object detected by the distance measuring sensor is point information, it is necessary to move the vehicle in order to draw a continuous outer edge. Since information cannot be displayed, there has been a problem that the position of the object cannot be accurately displayed to the driver.

  Further, when the object is a moving object, a difference occurs in past position information with the passage of time, and the position of the object may not be accurately displayed to the driver.

  Therefore, in the conventional vehicle surrounding display device described above, the driver cannot accurately recognize the situation around the own vehicle.

  In order to solve the above-described problem, a vehicle surroundings monitoring apparatus according to the present invention converts a situation around the own vehicle into a bird's-eye view image and presents it to the driver to monitor the surroundings of the vehicle. An apparatus for photographing a situation around the own vehicle; first conversion means for converting an image photographed by the photographing means into an overhead image of a ground height; and Detection means for detecting an existing object, distance calculation means for calculating a distance to the object detected by the detection means, and an image photographed by the photographing means based on the distance to the object Second conversion means for converting to a bird's-eye view image of a predetermined height, composition means for synthesizing the bird's-eye view image of the ground height and the bird's-eye view image of the predetermined height, and an image synthesized by the composition means And display means for displaying.

  Further, the vehicle surroundings monitoring method according to the present invention is a vehicle surroundings monitoring method for monitoring the surroundings of the own vehicle by converting the surroundings of the own vehicle into a bird's-eye view image and presenting it to the driver. A shooting step for shooting the situation around the vehicle, a first conversion step for converting the image shot in the shooting step into a bird's-eye view image of the height of the ground, and detecting an object existing around the vehicle A detecting step for calculating, a distance calculating step for calculating a distance to the object detected in the detecting step, and an image captured in the imaging step based on the distance to the object. A second converting step for converting to a bird's-eye view image; a combining step for combining the overhead image with the height of the ground and the overhead image having the predetermined height; and a display step for displaying the image combined in the combining step. When Characterized in that it contains.

  In the vehicle surrounding monitoring apparatus and method according to the present invention, an image captured based on the detected distance to the object is converted into an overhead image of a predetermined height, and the overhead image of the predetermined height Since it is synthesized and displayed with the overhead image of the height of the ground, information such as white lines drawn on the ground can be displayed accurately, and objects that are located higher than the ground can be displayed accurately, As a result, the driver can accurately recognize the situation around the vehicle.

  Embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of a vehicle surrounding monitoring apparatus according to the present embodiment. As shown in FIG. 1, the vehicle surroundings monitoring device 1 of the present embodiment includes a photographing unit (photographing means) 2 for photographing a situation around the vehicle, and an image photographed by the photographing unit 2 at the height of the ground. A first conversion unit (first conversion unit) 3 that converts an overhead image, a detection unit (detection unit) 4 that detects an object existing around the vehicle, and an object detected by the detection unit 4 A distance calculation unit (distance calculation unit) 5 that calculates a distance to the second object, and a second conversion unit that converts an image photographed by the photographing unit 2 into an overhead image of a predetermined height based on the distance to the object. (Second conversion means) 6, a composition unit (composition means) 7 that composes the overhead image of the height of the ground and the overhead image of a predetermined height, and a display that displays the image synthesized by the composition unit 7 Part (display means) 8.

  Here, the photographing unit 2 is a photographing unit such as a CCD camera or the like installed in the rear or front of the own vehicle, and the photographed image is formed as a collection of pixels one by one. Yes.

  The first conversion unit 3 indicates where on the ground each captured pixel is based on the lens characteristics of the camera constituting the imaging unit 2, the angle of attachment of the camera to the ground, and the relative distance. The regularity (correspondence) indicating which position corresponds to the converted overhead image is stored, and the overhead image is created by performing image processing called rearrangement of each pixel by a built-in arithmetic circuit. In particular, the first conversion circuit 3 converts the image captured by the imaging unit 2 into an overhead view image of the height of the ground.

  The detection unit 4 is detection means for detecting an object existing at a predetermined height set in advance, and can be configured by, for example, an ultrasonic sonar. In the case of this ultrasonic sonar, an object is detected by taking out an electric signal by receiving a reflected signal which is reflected by the transmitted ultrasonic wave and returning to the object.

  The distance calculation unit 5 detects the presence of an object from the presence or absence of an electrical signal output from the detection unit 4, and obtains a relative distance (distance to the object) from a time difference from when an ultrasonic wave is transmitted to when it is received. ing.

  The second conversion unit 6 performs the same processing as the first conversion unit 3 to convert the image captured by the imaging unit 2 into an overhead image, but the second conversion unit 6 is set in advance. The image is converted into an overhead image having a predetermined height, for example, the height at which the detection unit 4 is attached. Therefore, the second conversion unit 6 stores regularity for converting the captured image into a bird's-eye view image of a predetermined height, and the presence of the target detected by the distance calculation unit 5 and the target A bird's-eye view image of the corresponding area is created based on the relative distance to the object.

  The composition unit 7 correctly determines the vertical relationship between the overhead image of the ground height created by the first converter 3 and the overhead image of the predetermined height created by the second converter 6. These are overlapped to create a single image by overlaying the overhead view image of a predetermined height on the overhead view image of the height of the ground.

  The display unit 8 is a display device such as a monitor installed in the passenger compartment of the host vehicle, and presents the image combined by the combining unit 7 to the driver with appropriate brightness and contrast.

  Next, the vehicle periphery monitoring process by the vehicle periphery monitoring device 1 of the present embodiment will be described based on the flowchart of FIG. From here, the photographing unit 2 will be described as a camera, the detection unit 4 will be described as an ultrasonic sonar, and the display unit 8 will be described as a monitor.

  As shown in FIG. 2, first, when a driver turns on a switch that starts monitoring the surroundings of the vehicle (S201), the camera 2 takes a picture (S202). Then, as shown in FIG. 3A, the photographed image is usually converted as an overhead image of the ground level (S203).

  Here, it is determined whether or not an object is detected around the own vehicle by the sonar 4 (S204). When no object is detected, an overhead image of the ground height is displayed on the monitor 8 (S205). ).

  Then, it is determined whether or not the switch for monitoring the surroundings of the vehicle is turned off (S206). If not turned off, the process returns to step S204 to detect the object. Here, when an object is detected around the own vehicle by the sonar 4, when the detected object is a vehicle or the like, it is converted into an overhead view image of the height of the ground. It will be displayed in the distance. For example, as shown in FIG. 4, when another vehicle 41 exists as an object around the own vehicle 40, the end D point of the vehicle 41 is actually at a distance B from the own vehicle 40. Nevertheless, the bird's-eye view image is displayed so as to be at a distance C from the own vehicle 40.

  In view of this, not only the overhead image of the ground level but also the overhead image having a predetermined height is generated and combined with the overhead image of the ground level. However, here, the predetermined height is the mounting height of the sonar 4, but may be set to other heights.

  First, the detection area by the sonar 4 is fixed (S207). Here, the detection range of the sonar 4 will be described with reference to FIG. The ultrasonic sonar 4 has a detection distance from a distance 0 to a distance where the reflected signal is weak and cannot be received. However, the detection distance can be arbitrarily set according to the intensity of the transmitted ultrasonic wave. And the ultrasonic wave transmitted from the sonar 4 is transmitted in a fan shape from the main body as shown in FIGS. 5 (a) and 5 (b), and the angle of view is a detection range. The horizontal direction is wide with respect to the ground, and the vertical direction is narrow.

  In such an ultrasonic detection range, detection areas are set according to the distance, and in FIG. 5A, three stages of detection areas 51 to 53 are set. However, more detection areas may be set according to the size of the detection range, or one detection area may be used.

  And based on the time difference from the transmission of the ultrasonic wave transmitted from the sonar 4 to the reception, the distance to the target is specified, and the detection area where the target exists is determined.

  When the detection area is determined in this way, the next photographed image is converted into an overhead view image of the sonar 4 height (S208). As this conversion method, as shown in FIG. 6, the depression angle between the camera 2 and the object 61 is θc, the height from the ground to the camera is Hc, the height of the sonar 4 is Ho, and the distance to the object 61 Is Lo, and the distance to the object in the overhead image of the ground height is Ld, and Ld−Lo = Lg, the image taken by the camera 2 using the relationship of Lg: Ld = Ho: Hc is the sonar 4 Convert to a bird's-eye view image at a height of.

  Then, the overhead image of the height of the sonar 4 and the overhead image of the height of the ground are synthesized (S209) and displayed on the monitor 8 (S205).

  Here, an example of the overhead view image displayed on the monitor 8 is shown in FIGS. As shown in FIG. 7A, in the overhead image after synthesis, the overhead image having the height of the sonar 4 is displayed surrounded by a dotted line in the overhead image having the height of the ground.

  This bird's-eye view image is a diagram showing a case where a vehicle as a target object has entered a detection area 51 farthest from the sonar 4 among the detection areas set in three stages in FIG. A region surrounded by a dotted line in FIG. 7A corresponds to the detection area 51. Then, the image of the detection area 51 is converted into a bird's-eye view image having the height of the sonar 4, and is synthesized and displayed on the bird's-eye view image having the height of the ground. Accordingly, when the synthesized bird's-eye view image is converted into a side view, the side view is as shown in FIG. An image is displayed.

  Here, in FIG. 7 (a), the overhead view image of the inspection area is set to a rectangle, but this ensures the driver's visibility when the overhead view image of the inspection area is combined with the overhead view image of the ground height. In order to accurately correspond to the detection range of the sonar 4, it may be set in a sector shape. Further, if the width of the detection area is narrowed according to the reliability of the sonar (the certainty that the object is present in the detection range), the required minimum composition is achieved.

  When the combined image in step S205 is displayed on the monitor 8, it is determined whether or not the switch for monitoring the surroundings of the vehicle is turned off (S206). If not, the process returns to step S204. The above-described processing is repeated, and if it is OFF, the vehicle surrounding monitoring process by the vehicle surrounding monitoring device 1 of the present embodiment is terminated.

  As described above, in the vehicle surrounding monitoring apparatus 1 according to the present embodiment, an image captured based on the detected distance to the object is converted into an overhead view image having a predetermined height, and the overhead view having the predetermined height is displayed. Since the image and the overhead image of the height of the ground are combined and displayed, the ground image can be accurately displayed and the object existing at a position higher than the ground can also be accurately displayed. Can accurately recognize the situation around the vehicle.

  Moreover, in the vehicle surrounding monitoring apparatus 1 of this embodiment, the detection range of the sonar 4 is divided into a plurality of detection areas 51 to 53, and an overhead image with a predetermined height is generated for each detection area. An existing detection area can be identified and presented to the driver, and the driver can recognize the position of the object more accurately.

  Furthermore, in the vehicle surrounding monitoring device 1 of the present embodiment, only the detection area where the object exists is converted into an overhead image of a predetermined height and displayed, so the detection area where the object exists is narrowed down and the driver is selected. The driver can quickly recognize the position of the object.

<Second Embodiment>
Next, a second embodiment of the present invention will be described with reference to the drawings. In the vehicle surrounding monitoring apparatus of the present embodiment, when the image taken by the camera 2 is converted into an overhead image with the height of the sonar 4, only the detection area portion is converted into an overhead image in the first embodiment. In contrast, in the present embodiment, the entire range in the direction away from the detection area is converted into an overhead image.

  Therefore, when the overhead image of the height of the sonar 4 converted by this method and the overhead image of the height of the ground are combined, an overhead image as shown in FIG. 9A is obtained, as shown in FIG. 9B. The objects are all displayed in a direction away from the distance of point B. Therefore, when this synthesized bird's-eye view image is converted into a side view, the side view as shown in FIG. 10 is obtained, and all the directions away from the detection area are the bird's-eye view image having the height of the sonar 4, and the front side from the detection area. This is a bird's-eye view of the ground level.

  As a result, in the first embodiment, the same object may be displayed in two places as shown in FIG. 7A, and the driver may misunderstand which is the real position. On the other hand, in the present embodiment, the end D of the object can be accurately displayed and the object can be displayed as a single unit, so that the driver can reliably grasp the object.

  As described above, in the vehicle surrounding monitoring apparatus according to the present embodiment, the entire range in the direction away from the detection area where the object exists is converted into an overhead image with a predetermined height, so that an easy-to-see overhead image is presented to the driver. The driver can surely grasp the object.

<Third Embodiment>
Next, a third embodiment of the present invention will be described with reference to the drawings. In the vehicle surrounding monitoring apparatus according to the present embodiment, when the image captured by the camera 2 is converted into an overhead image with the height of the sonar 4, the height of the ground is determined at the boundary with the overhead image with the height of the ground. The image is converted into a bird's-eye view image whose height continuously changes up to the height of the sonar 4.

  Here, FIG. 11 shows a side view of the bird's-eye view image converted by the vehicle surrounding monitoring apparatus of the present embodiment. As shown in FIG. 11, in the overhead image of the present embodiment, the variable range 111 is provided at the boundary between the overhead image of the ground height and the overhead image of the height of the sonar 4 to continuously change the height.

  Therefore, in the first embodiment, when a white line is drawn on the ground, as shown in FIG. 12A, a white line is formed at the boundary between the overhead image of the ground height and the overhead image of the sonar 4 height. In contrast to 121 being discontinuous, the bird's-eye view image of the present embodiment can continuously display white lines 131 drawn on the ground as shown in FIG. In the first embodiment, since the overhead images having different heights are simply combined, the pixels captured by the camera 2 are lost at the boundary, and continuity and lack of effective information are generated. Therefore, in the present embodiment, this problem is solved by gradually changing the height of the bird's-eye view image at the boundary that is lost due to the composition, so that it can be continuously displayed.

  As described above, in the vehicle surrounding monitoring apparatus according to the present embodiment, the height of the ground is continuously increased from the ground level to the predetermined height at the boundary between the overhead image of the ground height and the overhead image of the predetermined height. Because it converts to a bird's-eye view image that changes, it is possible to smoothly connect and synthesize bird's-eye images with different heights, and present complex objects and non-vertical objects such as white lines drawn on the ground in an easy-to-understand manner. So that the driver can recognize it more accurately.

<Fourth Embodiment>
Next, a fourth embodiment of the present invention will be described with reference to the drawings. The vehicle surrounding monitoring apparatus of the present embodiment is different from the first embodiment in that the overhead image of the height of the sonar 4 has a transmission effect.

  Therefore, in the overhead view image of the present embodiment, as shown in FIG. 14A, the white line drawn on the ground is displayed with a dotted line in the detection area where the overhead view image of the sonar 4 is displayed. . If this overhead image is a side view, the side view as shown in FIG. 15 is obtained. In the detection area, the overhead image at the height of the sonar 4 becomes a transparent image, and the overhead image at the height of the ground can be displayed at the same time. Yes.

  As described above, in the vehicle surrounding monitoring apparatus according to the present embodiment, the overhead view image having a predetermined height is transmitted so as to be seen so that the overhead view image having the height of the ground can be seen. The bird's-eye view image and the ground-level bird's-eye view image can be presented at the same time, thereby increasing the information presented to the driver and giving the driver many choices.

  As described above, the vehicle surrounding monitoring apparatus of the present invention has been described based on the illustrated embodiment, but the present invention is not limited to this, and the configuration of each part is an arbitrary configuration having the same function. Can be replaced.

  The present invention relates to a vehicle surrounding monitoring device that monitors the surroundings of a vehicle, and is particularly useful as a technique for displaying an object present at a high position from the ground at a correct position on an overhead image.

1 is a block diagram showing a configuration of a vehicle surroundings monitoring device according to a first embodiment of the present invention. It is a flowchart for demonstrating the vehicle periphery monitoring process by the vehicle periphery monitoring apparatus which concerns on the 1st Embodiment of this invention. FIG. 3A and FIG. 3B are diagrams for explaining a bird's-eye view of the height of the ground after conversion and a side arrangement of the own vehicle. 4 (a) and 4 (b) are diagrams for explaining a bird's-eye view image of the height of the converted ground and a side arrangement of the own vehicle when there is an object at a high position from the ground. . FIG. 5A and FIG. 5B are a plan view and a side view for explaining a detection range of a sonar installed as a detection unit. It is a figure for demonstrating the conversion method from the bird's-eye view image of the height of the ground to the bird's-eye view image of the height of sonar. FIG. 7A and FIG. 7B are views showing a bird's-eye view image and a side arrangement of the own vehicle after being synthesized by the vehicle periphery monitoring device of the first embodiment. It is a side view for demonstrating the bird's-eye view image after composition by the perimeter monitoring device for vehicles of a 1st embodiment. FIG. 9A and FIG. 9B are views showing a bird's-eye view image and a side arrangement of the own vehicle after being synthesized by the vehicle periphery monitoring device of the second embodiment. It is a side view for demonstrating the bird's-eye view image after a synthesis | combination by the vehicle periphery monitoring apparatus of 2nd Embodiment. It is a side view for demonstrating the bird's-eye view image after a synthesis | combination by the vehicle periphery monitoring apparatus of 3rd Embodiment. FIGS. 12A and 12B are diagrams for explaining a bird's-eye view image and a side arrangement of the host vehicle after the synthesis by the vehicle surrounding monitoring apparatus of the first embodiment when a white line is drawn on the ground. FIG. FIG. 13A and FIG. 13B are views showing a bird's-eye view image and a side arrangement of the own vehicle after being synthesized by the vehicle periphery monitoring device of the third embodiment. FIG. 14A and FIG. 14B are views showing a bird's-eye view image and a side arrangement of the own vehicle after being synthesized by the vehicle periphery monitoring device of the fourth embodiment. It is a side view for demonstrating the bird's-eye view image after a synthesis | combination by the vehicle periphery monitoring apparatus of 4th Embodiment.

Explanation of symbols

1 Vehicle Ambient Monitoring Device 2 Camera (Camera, Camera)
3 1st conversion part (1st conversion means)
4 Detection part (sonar, detection means)
5 Distance calculation part (distance calculation means)
6 2nd conversion part (2nd conversion means)
7 Synthesizer (synthesizer)
8 Display section (monitor, display means)
40 Own vehicle 41, 61 Object 51, 52, 53 Detection area 111 Variable range 121, 131 White line

Claims (7)

A vehicle surrounding monitoring device that converts the situation around the vehicle into a bird's-eye view image and presents it to the driver, and monitors the surroundings of the vehicle,
Photographing means for photographing the situation around the vehicle;
First conversion means for converting an image photographed by the photographing means into an overhead view image of the height of the ground;
Detecting means for detecting an object existing around the vehicle;
Distance calculating means for calculating the distance to the object detected by the detecting means;
Second conversion means for converting an image photographed by the photographing means into an overhead image having a predetermined height based on the distance to the object;
Combining means for combining the overhead image of the height of the ground and the overhead image of the predetermined height;
A vehicle surrounding monitoring device comprising: display means for displaying an image synthesized by the synthesizing means.   The detection range of the detection means is divided into a plurality of detection areas, and the second conversion means generates the overhead image having the predetermined height for each detection area. Vehicle surrounding monitoring device.   The vehicle surrounding monitoring apparatus according to claim 2, wherein the second conversion unit converts only the detection area where the object is present into the overhead image having the predetermined height.   The vehicle surrounding monitoring apparatus according to claim 2, wherein the second conversion unit converts all ranges in a direction away from a detection area where the object exists into an overhead image having a predetermined height. .   The second conversion means continuously changes the height from the height of the ground to the predetermined height at the boundary between the overhead image of the ground height and the overhead image of the predetermined height. The vehicle surrounding monitoring apparatus according to any one of claims 1 to 4, wherein the vehicle surrounding monitoring apparatus is converted into a bird's-eye view image.   The said synthetic | combination means synthesize | combines so that the overhead image of the said height can be seen through the said overhead image of the said predetermined | prescribed height, The composite image of Claim 1 characterized by the above-mentioned. Vehicle surrounding monitoring device. A vehicle surroundings monitoring method for monitoring the surroundings of the vehicle by converting the situation around the vehicle into a bird's-eye view image and presenting it to the driver.
A shooting step of shooting the situation around the vehicle by shooting means;
A first conversion step of converting the image shot in the shooting step into an overhead view image of the height of the ground;
A detection step of detecting an object existing around the vehicle;
A distance calculating step for calculating a distance to the object detected in the detecting step;
A second conversion step of converting the image shot in the shooting step into the overhead image of the predetermined height based on the distance to the object;
A synthesis step of synthesizing the overhead image of the height of the ground and the overhead image of the predetermined height;
And a display step for displaying the image synthesized in the synthesis step.

Images