JP5073461B2 - Vehicle perimeter monitoring system - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring system, and more particularly to an improvement in a system that converts an original image taken by a camera into an overhead image.

  In order to monitor the surroundings of the vehicle without omission, it is necessary to install a total of four sensors at least in the front and rear and left and right sides of the vehicle.

For example, a millimeter wave radar is used as a sensor, and a monitoring system (Patent Document 1) that detects an object close to the vehicle using a millimeter wave radar, or a stereo camera is used as a sensor, and the distance to the object is converted into a captured image of the stereo camera. A system (Patent Document 2) that monitors the periphery of a vehicle by measuring based on the above is known.
JP 2004-142660 A JP 2000-261787 A

  By the way, the prior art disclosed by the above-mentioned patent document is a configuration for detecting an object around the vehicle using a millimeter wave radar or a stereo camera, but such a radar or stereo camera is special. There is a problem that the cost of the parts such as the radar is reflected in the entire manufacturing cost of the vehicle periphery monitoring system, resulting in an increase in product price.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an inexpensive vehicle periphery monitoring system using a general camera that is commercially available for in-vehicle use.

  The vehicle periphery monitoring system according to the present invention converts an original image obtained using a general vehicle-mounted camera into an overhead image, and detects an object other than a road surface around the vehicle based on the overhead image. The vehicle periphery monitoring system is provided at a low cost.

That is, the vehicle periphery monitoring system according to the present invention includes a camera that captures at least a part of the periphery of the vehicle, an image conversion processing device that converts an original image obtained by the camera into an overhead image, and the image conversion Based on the bird's-eye view image obtained by the processing device, an object detection processing device that detects the presence or absence of an object other than the road surface around the vehicle, and the bird's-eye view image obtained by the image conversion processing device are displayed. wherein the object obtained by the object detection apparatus, e Bei and an image display device for displaying by superimposing the display emphasizes the display portion during the overhead image, the object detecting apparatus, the overhead image in A non-road surface area detection unit that detects an area that is not a road surface in the vehicle, and a non-moving body area detection unit that detects an area that is not a mobile body in the overhead image. And a non-moving body region obtained by the non-road surface area detection unit and the non-moving body region obtained by the non-moving body region detection unit. And a stationary body detection unit that detects the above.

  According to the vehicle periphery monitoring system according to the present invention configured as described above, an image conversion processing device converts an original image obtained by photographing with a camera into a bird's-eye view image by conversion processing, and an object detection processing device By detecting an object other than the road surface around the vehicle based on the obtained bird's-eye view image, an object such as an obstacle around the vehicle can be easily detected.

  Therefore, the vehicle periphery monitoring system according to the present invention can be provided at low cost because it uses a general camera that is commercially available for in-vehicle use.

  According to the vehicle periphery monitoring system according to the present invention, since a general camera that is commercially available for in-vehicle use is used, it can be provided at low cost.

  Hereinafter, an embodiment of a vehicle periphery monitoring system 100 according to the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring system 100 according to an embodiment of the present invention, FIG. 2 is a block diagram showing the system 100 shown in FIG. 1 with a more specific configuration, and FIG. The figure which shows the window area w set to the bird's-eye view image, FIG. 4 is a flowchart which shows the process sequence of the system 100 shown in FIG.

  As shown in FIG. 1, the vehicle periphery monitoring system 100 shown in the figure is installed at each of the four corners of the vehicle, and the vehicle periphery (right side direction, left side direction, front direction, rear direction of the vehicle) Each of the four cameras 10 for photographing at least a part and the original images obtained by these four cameras 10 are converted into overhead images, and the obtained four overhead images are converted into a single centered on the vehicle. Based on the image conversion processing device 20 that performs composition processing on the bird's-eye view image, and the bird's-eye view image obtained by the image conversion processing device 20 (a single bird's-eye view image obtained by synthesis, the same applies hereinafter), The object detection processing device 30 that detects the presence or absence of an obstacle (an object other than the road surface) and the overhead image obtained by the image conversion processing device 20 are displayed on the object detection processing device 30. Of the resulting obstacle I, emphasizing the image display device 40 to display part in the overhead image, a configuration in which a.

  Here, the object detection processing device 30 includes a non-road surface area detection unit 31 that detects a non-road area in the overhead image, a non-moving body area detection unit 35 that detects a non-moving area in the overhead image, Based on the non-road surface region obtained by the non-road surface region detection unit 31 and the non-moving body region obtained by the non-moving body region detection unit 35, a stationary body that is not a road surface and is not a moving body is detected as the obstacle. A stationary object detection unit 39a, an n-gradation image conversion unit 38 that converts an overhead image into an image with a gradation number n, and an image with a gradation number n obtained by the n-gradation image conversion unit 38. And a moving body detection unit 39b that detects a moving body in the vicinity of the vehicle as an obstacle (object) as compared with each pixel between frames.

  More specifically, the non-road surface area detection unit 31 calculates in advance the brightness and color of the surrounding area of the vehicle in the overhead view image at a specific time as a reference value representing the road surface, and stores the calculated reference value A reference value calculation unit 32, a road surface index value calculation unit 33 that sequentially calculates the brightness and color of the surrounding area of the vehicle in successive bird's-eye views as index values, and a road surface index value calculated by the road surface index value calculation unit 33 A non-road surface area determination unit 34 that compares the road surface reference value stored in advance by the road surface reference value calculation unit 32 and determines whether or not the surrounding area of the vehicle in the sequential bird's-eye view image is a non-road surface area; It has.

  Here, the road surface reference value calculation unit 32 determines the surrounding environment of the vehicle (conditions of ambient light such as weather conditions and day and night, and the state of the road surface when the vehicle is stopped or running (whether it is a paved road, whether it is a paved road or ground surface). In consideration of changes in color conditions, etc.)), the calculation of the reference value is repeated at appropriate time intervals, and the reference value is updated.

  The non-moving body region detection unit 35 includes an index value difference calculation unit 36 that calculates a difference between the brightness and color index values of the peripheral region of the vehicle between corresponding pixels between successive bird's-eye views, and an index value difference calculation unit 36. A non-moving body determination unit 37 that determines the presence / absence of a moving body in a peripheral area of the vehicle in successive bird's-eye views based on the calculated index value difference.

  The stationary object detection unit 39a detects the area determined as a non-road surface by the non-road surface area determination unit 34 and determined as a non-moving object by the non-moving object determination unit 37 as the above-described stationary object (stationary obstacle). To do.

  Here, the correspondence between the configuration shown in FIG. 1 and the specific configuration shown in FIG. 2 will be described.

  That is, the camera 10 is a monocular camera 71 generally used as an in-vehicle camera, the image conversion processing device 20 is a first image processing unit 72, the road surface reference value calculating unit 32 is a second image processing unit 73, and a road surface index value is calculated. The unit 33 is a third image processing unit 74, the non-road surface area determination unit 34 is a first image recognition unit 75, the index value difference calculation unit 36 is a third image processing unit 74, and the non-moving body determination unit 37 is a first image recognition unit. 75, the stationary object detection unit 39a is the first image recognition unit 75, the n-tone image conversion unit 38 is the fourth image processing unit 76, the moving object detection unit 39b is the fifth image processing unit 77 and the second image recognition unit 78, The image display device 40 corresponds to the image display unit 79.

  Note that the number of monocular cameras 71 (cameras 10) is four in the above-described one, but the number is not limited to four as long as the entire area around the vehicle 200 (FIG. 3) can be photographed. Any number can be used.

  Then, the first image processing unit 72 generates an overhead view video (overhead image) as shown in FIG. 3 based on the video of the monocular camera 71 (step 1 (S1) in FIG. 4).

  This bird's-eye view image is converted into an image in which the periphery of the vehicle 200 is viewed from above the vehicle 200 based on the images (images) of the monocular cameras 71 installed on the front, rear, left and right of the vehicle 200.

  In the display on the image display unit 79 that is visually recognized by the driver or the like of the vehicle 200, for example, the vehicle 200 itself is superimposed and displayed on the overhead view image by superimpose as a picture imitating a plan view of the vehicle. The driver can grasp the surrounding situation.

  The display layout shown in FIG. 3 is merely an example, and various other methods for displaying the bird's-eye view around the vehicle 200 can be applied. Is as follows.

  That is, in the bird's-eye view video, the far region of the camera video (original image) is blurred by the image processing method. Therefore, the blurred area is unnecessary as a display image.

  Next, based on the bird's-eye view video, the second image processing unit 73 calculates the luminance and color reference values of the window area w (window area).

  Here, the window area w refers to a plurality of small windows (for example, several tens of pixel areas) in the peripheral area of the vehicle in the bird's-eye view image of FIG. 3 (24 areas (window areas w11 and w12 in the example of FIG. 3)). ,..., W25,..., W43,..., W46)) are set (S2 in FIG. 4), and the luminance and color of the pixels in these window areas w11 to w46 are acquired (S3 in FIG. 4).

  For example, the luminance and the color are acquired with values representing the luminance values and colors of a large number of pixels included in each window area.

  In order to reduce the influence of noise and the like, it is preferable to calculate with an average value of several frames as an example. In addition, the luminance and color acquisition is updated at appropriate time intervals in consideration of changes with time.

  For the luminance and color obtained for each window area w11 to w46, for example, the luminance and color of each pixel included in each window area w are frequently used as the reference value for each window area, and this reference value is The road surface brightness and color reference values corresponding to the window areas w11 to w46 are stored (S4 in FIG. 4).

  Depending on the road surface, the brightness and color of the area corresponding to each window in the overhead view video may not be the same between all window areas w (such as pavement and soil), in this case, Since the reference values are different for each window area w, a plurality of reference values are stored as reference values for each window area w in descending order of similar luminance and color, and they are stored as background information (vehicle surroundings). It is preferably used as information corresponding to the corresponding luminance and color.

  The third image processing unit 74 calculates brightness and color between frames of the camera video of the overhead view video, and uses these values corresponding to each frame as index values of sequential overhead view images (frames constituting the overhead view video). (S6 in FIG. 4).

  The frame in the camera video is usually 30 frames / second, but is preferably calculated at 15 frames / second in order to reduce the load of index value calculation.

  Then, the first image recognition unit 75 compares the index value calculated for each window area w and for each frame with the reference value stored in the second image processing unit 73, and determines the same index value as the reference value. Excluded pixels are excluded (S7 in FIG. 4). Thereby, the non-road surface area | region determined not to be a road surface is detected for every window area w.

  Further, the third image processing unit 74 calculates a difference in index value between frames, and the first image recognition unit 75 determines an area where the difference is 0 or a value close to 0 as an area where a difference occurs between frames. That is, it is determined that a region having changes in luminance and color has appeared around the vehicle 200.

  Here, when the vehicle is traveling, the above-described region where the luminance and the color change are the obstacle (stationary object) that is stationary or the obstacle (moving object) that is moving In any case, since the detection of the moving object is performed in a later process, the above-described region where the luminance and the color are changed is determined as a stationary obstacle (S8 in FIG. 4).

  By the above operation, the first image recognition unit 75 that is the stationary body detection unit 39a detects a stationary body (not a moving body) that is not a road surface, and displays information indicating the detected position of the stationary body as an image. The data is output to the display unit 79.

  In the stationary object detection process described above, in order to reduce the load of calculation amount, the position information of the moving object (coordinates in the overhead view image: S12 in FIG. 4) calculated by the second image recognition unit 78 described later is used. The area may be excluded in advance (S5 in FIG. 4).

  In addition, the above determination of a stationary object is an image recognition over several frames (for example, it is not a sudden appearance of luminance and color information, but is constantly present while performing an overhead view) It is preferable to determine that the object is stationary.

  Next, the fourth image processing unit 76 deletes the color information from the bird's-eye view video, and generates an achromatic gray scale (black and white n gradation) video (S10 in FIG. 4). Thereby, it is possible to reduce the amount of calculation in the second image recognition unit 78 in the subsequent stage, compared with the case where a color overhead view video is used.

  Note that the number n of gradations of the n-gradation video can be appropriately changed according to the arithmetic processing capability of the second image recognition unit 78 and the desired detection accuracy of the moving body.

  The fifth image processing unit 77 detects a motion between frames from the black and white n-gradation video (S11 in FIG. 4). Specifically, the obtained image having the number of gradations n is compared for each pixel between a plurality of frames, and an obstacle (moving body) moving around the vehicle 200 is detected as an object.

  Here, when the vehicle 200 is stopped, the fifth image processing unit 77 determines that a change in pixel value (brightness value) between frames is a candidate for a moving object, and coordinates of the region of the candidate for the moving object. Information is output to the second image recognition unit 78 (S11 in FIG. 4).

  On the other hand, when the vehicle 200 is moving, the second image recognizing unit 78 compares the corresponding pixels between the plurality of frames generated by the movement of the vehicle 200 itself before comparing the values for each pixel between the plurality of frames. It is only necessary to correct the value (or change the pixel position information as the vehicle 200 moves), and by interpolating the motion component of the vehicle 200, the position information (coordinates) of the area of the moving body is the same as when the vehicle 200 is stopped. Information) is acquired (S11 in FIG. 4).

  With the above operation, the coordinate information of the moving obstacle candidate is obtained, and the second image recognition unit 78 determines the moving object based on the coordinate information corresponding to the obstacle candidate. Here, the second image recognition unit 78 analyzes the change (motion) between the frames of coordinate information obtained in a single frame or a plurality of frames by a statistical process, so that a true moving object is obtained. Only the coordinates of the candidate determined to be (S12 in FIG. 4) are outputted.

  The coordinate information of the moving body is provided as mask information for reducing the calculation load in the third image processing unit 74 described above (S5 in FIG. 4) and is output to the image display unit 79.

  The image display unit 79 to which the coordinate information of the stationary body and the moving body is input by the above processing displays an overhead image as shown in FIG. 3 and also displays the stationary body and the second body detected by the first image recognition unit 75. The display part in the bird's-eye view image of the moving body detected by the image recognition unit 78 is displayed in an emphasized manner, and the driver or the like is cautioned about the presence of the stationary body and the moving body.

That is, an obstacle (moving body and stationary body) is displayed on the overhead view video (including a schematic display of the vehicle 200) output as a display on the image display unit 79 from the coordinate information of the moving body and stationary body to the corresponding coordinates on the display. (S9 in FIG. 4). As this explicit method,
(1) Surround the area corresponding to the obstacle with a frame,
(2) Place an icon in the area corresponding to the obstacle,
(3) Display a symbol in the area corresponding to the obstacle,
(4) The color of the area corresponding to the obstacle is different from the color of the other areas.
(5) blink the area corresponding to the obstacle,
Etc. can be applied.

  Further, in the configuration in which the vehicle periphery monitoring system 100 of the present embodiment is further provided with an alarm sound generator 80 as shown in FIG. 2, not only the obstacle is displayed on the image display unit 79, The warning sound generator 80 may issue a voice or other warning sound to further increase the level of alerting the driver or the like.

  As described above, according to the vehicle periphery monitoring system 100 according to the present embodiment, since the general camera 10 that is commercially available for in-vehicle use is used, it can be provided at low cost.

  That is, by using the monocular camera 71, it is possible to provide an advantageous product in terms of cost, and by detecting an obstacle around the vehicle, it is possible to assist the driver with safe driving.

  As a method for obtaining a bird's-eye view image based on an original image obtained with a general vehicle-mounted camera, for example, a technique already known in Japanese Patent Application Laid-Open No. 2002-034035 may be applied.

It is a block diagram which shows the structure of the vehicle periphery monitoring system which concerns on embodiment of this invention. FIG. 2 is a block diagram illustrating the system illustrated in FIG. 1 with a more specific configuration. It is a figure which shows the window area set to the bird's-eye view image | video. It is a flowchart which shows the processing flow of the vehicle periphery monitoring system shown in FIG.

Explanation of symbols

71 monocular camera 72 first image processing unit 73 second image processing unit 74 third image processing unit 75 first image recognition unit 76 fourth image processing unit 77 fifth image processing unit 78 second image recognition unit 79 image display unit 80 Alarm sound generator 100 Vehicle periphery monitoring system

Claims (7)

A camera for photographing at least a part of the periphery of the vehicle;
An image conversion processing device for converting an original image obtained by the camera into an overhead image;
An object detection processing device that detects the presence or absence of an object other than a road surface around the vehicle, based on the overhead image obtained by the image conversion processing device;
And displays the overhead image obtained by the image converter, Bei example and a highlight image display device a display portion in the object, the overhead in the image obtained by the object detecting apparatus,
The object detection processing device includes:
A non-road surface area detection unit for detecting an area that is not the road surface in the overhead view image;
A non-moving body region detection unit that detects a region that is not a moving body in the overhead view image;
Based on the non-road surface region obtained by the non-road surface region detection unit and the non-moving body region obtained by the non-moving body region detection unit, a stationary body that is not a road surface and is not a moving body is detected as the object. A vehicle periphery monitoring system comprising: a stationary body detection unit . The non-road surface area detection unit is
A road surface reference value calculation unit that pre-calculates the brightness and color of the surrounding area of the vehicle in the overhead image at a specific time as a reference value representing the road surface, and stores the calculated reference value;
A road surface index value calculation unit that sequentially calculates the brightness and color of the surrounding area of the vehicle in the sequential overhead image as an index value;
The road surface index value calculated by the road surface index value calculation unit is compared with the road surface reference value stored in advance by the road surface reference value calculation unit. A non-road surface area determination unit that determines whether or not it is a non-road surface area,
The non-moving body region detecting unit is
An index value difference calculation unit that calculates a difference between brightness and color index values of the surrounding area of the vehicle between the sequential overhead images;
A non-moving body determining unit that determines the presence or absence of the moving body in a peripheral area of the vehicle in the sequential overhead image based on the index value difference calculated by the index value difference calculating unit;
The stationary body detection unit detects, as the stationary body, an area that is determined as a non-road surface by the non-road surface area determination unit and is determined as a non-moving body by the non-moving body determination unit. Item 4. The vehicle periphery monitoring system according to Item 1 . In the peripheral area of the vehicle in the overhead image, a plurality of different small areas are preset as window areas,
The road surface reference value calculation unit calculates and stores the reference value for each of the plurality of window regions,
The road surface index value calculation unit calculates the index value for each of the plurality of window regions,
The vehicle periphery monitoring system according to claim 2, wherein the non-road surface area determination unit determines whether the plurality of window areas are the non-road surface area. The road surface reference value calculation unit
The vehicle periphery monitoring system according to claim 2 or 3, wherein an average value of the reference values for each of the corresponding areas in a plurality of frames is calculated as the road surface reference value. The object detection processing device includes:
An n-gradation image converter for converting the overhead image into an image having a gradation number n;
A moving body detection unit that compares the image of the number of gradations n obtained by the n-tone image conversion unit for each pixel between a plurality of frames, and detects a moving body around the vehicle as the object; vehicle environment monitoring system according to claim 1 or et 4, characterized in that with a. When the vehicle itself moves, the moving body detecting unit compares the values of corresponding pixels between the plurality of frames generated by the movement of the vehicle itself before comparing the pixels for each pixel between the plurality of frames. The vehicle periphery monitoring system according to claim 5, wherein the vehicle periphery monitoring system is corrected. 7. The vehicle periphery monitoring system according to claim 5, wherein the n gradation image conversion unit converts the image having the number of gradations n into an achromatic gray scale image.

Images