JP4013872B2 - Obstacle detection device - Google Patents

Description

  The present invention relates to an obstacle detection device that detects an obstacle, and more particularly, to an obstacle detection device that detects an obstacle based on the amount of movement of an image and the similarity of images.

  There is a device that mounts a camera device on a vehicle, images the surroundings of the host vehicle, detects an obstacle, and notifies the vehicle driver. In this type of apparatus, it is required to determine the possibility of a collision from a captured image and detect an obstacle with a possibility of a collision.

For example, in Patent Document 1, an image of the periphery of a vehicle is captured by a pair of stereo CCD cameras mounted on the left and right of the vehicle. There is described a vehicle obstacle alarm device that activates an alarm when there is an obstacle that may collide based on the distance between the object and the vehicle and the relative speed of the obstacle calculated from the distance.
JP-A-8-110224

However, in the conventional technique, since the distance is detected by stereo image processing, two cameras are required, and there is a problem in that the device configuration is complicated and the cost is increased. In addition, in order to accurately measure the distance, it is necessary to accurately align the optical axes of the two cameras, and there is a disadvantage that man-hours and costs are required.
The present invention has been proposed to solve the above-described problems, and provides an obstacle detection method and an obstacle detection apparatus capable of detecting an obstacle that may collide with a single camera. Objective.

  According to the present invention, an imaging unit that images the surroundings, a trackable region detection unit that detects at least a tracking region corresponding to an image of an obstacle moving relatively from the image captured by the imaging unit, and the imaging unit The movement amount calculation means for calculating the movement amount of the tracking area based on two images captured at different timings according to the above, and the similarity between the tracking area before the movement and the tracking area after the movement in which the movement amount is calculated Area similarity calculating means for calculating the degree, obstacle detecting means for detecting an obstacle based on the movement amount calculated by the area movement amount calculating means and the similarity calculated by the area similarity calculating means It is possible to provide an obstacle detection device having

  Thereby, the presence of an obstacle can be detected with one camera.

  Hereinafter, the obstacle detection apparatus 100 of the present embodiment will be described with reference to the drawings. The obstacle detection apparatus 100 according to the present embodiment is mounted on a vehicle on which a user is boarded, and detects obstacles existing around the vehicle.

  FIG. 1 shows a block configuration of an obstacle detection apparatus 100 according to the present embodiment.

  As shown in FIG. 1, the obstacle detection apparatus 100 includes an imaging unit 1, a trackable region detection unit 2, a region movement amount calculation unit 3, a region similarity calculation unit 4, and an obstacle detection unit 5. Have. Specifically, at least the tracking area is determined based on the captured image information, the movement amount and similarity of the determined tracking area are calculated, and the presence of an obstacle is determined based on the calculated movement amount and similarity. ROM that stores a program to be determined, and functions as a traceable area detection unit 2, an area movement amount calculation unit 3, an area similarity calculation unit 4, and an obstacle detection unit 5 by executing the program stored in the ROM And a RAM functioning as the storage means 6.

  In addition, the obstacle detection device 100 can exchange information with at least one of the warning device 200 that issues various warnings based on the detection result of the obstacle and the driving support device 300 that supports driving. It is connected by an in-vehicle LAN.

  Each configuration will be described below.

  The “imaging means 1” is, for example, a video camera, and images the front of the vehicle at a constant cycle (time interval Δt). The captured image is stored in the image storage unit 61. The left and right optical axes of the camera serving as the imaging unit 1 are set to coincide with the straight traveling direction of the vehicle, and the vertical optical axis of the camera is set to be horizontal.

  The “trackable area detecting unit 2” detects an obstacle moving at least relative to the vehicle based on two images captured by the imaging unit 1 at different timings. Therefore, the tracking corresponding to the image of the obstacle is performed. Detect areas. The tracking area may be arbitrarily set, but in the present embodiment, an area where the moving body can be tracked is detected based on a change in lightness and darkness of the image. That is, the traceable area detecting unit 2 of the present embodiment detects an image area having a change in shading in a plurality of directions as a tracking area. In this way, the area that can be tracked by the detection target is an area that has light and shade changes in multiple directions. This is because it is difficult to detect the amount of movement of the tracking target when is perpendicular to the direction of light and shade change.

  Specifically, the traceable area detection unit 2 includes an edge detection unit 21, a variance value calculation unit 22, and a determination unit 23. The edge detection unit 21 detects the edge of the image captured by the imaging unit 1. The edge detection unit 21 of the present embodiment reads an image stored in the image storage unit 61, applies a filter for detecting vertical and horizontal edges, and determines the vertical and horizontal component sizes ( dI / dx, dI / dy: I is a pixel value). Next, in order to remove a noise component included in the detected edge, a pixel whose sum of squares of the obtained vertical and horizontal edge components is greater than or equal to a threshold th1 is extracted, and based on the extracted pixel To detect edges.

  Since the variance value calculation unit 22 detects a region having light and shade changes in a plurality of directions, the variance value of the ratio of the size of the vertical component and the horizontal component of the edge detected by the edge detection unit is calculated. Calculation is performed for each predetermined area of the image. Specifically, the variance value calculation unit 22 obtains the density gradient direction by obtaining the ratio of the magnitudes of the vertical and horizontal components of the edge. Next, the image in which the edge from which the noise has been removed is shown is divided into a lattice shape at equal intervals along the vertical direction and the horizontal direction. FIG. 2 is a schematic diagram illustrating a state in which an image in which an edge is detected from a captured image using the above-described method is divided into a lattice shape. The divided one grid shown in FIG. 2 is a unit area for performing the process of detecting the traceable area. The variance value calculation unit 22 obtains a variance value of the ratio of the size of the vertical component and the horizontal component of the edge for each of the divided unit regions.

  The determination unit 23 determines a predetermined area in which the variance value calculated by the variance value calculation unit 22 is larger than a predetermined value as a tracking area for tracking the movement of the obstacle. This is based on the tendency that if the edge gradient direction in the divided rectangular area is one direction, the variance value is small, and if the edge gradient direction is in a plurality of directions, the variance value is large. In other words, the determination unit 23 determines that the edge gradient direction is likely to be one direction if the obtained variance value of the unit region is smaller than the predetermined threshold th2. On the other hand, if the obtained dispersion value is equal to or greater than the predetermined threshold th2, the moving body moves in any direction because it has edge gradients in multiple directions. Also, it is determined that the area corresponding to the obstacle can be tracked. That is, the determination unit 23 determines that an area where the obtained variance value is equal to or greater than the predetermined threshold th2 is a tracking area. FIG. 3 shows the tracking area determined by the determination unit 23 based on the captured image shown in FIG.

  The determination unit 23 of the traceable area detecting unit 2 identifies each unit area by specifying the start point position (for example, the upper left corner), the image size (width and height) of the area determined as the tracking area, and the image pattern of the original image. The tracking area information storage unit 62 stores the identification number in association with the identification number. Hereinafter, the image pattern of the tracking area before movement, which is a reference for calculating the movement amount and the correlation value, is referred to as “reference pattern”.

  The trackable area detection unit 2 detects a new tracking area for each image captured by the imaging unit 1 at an interval of time Δt. If the newly detected tracking area is an area different from the area where the area movement amount is being calculated, the starting position, image size, and image pattern are stored in the tracking area information storage unit 62 as a new tracking area. To do. Whether or not it is a new tracking area is determined based on the starting point position and size of the tracking area stored in the tracking area information storage unit 62.

  When information related to a new image captured by the imaging unit 1 is input to the image storage unit 61, the “region movement amount calculation unit 3” tracks based on the two images captured at different timings by the imaging unit 1. The amount of movement of the area is calculated. The area movement amount calculation means 3 includes a correlation value calculation unit 31, a determination unit 32, and a calculation unit 33.

  The movement amount detection method will be described. First, the correlation value calculation unit 31 sets a search area in order to detect the movement destination of the tracking area. This search area is an area that includes the tracking area and is extended by a predetermined number of pixels toward the outside of the tracking area. In the present embodiment, a search area is set that is expanded by one pixel outward in the vertical direction with respect to the four sides of the tracking area. The correlation value between the reference pattern and the tracking area of the new image data is calculated while shifting the reference pattern (the stored tracking area image pattern) by one pixel within the search area. In calculating the correlation value, the position of the reference pattern is moved (shifted) in units of one pixel, so the position where the correlation value is obtained is also in units of one pixel, but interpolation calculation is performed using a plurality of correlation values. Thus, the position where the correlation value is maximized can be obtained with an accuracy of 1 pixel or less. As a result, the movement destination of the tracking area can be obtained with an accuracy of 1 pixel or less.

The determination unit 32 determines that the obstacle image after movement exists in the search region where the correlation value calculated by the correlation value calculation unit 31 is maximum, and determines the search region as the tracking region after movement. In the present embodiment, the area where the calculated correlation value is equal to or greater than the threshold th3 and is the maximum is determined as the tracking area after the movement of the reference pattern.

  The calculation unit 33 compares the position of the tracking region after movement determined by the determination unit 32 with the position of the tracking region (tracking region corresponding to the reference pattern) detected by the tracking region detection unit 2, and the amount of movement Is calculated. The movement amount of the tracking area is expressed as each direction component in the horizontal direction x and the vertical direction y based on the image coordinates. In FIG. 4, the movement component of the movement amount calculated for each tracking region shown in FIG. 3 is shown as a vector.

  The tracking region position after movement determined by the determination unit 32 and the movement amount calculated by the calculation unit 33 are stored in the tracking region information storage unit 62. However, if the correlation value with the new image is less than the threshold th3, it cannot be determined that the same part in the image has moved, so the information regarding the corresponding tracking area is initialized. Also, when the start point position (start point coordinates) of the tracking area deviates from a predetermined range (horizontal direction x, vertical direction y), information on the corresponding tracking area is initialized. This is because the detection target moving in a direction different from the moving direction of the stationary object or the host vehicle on the image moves from the vanishing point (center of the captured image) to the peripheral direction (outside of the captured image) with the movement of the host vehicle. This is because the camera goes out of the imaging range as time passes. With respect to the initialized tracking area, detection processing by the obstacle detection means 5 described later is not performed.

  The “region similarity calculation unit 4” calculates the similarity between the “tracking region before movement (reference pattern)” whose movement amount is calculated by the region movement amount calculation unit 3 and the “tracking region after movement”. . The area similarity calculation unit 4 according to the present embodiment calculates the similarity based on the correlation value between the tracking area before movement and the tracking area after movement. Specifically, the region similarity calculation unit 4 acquires the correlation value obtained by the region movement amount calculation unit 3 in the movement amount calculation processing (movement destination detection processing), and uses the correlation value to calculate the similarity. calculate. In the present embodiment, since the correlation value calculated in advance in the tracking area detection process can be used, the calculation process can be performed quickly.

  The “obstacle detection unit 5” detects an obstacle based on the movement amount calculated by the region movement amount calculation unit 3 and the similarity calculated by the region similarity calculation unit 4. The reason for this is to distinguish between an obstacle with a possibility of collision and an obstacle without a possibility of collision, which cannot be distinguished only by the amount of movement. Examining the relationship between the possibility of collision and the amount of movement, the “car or person relatively moving toward the own vehicle” with the possibility of collision is the imaging means 1 (the own vehicle on which the obstacle detection device 100 is mounted). Therefore, the amount of movement in the horizontal direction on the image is small. Also, the amount of movement of an object with low possibility of collision such as “distant object” and “front traveling vehicle moving in the same direction as the own vehicle” is small. Furthermore, since the obstacle has a height, the amount of movement in the vertical direction on the image is smaller than that on the road surface. That is, an area with a small amount of movement includes both obstacles with high possibility of collision and obstacles with low possibility of collision, and these cannot be distinguished based only on the amount of movement of the tracking area.

  In this embodiment, in order to determine the possibility of a collision based on the amount of movement and the similarity, an obstacle with a possibility of collision is distinguished from an obstacle without a possibility of collision, and an obstacle with a possibility of collision is identified. Can be detected.

  Next, considering the relationship between the possibility of collision and the degree of similarity, as shown in FIG. 5, the distance to the object (including the obstacle) and the size of the image are inversely proportional, and the object approaches. As the image size changes abruptly, it can be seen that the similarity of the image changes greatly. Considering the similarity as a normalized correlation value, if the distance from the object is close, the normalized correlation value changes greatly.

  FIG. 6 shows temporal changes in correlation values during tracking of an object (obstacle) for the tracking areas a, b, c, and d shown in FIG. The “distant object” as in (b) and the “front vehicle moving in the same direction as the host vehicle” as in (c) are the distance change rate before and after tracking, that is, before and after tracking with respect to the distance to the object. Since the ratio of the distance change amount is small and the change of the enlargement ratio is small, the change of the normalized correlation value is small. On the other hand, the “height obstacle” such as “a car or a person relatively moving toward the vehicle” as in (a) or the “height obstacle” as in (d) has a large change in distance before and after tracking, and the enlargement ratio Since the change in is large, the change in the normalized correlation value is large.

  The inventor noticed that the amount of change in similarity or correlation value is large in the tracking area corresponding to the obstacle image in the vicinity. And the presence of an obstacle based on the similarity of the area. As a result, the collision amount can be selected from among “obstacles that move relatively toward the own vehicle (collision determination subject)”, “distant objects”, and “front vehicles that move in the same direction as the own vehicle” with a small amount of movement. It is possible to selectively detect an obstacle present near a possible host vehicle.

  Specifically, in the present embodiment, there is a tracking region in which the movement amount of the tracking region is equal to or smaller than the first threshold value (th4) and the amount of change in similarity is equal to or larger than the second threshold value (th5). At this time, the presence of an obstacle with a possibility of collision is detected, and the object recognized in the tracking area is determined as an obstacle. The amount of change in the similarity of the tracking region is calculated based on the amount of change in the correlation value. However, since the individual values of the correlation value vary, in an actual process, an approximate straight line with a certain number of samples is obtained, and the straight line is obtained. Is preferably used as the amount of change in similarity or the amount of change in correlation value. This makes it possible to perform highly reliable detection processing.

  FIG. 7 shows the relationship between the movement amount and the correlation value change amount for the tracking areas a to d. In the hatched portion of FIG. 7, the amount of movement of the tracking region is equal to or less than the first threshold (th4), and the amount of change in the similarity (correlation value) is equal to or greater than the second threshold (th5). This is the part that satisfies the conditions. Since the movement amount and the similarity change amount of the tracking area a (FIG. 3) satisfy this condition, the obstacle detection unit 5 detects the object included in the image of the tracking area a as an obstacle. The threshold value (th4) and threshold value (th5) used here are preferably set individually and specifically, so experiments were conducted and set based on the experimental results. In addition to the first threshold value and the second threshold value, the threshold value used in this embodiment is preferably stored in the threshold value storage unit 63 in advance.

Next, a processing procedure of the obstacle detection apparatus 100 according to the present embodiment will be described.
FIG. 8 is a flowchart showing a processing procedure of the obstacle detection apparatus 100 according to the present embodiment. The process starting from step S10 starts when an ignition switch (not shown) is turned on. In step S10, the tracking area information stored in the tracking area information storage unit 62 is initialized, and the process proceeds to step S20. In step S20, the imaging means 1 images the periphery of the vehicle, particularly the front of the vehicle, stores the captured image in the image storage unit 61, and proceeds to step S30.

  The traceable area detection unit 2 detects the tracking area based on the image stored in the image storage unit 61 (step S30). Next, the region movement amount calculation means 3 calculates the movement amount of the tracking region corresponding to the moving obstacle based on the two images captured at different timings at the imaging interval Δt (step S40). The area similarity calculation means 4 calculates the similarity of the tracking area before and after the movement for which the movement amount is calculated (step S50). The obstacle detection means 5 determines the possibility of collision based on the amount of movement and the similarity, and detects an obstacle with a possibility of collision (step S60). Specifically, when the movement amount of the tracking area is equal to or smaller than the first threshold value and the amount of change in the similarity of the tracking area is equal to or larger than the second threshold value, it is determined that there is a possibility of collision, An object imaged in the tracking area is detected as an obstacle.

  The detection result by the obstacle detection means 5 is output to the warning device 200 and / or the driving support device 300 which are external devices. The obstacle detection result may be information that “the obstacle exists”, or information “where the obstacle exists” derived by referring to the coordinates associated with the image information. It may be information such as “what is an obstacle” derived by pattern matching processing.

  Depending on the detection result of the obstacle detection means 5, the warning device 200 outputs an alarm to the driver (step S70). The alarm may be a screen display or a voice guidance.

  When the process in step S70 ends, the process returns to step S20. Thereafter, the processing after step S20 is repeatedly performed at a cycle of the imaging interval Δt.

  According to the obstacle detection apparatus 100 of the present embodiment, since an obstacle is detected based on the movement amount and similarity of the tracking area, it is possible to detect an obstacle that may collide with one camera. That is, it is not necessary to provide two or more cameras. For this reason, it is not necessary to provide a device for measuring the distance to the object as in a device having two or more cameras, and the device configuration can be simplified. In addition, it is not necessary to perform precise optical axis alignment between cameras, and costs and man-hours can be reduced.

  In the obstacle detection process, the threshold value for the amount of movement of the tracking area and the threshold value for the change in similarity are set, and the possibility of collision is determined based on the set threshold value. It is possible to accurately detect an obstacle that may collide among the detected objects.

  In the process of calculating the movement amount of the tracking area, the correlation value between the tracking area and the search area is calculated, and the movement area of the tracking area is accurately calculated by determining the tracking area after movement based on the correlation value. Can do. Also, by calculating the similarity using the correlation value in the similarity calculation process, the correlation value calculated in the tracking area movement amount calculation process can be used, and the obstacle detection process can be speeded up. be able to.

  Furthermore, in order to track the movement of the obstacle, a region having a plurality of shade changes is set as the tracking region, and the obstacle can be detected in the region where the obstacle can be accurately detected, so that highly accurate detection processing can be performed. it can.

  In the present embodiment, the obstacle detection apparatus 100 that is mounted on a vehicle and detects obstacles around the vehicle has been described. However, the present invention is not limited to this, and the obstacle detection apparatus 100 can also be used for a robot, a machine tool, or the like.

  The embodiment described above is described in order to facilitate understanding of the present invention, and is not described in order to limit the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

It is a block block diagram of the obstacle detection apparatus which concerns on this embodiment. It is a 1st figure for demonstrating the detection process of a tracking area | region. It is a 2nd figure for demonstrating the detection process of a tracking area | region. It is a figure for demonstrating the calculation process of the movement amount of a tracking area | region. It is a figure which shows the relationship between the distance of an obstruction, and an expansion rate. It is a figure which shows the time change of a correlation value. It is a figure for demonstrating the detection process of an obstruction. It is a flowchart figure which shows the process sequence of the obstruction detection apparatus which concerns on this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Obstacle detection apparatus 1 ... Imaging means 2 ... Traceable area detection means 3 ... Area movement amount calculation means 4 ... Area similarity calculation means 5 ... Obstacle detection means 6 ... Storage means 61 ... Image memory | storage part 62 ... Tracking area | region Information storage unit 63 ... threshold value storage unit 200 ... warning device 300 ... travel support device

Claims (4)

Imaging means for imaging the front of the host vehicle ;
A traceable area detecting means for detecting a tracking area corresponding to an image of an obstacle moving relatively at least from an image captured by the imaging means;
Area movement amount calculation means for calculating the movement amount of the tracking area based on two images imaged at different timings by the imaging means;
Area similarity calculating means for calculating the similarity between the tracking area before movement and the tracking area after movement, in which the movement amount is calculated;
The movement amount on the image of the tracking area calculated by the area movement amount calculation means is equal to or less than a first threshold value for excluding surrounding structures in the captured image , and is calculated by the similarity calculation means. When there is a tracking area in which the amount of change in the similarity of the image of the tracking area is equal to or greater than the second threshold, an object included in the image corresponding to the tracking area is regarded as an obstacle that may collide. obstruction detection system having an obstacle detection means for detecting. The area movement amount calculation means includes:
A correlation value calculation unit for calculating a correlation value between the detected tracking region and a search region set with reference to the tracking region;
A determination unit that determines a search region where the correlation value calculated by the correlation value calculation unit is maximum as a tracking region after movement;
Claim with a calculation unit and the position of the tracking area after movement determined by the determining unit compares the position of the tracking area before movement detected by the tracking area detecting unit calculates a movement amount obstacle detection device according to 1. The obstacle detection device according to claim 2 , wherein the area similarity calculation unit calculates a similarity based on a correlation value between the tracking area before the movement and the tracking area after the movement. The traceable area detection means includes an edge detection unit that detects an edge of the image;
A variance value calculating unit that calculates a variance value of a ratio of magnitudes of vertical and horizontal components of an edge detected by the edge detecting unit for each predetermined region of the image;
The determination unit according to any one of claims 1 to 3 , further comprising: a determination unit that determines a predetermined region in which the variance value calculated by the variance value calculation unit is greater than a predetermined value as a tracking region for tracking the movement of the obstacle. The obstacle detection apparatus according to 1.

Images