JP3988758B2 - Moving body detection device - Google Patents

Description

  The present invention relates to a moving body detection apparatus that detects a moving body based on a captured image.

  An apparatus that detects a moving body based on an image captured by a camera mounted on a vehicle is known (see Patent Document 1). In this conventional detection device, a horizontal edge component and a vertical edge component in a captured image are extracted, and based on the extracted edge component, a vertical edge component representing both side ends of the vehicle is detected.

Japanese Unexamined Patent Publication No. 2000-113201

  However, the conventional detection device detects a vertical edge component representing both ends of the vehicle based on conditions representing the vehicle (reference conditions relating to the vehicle width and height) from among the vertical edge components in the image. Therefore, if the type of vehicle or the positional relationship with the vehicle changes, the conditions that represent the vehicle also change, so the edge component that represents both ends of the vehicle cannot be detected, and the vehicle in the captured image is captured. There was a problem that it may not be possible to identify the area.

  The moving body detection apparatus according to the present invention first divides a captured image into a plurality of areas, and detects a trackable area in which the moving body can be tracked among the divided areas. Then, when the vehicle moves for a predetermined time, it moves on the captured image when it is assumed that the trackable area moves on the captured image and the object in the trackable area is a stationary object. Based on the moving vector to be detected, the moving object region in which the moving object is reflected is detected from the trackable regions. The distance in real space to the object in the trackable area based on the amount of movement of the vehicle during the predetermined time and the amount of movement of the trackable area on the captured image during the predetermined time An index value is calculated, and the moving object area is classified for each area belonging to the same moving object based on the distance index value.

  According to the moving object detection device of the present invention, a moving object region in which a moving object is reflected is detected from the captured image divided into a plurality of areas, and the plurality of detected moving object areas are detected as the same moving object. Therefore, the moving object in the captured image can be reliably detected.

  FIG. 1 is a diagram illustrating a configuration of a moving object detection device according to an embodiment. The moving body detection device in one embodiment is used by being mounted on a vehicle, and includes an imaging unit 10, an image storage unit 20, a trackable region detection unit 30, and a tracking region information storage unit 40. The local region tracking unit 50, the own vehicle movement amount detection unit 60, the distance index value calculation unit 70, the movement region detection unit 80, the region classification unit 90, the distance calculation unit 100, and the speed calculation unit 110 Prepare.

  The imaging unit 10 is, for example, a video camera, and images the front of the vehicle at regular time intervals. The camera is installed so that the optical axis in the left-right direction of the camera matches the straight direction of the vehicle, and the optical axis in the vertical direction is horizontal. An image captured by the imaging unit 10 is stored in the image storage unit 20.

  The traceable area detection unit 30 detects an area where the mobile body can be tracked based on the image captured by the imaging unit 10 and stored in the image storage unit 20. In a region with a uniform shading change, such as a white line on a road, even if the target (moving body) moves in a direction perpendicular to the shading direction (the direction in which the white line extends) on the image, The amount of movement cannot be detected. Therefore, the region where the target can be tracked is a region having a change in shading in a plurality of directions. A method for detecting such a region will be described.

  The traceable area detection unit 30 reads an image stored in the image storage unit 20, applies a filter for detecting vertical and horizontal edges, and determines the size of the vertical and horizontal components (dI / dx, dI / dy: I is a pixel value). Next, in order to remove the noise component included in the detected edge, a pixel whose sum of squares of the obtained vertical and horizontal edge components is greater than or equal to the threshold th1 is extracted. The density gradient direction is obtained by obtaining the ratio of the size of the vertical component and the horizontal component of the edge.

  Next, the image in which the edge after the noise is removed is divided into a lattice shape at equal intervals in the vertical direction and the horizontal direction. FIG. 2 is a diagram illustrating regions obtained by dividing the image in which the edge is detected from the captured image by the above-described method into a lattice shape in the vertical direction and the horizontal direction. For each of the divided areas, a variance value of the ratio of the size of the vertical component and the horizontal component of the edge is obtained. In this case, if the edge gradient direction in the divided rectangular area is only one direction, the variance value is small, and if it has edge gradients in a plurality of directions, the variance value is large. Therefore, if the obtained dispersion value is smaller than the predetermined threshold th2, it is highly likely that the edge gradient direction is one direction, and it is difficult to detect an object moving in a direction perpendicular to the density gradient. On the other hand, if the obtained dispersion value is equal to or greater than the predetermined threshold value th2, it has an edge gradient in a plurality of directions, and it can be determined that tracking is possible regardless of which direction the moving body moves. The traceable area detection unit 30 determines that an area having a calculated variance value equal to or greater than a predetermined threshold th2 is a traceable area. A rectangular area determined as a traceable area based on the captured image of FIG. 2 is shown in FIG.

  The traceable area detection unit 30 detects a new traceable area based on images captured by the imaging unit 10 at intervals of time Δt. The area determined as the traceable area by the traceable area detection unit 30 is the tracking area information in which the position of the area and the image pattern of the original image are associated with the identification number assigned to each rectangular area. Stored in the storage unit 40. Hereinafter, the image pattern stored in the tracking area information storage unit 40 is referred to as a reference pattern.

  The local region tracking unit 50 uses the information stored in the tracking region information storage unit 40 and the new image captured by the imaging unit 10 to detect the trackable region detected by the trackable region detection unit 30. Detect the destination and amount of movement. A method of detecting the movement destination and the movement amount will be described.

  A search area for detecting the amount of movement is provided around the traceable area, and the correlation value with the new image is calculated while shifting the reference pattern position by one pixel within the search area. In this case, the calculated correlation value is equal to or greater than the threshold th3 and the maximum area is detected as the destination of the traceable area. Since the reference pattern position is moved (shifted) in units of one pixel, the position of the movement destination can be obtained only in units of one pixel. However, by performing an interpolation operation using a plurality of correlation values with respect to the surrounding area, correlation is performed. The position (movement destination) where the value is maximum can be obtained with an accuracy of 1 pixel or less. The movement amount of the traceable area calculated based on the obtained movement destination is expressed as each direction component in the horizontal direction x and the vertical direction y. FIG. 4 is a diagram showing the moving component of each trackable area by an arrow.

  The position and amount of movement determined by the local area tracking unit 50 are stored in the tracking area information storage unit 40 for each identification number. Therefore, the tracking area information storage unit 40 stores the identification number, the current coordinates, the movement vector, and the reference pattern for each trackable area. However, if the correlation value with the new image is less than the threshold th3, it cannot be determined that the same part in the image has moved, so information on the target traceable area is initialized. . Also, when the start point coordinates of the traceable area deviate from a certain range in the horizontal direction x and the vertical direction y, the information regarding the traceable area is initialized. This is because a moving object moving in a direction different from the moving direction of the stationary object or the own vehicle on the image moves from the vanishing point to the peripheral direction with the movement of the own vehicle. Because it goes out. With respect to the initialized traceable area, it is not determined whether or not the area is a moving object area.

  The own vehicle movement amount detection unit 60 detects the movement amount of the own vehicle during the imaging interval Δt by the following method. First, the number of rotations of the wheel during the imaging interval Δt is measured, and the product of this number of rotations and the outer peripheral length of the tire is calculated. In addition, if a plurality of trigger positions to the sensor for measuring the number of rotations of the wheel are set at an equal angle around one rotation of the wheel, the number of rotations can be measured with a resolution of one rotation or less.

The distance index value calculation unit 70 calculates the position of the tracking target in the trackable area based on the information stored in the tracking area information storage unit 40. Specifically, the distance L from the center of the lens of the imaging unit 10 to the tracking target reflected in the trackable area is calculated. FIG. 5 is a diagram for explaining a method of calculating the distance L from the imaging unit 10 to the tracking target, and is a diagram of the imaging unit 10 and the tracking target before and after the imaging interval Δt viewed from the side. As shown in FIG. 5, it is assumed that the host vehicle has moved by ΔL during the imaging interval Δt. Here, the focal length of the lens of the imaging unit 10 is f, the y coordinate on the image to be tracked before the own vehicle moves (time T1), y1, and the tracking target after the own vehicle moves (time T2). If the y coordinate is y2, and the amount of movement in the y direction on the image to be tracked is Δy (= y2−y1), the distance L is expressed by the following equation (1). Note that the horizontal axis in the captured image is the x-axis and the vertical axis is the y-axis.
L = f + ΔL × y1 / Δy (1)

  The moving area detection unit 80 detects a moving object area in which a moving object is shown among the trackable areas stored in the tracking area information storage unit 40. A method for detecting the moving object region will be described. First, the vanishing point is obtained from the movement vector (see FIG. 4) of the traceable area detected by the local area tracking unit 50. That is, a plurality of position coordinates of points where straight lines obtained by extending the movement vectors of the respective traceable areas intersect are obtained, and a position where the frequency of the obtained plurality of coordinate values is maximized is set as a vanishing point. FIG. 6 is a diagram showing vanishing points obtained based on the movement vector shown in FIG.

  Next, when it is assumed that the object shown in each trackable area is a stationary object, a movement vector of the trackable area obtained by the host vehicle traveling during the imaging interval Δt is obtained. This movement vector is uniquely determined according to the position of the traceable area on the captured image and the amount of movement of the host vehicle. The moving region detection unit 80 calculates a difference between the movement vector (see FIG. 4) of each trackable region detected by the local region tracking unit 50 and the movement vector when each trackable region is assumed to be a stationary object region. By doing this, it is determined whether the trackable area is a moving object area or a stationary object area. That is, when the trackable area is a stationary object area, the above-described difference between the movement vectors is zero, but is not zero in the case of a moving object area.

  FIG. 7 is a diagram illustrating a result of obtaining the above-described difference between the movement vectors for each trackable region based on the movement vector illustrated in FIG. Actually, as shown in FIG. 7, the difference in the movement vector may not become zero even in the stationary object region due to the influence of the detection error of the movement vector. Therefore, the movement area detection unit 80 compares the absolute value of the magnitude of the above-described movement vector difference with a predetermined threshold value, and the absolute value of the magnitude of the movement vector difference is greater than or equal to the predetermined threshold value. In addition, the trackable area is determined to be a moving object area, and if the trackable area is smaller than a predetermined threshold value, the trackable area is determined to be a stationary object area.

  The area classification unit 90 determines whether or not the moving object area detected by the moving area detection unit 80 is an area belonging to the same moving object based on the distance L calculated by the distance index value calculation unit 70, that is, It is determined whether or not the moving objects shown in the plurality of areas determined as the moving object area are the same moving object. FIG. 8 shows (i) a moving object traveling in the same direction as the own vehicle at a slower speed than the own vehicle, (ii) a moving object proceeding in the same direction as the own vehicle at a higher speed than the own vehicle, It is a figure which shows distribution of the distance data calculated by the distance index value calculation part 70 for the moving object which is advancing in the opposite direction to a vehicle.

  As described above, the distance index value calculation unit 70 calculates the distance L to the moving object from Expression (1). When the distance L is calculated for the same moving object at the same distance from the host vehicle, the distance L calculated by the equation (1) is the same value. However, when the distance L is calculated for different moving objects at the same distance from the own vehicle, the movement amounts Δy in the y direction on the images of the moving objects (i) to (iii) are different from each other. Therefore, the distance L calculated by the equation (1) also has different values.

  That is, when the data of the distance L is obtained for each moving object area, the areas belonging to the same moving object have a distribution having data within a certain range with a certain distance as a peak, as shown in FIG. Therefore, the area classification unit 90 determines whether each moving object area is an area where the same moving object is imaged based on the frequency distribution of the distance L calculated for each moving object area. it can. FIG. 9 is a diagram illustrating a result of classification by the region classification unit 90 for each identical moving object. In FIG. 9, it is classified into two areas.

  As described above, when the distance to the moving object is calculated using the equation (1), an error occurs from the actual distance, but for the region at the same distance moving at the same speed, The error is the same. Therefore, when the frequency distribution is obtained with the distance L obtained from Equation (1) as the horizontal axis, the regions belonging to the same moving object become a distribution concentrated in one place as shown in FIG.

The distance calculation unit 100 calculates the distance S between the moving object in the area determined to belong to the same moving object by the area classification unit 90 and the host vehicle. That is, among the plurality of moving object areas determined to belong to the same moving object, the y coordinate y2 of the area at the lowest position on the captured image (the area with the smallest y coordinate value) is obtained, and the following equation ( From 2), the distance S is calculated.
S = h × f / y2 (2)
However, f is the focal length of the lens of the imaging unit 10, and h is the height from the road surface to the optical axis of the imaging unit 10.

  The reason why the y-coordinate y2 of the region at the lowest position on the captured image is obtained is because the lowest position of the moving object is a position in contact with the road surface. Unlike the distance calculation method of equation (1), the method of calculating the distance S using the equation (2) by the distance calculation unit 100 can accurately calculate the distance even if the distance detection target is a moving object. . FIG. 10 is a diagram showing a relationship between the distance L calculated by the equation (1) and the distance S calculated by the equation (2) when the object shown in the trackable area is a moving object. .

The speed calculation unit 110 uses the distance L calculated by the distance index value calculation unit 70, the own vehicle movement amount ΔL detected by the own vehicle movement amount detection unit 60, and the distance S calculated by the distance calculation unit 100. The velocity V of the moving object is calculated from the following equation (3).
V = S × [−1 / {(L−f) × ΔL} + f × y2 / y1 + ΔL] / Δt (3)
However, y1 is the y coordinate on the image of the tracking target before the own vehicle moves (time T1), and y2 is the tracking target after the own vehicle moves (time T2) during the imaging interval Δt. The y coordinate.

  FIG. 11 is a flowchart illustrating a processing procedure performed by the moving object detection device according to the embodiment. The process starting from step S10 starts when an ignition switch (not shown) is turned on. In step S10, the tracking area information stored in the tracking area information storage unit 40 is initialized, and the process proceeds to step S20. In step S20, the image captured by the imaging unit 10 is stored in the image storage unit 20, and the process proceeds to step S30.

  In step S <b> 30, the traceable area detection unit 30 detects the traceable area based on the image stored in the image storage unit 20. Since the method for detecting the traceable area has been described above, a detailed description thereof will be omitted. When the traceable area is detected, the process proceeds to step S40. In step S40, the local region tracking unit 50 detects a movement amount (movement vector) on the image in which the trackable region moves during the imaging interval Δt. Since the movement vector detection method has been described above, a detailed description thereof will be omitted. When the movement vector of the traceable area is detected, the process proceeds to step S50.

  In step S50, the movement area detecting unit 80 obtains a movement vector in which the stationary object moves during the imaging interval Δt for each trackable area, and based on the movement vector and the movement vector obtained in step S40. The moving object area is detected from the traceable area. Since the moving object region detection method has been described above, a detailed description thereof will be omitted.

  In step S60, the distance index value calculation unit 70 calculates the distance L to the object shown in all the moving object areas detected in step S50. Since the calculation method of the distance L was mentioned above, detailed description is abbreviate | omitted. When the distance L is calculated, the process proceeds to step S70. In step S70, the area classification unit 90 determines whether or not the moving objects shown in the plurality of moving object areas belong to the same object based on the distance L calculated for each moving object area. Classify the area. When the moving object area is classified, the process proceeds to step S80.

  In step S80, the distance calculation unit 100 calculates the distance S to the moving object shown in the moving object region classified by the region classification unit 90. Since the method for calculating the distance S to the moving object has been described above, a detailed description thereof will be omitted. In step S90 following step S80, the velocity calculation unit 110 calculates the velocity V of the moving object. Since the method for calculating the velocity V of the moving object has been described above, a detailed description thereof will be omitted.

  The contents of processing performed by the mobile body detection device in one embodiment will be summarized. The traceable area detection unit 30 divides the image captured by the imaging unit 10 into a plurality of areas, and detects a traceable area where the mobile body can be tracked for each area. The local area tracking unit 50 detects a movement vector of the traceable area based on at least two images captured by the imaging unit 10 at an imaging interval of Δt. When it is assumed that the object shown in the traceable area is a stationary object, the movement area detection unit 80 calculates a movement vector in which the traceable area moves on the captured image at the imaging interval Δt. The moving object region is detected based on the movement vector detected by the local region tracking unit 50. The region classifying unit 90 calculates, for each region where the same moving object is imaged, based on the distance L to the moving object shown in the moving object region, which is calculated by the distance index value calculating unit 70 using the equation (1). Classify moving object areas. After that, the distance calculation unit 100 calculates the distance S to the moving object from the coordinate value of the area where the same moving object is reflected, using Equation (2). Moreover, the speed calculation part 110 calculates the moving speed V of a moving object from Formula (3).

  According to the moving object detection device in the embodiment, the presence of the moving object can be detected based on the captured image regardless of the type of the moving object and the positional relationship between the moving object and the imaging unit 10. The moving object region on the detected image can be classified for each region belonging to the same moving object.

  In addition, according to the moving object detection device in one embodiment, based on the coordinate value of the area at the lowest position on the captured image among the plurality of areas determined to belong to the same moving object, Since the distance to the moving object in space is calculated, the distance can be accurately calculated even when the object whose distance is to be calculated is a moving object.

  Furthermore, according to the moving object detection apparatus in the embodiment, the moving speed of the moving object is based on at least the distance L calculated by the distance index value calculation unit 70 and the distance S calculated by the distance calculation unit 100. Since V is calculated, the speed of the moving object can be accurately calculated.

  The present invention is not limited to the above-described embodiment, and modified examples in a range that does not impair the essence of the invention are included in the technical scope of the present invention.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the imaging unit 10 is an imaging unit, the trackable region detection unit 30 is a region segmentation unit and a trackable region detection unit, the local region tracking unit 50 is a movement vector detection unit, and the moving region detection unit 80 is a moving object region detection. The vehicle movement amount detection unit 60 is a vehicle movement amount detection unit, the distance index value calculation unit 70 is a distance index value calculation unit, the region classification unit 90 is a region classification unit, and the distance calculation unit 100 calculates a distance. The speed calculation unit 110 constitutes a speed calculation means. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.

The figure which shows the structure of the mobile body detection apparatus in one embodiment. The figure which shows the several area | region divided | segmented into the grid | lattice form in the vertical direction and the horizontal direction with respect to the image which detected the edge from the captured image The figure which shows the rectangular area | region judged to be a traceable area | region based on the captured image of FIG. Diagram showing moving components of each traceable area with arrows The figure which looked at the image pick-up part and tracking object before and behind imaging interval deltat from the side The figure which shows the vanishing point calculated | required based on the movement vector shown in FIG. The figure which shows the result which calculated | required the difference of the movement vector mentioned above for every trackable area | region For moving objects traveling in the same direction as the own vehicle at a slower speed than the own vehicle, moving objects proceeding in the same direction as the own vehicle at a speed faster than the own vehicle, and moving objects proceeding in the opposite direction to the own vehicle The figure which shows distribution of the distance data calculated by the distance index value calculation part 70 The figure which shows the result of having classified the plural moving object area for every moving object The figure which shows the relationship between the distance L calculated by Formula (1), and the distance S calculated by Formula (2) when the target object reflected in the trackable area is a moving object. The flowchart which shows the process sequence performed by the mobile body detection apparatus in one embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Imaging part 20 ... Image memory | storage part 30 ... Traceable area | region detection part 40 ... Tracking area | region information storage part 50 ... Local area | region tracking part 60 ... Own-vehicle movement amount detection part 70 ... Distance index value calculation part 80 ... Movement area | region detection part 90 ... area classification unit 100 ... distance calculation unit 110 ... speed calculation unit

Claims (6)

An imaging means installed in the vehicle for imaging the surroundings of the vehicle;
Area dividing means for dividing the image captured by the imaging means into a plurality of areas;
A trackable area detecting means for detecting a trackable area capable of tracking a moving body among the areas divided by the area dividing means;
The tracking is possible based on a first image captured at a first timing by the imaging unit and a second image captured at a second timing after a predetermined time has elapsed from the first timing. Movement vector detection means for detecting movement vectors of a plurality of trackable areas detected by the area detection means;
When it is assumed that the movement vector detected by the movement vector detection means and the object reflected in the trackable area are stationary objects, the movement vector is obtained based on the first image and the second image. A moving object region detecting means for detecting a moving object region in which a moving object is reflected from the trackable region based on a movement vector of the trackable region;
Own vehicle movement amount detection means for detecting the distance that the vehicle moves during the predetermined time;
Based on the own vehicle movement amount detected by the own vehicle movement amount detection unit and the movement amount on the image of the trackable region detected by the movement vector detection unit, the object up to the object shown in the trackable region is detected. A distance index value calculating means for calculating a distance in real space;
Area classification means for classifying the moving object area detected by the moving object area detection means for each area belonging to the same moving object based on the distance index value calculated by the distance index value calculation means; A moving body detecting apparatus characterized by the above. The moving object detection apparatus according to claim 1,
The moving object area detecting means is the absolute magnitude of the difference between the moving vector detected by the moving vector detecting means and the moving vector when it is assumed that the object shown in the trackable area is a stationary object. A moving body detection apparatus, wherein when the value is equal to or greater than a predetermined value, the traceable area is determined to be a moving object area. The moving body detection device according to claim 1 or 2,
The area classification means obtains a distribution of distance index values calculated by the distance index value calculation means for each moving object area, and moves for each area belonging to the same moving object according to an area where the distribution is concentrated. A moving object detection apparatus characterized by classifying an object area. In the moving body detection apparatus in any one of Claims 1-3,
The distance to the moving object in the real space is calculated based on the coordinate value of the area at the lowest position on the captured image among the plurality of areas determined to belong to the same moving object by the area classifying means. A moving body detecting apparatus further comprising a distance calculating means. In the moving body detection apparatus according to claim 4,
The movement further comprising speed calculation means for calculating a moving speed of the moving object based on at least the distance index value calculated by the distance index value calculation means and the distance calculated by the distance calculation means. Body detection device. In the moving body detection apparatus in any one of Claims 1-5,
The trackable area detecting unit detects an area having light and shade changes in a plurality of directions as the trackable area.

Images