JP4062290B2 - Moving body detection device - Google Patents

Description

  The present invention relates to a moving body detection apparatus that detects a moving body based on a captured image.

  An apparatus that detects a moving body based on an image captured by a camera mounted on a vehicle is known (see Patent Document 1). In this conventional detection device, a horizontal edge component and a vertical edge component in a captured image are extracted, and based on the extracted edge component, a vertical edge component representing both side ends of the vehicle is detected.

Japanese Unexamined Patent Publication No. 2000-113201

  However, the conventional detection device detects a vertical edge component representing both ends of the vehicle based on conditions representing the vehicle (reference conditions relating to the vehicle width and height) from among the vertical edge components in the image. Therefore, if the type of vehicle or the positional relationship with the vehicle changes, the conditions that represent the vehicle also change, so that it is not possible to detect the edge component that represents the both ends of the vehicle, and the vehicle cannot be detected. There was a problem of having sex.

  The moving object detection device according to the present invention detects a trackable area in which a moving object can be tracked in a captured image divided into a plurality of areas, and the distance to the object reflected in the trackable area, the height, and Based on the distance traveled by the vehicle during a predetermined time, the amount of movement in the vertical direction is calculated out of the amount of movement that the traceable area moves on the captured image when the vehicle moves during the predetermined time. Based on the difference between the vertical movement amount calculated in this way and the vertical movement amount of the traceable area detected based on the captured image, it is determined whether or not the moving object is reflected in the traceable area. It is characterized by determining.

  According to the moving object detection device of the present invention, it is possible to reliably distinguish a moving object and a stationary object on a captured image.

  FIG. 1 is a diagram illustrating a configuration of a moving object detection device according to an embodiment. The moving body detection device in one embodiment is used by being mounted on a vehicle, and includes an imaging unit 10, an image storage unit 20, a trackable region detection unit 30, and a tracking region information storage unit 40. The local region tracking unit 50, the own vehicle movement amount detection unit 60, the distance index value calculation unit 70, the height index value calculation unit 80, and the movement region detection unit 90 are provided.

  The imaging unit 10 is, for example, a video camera, and images the front of the vehicle at regular time intervals. The camera is installed so that the optical axis in the left-right direction of the camera matches the straight direction of the vehicle, and the optical axis in the vertical direction is horizontal. An image captured by the imaging unit 10 is stored in the image storage unit 20.

  The traceable area detection unit 30 detects an area where the mobile body can be tracked based on the image captured by the imaging unit 10 and stored in the image storage unit 20. In a region with a uniform shading change, such as a white line on a road, even if the target (moving body) moves in a direction perpendicular to the shading direction (the direction in which the white line extends) on the image, The amount of movement cannot be detected. Therefore, the region where the target can be tracked is a region having a change in shading in a plurality of directions. A method for detecting such a region will be described.

  The traceable area detection unit 30 reads an image stored in the image storage unit 20, applies a filter for detecting vertical and horizontal edges, and determines the size of the vertical and horizontal components (dI / dx, dI / dy: I is a pixel value). Next, in order to remove the noise component included in the detected edge, a pixel whose sum of squares of the obtained vertical and horizontal edge components is greater than or equal to the threshold th1 is extracted. The density gradient direction is obtained by obtaining the ratio of the size of the vertical component and the horizontal component of the edge.

  Next, the image in which the edge after the noise is removed is divided into a lattice shape at equal intervals in the vertical direction and the horizontal direction. FIG. 2 is a diagram illustrating regions obtained by dividing the image in which the edge is detected from the captured image by the above-described method into a lattice shape in the vertical direction and the horizontal direction. For each of the divided areas, a variance value of the ratio of the size of the vertical component and the horizontal component of the edge is obtained. In this case, if the edge gradient direction in the divided rectangular area is only one direction, the variance value is small, and if it has edge gradients in a plurality of directions, the variance value is large. Therefore, if the obtained dispersion value is smaller than the predetermined threshold th2, it is highly likely that the edge gradient direction is one direction, and it is difficult to detect an object moving in a direction perpendicular to the density gradient. On the other hand, if the obtained dispersion value is equal to or greater than the predetermined threshold value th2, it has an edge gradient in a plurality of directions, and it can be determined that tracking is possible regardless of which direction the moving body moves. The traceable area detection unit 30 determines that an area having a calculated variance value equal to or greater than a predetermined threshold th2 is a traceable area. A rectangular area determined as a traceable area based on the captured image of FIG. 2 is shown in FIG.

  The traceable area detection unit 30 detects a new traceable area based on images captured by the imaging unit 10 at intervals of time Δt. The area determined as the traceable area by the traceable area detection unit 30 is the tracking area information in which the position of the area and the image pattern of the original image are associated with the identification number assigned to each rectangular area. Stored in the storage unit 40. Hereinafter, the image pattern stored in the tracking area information storage unit 40 is referred to as a reference pattern.

  When a new image captured by the imaging unit 10 is input to the image storage unit 20, the local region tracking unit 50 is detected by the traceable region detection unit 30 based on two images captured at different timings. The destination and amount of movement of the trackable area thus detected are detected. A method of detecting the movement destination and the movement amount will be described.

  A search area for detecting the movement destination is provided around the traceable area, and the correlation value with the new image is calculated while shifting the reference pattern position by one pixel within the search area. In this case, the calculated correlation value is equal to or greater than the threshold th3 and the maximum area is detected as the destination of the traceable area. Since the reference pattern position is moved (shifted) in units of one pixel, the position of the movement destination can be obtained only in units of one pixel. However, by performing an interpolation operation using a plurality of correlation values with respect to the surrounding area, correlation is performed. The position (movement destination) where the value is maximum can be obtained with an accuracy of 1 pixel or less. The movement amount of the traceable area calculated based on the obtained movement destination is expressed as each direction component in the horizontal direction x and the vertical direction y. FIG. 4 is a diagram showing the moving component of each trackable area by an arrow.

  The position and amount of movement determined by the local area tracking unit 50 are stored in the tracking area information storage unit 40 for each identification number. Therefore, the tracking area information storage unit 40 stores the identification number, the current coordinates, the movement vector, and the reference pattern for each trackable area. However, if the correlation value with the new image is less than the threshold th3, it cannot be determined that the same part in the image has moved, so information on the target traceable area is initialized. . Also, when the start point coordinates of the traceable area deviate from a certain range in the horizontal direction x and the vertical direction y, the information regarding the traceable area is initialized. This is because a moving object moving in a direction different from the moving direction of the stationary object or the own vehicle on the image moves from the vanishing point to the peripheral direction with the movement of the own vehicle. Because it goes out. With respect to the initialized traceable area, it is not determined whether or not the area is a moving object area.

  The own vehicle movement amount detection unit 60 detects the movement amount of the own vehicle during the imaging interval Δt by the following method. First, the number of rotations of the wheel during the imaging interval Δt is measured, and the product of this number of rotations and the outer peripheral length of the tire is calculated. In addition, if a plurality of trigger positions to the sensor for measuring the number of rotations of the wheel are set at an equal angle around one rotation of the wheel, the number of rotations can be measured with a resolution of one rotation or less.

The distance index value calculation unit 70 calculates the position of the tracking target in the trackable area. Specifically, the distance L from the center of the lens of the imaging unit 10 to the tracking target reflected in the trackable area is calculated. FIG. 5A is a diagram for explaining a method of calculating the distance L from the imaging unit 10 to the tracking target, and is a diagram looking down on the imaging unit 10 and the tracking target before and after the imaging interval Δt. . As shown in FIG. 5A, it is assumed that the host vehicle has moved by ΔL during the imaging interval Δt. Here, the focal length of the lens of the imaging unit 10 is f, the x coordinate on the image of the tracking target before the host vehicle moves is x1, the x coordinate of the tracking target after the host vehicle moves is x2, and the tracking target If the amount of movement in the x direction on the image is Δx (= x2−x1), the distance L is expressed by the following equation (1). Note that the horizontal axis in the captured image is the x-axis, and the vertical axis is the y-axis.
L = f + ΔL × x1 / Δx (1)

FIG. 5B is a diagram of the imaging unit 10 and the tracking target before and after the imaging interval Δt as viewed from the side. The height index value calculation unit 80 calculates the height H from the track surface to be tracked. Here, h1 is the height from the road surface to the optical axis of the imaging unit 10, h2 is the height to the optical axis of the imaging unit 10 when the height of the tracking target is used as a reference, and tracking before the host vehicle moves If the y coordinate on the target image is y1, and the y coordinate of the tracking target after the host vehicle moves during the time Δt is y2, the height H is expressed by the following equation (2).
H = h1-h2
= H1-y2 * L / f (2)

The moving area detection unit 90 determines whether the object shown in the trackable area is a stationary object or a moving object, that is, whether the trackable area is a stationary object area or a moving object area. From the relationship of FIG. 5B, if the amount of movement in the y direction on the image is Δy, Δy is expressed by the following equation (3).
Δy = f × ΔL × (h1−H) / (L × (L + ΔL)) (3)

  In Expression (3), the distance L to the tracking target is calculated from Expression (1), and the height H of the tracking target is calculated from Expression (2). In addition, since the values of h1 and ΔL can be obtained in advance, the values of h1 and ΔL on the image to be tracked can be obtained from the equation (3) using the values of equations (1) and (2) and h1 and ΔL. The amount of movement Δy in the direction can be calculated.

  When the traceable area is a stationary object area, the distance L obtained from the equation (1) and the height H obtained from the equation (2) coincide with the actual distance and height. The Δy obtained from the equation (3) and the movement amount Δy (= y2−y1) in the y direction obtained from the coordinate values on the image by the local region tracking unit 50 coincide with each other. However, when the trackable area is a moving object area, the distance L obtained from Expression (1) and the height H obtained from Expression (2) are different from the actual distance and height. The Δy obtained from the equation (3) does not coincide with the movement amount Δy in the y direction on the image obtained by the local region tracking unit 50.

  FIG. 6 is a diagram showing the relationship between the distance L to the tracking target and the movement amount Δy in the y direction obtained from the equation (3). When the height of the object is constant, the distance L to the object and the movement amount Δy in the y direction on the image are expressed by a certain relational expression (see Expression (3)). The relationship with the amount of movement Δy in the y direction can be represented by a curve as shown in FIG. Here, when the traceable area is a stationary object area, a point indicating the relationship between the distance L and the movement amount Δy in the y direction exists on the curve shown in FIG. 6 or in the vicinity of the curve. However, when the trackable area is a moving object area, the point indicating the relationship between the distance L and the movement amount Δy in the y direction exists at a position away from the curve shown in FIG.

  Accordingly, the moving region detection unit 90 obtains a difference between the movement amount Δy in the y direction obtained from the equation (3) and the movement amount Δy in the y direction obtained from the coordinate value on the image by the local region tracking unit 50. Based on the magnitude of the difference, it is determined whether the trackable area is a stationary object area or a moving object area. However, since it is difficult to accurately determine whether it is a stationary object region or a moving object region by calculating the difference described above once, the above difference is calculated a predetermined number of times, and the sum of the absolute values is a predetermined threshold value. If it is larger, it is determined as a moving object region, and if it is below a predetermined threshold value, it is determined as a stationary object region.

  FIG. 7 illustrates the difference between the movement amount Δy in the y direction obtained from Equation (3) and the movement amount Δy in the y direction obtained by the local region tracking unit 50 a plurality of times for the moving object region and the stationary object region. It is a figure which calculates | requires over and shows transition of the sum of the absolute value of the difference. As shown in FIG. 7, when the calculation is performed on the moving object region, the total value of the absolute values of the above differences exceeds a predetermined threshold value in the number of calculations n times. Therefore, the difference between the movement amount Δy in the y direction obtained from the equation (3) and the movement amount Δy in the y direction obtained by the local region tracking unit 50 is calculated a predetermined number of times m (m> n) times, By comparing the sum of absolute values with a predetermined threshold value, it is possible to determine whether the trackable area is a stationary object area or a moving object area.

  FIG. 8 is a diagram illustrating a result of classification into a moving object area and a stationary object area by the moving area detection unit 90. In FIG. 8, the moving object region is subjected to a shading process to be distinguished from the stationary object region.

  FIG. 9 is a flowchart illustrating a processing procedure performed by the moving object detection device according to the embodiment. The process starting from step S10 starts when an ignition switch (not shown) is turned on. In step S10, the tracking area information stored in the tracking area information storage unit 40 is initialized, and the process proceeds to step S20. In step S20, the image captured by the imaging unit 10 is stored in the image storage unit 20, and the process proceeds to step S30.

  In step S <b> 30, the traceable area detection unit 30 detects the traceable area based on the image stored in the image storage unit 20. Since the method for detecting the traceable area has been described above, a detailed description thereof will be omitted. When the traceable area is detected, the process proceeds to step S40. In step S40, the local area tracking unit 50 detects the movement destination and movement amount on the image in which the trackable area moves during the imaging interval Δt. Since the movement destination of the traceable area and the detection method of the movement amount have been described above, detailed description thereof will be omitted. When the movement destination and movement amount of the traceable area are detected, the process proceeds to step S50.

  In step S50, the distance index value calculation unit 70 calculates the distance L from the center of the lens of the imaging unit 10 to the tracking target. Since the calculation method of the distance L was mentioned above, detailed description is abbreviate | omitted. In step S60 following step S50, the height index value calculation unit 80 calculates the height H from the track surface to be tracked. Since the method for calculating the height H has been described above, a detailed description thereof will be omitted. When the height H of the tracking target is calculated, the process proceeds to step S70.

  In step S70, the moving area detection unit 90 determines whether the trackable area detected in step S30 is a stationary object area or a moving object area. Since this determination method has also been described above, a detailed description thereof will be omitted. Thereafter, the processing after step S20 is repeatedly performed at a cycle of the imaging interval Δt.

  When the moving area detection unit 90 determines whether the trackable area is a stationary object area or a moving object area, for example, the distance and direction between the building determined to be a stationary object and the host vehicle. It is possible to obtain a positional relationship such as, and to measure the distance between the other vehicle determined to be a moving object and the host vehicle.

  A method of detecting a moving body region by the moving body region detection apparatus in one embodiment will be summarized. The traceable area detection unit 30 divides the image captured by the imaging unit 10 into a plurality of areas, and detects a traceable area where the mobile body can be tracked for each area. The local region tracking unit 50 detects the movement amount of the trackable region on the image based on at least two images captured by the imaging unit 10 at an imaging interval of Δt. The distance index value calculation unit 70 calculates the distance ΔL that the host vehicle moves between the focal length f of the lens of the imaging unit 10 and the imaging interval Δt, and the x of the trackable region detected by the local region tracking unit 50. Based on the amount of movement Δx in the direction, the distance L to the tracking target is calculated from the above equation (1). The height index value calculation unit 80 is based on the height h1 of the optical axis of the imaging unit 10, the focal length f of the lens, the distance L calculated by the equation (1), and the y coordinate value of the trackable area. From formula (2), the height H to the tracking target is calculated. The moving area detection unit 90 uses the distance L calculated by Expression (1) and the height H calculated by Expression (2) to obtain an image of the trackable area between Δt according to Expression (3). The distance Δy that moves in the upper y direction is calculated, and tracking is possible based on the difference between this Δy and the amount of movement Δy in the y direction calculated by the local region tracking unit 50 based on the coordinate value on the image. It is determined whether the area is a moving object area or a stationary object area.

  According to the moving body region detection device in one embodiment, it is possible to detect a moving body based on a captured image regardless of the type of the moving body and the positional relationship between the moving body and the imaging unit 10. In a conventional apparatus for detecting a moving object based on the characteristics of edge components in an image, if a building having an edge similar to the edge constituting the moving object exists in the image, that building is used as the moving object. There is a possibility of false detection. However, in the moving body region detection apparatus according to the embodiment, even when a building or the like exists in the image, it is possible to reliably detect the region where the moving body is shown as the moving body region.

  The present invention is not limited to the embodiment described above. For example, the moving region detection unit 90 calculates a plurality of differences between the movement amount Δy in the y direction obtained from Expression (3) and the movement amount Δy in the y direction obtained from the coordinate values on the image by the local region tracking unit 50. If the sum of the absolute values of the differences is greater than a predetermined threshold value, the moving object region is determined. However, the above-described difference is calculated only once, and the absolute value of the difference is determined to be the predetermined value. If it is larger than the threshold value, it may be determined as a moving object region. However, as described above, it is possible to improve the determination accuracy between the stationary object region and the moving object region by calculating the difference a plurality of times and comparing the sum of the absolute values of the differences with a predetermined threshold value. Can do.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the imaging unit 10 is an imaging unit, the trackable region detection unit 30 is a region classification unit and a trackable region detection unit, the local region tracking unit 50 is a trackable region movement amount detection unit, and the vehicle movement amount detection unit 60 is. Is a vehicle movement amount detection unit, a distance index value calculation unit 70 is a distance calculation unit, a height index value calculation unit 80 is a height calculation unit, and a moving region detection unit 90 is a determination unit. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.

The figure which shows the structure of the mobile body detection apparatus in one embodiment. The figure which shows the several area | region divided | segmented into the grid | lattice form in the vertical direction and the horizontal direction with respect to the image which detected the edge from the captured image The figure which shows the rectangular area | region judged to be a traceable area | region based on the captured image of FIG. Diagram showing moving components of each traceable area with arrows FIG. 5A is a view of the imaging unit and the tracking target before and after the imaging interval Δt, and FIG. 5B is a side view of the imaging unit and the tracking target before and after the imaging interval Δt. Figure The figure which shows the relationship between the distance L to the tracking object in a certain height, and the moving amount (DELTA) y of the y direction of the tracking object on an image. For a moving object region and a stationary object region, a difference between a movement amount Δy in the y direction obtained by calculation and a movement amount Δy in the y direction obtained from a difference between coordinate values of two points on the image is obtained a plurality of times. , A diagram showing the transition of the sum of absolute values of the differences The figure which shows the result classified into the moving object area | region and the stationary object area | region by the moving area detection part. The flowchart which shows the process sequence performed by the mobile body detection apparatus in one embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Imaging part 20 ... Image memory | storage part 30 ... Traceable area | region detection part 40 ... Tracking area | region information storage part 50 ... Local area | region tracking part 60 ... Own-vehicle movement amount detection part 70 ... Distance index value calculation part 80 ... Height index value Calculation unit 90 ... moving region detection unit

Claims (4)

An imaging means installed in the vehicle for imaging the surroundings of the vehicle;
Area dividing means for dividing the image captured by the imaging means into a plurality of areas;
A trackable area detecting means for detecting a trackable area capable of tracking a moving body among the areas divided by the area dividing means;
The tracking is possible based on a first image captured at a first timing by the imaging unit and a second image captured at a second timing after a predetermined time has elapsed from the first timing. A traceable area movement amount detection means for detecting a movement amount by which the traceable area detected by the area detection means moves on the captured image;
Own vehicle movement amount detecting means for detecting a distance that the own vehicle moves during the predetermined time;
Based on the own vehicle movement amount detected by the own vehicle movement amount detection means and the movement amount detected by the trackable area movement amount detection means, from the imaging means to the object shown in the trackable area Distance calculating means for calculating the distance of;
Height for calculating the height in real space of the object shown in the trackable area based on the distance calculated by the distance calculation means and the vertical position on the captured image of the trackable area A calculation means;
Based on at least the own vehicle movement amount detected by the own vehicle movement amount detection unit, the distance calculated by the distance calculation unit, and the height calculated by the height calculation unit, the vehicle A vertical movement amount calculating means for calculating a vertical movement amount of the traceable region on the captured image when moved in between;
Based on the difference between the vertical movement amount of the movement amount on the captured image detected by the traceable region movement amount detection means and the vertical movement amount calculated by the vertical movement amount calculation means, A moving body detection apparatus comprising: a determination unit that determines whether or not a moving object is reflected in the traceable area. The moving object detection apparatus according to claim 1,
The determination means includes a vertical movement amount of movement amounts on the captured image detected by the traceable area movement amount detection means and a vertical movement amount calculated by the vertical movement amount calculation means. A moving body detection apparatus, wherein when the difference is equal to or greater than a predetermined threshold value, it is determined that a moving object is reflected in the traceable area. The moving object detection apparatus according to claim 1,
The determination means includes a vertical movement amount of movement amounts on the captured image detected by the traceable area movement amount detection means and a vertical movement amount calculated by the vertical movement amount calculation means. The difference is calculated a plurality of times using different captured images, and when the sum of the absolute values of the calculated differences is equal to or greater than a predetermined threshold, it is determined that the moving object is reflected in the trackable area A moving body detection apparatus characterized by In the moving body detection apparatus in any one of Claims 1-3,
The trackable area detecting unit detects an area having light and shade changes in a plurality of directions as the trackable area.

Images