JP3992033B2 - Image area classification device - Google Patents

Description

  The present invention relates to an image region classification device that classifies each region in which an identical object is captured in an image captured by an imaging unit.

  An apparatus that detects a moving body based on an image captured by a camera mounted on a vehicle is known (see Patent Document 1). In this conventional detection device, a horizontal edge component and a vertical edge component in a captured image are extracted, and based on the extracted edge component, a vertical edge component representing both side ends of the vehicle is detected.

Japanese Unexamined Patent Publication No. 2000-113201

  However, the conventional detection device detects a vertical edge component representing both ends of the vehicle based on conditions representing the vehicle (reference conditions relating to the vehicle width and height) from among the vertical edge components in the image. Therefore, when the type of vehicle and the positional relationship with the vehicle change, the conditions representing the vehicle also change, and thus there is a problem in that an image region in which the vehicle is captured cannot be specified in the captured image.

  An image region classification apparatus according to the present invention detects a trackable region in which a moving body can be tracked in a captured image divided into a plurality of regions, and targets two different trackable regions in the height direction in real space. It is determined whether or not the two traceable areas where the ratio of the coordinate values is calculated are areas where the same object is imaged based on the temporal change amount of the ratio of the coordinate values on the image corresponding to Features.

  According to the image region classification device of the present invention, the image region where the same object is imaged can be determined based on the temporal change amount of the ratio of the coordinate values of two different trackable regions. It is possible to specify an image region in which the same object is shown regardless of characteristics such as the structure of the existing object.

  FIG. 1 is a diagram illustrating a configuration of an image region classification device according to an embodiment. The image region classification device according to the embodiment includes an imaging unit 10, an image storage unit 20, a trackable region detection unit 30, a tracking region information storage unit 40, a local region tracking unit 50, and a height ratio calculation unit. 60 and an area classification unit 70.

  The imaging unit 10 is, for example, a video camera, and images the front of the vehicle at regular time intervals. The camera is installed so that the optical axis in the left-right direction of the camera matches the straight direction of the vehicle, and the optical axis in the vertical direction is horizontal. An image captured by the imaging unit 10 is stored in the image storage unit 20.

  The traceable area detection unit 30 detects an area where the mobile body can be tracked based on the image captured by the imaging unit 10 and stored in the image storage unit 20. In a region with a uniform shading change, such as a white line on a road, even if the target (moving body) moves in a direction perpendicular to the shading direction (the direction in which the white line extends) on the image, The amount of movement cannot be detected. Therefore, the region where the target can be tracked is a region having a change in shading in a plurality of directions. A method for detecting such a region will be described.

  The traceable area detection unit 30 reads an image stored in the image storage unit 20, applies a filter for detecting vertical and horizontal edges, and determines the size of the vertical and horizontal components (dI / dx, dI / dy: I is a pixel value). Next, in order to remove the noise component included in the detected edge, a pixel whose sum of squares of the obtained vertical and horizontal edge components is greater than or equal to the threshold th1 is extracted. The density gradient direction is obtained by obtaining the ratio of the size of the vertical component and the horizontal component of the edge.

  Next, the image in which the edge after the noise is removed is divided into a lattice shape at equal intervals in the vertical direction and the horizontal direction. FIG. 2 is a diagram illustrating regions obtained by dividing the image in which the edge is detected from the captured image by the above-described method into a lattice shape in the vertical direction and the horizontal direction. For each of the divided areas, a variance value of the ratio of the size of the vertical component and the horizontal component of the edge is obtained. In this case, if the edge gradient direction in the divided rectangular area is only one direction, the variance value is small, and if it has edge gradients in a plurality of directions, the variance value is large. Therefore, if the obtained dispersion value is smaller than the predetermined threshold th2, it is highly likely that the edge gradient direction is one direction, and it is difficult to detect an object moving in a direction perpendicular to the density gradient. On the other hand, if the obtained dispersion value is equal to or greater than the predetermined threshold value th2, it has an edge gradient in a plurality of directions, and it can be determined that tracking is possible regardless of which direction the moving body moves. The traceable area detection unit 30 determines that an area having a calculated variance value equal to or greater than a predetermined threshold th2 is a traceable area. A rectangular area determined as a traceable area based on the captured image of FIG. 2 is shown in FIG.

  The traceable area detection unit 30 detects a new traceable area based on images captured by the imaging unit 10 at intervals of time Δt. The area determined as the traceable area by the traceable area detection unit 30 is the tracking area information in which the position of the area and the image pattern of the original image are associated with the identification number assigned to each rectangular area. Stored in the storage unit 40. Hereinafter, the image pattern stored in the tracking area information storage unit 40 is referred to as a reference pattern.

  When a new image captured by the imaging unit 10 is input to the image storage unit 20, the local region tracking unit 50 is detected by the traceable region detection unit 30 based on two images captured at different timings. The destination and amount of movement of the trackable area thus detected are detected. A method of detecting the movement destination and the movement amount will be described.

  A search area for detecting the movement destination is provided around the traceable area, and the correlation value with the new image is calculated while shifting the reference pattern position by one pixel within the search area. In this case, the calculated correlation value is equal to or greater than the threshold th3 and the maximum area is detected as the destination of the traceable area. Since the reference pattern position is moved (shifted) in units of one pixel, the position of the movement destination can be obtained only in units of one pixel. However, by performing an interpolation operation using a plurality of correlation values with respect to the surrounding area, correlation is performed. The position (movement destination) where the value is maximum can be obtained with an accuracy of 1 pixel or less. The movement amount of the traceable area calculated based on the obtained movement destination is expressed as each direction component in the horizontal direction x and the vertical direction y.

  The position and amount of movement determined by the local area tracking unit 50 are stored in the tracking area information storage unit 40 for each identification number. Accordingly, the tracking area information storage unit 40 stores an identification number, current coordinates, and reference pattern for each trackable area, and also stores a classification number described later. However, if the correlation value with the new image is less than the threshold th3, it cannot be determined that the same part in the image has moved, so information on the target traceable area is initialized. . Also, when the start point coordinates of the traceable area deviate from a certain range in the horizontal direction x and the vertical direction y, the information regarding the traceable area is initialized. This is because a moving object moving in a direction different from the moving direction of the stationary object or the own vehicle on the image moves from the vanishing point to the peripheral direction with the movement of the own vehicle. Because it goes out. Regarding the initialized traceable area, it is not determined whether the area is a moving body area, that is, a moving body area.

The height ratio calculation unit 60 can perform different tracking based on the y coordinate value on the image of the trackable area stored in the tracking area information storage unit 40 (coordinate value corresponding to the height direction in the real space). The ratio of the y coordinate values of the area is calculated. FIG. 4 is a model diagram of the tracking target and the imaging unit 10 viewed from the side. The height from the road surface to the optical axis of the imaging unit 10 is Hc, the height from the road surface to the tracking portion of the object is H, the distance from the lens of the imaging unit 10 to the object is L, and the focal length of the lens is f. Then, following Formula (1) is materialized. Y in the formula (1) is a y-coordinate value on the image of the tracking region.
Hc−H = y × L / f (1)

Here, the distance to the first tracking object is L1, the height is H1, the y coordinate value on the captured image is y1, the distance to the second tracking object is L2, the height is H2, and the captured image If the y coordinate value above is y2, the relationship of the following equation (2) is established from the equation (1).
{(Hc−H1) × L2} / {(Hc−H2) × L1} = y1 / y2 (2)
Further, when L1 = L2, the following expression (3) is established.
(Hc−H1) / (Hc−H2) = y1 / y2 (3)

  FIG. 5 shows a temporal change in the ratio of y coordinates of two tracking target parts at the same distance from the lens of the imaging unit 10 and a temporal change in the ratio of the y coordinates of two tracking target parts at different distances. FIG. As is clear from Expression (3), when the imaging unit 10 moves, the distances L1 and L2 change, so the y-coordinate ratio on the captured image of an object existing at a different distance changes. However, when the distances L1 and L2 are equal, the y-coordinate ratio on the captured image does not change even when the imaging unit 10 moves, as is apparent from Equation (3). When the same object is imaged by the imaging unit 10, the distance to each part to be imaged is considered to be within a predetermined range, and thus the amount of change in the y-coordinate ratio is also within the predetermined range. Accordingly, the ratio of the y coordinates of two trackable areas is obtained, and when the amount of time change is within a predetermined range, the two areas are areas where the same object is imaged (hereinafter, the same object area). It is determined that

  The height ratio calculation unit 60 calculates the ratio of y coordinates in the images of two trackable regions among the plurality of trackable regions stored in the tracking region information storage unit 80. For example, when there are n traceable areas, there are n · (n−1) / 2 combinations for extracting two traceable areas, and therefore n · (n−1) / 2 y-coordinates The ratio is calculated and stored in a memory (not shown). In a memory (not shown), y-coordinate ratios calculated every predetermined time are sequentially stored, and past m calculation results are stored.

  The area classification unit 70 first obtains the temporal change amount of the y-coordinate ratio calculated by the height ratio calculation unit 60. Here, an y-coordinate is obtained by calculating an approximate line based on the calculated values of m y-coordinate ratios stored in a memory (not shown) of the height ratio calculation unit 60 and calculating the inclination of the obtained approximate line. Find the time change of the ratio. Next, the absolute value of the temporal change amount of the obtained y coordinate ratio is compared with a predetermined threshold value, and the absolute value of the temporal change amount of the y coordinate ratio is equal to or smaller than the predetermined threshold value. In this case, it is determined that the two trackable areas that have been calculated are the same object area. The area classification unit 70 further classifies each area by a method to be described later based on the determination result of whether or not they are the same object area.

  As described above, identification numbers are sequentially assigned to the traceable areas stored in the tracking area information storage unit 40. The region classification unit 70 determines the same object region in order from the traceable region with the lowest identification number, and assigns region classification number 1 to all the regions determined to be the same object region. Subsequently, it is determined whether or not the traceable area having the next lowest identification number is the same object area, and the area classification number 2 is assigned to the area determined to be the same object area. Using the same procedure, it is determined whether or not all areas not assigned with area classification numbers are the same object area, and area classification numbers are assigned to areas determined to be the same object area. I will do it.

  FIG. 6 is a diagram showing a result of classification for each same area based on the traceable area shown in FIG. In FIG. 6, in order to make the classification result visually clear, the same pattern is used to represent the areas determined to be the same object area.

  Once the same object region determination process is performed, it is not necessary to perform a determination process for determining whether or not the same object region is the same object region in the next input captured image. . Accordingly, the region classification unit 70 performs the same object region determination process on the region around the trackable region newly detected by the trackable region detection unit 30 (the region within a predetermined range from the trackable region). . Thereby, the amount of calculation of the determination process of the same object area | region performed after the determination process of the same object area | region can be reduced significantly.

  FIG. 7 is a flowchart illustrating a processing procedure performed by the image region classification device according to the embodiment. The process starting from step S10 starts when an ignition switch (not shown) is turned on. In step S10, the tracking area information stored in the tracking area information storage unit 40 is initialized, and the process proceeds to step S20. In step S20, the image captured by the imaging unit 10 is stored in the image storage unit 20, and the process proceeds to step S30.

  In step S <b> 30, the traceable area detection unit 30 detects the traceable area based on the image stored in the image storage unit 20. Since the method for detecting the traceable area has been described above, a detailed description thereof will be omitted. When the traceable area is detected, the process proceeds to step S40. In step S40, the local area tracking unit 50 detects the position of the destination of the trackable area based on the image captured in step S20 and the image newly captured at the imaging interval Δt. Since the destination detection method has been described above, a detailed description thereof will be omitted. When the destination of the traceable area is detected, the process proceeds to step S50.

  In step S50, the ratio of y coordinates of two different trackable areas is calculated. Since the method for calculating the ratio of the y-coordinate has been described above, detailed description thereof is omitted here. In step S60 following step S50, the region classification unit 70 classifies the traceable region based on the temporal change in the y-coordinate ratio calculated in step S50. Since the traceable area classification method has been described above, a detailed description thereof will be omitted here. Thereafter, the processing after step S20 is repeatedly performed at a cycle of the imaging interval Δt.

  The region classification method by the image region classification device in one embodiment will be summarized. The traceable area detection unit 30 divides the image captured by the imaging unit 10 into a plurality of areas, and detects a traceable area where the mobile body can be tracked for each area. The height ratio calculation unit 60 calculates a ratio of y coordinates on the captured image of two different trackable areas detected by the trackable area detection unit 30. The region classification unit 70 determines whether the two trackable regions whose y-coordinate ratios are calculated based on the time change amount of the y-coordinate ratios calculated by the height ratio calculating unit 60 are the same object region. judge. As a result, when a moving body such as a vehicle is detected on the captured image, it is not necessary to perform the moving body detection process on all the regions of the captured image divided into a plurality of regions, and it is determined as the same object region. Since it is only necessary to perform the moving object detection process for these areas as a group, the amount of calculation can be greatly reduced.

  In the conventional method of detecting the vehicle on the captured image based on the feature of the edge of the vehicle, when a part of the vehicle to be detected is hidden by another vehicle or a building and a part of the edge cannot be detected, The vehicle may not be detected. In this case, the captured image cannot be classified for each same object area. However, in the image area classification apparatus according to the embodiment, even if a part of the vehicle is hidden, for each area in which the same object is reflected. It is possible to classify captured images.

  Further, according to the image region classification device in one embodiment, after the same object region determination process is performed once, the region around the trackable region newly detected by the trackable region detection unit 30 ( Since the same object region determination process is performed on a region within a predetermined range from the traceable region), the calculation amount of the same object region determination process can be greatly reduced.

  The present invention is not limited to the embodiment described above. For example, you may make it further provide the moving body detection part which detects moving bodies, such as a vehicle, based on the image classified for every same object area | region with the image area classification | category apparatus in one Embodiment.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the imaging unit 10 is an imaging unit, the trackable region detection unit 30 is a region classification unit and a trackable region detection unit, the height ratio calculation unit 60 is a coordinate ratio calculation unit, and the region classification unit 70 is a coordinate ratio time change. A quantity calculating unit and an identical object region determining unit are configured. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.

The figure which shows the structure of one Embodiment of the image area classification device by this invention. The figure which shows a mode that it divided | segmented into the grid | lattice form to the vertical direction and the horizontal direction with respect to the image which detected the edge from a captured image The figure which shows the rectangular area | region judged to be a traceable area | region based on the captured image of FIG. Model view of the tracking target and imaging unit viewed from the side The figure which shows the time change of the ratio of the y coordinate of two tracking object site | parts in the same distance, and the time change of the ratio of the y coordinate of two tracking object parts in a different distance, respectively. The figure which shows the result of classifying the traceable area for each identical object area The flowchart which shows the process sequence performed by the image area classification device in one embodiment

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Imaging part, 20 ... Image memory | storage part, 30 ... Traceable area | region detection part, 40 ... Tracking area | region information storage part, 50 ... Local area tracking part, 60 ... Height ratio calculation part, 70 ... Area classification | category part

Claims (5)

An imaging means installed in the vehicle for imaging the surroundings of the vehicle;
Area dividing means for dividing the image captured by the imaging means into a plurality of areas;
A trackable area detecting means for detecting a trackable area capable of tracking a moving body among the areas divided by the area dividing means;
A coordinate ratio calculation unit that calculates a ratio of coordinate values on an image corresponding to a height direction in real space for two different trackable regions detected by the trackable region detection unit;
A coordinate ratio time change amount calculating means for calculating a time change amount of the ratio of the coordinate values calculated by the coordinate ratio calculation means based on a plurality of images picked up at different timings by the image pickup means;
Based on the time change amount of the ratio of the coordinate values calculated by the coordinate ratio time change amount calculating means, two traceable areas where the ratio of the coordinate values is calculated are areas where the same object is imaged (hereinafter, An image region classification apparatus comprising: an identical object region determining unit that determines whether or not the same object region is referred to. The image region classification device according to claim 1,
When the absolute value of the time change amount of the coordinate value ratio is equal to or less than a predetermined threshold, the same object region determination unit is configured to determine that the two trackable regions in which the coordinate value ratio is calculated are the same object region. An image region classifying apparatus characterized by determining that there is one. In the image area classification device according to claim 1 or 2,
Once the same object region determining means determines whether or not the same object region is once, thereafter, the same object region determining means is applied to an area within a predetermined range from the trackable area newly detected by the trackable area detecting means. An image area classification apparatus that performs a process of determining whether or not the same object area. In the image region classification device according to any one of claims 1 to 3,
The coordinate ratio calculation means calculates a ratio of y coordinates of two different traceable areas detected by the traceable area detection means when a vertical direction of an image taken by the imaging means is a y-axis,
The coordinate ratio time change amount calculating means calculates a time change amount of the y-coordinate ratio calculated by the coordinate ratio calculation means,
The same object region determining unit is configured to determine that the two traceable regions in which the y coordinate ratio is calculated based on the time change amount of the y coordinate ratio calculated by the coordinate ratio time change amount calculating unit are the same object region. An image region classification device for determining whether or not there is an image region. In the image region classification device according to any one of claims 1 to 4,
The traceable area detecting means detects an area having a change in shading in a plurality of directions as the traceable area.

Images