JP5505439B2 - Moving object detection apparatus, computer program, and moving object detection method - Google Patents

Description

  The present invention relates to a moving object detection device that detects a moving object using a captured image in which a plurality of pixels are arranged in a matrix, a computer program for realizing the moving object detection device with a computer, and a moving object detection method.

  It is demanded to provide detailed traffic information in order to realize smooth traffic or prevent traffic accidents. For example, not only a device that can acquire macro traffic parameters such as the number of vehicles passing through a predetermined point on a road or an average speed, but also a device that can accurately measure the position or movement of each vehicle. Is required.

  These measuring devices are often installed outdoors, and stable measuring accuracy corresponding to various environmental changes is required. In order to spread social infrastructure using these devices, it is also essential to provide inexpensive devices.

  As an example of such an apparatus, an image-type vehicle detector using an image processing technique has been widely used. For example, an apparatus for measuring traffic flow by detecting the presence of a vehicle, a vehicle type such as a small-sized vehicle or a medium-sized vehicle, vehicle speed, and the like from image information captured by a video camera is disclosed (see Patent Document 1).

JP-A-5-307695

  However, the apparatus disclosed in Patent Document 1 compares the captured image in the current frame with the captured image in the past frame, for example, when the difference between the pixel values of both exceeds a predetermined threshold value. Determines that a vehicle is present. Further, as a conventional technique, there is a technique in which an image of a road where no vehicle exists is stored in advance as a background image, and the vehicle is detected based on a difference image between the captured image and the background image.

  Depending on the daytime time zone, season, etc., the shadow of the vehicle may be generated in front of the vehicle, on the side of the road, etc., and the shadow also moves as the vehicle moves. Further, since there is a luminance difference between a shadow portion and a non-shadow portion, there is a problem that when detecting a vehicle based on the difference image, the shadow portion is erroneously detected as a vehicle. For this reason, it has been desired to detect a moving body such as a vehicle with high accuracy.

  The present invention has been made in view of such circumstances, and a mobile object detection device capable of detecting a mobile object with higher accuracy than before, a computer program for realizing the mobile object detection device with a computer, and It is an object to provide a moving body detection method.

According to a first aspect of the present invention, there is provided a moving body detection device that detects a moving body using a captured image in which a plurality of pixels are arranged in a matrix, and a feature point based on a pixel value of each pixel of the captured image. Extraction means for extracting, first count means for counting the number of feature points extracted by the extraction means for each unidirectional line of the captured image, and the number of feature points counted by the first count means is a predetermined number. Detection means for detecting the vertical position of the moving body based on the position of the one-direction line that is equal to or greater than the first feature point threshold, and a height having a predetermined vertical dimension based on the vertical position detected by the detection means A height area setting means for setting an area in the captured image, and a second for counting the number of feature points extracted by the extraction means for each other direction line in the height area set by the height area setting means. Counting means, and the detection means includes the second meter. Based on the position of the other direction line feature points counted in unit is present, Yes so as to detect the lateral position of the movable body, the predetermined lateral dimension relative to the transverse position of said detecting means detects And a width region setting unit that sets a width region different from the height region in the captured image, wherein the first counting unit includes the one-way line of the width region set by the width region setting unit. The number of feature points extracted by the extraction unit is counted every time, and the detection unit replaces the detected vertical position with each one-way line of the width region set by the width region setting unit. The vertical position of the moving body is further detected based on the position of the one-way line where the number of feature points extracted in the above is equal to or greater than a predetermined second feature point threshold value .

Moving object detection apparatus according to the second invention, Oite the first shot bright, first captured image and the difference of the second image of each imaging of the corresponding pixel value of each pixel is first difference threshold imaging point is different First pixel specifying means for specifying a larger first pixel, and a second difference in which a difference between pixel values of corresponding pixels of each of the first captured image and the second captured image is smaller than the first difference threshold. A second pixel specifying unit that specifies a second pixel that is larger than the threshold; a first pixel that is specified by the first pixel specifying unit; and a pixel that is adjacent to the first pixel and includes the second pixel. Difference image generation means for generating a difference image to be performed, and processing means for filtering either the first captured image or the second captured image with the difference image generated by the difference image generation means, The extraction means is a filter by the processing means. Characterized in that is arranged to extract a feature point on the basis of the pixel value of each pixel of a captured image obtained by sense.

Moving object detection apparatus according to the third invention, Oite the first invention or the second shot bright, the detection means, the number of the counted feature points in the first counting means and the first or second feature point threshold The vertical position of the moving body is detected on the basis of the position of the lowermost unidirectional line among the unidirectional lines as described above.

A moving body detection apparatus according to a fourth aspect of the present invention is the moving body detection apparatus according to any one of the first to third aspects of the present invention, wherein the moving direction detection apparatus continuously adjoins among the other direction lines where the feature points counted by the second counting means are present. A range specifying unit that specifies a horizontal position candidate range configured by a plurality of other direction lines is provided, and the detection unit is configured to display the other direction line that represents a feature point existing in the horizontal position candidate range specified by the range specifying unit. It is configured to detect the lateral position of the moving body based on the position.

According to a fifth aspect of the present invention, in the mobile device detection apparatus according to the fourth aspect of the present invention, when the plurality of lateral position candidate ranges exist apart from each other, the range specifying means determines a lateral position candidate range having the maximum number of other direction lines. It is configured to be specified.

According to a sixth aspect of the present invention, in the fifth aspect , the range specifying means is configured such that, when there are a plurality of lateral position candidate ranges apart from each other, the distance between adjacent lateral position candidate ranges is a predetermined separation threshold. In the following cases, the adjacent lateral position candidate ranges are specified as one lateral position candidate range.

According to a seventh aspect of the present invention, in the sixth aspect of the present invention, the separation threshold setting means for setting the separation threshold according to the larger number of the other direction lines in each of the adjacent lateral position candidate ranges. It is characterized by providing.

A computer program according to an eighth invention is a computer program for causing a computer to detect a moving body using a captured image in which a plurality of pixels are arranged in a matrix. Extracting the feature points based on the step, counting the number of feature points extracted for each unidirectional line of the captured image, and the one feature number being equal to or greater than a predetermined first feature point threshold value. A step of detecting the vertical position of the moving body based on the position of the direction line; a step of setting a height region having a predetermined vertical dimension with reference to the detected vertical position; The step of counting the number of feature points extracted for each other direction line in the area, and the position of the other direction line where the counted feature points exist, Detecting a lateral position, based on the detected lateral position having a predetermined transverse dimension, and setting the height region different width region in the captured image, the one width area set A step of counting the number of feature points extracted for each direction line, and the number of feature points extracted for each one-direction line of a set width region is a predetermined second feature point threshold value instead of the detected vertical position The step of further detecting the vertical position of the moving body is executed based on the position of the one-way line as described above .

A moving object detection method according to a ninth aspect of the present invention is the moving object detection method by a moving object detection device that detects a moving object using a picked-up image in which a plurality of pixels are arranged in a matrix. Extracting feature points based on the values; counting the number of feature points extracted for each unidirectional line of the captured image; and counting the number of feature points equal to or greater than a predetermined first feature point threshold value. A step of detecting a vertical position of the moving body based on the position of the one-directional line, and a step of setting a height region having a predetermined vertical dimension in the captured image with reference to the detected vertical position; A step of counting the number of feature points extracted for each other direction line in the set height region, and a lateral position of the moving body based on the position of the other direction line where the counted feature point exists and the step of detecting the, A step of setting a width area different from the height area in the captured image with a predetermined horizontal dimension based on the horizontal position, and a feature point extracted for each directional line of the set width area And the number of feature points extracted for each one-way line in the set width area is equal to or greater than a predetermined second feature point threshold value instead of the detected vertical position. And further detecting the vertical position of the moving body based on the position .

In the first invention, the eighth invention, and the ninth invention, the extracting means extracts the feature point based on the pixel value of each pixel (for example, each adjacent pixel) of the captured image. If the horizontal direction of the captured image is the x-axis, the vertical direction is the y-axis, and the origin of the xy coordinates is the upper left of the captured image, each pixel is, for example, each pixel adjacent to the horizontal direction (x-axis direction) (for example, adjacent Pixels adjacent to each other in the vertical direction (y-axis direction) (for example, adjacent pixels). The pixel value is, for example, the luminance value Y. A feature point is extracted as a feature point when the absolute value of the difference between the luminance values of an arbitrary pixel of interest and a pixel adjacent to the pixel of interest is larger than a predetermined luminance threshold. The first counting unit counts the number of feature points extracted for each unidirectional line (for example, horizontal line) of the captured image. In this case, if the absolute value of the difference between the luminance values of an arbitrary pixel of interest and a pixel adjacent in the lateral direction (adjacent) to the pixel of interest is greater than a predetermined luminance threshold Yth1, the feature point is the feature point. Extract as The horizontal line is a line along the horizontal direction (x-axis). For example, when the size of the captured image is (vertical × horizontal) pixels, the horizontal line exists as many as the number of vertical strokes. That is, the first counting means counts and adds the number of feature points on the captured image in the horizontal direction (each horizontal line), and sums the histograms of the feature points using the total value as a frequency (horizontal image feature projection histogram). Is generated.

  The detecting means detects the vertical position (y coordinate) of the moving body based on the position of the horizontal line (one-direction line) where the number of counted feature points (frequency of the histogram) is equal to or greater than a predetermined first feature point threshold. To do. The first feature point threshold value may be, for example, 70% of the maximum number of feature points counted (maximum value of frequency), or may be set in advance. The vertical position (y-coordinate) of the moving body can be, for example, the lowest position among horizontal lines in which the frequency of the horizontal image feature projection histogram is equal to or greater than the first feature point threshold. For example, if the size of the captured image is (vertical × horizontal) pixels and the y coordinate of the lowermost horizontal line is y1, the y coordinate (vertical position) of the moving body is y = y1. The height region determining means sets a height region (vehicle head height range) having a predetermined vertical dimension in the captured image with reference to the detected vertical position. The predetermined vertical dimension can be set assuming a range of the height of the moving body. The height region can be, for example, a region expanded by a predetermined number of pixels in the vertical direction with respect to the vertical position (y coordinate) of the moving body.

  The second counting means counts the number of extracted feature points for each other direction line (for example, vertical line) in the height region (vehicle head height range) of the captured image. In this case, if the absolute value of the difference between the luminance values of an arbitrary target pixel and a pixel adjacent in the vertical direction (adjacent) to the target pixel is greater than a predetermined luminance threshold Yth2, the feature point is the feature point. Extract as The vertical line is a line along the vertical direction (y-axis). For example, when the size of the captured image is (vertical × horizontal) pixels, there are as many vertical lines as there are horizontal pixels. That is, the second counting means counts the number of feature points in the height region (vehicle head height range) of the captured image in the vertical direction (for each vertical line) and sums the feature points, and the summed value is the feature point. (Vertical head characteristic projection histogram) is generated. The detecting means detects the horizontal position (x coordinate) of the moving body based on the position of the vertical line where the counted feature points (frequency of the histogram) are present. The lateral position (x coordinate) of the moving body is determined based on the longitudinal vehicle head characteristic projection histogram, and the predetermined position of the determined vehicle head width can be set as the lateral position (x coordinate) of the moving body. For example, a portion having a continuous distribution of feature point histograms (vertical vehicle head feature projection histogram) or a continuous portion of the distribution is determined as the vehicle head width. The position (lateral position) at which the total value of the frequencies in the histogram within the vehicle head width is approximately equally divided can be set as the lateral position (x coordinate) of the moving body.

  Since a pixel whose pixel value (luminance value) difference between adjacent (adjacent) pixels is equal to or greater than a predetermined threshold is extracted as a feature point, it is on the road surface caused by the influence of external environment changes such as sunshine changes and weather changes. Shadow portions are not extracted as feature points, and changes in luminance of moving objects (for example, vehicles), headlights, front grills, tail lamps, etc. can be quantified without being affected by shadows on the road surface. . Since the license plate as the reference position of the moving body is sandwiched between headlights, tail lamps, and the like, the moving body (for example, the license plate, for example) is obtained by using the distribution along the vertical and horizontal directions of the extracted feature points. ) Position can be detected with high accuracy.

The width area setting means sets a width area (four-wheeled vehicle width range) having a predetermined horizontal dimension in the captured image with reference to the detected horizontal position (x coordinate). The predetermined lateral dimension can be set assuming a range of the width of the moving body (vehicle width of a four-wheeled vehicle). The width region can be, for example, a region expanded by a predetermined number of pixels in the left-right direction with reference to the horizontal position (x coordinate) of the moving body. The first counting means counts the number of extracted feature points for each horizontal line of the set width region. In this case, if the absolute value of the difference between the luminance values of an arbitrary target pixel and a pixel that is adjacent to (adjacent to) the target pixel in the horizontal direction is greater than a predetermined luminance threshold Yth3, the feature point is the feature point. Extract as The horizontal line is a line along the horizontal direction (x-axis). For example, if the size of the captured image is (vertical × horizontal) pixels, the horizontal line exists as many as the number of vertical pixels. That is, the first counting means counts the number of feature points in the width region (four-wheeled vehicle width range) of the captured image in the horizontal direction (each horizontal line) and sums the feature points, and the summed value is the feature point. The histogram (lateral four-wheel feature projection histogram) is generated.

  The detection means replaces the vertical position (y coordinate) detected based on the horizontal image feature projection histogram, based on the position of the horizontal line where the number of counted feature points is equal to or greater than the second feature point threshold. The vertical position (y coordinate) is detected. The second feature point threshold value may be, for example, 70% of the maximum number of counted feature points (maximum value of frequency), or may be set in advance. The vertical position (y coordinate) of the moving body can be, for example, the lowest position among the horizontal lines in which the frequency of the horizontal four-wheel feature projection histogram is equal to or greater than the second feature point threshold. For example, if the size of the width area (four-wheeled vehicle width range) of the captured image is (vertical × horizontal) pixels and the y coordinate of the lowermost horizontal line is y2, the y coordinate (vertical position) of the moving object is y = y2. Since the range of counting the feature points in the horizontal direction is limited to the width region (four-wheeled vehicle width range) instead of the entire captured image, the second feature point threshold is a value smaller than the first feature point threshold. can do.

  By limiting the range in which the feature points are counted in the horizontal direction to the width area (four-wheeled vehicle width range) based on the horizontal position (x coordinate) of the moving body, Alternatively, when there is another vehicle that runs in the vicinity of the diagonal rear, the feature points extracted by the other vehicle can be excluded, so that the distribution of the feature points can be prevented from being influenced by another vehicle, Furthermore, the position of the moving body can be detected with high accuracy.

In the second invention, the pixel values (for example, luminance values) of the respective corresponding pixels of the first captured image (for example, time t) and the second captured image (for example, time t + 1) having different imaging time points. The first pixel is larger than the first difference threshold (strong difference threshold), and the difference between the pixel values of the corresponding pixels of the first captured image and the second captured image is the first difference threshold. A second pixel greater than the smaller second difference threshold (weak difference threshold) is identified. The number plate, headlight, front grille, tail lamp, and other parts show a large difference in luminance values on the captured images obtained in time series. Therefore, by using the first difference threshold (strong difference threshold), the license plate The first pixel corresponding to the headlight, the front grill, the tail lamp, and the like can be specified. In addition, shadows on the road surface caused by changes in the external environment such as sunshine changes and weather changes, and parts with little texture such as bonnets have a large difference in luminance values on the captured images obtained in time series. Since it does not appear, by using the second difference threshold value (weak difference threshold value), it is possible to specify the second pixel corresponding to a shadowed part on the road surface, a part with less texture such as a hood.

  The difference image generation means generates a difference image composed of the identified first pixel and the second pixel adjacent to the first pixel. The difference image composed of the first pixel and the second pixel adjacent (continuous) to the first pixel can remove a shadow portion that is separated from the moving body (vehicle) portion, A low-textured portion such as a bonnet adjacent (continuous) to a portion (for example, a vehicle head portion or a vehicle tail portion) such as a license plate, a headlight, a front grille, and a tail lamp can be included. As a result, the vehicle body portion including the vehicle head (or vehicle tail) portion can be specified (leaved) as the difference image. The difference image is, for example, a binarized image in which the vehicle body portion is “1” and the portions other than the vehicle body portion are represented by “0”. The extraction means filters either the first captured image or the second captured image (for example, the captured image at time t) with the generated difference image (for example, “1” of the binarized difference image). Based on the pixel value of each pixel (for example, each adjacent pixel) of the captured image obtained by leaving the corresponding captured image pixel and removing the captured image pixel corresponding to “0” of the difference image) Extract feature points. As a result, shadows on the road surface caused by changes in the external environment such as sunshine changes and weather changes can be deleted, the vehicle body part can be included, and a moving body can be detected with high accuracy.

In the third aspect of the invention, the detection means includes the lowest unidirectional line (horizontal line) among the unidirectional lines (horizontal lines) in which the number of feature points counted is equal to or greater than the first feature point threshold or the second feature point threshold ( Based on the position of the horizontal line), the vertical position (y coordinate) of the moving body is detected. Generally, the position of the license plate is located between the headlight or the tail lamp, and there is no vehicle body below the license plate. Therefore, the position of the lowest horizontal line among the horizontal lines in which the number of feature points counted for each horizontal line is equal to or greater than the first feature point threshold value or the second feature point threshold value is the vehicle body part and the vehicle body front (or vehicle body rear). ) Near the boundary with the road surface portion and near the position where the license plate is installed. By detecting the position as the vertical position (y coordinate) of the moving body, the position of the moving body is determined. It can be detected with high accuracy.

In the fourth invention, the range specifying means is a horizontal position constituted by a plurality of other direction lines (vertical lines) adjacent to each other among the other direction lines (vertical lines) in which the counted feature points exist. Identify candidate ranges. The lateral position candidate range is a range of the width (vehicle head width or vehicle tail width) of the moving object including the horizontal position of the moving object. That is, a portion (a plurality of adjacent vertical lines) in which the distribution of feature point histograms (vertical vehicle head feature projection histogram) counted in the vertical direction (for each vertical line) is continuous is defined as the vehicle head width or the vehicle tail width. Identify. The detecting means detects the horizontal position of the moving body based on the position of the vertical line representing the feature point existing in the specified horizontal position candidate range. The position of the vertical line representing the feature point is, for example, the position of the vertical line that substantially divides the total number of feature points, but is not limited thereto. For example, the position of the vertical line that roughly divides the total number (total value) of feature points existing in the portion where the distribution of the feature point histogram (vertical vehicle head feature projection histogram) continues is the horizontal position (x coordinate) of the moving object. Detect as. Note that if the distribution of the feature point histogram (vertical vehicle head feature projection histogram) is substantially uniform (for example, the frequency of the feature points is approximately equal), the approximate center of the portion where the distribution continues is the side of the moving object. Position. Thereby, the position of the moving body can be detected with high accuracy.

In the fifth invention, the range specifying means specifies the horizontal position candidate range having the maximum number of vertical lines when a plurality of horizontal position candidate ranges exist apart from each other. That is, when there are a plurality of feature points histograms (vertical vehicle head feature projection histograms) with continuous distribution (several adjacent vertical lines) separated from each other, the longest part is the vehicle width or vehicle width. Specify as tail width. As a result, the most probable vehicle head width or vehicle tail width can be specified, and the position of the moving body can be detected with high accuracy.

In the sixth invention, the range specifying means, when there are a plurality of lateral position candidate ranges apart, and when the separation dimension of the adjacent lateral position candidate ranges is equal to or less than a predetermined separation threshold, the adjacent lateral position The candidate range is specified as one horizontal position candidate range. That is, when there are a plurality of portions (continuous adjacent vertical lines) in which the distribution of the histogram of feature points (vertical vehicle head feature projection histogram) is separated, there is a separation dimension between the portions in which the distribution is continuous. When the distance is equal to or less than the predetermined separation threshold, the portions where the distributions are continuous are specified as one continuous portion. Thereby, the part contained in a vehicle body and the part other than a vehicle body can be distinguished, and the position of a moving body can be detected with a sufficient precision.

In the seventh invention, the separation threshold setting means sets the separation threshold according to the larger number of the vertical lines in each of the adjacent horizontal position candidate ranges. For example, when the distribution of feature point histograms (vertical vehicle head feature projection histograms) exists continuously (a plurality of adjacent vertical lines) adjacent to each other, the longer width (length) continues. The separation threshold is increased as the width (length) of the portion is longer. Thus, according to the length of the portion where the distribution of the feature point histogram (vertical vehicle head feature projection histogram) continues, it can be combined into one portion, and the most probable vehicle head width or vehicle tail width is specified. And the position of the moving body can be detected with high accuracy.

  According to the present invention, the position of the moving body can be detected with high accuracy.

It is a block diagram which shows an example of a structure of the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the production | generation method of the difference image by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the production | generation method of the horizontal direction image feature projection histogram by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the detection method of the vehicle head reference position by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the production | generation method of the vertical direction head characteristic projection histogram by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the detection method of the horizontal position of the mobile body by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the identification method of the vehicle head width or the vehicle tail width by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows the other example of the identification method of the vehicle head width or the vehicle tail width by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the production | generation method of the horizontal direction four-wheels feature projection histogram by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the detection method of the horizontal position of the mobile body by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of a mobile body coordinate offset range. It is a flowchart which shows an example of the process sequence of the mobile body detection by the mobile body detection apparatus of this Embodiment. It is a flowchart which shows an example of the process sequence of the mobile body detection by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the production | generation method of the horizontal direction two-wheels characteristic projection histogram by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the determination method of whether the type determination by the moving body detection apparatus of this Embodiment is performed. It is explanatory drawing which shows an example of the feature-value of the horizontal direction four-wheels feature projection histogram and the horizontal direction two-wheels feature projection histogram by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the classification determination method of the mobile body by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the determination method of the lane by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the production | generation method of the positional information by the moving body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the tracking method of the mobile body by the mobile body detection apparatus of this Embodiment. It is explanatory drawing which shows an example of the tracking information by the mobile body detection apparatus of this Embodiment. It is a flowchart which shows an example of the process sequence of the classification determination of the mobile body by the mobile body detection apparatus of this Embodiment.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments. FIG. 1 is a block diagram showing an example of the configuration of the moving object detection device 100 of the present embodiment. The mobile body detection device 100 has a function as a mobile body type determination device that determines the type of the mobile body. In the following, it is assumed that the mobile body detection device 100 can also include a mobile body type determination device. The moving body detection apparatus 100 includes a control unit 10 that controls the entire apparatus, a difference image generation unit 11, a feature point extraction unit 12, a lateral direction feature point counting unit 13, a vehicle head height region setting unit 14, and a longitudinal direction feature point counting unit. 15, lateral position candidate range specifying unit 16, vehicle width region setting unit 17, moving body coordinate detection unit 18, vehicle type information generation unit 20, lane information generation unit 31, position information generation unit 32, tracking information generation unit 33, and the like. . The vehicle type information generation unit 20 includes a first type region setting unit 21, a second type region setting unit 22, a feature point distribution position specifying unit 23, a moving body type determining unit 24, and the like.

  In the following description, the horizontal direction is the x-axis and the vertical direction is the y-axis with the upper left corner of the captured image as the origin. The captured image is, for example, a gray scale image, but is not limited thereto. Further, the moving body includes vehicles such as a four-wheel vehicle and a two-wheel vehicle.

  An imaging device (not shown) such as a video camera or a camera is located near the road in a state where imaging conditions such as a predetermined height and a lens optical axis direction (for example, depression angle and rotation angle) are set with the road as a field of view. It is installed at the required point. For example, the imaging device may take an image from the front of the traveling moving body, or may take an image from the rear of the moving body. When imaging from the front of the moving body, the vehicle head portion of the moving body is detected, and when capturing from the rear of the moving body, the tail portion of the moving body is detected.

  The size of the original image acquired by the imaging device is larger than the size of the above-described captured image, for example. That is, a reduced image obtained by reducing the original image is used as the above-described captured image. Thereby, it is possible to reduce the load of image processing required for detection or determination of the moving body. When the size of the original image does not require a reduction in image processing load, the original image can be used as a captured image without being reduced.

  Moreover, in the following description, the position (horizontal position, vertical position) of the moving body is represented by the x coordinate and y coordinate on the captured image. In addition, the position of the moving body is assumed to be the position of the license plate (for example, the position of the character portion of the license plate, the center position of the license plate, etc.), but the position of the moving body is limited to the position of the license plate. It is not something.

  The moving object detection device 100 determines the position (coordinates) of the moving object from the distribution of the feature points with respect to the area with change in the captured image captured in time series, and the object information of the moving object, that is, the type of the moving object (for example, Whether the vehicle is a vehicle or, if it is a vehicle, whether it is a four-wheel vehicle or a two-wheel vehicle).

  In the moving object detection process, a difference image is generated based on two captured images having different imaging time points in time series in order to detect a moving part of the moving object. Hereinafter, the generation of the difference image will be described. FIG. 2 is an explanatory diagram illustrating an example of a method for generating a difference image by the moving object detection apparatus 100 according to the present embodiment.

  The difference image generation unit 11 functions as a difference image generation unit, a first pixel specifying unit, a second pixel specifying unit, and a processing unit.

  As illustrated in FIG. 2A, the difference image generation unit 11 has a pixel value of each pixel corresponding to each of the first captured image (for example, time t) and the second captured image (for example, time t + 1) at different capturing points. The first pixel having a difference (for example, luminance value) greater than the first difference threshold (strong difference threshold) is specified, and the pixel value of each corresponding pixel of each of the first captured image and the second captured image is specified. A second pixel having a difference larger than a second difference threshold (weak difference threshold) smaller than the first difference threshold is specified. For example, when the luminance value range is 0 to 255 (0 corresponds to black and 255 corresponds to white), the first difference threshold (strong difference threshold) is the shape and contour of the moving body (vehicle body). It can be a relatively large value as specified. Further, the second difference threshold value (weak difference threshold value) can be set to a relatively small value so that the inside of the moving body (vehicle body), that is, the inner part of the contour is specified.

  The number plate, headlight, front grille, tail lamp, and other parts show a large difference in luminance values on the captured images obtained in time series. Therefore, by using the first difference threshold (strong difference threshold), the license plate The first pixel corresponding to the headlight, the front grill, the tail lamp, and the like can be specified. In addition, shadows on the road surface caused by changes in the external environment such as sunshine changes and weather changes, and parts with little texture such as bonnets have a large difference in luminance values on the captured images obtained in time series. Since it does not appear, by using the second difference threshold value (weak difference threshold value), it is possible to specify the second pixel corresponding to a shadowed part on the road surface, a part with less texture such as a hood.

  In the example of FIG. 2B, a pixel having a luminance greater than the first difference threshold (strong difference threshold) is indicated by reference numeral WT. A pixel having a luminance smaller than the first difference threshold (strong difference threshold) and larger than the second difference threshold (weak difference threshold) is denoted by reference symbol GR1. A pixel having a luminance smaller than the second difference threshold (weak difference threshold) is indicated by reference numeral BK.

  As illustrated in FIGS. 2B and 2C, the difference image generation unit 11 generates a difference image composed of the identified first pixel and the second pixel adjacent to the first pixel. In the example of FIGS. 2B and 2C, the pixel GR1 (second pixel) adjacent to the pixel WT (first pixel) is the pixel WT, and the pixel GR1 separated from the pixel WT is the pixel BK. By doing so, a difference image (binarized image) composed of the pixel WT and the pixel BK is generated.

  The difference image composed of the first pixel and the second pixel adjacent (continuous) to the first pixel can remove a shadow portion that is separated from the moving body (vehicle) portion, A low-textured portion such as a bonnet adjacent (continuous) to a portion (for example, a vehicle head portion or a vehicle tail portion) such as a license plate, a headlight, a front grille, and a tail lamp can be included. As a result, the vehicle body portion including the vehicle head (or vehicle tail) portion can be specified (leaved) as the difference image. The difference image is, for example, a binarized image in which the vehicle body portion is “1” and the portions other than the vehicle body portion are represented by “0”.

  The feature point extraction unit 12 has a function as extraction means for extracting feature points. The feature point extraction unit 12 extracts a feature point based on the pixel value of each pixel of the captured image (for example, each adjacent pixel, but not limited to the adjacent pixel). Assuming that the horizontal direction of the captured image is the x-axis and the vertical direction is the y-axis, the adjacent pixels are pixels adjacent in the horizontal direction (x-axis direction) (for example, adjacent pixels) and the vertical direction (y-axis). Each pixel adjacent in the (direction) (for example, adjacent pixels). The pixel value is, for example, the luminance value Y. A feature point is extracted as a feature point when the absolute value of the difference between the luminance values of an arbitrary pixel of interest and a pixel adjacent to the pixel of interest is larger than a predetermined luminance threshold.

  When extracting feature points, the difference image generated by the difference image generation unit 11 may not be used. However, when a difference image is used, the following can be performed. In other words, the differential image generation unit 11 selects either the first captured image or the second captured image at different imaging time points when generating the differential image (for example, the captured image at time t in the example of FIG. 2). Then, filtering is performed on the generated difference image (for example, the pixel of the captured image corresponding to “1” of the binarized difference image is left and the pixel of the captured image corresponding to “0” of the difference image is removed). The feature point extraction unit 12 extracts feature points based on the pixel values of adjacent pixels of the captured image obtained by the filtering process. By performing the filtering process using the difference image, it is possible to delete the shadow portion on the road surface caused by the influence of external environment changes such as sunshine change and weather change, and it is possible to include the vehicle body part and accurately move the moving object. Can be detected.

  Next, a method for generating a histogram that is a distribution of feature points will be described. Histograms generated for detecting a moving body include a horizontal image feature projection histogram, a vertical vehicle head feature projection histogram, and a horizontal four-wheel feature projection histogram. In addition to the horizontal four-wheel feature projection histogram described above, the histogram generated to determine the type of the moving body includes a horizontal two-wheel feature projection histogram.

  First, a method for generating a horizontal image feature projection histogram will be described. FIG. 3 is an explanatory diagram showing an example of a method for generating a horizontal image feature projection histogram by the moving object detection apparatus 100 according to the present embodiment. In FIG. 3, the right diagram shows an example of a captured image, and the left diagram shows an example of a horizontal image feature projection histogram.

  The lateral feature point counting unit 13 has a function as a first counting unit that counts the number of feature points. As shown in FIG. 3, the horizontal feature point counting unit 13 counts the number of extracted feature points for each horizontal line of the captured image. In this case, if the absolute value of the difference between the luminance values of an arbitrary pixel of interest and a pixel adjacent in the lateral direction (adjacent) to the pixel of interest is greater than a predetermined luminance threshold Yth1, the feature point is the feature point. Extract as In the example of FIG. 3, when | Y (x, y) −Y (x + 1, y) |> Yth1 is satisfied, the pixel (x, y) is extracted as a feature point.

  The horizontal line is a line along the horizontal direction (x-axis). For example, if the size of the captured image is (vertical × horizontal) pixels, the horizontal line exists as many as the number of vertical pixels. That is, the horizontal direction feature point counting unit 13 counts and sums the number of feature points on the captured image in the horizontal direction (each horizontal line), and a histogram of the feature points using the total value as a frequency (horizontal image feature). Projection histogram). That is, the horizontal image feature projection histogram represents the distribution of feature points obtained by projecting the feature points of the entire captured image in the horizontal direction.

  When generating the horizontal image feature projection histogram, it is necessary to extract the vehicle head portion or the vehicle tail portion from the entire captured image, and therefore it is desirable that the luminance threshold value Yth1 be a slightly large value.

  Next, a method for detecting the vehicle head reference position will be described. FIG. 4 is an explanatory diagram showing an example of a vehicle head reference position detection method by the moving object detection device 100 of the present embodiment. In FIG. 4, the left diagram shows an example of the generated horizontal image feature projection histogram, and the right diagram shows an example of a captured image. Since the vehicle head reference position can be the vertical position (y coordinate) of the moving object, the vertical position (y coordinate) of the moving object can also be detected by detecting the vehicle head reference position.

  The moving body coordinate detection unit 18 has a function as detection means for detecting a moving body. As shown in FIG. 4, the moving body coordinate detection unit 18 has the number of feature points (frequency of the horizontal image feature projection histogram) counted by the lateral feature point counting unit 13 equal to or greater than a predetermined first feature point threshold value Pth1. The vertical position (y coordinate) of the moving body is detected based on the position of the horizontal line.

  The first feature point threshold value Pth1 may be, for example, 70% of the maximum value of the number of feature points counted (maximum value of the frequency) or may be set in advance. The vehicle head reference position (vertical position of the moving body) can be, for example, the lowest position in the horizontal line where the frequency of the horizontal image feature projection histogram is equal to or higher than the first feature point threshold Pth1. For example, if the size of the captured image is (vertical × horizontal) pixels and the y coordinate of the lowermost horizontal line is y1, the vehicle head reference position, that is, the y coordinate (vertical position) of the moving body is y = y1. . The vehicle head reference position can be a position where a license plate is mounted. Further, when the maximum value of the counted number of feature points (the maximum value of the frequency) is equal to or less than a predetermined constant Vth, it is possible to disable the coordinate detection of the moving object.

  As described above, the lowest position where the frequency of the horizontal image feature projection histogram is equal to or higher than the first feature point threshold Pth1 is detected as the vertical position (y coordinate) of the moving object. Generally, the position of the license plate is located between the headlight or the tail lamp, and there is no vehicle body below the license plate. Therefore, the position of the lowest horizontal line among the horizontal lines in which the number of feature points in the lateral direction is equal to or greater than the first feature point threshold Pth1 is near the boundary between the vehicle body portion and the road surface portion in front of the vehicle body (or the rear of the vehicle body). And since it can be considered near the position where the license plate is installed, the position of the moving body can be detected with high accuracy by detecting the position as the vertical position (y coordinate) of the moving body.

  Next, a method for generating a longitudinal vehicle head feature projection histogram will be described. FIG. 5 is an explanatory diagram showing an example of a method of generating a longitudinal vehicle head feature projection histogram by the moving object detection device 100 of the present embodiment. In FIG. 5, the upper diagram shows an example of a captured image, and the lower diagram shows an example of a longitudinal vehicle head feature projection histogram.

  The vehicle head height region setting unit 14 has a function as height region setting means for setting a vehicle head height range as a height region in the captured image. As shown in FIG. 5, the vehicle head height region setting unit 14 captures a vehicle head height range (height region) having a predetermined vertical dimension with reference to the detected vehicle head reference position (vertical position of the moving body). Set in. The predetermined vertical dimension can be set assuming a range of the height of the moving body. The vehicle head height range can be, for example, a region widened by a predetermined number of pixels in the vertical direction with respect to the vehicle head reference position.

  The vertical feature point counting unit 15 has a function as second counting means for counting the number of feature points. As shown in FIG. 5, the vertical feature point counting unit 15 counts the number of feature points extracted for each vertical line within the vehicle head height range (height region) of the captured image. In this case, if the absolute value of the difference between the luminance values of an arbitrary target pixel and a pixel adjacent in the vertical direction (adjacent) to the target pixel is greater than a predetermined luminance threshold Yth2, the feature point is the feature point. Extract as In the example of FIG. 5, when | Y (x, y) −Y (x, y + 1) |> Yth2 is satisfied, the pixel (x, y) is extracted as a feature point.

  The vertical line is a line along the vertical direction (y-axis). For example, when the size of the captured image is (vertical × horizontal) pixels, there are as many vertical lines as there are horizontal pixels. That is, the vertical direction feature point counting unit 15 counts and adds the number of feature points in the vehicle head height range of the captured image in the vertical direction (for each vertical line), and a histogram of the feature points with the total value as the frequency. (Vertical head characteristic projection histogram) is generated. That is, the longitudinal vehicle head feature projection histogram represents a distribution of feature points obtained by projecting feature points within the vehicle head height range in the vertical direction.

  When generating the longitudinal vehicle head feature projection histogram, since the vicinity of the vehicle head is already limited in the entire captured image, it is only necessary to extract all of the headlights, the front grille, the license plate, etc. Yth2 can be made smaller than the luminance threshold Yth1.

  Next, a method for detecting the lateral position (x coordinate) of the moving body will be described. FIG. 6 is an explanatory diagram showing an example of a method for detecting the lateral position of the moving body by the moving body detection apparatus 100 according to the present embodiment. In FIG. 6, the lower diagram shows an example of the generated longitudinal vehicle head feature projection histogram, and the upper diagram shows an example of a captured image.

  The moving object coordinate detection unit 18 detects the horizontal position (x coordinate) of the moving object based on the position of the vertical line where the counted feature point (the frequency of the longitudinal vehicle head feature projection histogram) exists. The lateral position (x coordinate) of the moving body is determined based on the longitudinal vehicle head characteristic projection histogram, and the predetermined position of the determined vehicle head width can be set as the lateral position (x coordinate) of the moving body. For example, a portion where the distribution of the longitudinal vehicle head feature projection histogram continues is determined as the vehicle head width. The position (lateral position) at which the total value of the frequencies in the histogram within the vehicle head width is approximately equally divided can be set as the lateral position (x coordinate) of the moving body.

  As described above, pixels whose pixel value (brightness value) difference between adjacent (adjacent) pixels is equal to or greater than a predetermined threshold are extracted as feature points, so the influence of changes in the external environment such as sunshine changes and weather changes The portion of the shadow on the road caused by is not extracted as a feature point, and the luminance change of the characters, headlights, front grille, tail lamp, etc. of the moving body (eg, vehicle) is quantified without being affected by the shadow on the road Can be Since the license plate as the reference position of the moving body is sandwiched between headlights, tail lamps, and the like, the moving body (for example, the license plate, for example) is obtained by using the distribution along the vertical and horizontal directions of the extracted feature points. ) Position can be detected with high accuracy.

  Further, the lateral position candidate range specifying unit 16 has a function as range specifying means for specifying the vehicle head width or the vehicle tail width as the horizontal position candidate range. The lateral position candidate range is a range of the width (vehicle head width or vehicle tail width) of the moving object including the horizontal position of the moving object. The horizontal position candidate range specifying unit 16 specifies a vehicle head width or a vehicle tail width composed of a plurality of adjacent vertical lines among the vertical lines where the feature points counted by the vertical feature point counting unit 15 exist. To do. That is, a portion (a plurality of adjacent vertical lines) in which the distribution of feature point histograms (vertical vehicle head feature projection histogram) counted in the vertical direction (for each vertical line) is continuous is defined as the vehicle head width or the vehicle tail width. Identify.

  Then, the moving object coordinate detection unit 18 detects the horizontal position of the moving object based on the position of the vertical line representing the feature point existing in the specified vehicle head width or vehicle width. The position of the vertical line representing the feature point can be, for example, the position of the vertical line that substantially divides the total number of feature points. For example, the position of the vertical line that substantially divides the total number (total value) of feature points existing in the portion where the distribution of the longitudinal vehicle head feature projection histogram continues is detected as the horizontal position (x coordinate) of the moving object. If the distribution of the longitudinal vehicle head feature projection histogram is substantially uniform (for example, the frequency of feature points is substantially equal), the approximate center of the portion where the distribution continues is the horizontal position of the moving body. Thereby, the position of the moving body can be detected with high accuracy.

  In addition, when there are a plurality of parts with continuous vertical vehicle head feature projection histogram distribution (a plurality of adjacent vertical lines) separated from each other, the longest part is specified as the car head width or the car tail width. You can also. As a result, the most probable vehicle head width or vehicle tail width can be specified, and the position of the moving body can be detected with high accuracy.

  In the above-described example, the vertical vehicle head feature projection histogram distribution is determined as the vehicle head width, or the portion having the longest width (horizontal length) is determined as the vehicle head width. However, the present invention is not limited to this. Hereinafter, another example of the method for determining the vehicle head width will be described.

  FIG. 7 is an explanatory diagram showing an example of a method for specifying the vehicle head width or the vehicle tail width by the moving body detection device 100 of the present embodiment. FIG. 7 schematically shows a longitudinal vehicle head feature projection histogram, and shows only the presence / absence of the frequency of feature points. In FIG. 7A, symbols W1, W2, W3, and W4 schematically represent portions where the distribution of the longitudinal vehicle head feature projection histogram continues.

  As shown in FIG. 7A, the horizontal position candidate range specifying unit 16 determines that the separation size of the continuous portions is different when there are a plurality of portions (horizontal position candidate ranges) in which the distribution of the longitudinal vehicle head feature projection histogram is separated. When the distance is equal to or less than the predetermined separation threshold, the two consecutive portions are specified as one continuous portion (lateral position candidate range). For example, in the example of FIG. 7A, if the separation threshold is 1, the separation dimension between the two consecutive portions W3 and W4 is “1”, so that the two consecutive portions are shown in FIG. 7B. W3 and W4 are combined into one continuous portion W5. Further, the distance between the two adjacent portions W1 and W2 is “5”, and the distance between the two adjacent portions W2 and W3 is “3”. Do n’t merge.

  That is, when there are a plurality of portions (continuous adjacent vertical lines) in which the distribution of the histogram of feature points (vertical vehicle head feature projection histogram) is separated, there is a separation dimension between the portions in which the distribution is continuous. When the distance is equal to or less than the predetermined separation threshold, the portions where the distributions are continuous are specified as one continuous portion (vehicle head width or vehicle tail width). Thereby, the part contained in a vehicle body and the part other than a vehicle body can be distinguished, and the position of a moving body can be detected with a sufficient precision.

  FIG. 8 is an explanatory diagram showing another example of the method for specifying the vehicle head width or the vehicle tail width by the moving body detection device 100 of the present embodiment. The lateral position candidate range specifying unit 16 has a function as a separation threshold setting means for setting a separation threshold. The lateral position candidate range specifying unit 16 sets the separation threshold according to the longer length (width) of two adjacent portions adjacent to each other. For example, as shown in FIG. 8, when the length (number of vertical lines) of the portion in which the distribution of the longitudinal vehicle head feature projection histogram is continuous is L to L1, the separation threshold is set to 1. Further, when the length (number of vertical lines) of the portion in which the distribution of the longitudinal vehicle head feature projection histogram continues is L1 + 1 to L2, the separation threshold is set to 2. Further, when the length (number of vertical lines) of the portion in which the distribution of the longitudinal vehicle head feature projection histogram continues is L2 + 1 to L3, the separation threshold is set to 3.

  That is, when the distribution of the longitudinal vehicle head feature projection histogram is continuous (a plurality of adjacent vertical lines) are adjacent to each other, the width (length) of the continuous portion having the longer width (length) ) Is longer, the separation threshold is increased. As a result, according to the length of the portion where the distribution of the longitudinal vehicle head feature projection histogram continues, it can be combined into one portion, the most probable vehicle head width or vehicle tail width can be specified, The position can be detected with high accuracy.

  Next, a method for generating a lateral four-wheel feature projection histogram will be described. FIG. 9 is an explanatory diagram showing an example of a method for generating a lateral four-wheel feature projection histogram by the moving object detection apparatus 100 of the present embodiment. In FIG. 9, the right diagram shows an example of a captured image, and the left diagram shows an example of a lateral four-wheel feature projection histogram.

  The vehicle width region setting unit 17 has a function as a width region setting means for setting a four-wheeled vehicle width range as a width region in a captured image. The vehicle width region setting unit 17 sets a width region (four-wheel vehicle width range) having a predetermined lateral dimension in the captured image based on the lateral position (x coordinate) of the moving body detected as in the example of FIG. Set to. The predetermined lateral dimension can be set assuming a range of the width of the moving body (vehicle width of a four-wheeled vehicle). The four-wheeled vehicle width range (width region) can be, for example, a region widened by a predetermined number of pixels in the left-right direction with reference to the lateral position (x coordinate) of the moving body.

  The lateral feature point counting unit 13 counts the number of extracted feature points for each horizontal line in the four-wheeled vehicle width range (width region) set by the vehicle width region setting unit 17. In this case, if the absolute value of the difference between the luminance values of an arbitrary target pixel and a pixel that is adjacent to (adjacent to) the target pixel in the horizontal direction is greater than a predetermined luminance threshold Yth3, the feature point is the feature point. Extract as In the example of FIG. 9, when | Y (x, y) −Y (x + 1, y) |> Yth3 is satisfied, the pixel (x, y) is extracted as a feature point.

  The horizontal line is a line along the horizontal direction (x-axis). For example, if the size of the captured image is (vertical × horizontal) pixels, the horizontal line exists as many as the number of vertical pixels. That is, the horizontal direction feature point counting unit 13 counts and adds the number of feature points within the four-wheeled vehicle width range of the captured image in the horizontal direction (for each horizontal line), and adds the total value as the frequency. A histogram (horizontal four-wheel feature projection histogram) is generated.

  When generating the lateral four-wheel feature projection histogram, the luminance threshold Yth3 can be made slightly smaller than the luminance threshold Yth1 because the vicinity of the vehicle head is already limited in the entire captured image.

  Next, a method for detecting the lateral position (x coordinate) of the moving body using the lateral four-wheel feature projection histogram will be described. FIG. 10 is an explanatory diagram illustrating an example of a method for detecting a lateral position of a moving body by the moving body detection apparatus 100 according to the present embodiment. In FIG. 10, the left diagram shows an example of the generated lateral four-wheel feature projection histogram, and the right diagram shows an example of a captured image.

  The moving object coordinate detection unit 18 uses the frequency of the feature points of the horizontal four-wheel feature projection histogram as the second feature instead of the vertical position (y coordinate) detected based on the horizontal image feature projection histogram illustrated in FIG. The vertical position (y coordinate) of the moving body can be detected based on the position of the horizontal line that is greater than or equal to the point threshold value Pth2.

  The second feature point threshold Pth2 may be, for example, 70% of the maximum number of counted feature points (maximum value of frequency) or may be set in advance. The vertical position (y-coordinate) of the moving body can be, for example, the lowest position in the horizontal line where the frequency of the horizontal four-wheel feature projection histogram is equal to or greater than the second feature point threshold value Pth2. For example, if the size of the width area (four-wheeled vehicle width range) of the captured image is (vertical × horizontal) pixels and the y coordinate of the lowermost horizontal line is y2, the y coordinate (vertical position) of the moving object is y = y2. Further, when the maximum value of the counted number of feature points (the maximum value of the frequency) is equal to or less than a predetermined constant Vth, it is possible to disable the coordinate detection of the moving object. In addition, since the range in which the feature points are counted in the horizontal direction is limited to the four-wheeled vehicle width range, not the entire captured image, the second feature point Pth2 can be set to a value smaller than the first feature point Pth1. .

  By limiting the range for counting feature points in the horizontal direction to the four-wheeled vehicle width range (width region) based on the horizontal position (x coordinate) of the moving body, Alternatively, when there is another vehicle that runs in the vicinity of the diagonal rear, the feature points extracted by the other vehicle can be excluded, so that the distribution of the feature points can be prevented from being influenced by another vehicle, Furthermore, the position of the moving body can be detected with high accuracy.

  As described above, the vertical position (y coordinate) of the moving body is detected based on the position of the lowest horizontal line among the horizontal lines in which the counted number of feature points is equal to or greater than the second feature point threshold Pth2. Generally, the position of the license plate is located between the headlight or the tail lamp, and there is no vehicle body below the license plate. Therefore, the position of the lowest horizontal line among the horizontal lines in which the number of feature points in the horizontal direction is equal to or greater than the second feature point threshold Pth2 is near the boundary between the vehicle body portion and the road surface portion in front of the vehicle body (or the rear of the vehicle body). And since it can be considered near the position where the license plate is installed, the position of the moving body can be detected with high accuracy by detecting the position as the vertical position (y coordinate) of the moving body.

  Using the pixels whose luminance difference between adjacent pixels is equal to or greater than the feature point threshold when generating the aforementioned horizontal image feature projection histogram, vertical vehicle head feature projection histogram, and lateral four-wheel feature projection histogram, as feature points Yes. This is intended to quantify luminance changes in characters, headlights, front grills, and the like. However, a continuous luminance change appears as a gradation at the boundary between the shadow and the sun. Since such a gradation portion has many feature points, an error may occur in detection of a moving object. Therefore, it is determined whether or not the extracted feature points are adjacent to each other, and if the number of adjacent consecutive feature points exceeds a predetermined threshold Cmax, the consecutive feature points should not be counted. You can also exclude the gradation part.

  Next, the detection range of the moving body coordinates will be described. FIG. 11 is an explanatory diagram showing an example of the moving object coordinate offset range. The position (coordinates) of the moving body detected by the moving body detection apparatus 100 of the present embodiment can be set to the position of the license plate, for example. When performing license plate detection (character detection, car number detection, etc.), the position (coordinates) of the detected license plate is converted to the coordinates in the original image acquired by the imaging device, and then the license plate of the original image is detected. A required feature amount is calculated based on the position. However, if the position of the license plate on the original image is in the vicinity of the top, bottom, left, and right edges of the original image, it is not possible to calculate a part of the feature amount and the license plate detection result is incorrect, Plate detection may not be possible.

  Therefore, as shown in FIG. 11, a predetermined range (S1, S2, S3, S4) from the top, bottom, left, and right ends of the captured image is set as the moving body coordinate offset range (S1, S2, S3, S4) and detected. When the coordinates of the moving body are within the moving body coordinate offset range, the detection of the moving body is not performed as a coordinate detection failure (for example, a determination result that the moving body does not exist). Thereby, it is possible to prevent erroneous detection when performing license plate detection based on the detection result of the moving body (coordinates of the moving body). Note that when the vehicle head portion is imaged, the moving body coordinate offset range is set so that the vehicle head portion completely enters the field of view. Further, when imaging the vehicle rear portion, the moving object coordinate offset range is set so that the vehicle rear portion is completely within the field of view.

  Next, the operation of the moving object detection apparatus 100 of the present embodiment will be described. 12 and 13 are flowcharts illustrating an example of a moving object detection processing procedure performed by the moving object detection apparatus 100 according to the present embodiment. Hereinafter, the main subject of the processing will be described as the control unit 10 for convenience. The control unit 10 acquires a captured image (S11), generates a differential image based on the acquired captured image (S12), and filters the captured image with the differential image (S13). The generation of the difference image and the filtering process using the difference image may be omitted.

  The control unit 10 extracts feature points for each horizontal line (S14), and generates a horizontal image feature projection histogram by counting the extracted feature points in the horizontal direction (S15).

  The control unit 10 specifies the vehicle head reference position (the lowest position where the frequency of the horizontal image feature projection histogram satisfies the threshold) based on the distribution of the feature points of the horizontal image feature projection histogram (S16), and the specified vehicle head A vehicle head height range widened by a predetermined range in the vertical direction from the reference position is set (S17).

  The control unit 10 extracts feature points for each vertical line within the set vehicle head height range (S18), and counts the extracted feature points in the vertical direction to generate a vertical vehicle head feature projection histogram ( S19).

  The control unit 10 identifies the longest continuous portion of the distribution of feature points in the longitudinal vehicle head feature projection histogram as the vehicle head width (S20). The control unit 10 detects, as the x-coordinate of the moving object, a position at which the distribution of the longitudinal vehicle head feature projection histogram is approximately equally divided in the specified vehicle head width (S21).

  The control unit 10 sets a four-wheeled vehicle width range that is widened by a predetermined range in the left-right direction with reference to the detected x coordinate (S22), and the feature extracted in step S14 within the set four-wheeled vehicle width range. By counting the points in the horizontal direction, a horizontal four-wheel feature projection histogram is generated (S23).

  The control unit 10 detects the y coordinate of the moving object (the lowest position at which the frequency of the horizontal four-wheel feature projection histogram satisfies the threshold) based on the distribution of the feature points of the horizontal four-wheel feature projection histogram (S24). The process is terminated.

  The moving body detection apparatus 100 can also be realized using a general-purpose computer including a CPU, a RAM, and the like. That is, a computer program that defines each processing procedure as shown in FIGS. 12 and 13 is recorded on a recording medium such as a DVD. Then, the computer program recorded on the recording medium is read by an optical disk device or the like, and loaded into a RAM provided in the computer, and the computer program is executed by the CPU, thereby realizing a moving body detection device on the computer. be able to.

  The mobile body detection device 100 according to the present embodiment performs each process of “vehicle type information generation”, “lane information generation”, “position information generation”, and “tracking information generation” in order to determine the vehicle likeness of the mobile body. First, vehicle type information generation, that is, type determination of a moving object will be described.

  The mobile body detection device 100 classifies the mobile body from which the position (coordinates) is detected into a mobile body type (for example, four-wheel vehicle, two-wheel vehicle, unknown). That is, the mobile body detection device 100 also functions as a mobile body determination device that determines the type of the detected mobile body. In addition, as the feature points used when determining the type of the moving object, the feature points extracted in the moving object detection process can be used as they are.

  First, a method for generating a horizontal two-wheel feature projection histogram will be described. FIG. 14 is an explanatory diagram showing an example of a method for generating a lateral two-wheel feature projection histogram by the moving object detection apparatus 100 of the present embodiment. In FIG. 14, the right diagram shows an example of a captured image, and the left diagram shows an example of a horizontal two-wheel feature projection histogram.

  The first type region setting unit 21 sets a four-wheeled vehicle width range as a first partial region, which is a region having a relatively long width (lateral length) on the captured image. The width range of the four-wheeled vehicle is the same as that illustrated in FIG. In the moving object detection process, since the four-wheeled vehicle width range is already set by the vehicle width region setting unit 17, the set four-wheeled vehicle width range can be used as it is.

  As shown in FIG. 14, the second type region setting unit 22 sets a two-wheeled vehicle width range as a second partial region that is a region having a relatively short width (length in the horizontal direction) on the captured image. As the two-wheeled vehicle width range, for example, a range having a predetermined lateral dimension is set in the captured image with reference to the detected lateral position (x coordinate) of the moving body. The predetermined lateral dimension can be set assuming a range of the width of the moving body (vehicle width of the two-wheeled vehicle). The two-wheeled vehicle width range can be, for example, a region widened by a predetermined number of pixels in the left-right direction with reference to the lateral position (x coordinate) of the moving body. Hereinafter, the first partial region is also referred to as a four-wheeled vehicle width range, and the second partial region is also referred to as a two-wheeled vehicle width range.

  The four-wheeled vehicle width range is a region where the vehicle width is relatively long, whereas the two-wheeled vehicle width range is a region where the vehicle width is relatively short, and the type of the moving object (the type is, for example, the width length). Corresponding to different ones). For example, when the four-wheeled vehicle width range corresponds to the first type having a relatively long width, the two-wheeled vehicle width range can correspond to the second type having a relatively short width. In addition, the type of the moving body is not limited to a four-wheeled vehicle and a two-wheeled vehicle. For example, when the first partial region corresponds to the width of a large vehicle, the second partial region corresponds to the width of a normal vehicle. be able to.

  The horizontal feature point counting unit 13 counts the number of extracted feature points for each horizontal line in the two-wheeled vehicle width range set by the second type region setting unit 22. In this case, if the absolute value of the difference between the luminance values of an arbitrary target pixel and a pixel that is adjacent to (adjacent to) the target pixel in the horizontal direction is greater than a predetermined luminance threshold Yth3, the feature point is the feature point. Extract as In the example of FIG. 9, when | Y (x, y) −Y (x + 1, y) |> Yth3 is satisfied, the pixel (x, y) is extracted as a feature point.

  The horizontal line is a line along the horizontal direction (x-axis). For example, if the size of the captured image is (vertical × horizontal) pixels, the horizontal line exists as many as the number of vertical pixels. That is, the horizontal direction feature point counting unit 13 counts and sums the number of feature points within the two-wheeled vehicle width range of the captured image in the horizontal direction (each horizontal line), and a histogram of feature points with the total value as a frequency ( A horizontal two-wheel feature projection histogram) is generated.

  When the horizontal two-wheel feature projection histogram is generated, the luminance threshold value Yth3 is slightly smaller than the luminance threshold value Yth1 because the vicinity of the vehicle head is already limited in the entire captured image as in the horizontal four-wheel feature projection histogram. can do.

  Since the horizontal four-wheel feature projection histogram has already been generated in the moving object detection process, it is not necessary to generate it again.

  Next, a method for determining whether or not to determine the type of the moving object using the generated lateral two-wheel feature projection histogram will be described.

  The feature point distribution position specifying unit 23 has a function as position specifying means for specifying the highest position and the lowest position. The feature point distribution position specifying unit 23 specifies the highest position and the lowest position among horizontal line positions where the number of feature points counted by the horizontal feature point counting unit 13 is equal to or greater than a predetermined feature point threshold.

  The feature point threshold can be set as appropriate, and can be set to 1, for example. When the feature point threshold is 1, it is determined whether or not a feature point exists. That is, among the positions of the horizontal lines where the frequency of the horizontal four-wheel feature projection histogram exceeds 0 (1 or more), the uppermost position is the highest position, and the position of the horizontal line where the frequency exceeds 0 (1 or more). Of these, the lowest position is the lowest position. Similarly, in the horizontal two-wheel feature projection histogram, among the positions of the horizontal lines where the frequency exceeds 0 (1 or more), the uppermost position is the highest position, and the frequency of the horizontal line where the frequency exceeds 0 (1 or more). Of the positions, the lowest position is the lowest position.

  FIG. 15 is an explanatory diagram illustrating an example of a method for determining whether or not to perform type determination by the moving object detection device 100 according to the present embodiment. In FIG. 15, the start point indicates the position of the upper end of the captured image, and the end point indicates the position of the lower end of the captured image. The middle is a center position of the captured image, that is, a position obtained by dividing the distance between the start point and the end point into two equal parts. The moving body type determination unit 24 has a function as a determination unit that determines the type of the moving body.

  As shown in FIG. 15A, the moving body type determination unit 24 has the uppermost position specified by the feature point distribution position specifying unit 23 in the upper half area of the captured image, and the specified lowest position is the lower half of the captured image. If it is in the area, the type of the moving object is determined (determined if determined).

  Length of feature point distribution (histogram) when the vehicle head part or car tail part such as license plate, headlight, tail lamp, bumper, front grille, etc. of the moving object (vehicle) is reflected near the center of the captured image (Number of horizontal lines) becomes longer, the highest position is in the upper half area of the captured image, and the lowest position is in the lower half area of the captured image. it can.

  Further, as shown in FIGS. 15B and 15C, when the vehicle head portion or the vehicle tail portion is not reflected near the center of the captured image, the highest position is in the lower half area of the captured image, or Since the lower position is in the upper half area of the captured image, the type of the moving body cannot be accurately determined, and therefore the type of the moving body is not determined. Thereby, while being able to determine the classification of a mobile body accurately, it can prevent determining incorrectly. Note that the horizontal four-wheel feature projection histogram is the same as that in FIG.

  Next, a method for determining the type of mobile object will be described. FIG. 16 is an explanatory diagram illustrating an example of feature amounts of the lateral four-wheel feature projection histogram and the lateral two-wheel feature projection histogram by the moving object detection device 100 of the present embodiment, and FIG. 17 illustrates the moving object of the present embodiment. It is explanatory drawing which shows an example of the classification determination method of the mobile body by the detection apparatus. The feature amount includes a total value N1 of feature points included in the lateral four-wheel feature projection histogram, a total value N2 of feature points included in the lateral two-wheel feature projection histogram, and an average value V of the frequencies of the lateral two-wheel feature projection histogram. Etc.

  The lateral feature point counting unit 13 has a function as a counting unit, and includes the number N1 of feature points included in the four-wheeled vehicle width range (first partial region) having a predetermined lateral width in the captured image, and the predetermined The number N2 of feature points included in the two-wheeled vehicle width range (second partial region) having a width smaller than the width is counted. That is, the number N1 of feature points is, for example, the total value of the feature points included in the lateral four-wheel feature projection histogram, and the number N2 of feature points is, for example, the number of feature points included in the lateral two-wheel feature projection histogram. It is the total value.

  The moving body type determination unit 24 determines the type of the moving body based on the ratio (for example, N2 / N1) of the number of feature points N1 and N2 counted by the horizontal direction feature point counting unit 13. For example, when the second type mobile body with a short width is shown on the captured image, the number N1 of feature points included in the four-wheeled vehicle width range (first partial region) is reduced, so the ratio (N2 / N1 ) Value increases. In addition, when the first type mobile body having a long width is reflected on the captured image, the number N1 of feature points included in the four-wheeled vehicle width range (first partial region) increases, so the ratio (N2 / N1 ) Value becomes smaller. Since a pixel whose pixel value (luminance value) difference between adjacent (adjacent) pixels is equal to or greater than a predetermined threshold is extracted as a feature point, it is on the road surface caused by the influence of external environment changes such as sunshine changes and weather changes. The shadow part is not extracted as a feature point, and it is not affected by the shadow on the road surface. The brightness change of the moving object (for example, vehicle), headlights, front grille, tail lamp, etc., depends on the moving object type. Therefore, the type of the moving object can be accurately determined according to the ratio of the number of feature points (N2 / N1).

  More specifically, in FIG. As shown in FIG. 1, the moving body type determination unit 24 determines that the moving body is the first type when the ratio (N2 / N1) of the number N2 to the number N1 of feature points is smaller than a predetermined first threshold Th1. judge. The first type is, for example, a relatively long moving body, which is a four-wheeled vehicle. When the first type mobile body having a long width is reflected on the captured image, the number N1 of feature points included in the first partial region is increased, and thus the value of the ratio (N2 / N1) is decreased. Therefore, when the ratio (N2 / N1) is smaller than the predetermined first threshold Th1, it can be determined that the moving body is the first type.

  In addition, in FIG. As shown in FIG. 2, when the ratio (N2 / N1) of the number N2 to the number N1 of feature points is equal to or greater than the second threshold Th2 (Th2> Th1), the moving body type determination unit 24 determines that the moving body is the first. It is determined that the second type is different from the type. The second type is, for example, a relatively short movable body and is a two-wheeled vehicle. When the second type moving body with a short width is shown on the captured image, the number N1 of feature points included in the first partial region is reduced, and thus the value of the ratio (N2 / N1) is increased. Therefore, when the ratio (N2 / N1) is larger than the predetermined second threshold Th2, it can be determined that the moving body is the second type.

  In addition, when the ratio (N2 / N1) of the number N2 to the number N1 of feature points is equal to or larger than the first threshold Th1 and smaller than the second threshold Th2, the moving body type determination unit 24 counts the number N1 and the number of feature points. Whether the moving body is the first type or the second type is determined according to the difference between N2 (for example, N1-N2) and a predetermined third threshold Th3. When the ratio (N2 / N1) is larger than the first threshold Th1 and smaller than the second threshold Th2, it cannot be determined whether the ratio is the first type or the second type. Therefore, the type of the moving body is determined according to the difference between the number N1 and the number N2 of feature points (N1−N2) and the third threshold Th3.

  For example, when the second type mobile body with a short width is shown on the captured image, the number N1 of feature points included in the four-wheeled vehicle width range (first partial region) is reduced, so the difference (N1-N2 ) Value becomes smaller. In addition, when the first type mobile body having a long width is reflected on the captured image, the number N1 of feature points included in the four-wheeled vehicle width range (first partial region) increases, so the difference (N1-N2 ) Value increases.

  And No. of FIG. As shown in FIG. 4, when the difference (N1-N2) between the number N1 and the number N2 of feature points is equal to or greater than the third threshold Th3, it is determined that the type is a long first type (four-wheeled vehicle). can do. When the difference (N1−N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3, it may be determined that the second type (two-wheeled vehicle) has a short width.

In addition, in FIG. As shown in FIG. 3, the moving body type determination unit 24 determines the number N2 of feature points (more specifically, when the difference (N1-N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3. When the average value V) of the frequency in the horizontal two-wheel feature projection histogram is smaller than the predetermined fourth threshold Th4, it is determined that the moving body is the second type. When the difference (N1−N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3, it can be determined that the second type has a short width. In the first type mobile body having a long width such as a four-wheeled vehicle, since there are many front grills, characters, etc., the number N2 of feature points increases, and the second type mobile body having a short width such as a two-wheel vehicle. Then, since there are few front grills, letters, etc., the number N2 of feature points is reduced. For this reason, when the number N2 of feature points (the average value V of the frequency of the horizontal two-wheel feature projection histogram) is smaller than the predetermined fourth threshold Th4, it is determined that the second type has a short width such as a two-wheel vehicle. be able to. Therefore, when the difference (N1-N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3, the number N2 of feature points (the average value V of the frequency of the horizontal two-way feature projection histogram) is further determined. By determining whether or not it is smaller than the fourth threshold Th4, it is possible to accurately determine the type of the moving object. Note that the number N2 of feature points may be used in place of the average value V of the frequencies in the horizontal two-wheel feature projection histogram.

  The mobile body type determination unit 24 determines the type of mobile body in time series. As shown in FIG. 5, when the difference (N1-N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3, the number N2 of feature points (the average value V of the frequency of the lateral two-wheel feature projection histogram) ) Is equal to or greater than the fourth threshold Th4, it is determined that the type of the moving object is the same as the type most recently determined. When the difference (N1−N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3, it can be determined that the second type has a short width. However, when the number N2 of feature points (the average value V of the frequency of the horizontal two-wheel feature projection histogram) is equal to or greater than a predetermined fourth threshold Th4, a long-width first type mobile body such as a four-wheel vehicle is used. It can also be determined that there is. Therefore, when the difference (N1−N2) between the number N1 and the number N2 of feature points is smaller than the third threshold Th3, the number N2 of feature points (the average value V of the frequency of the horizontal two-way feature projection histogram) is the fourth. When it is equal to or greater than the threshold Th4, it is impossible to accurately determine whether the type is the first type or the second type. Therefore, it is determined that the type of the moving object is not different from the type determined most recently. It is determined that the type is the same. As a result, even when a timing at which it is difficult to determine the type of the moving object, the latest determination result can be used, and it is possible to prevent the type from being erroneously determined. Note that the number N2 of feature points may be used in place of the average value V of the frequencies in the horizontal two-wheel feature projection histogram.

  Next, a method for generating lane information will be described. When there are a plurality of lanes within the field of view of the imaging device, only a part of the vehicle body is photographed for vehicles in adjacent lanes, which is inappropriate as an image of the vehicle. Therefore, in order not to output an inappropriate image, the lane in which the moving body on the captured image is traveling is determined.

  FIG. 18 is an explanatory diagram illustrating an example of a lane determination method performed by the moving object detection device 100 according to the present embodiment. As shown in FIG. 18A, the captured image is divided into a central region, two separation regions (a left separation region and a right separation region), a left end vicinity region, and a right end vicinity region.

  The central region corresponds to a region where a lane in the field of view is shown, and is a region having a predetermined first width in the lateral direction from the center of the captured image.

  The left end vicinity region and the right end vicinity region correspond to regions where adjacent lanes are shown, and are regions having a predetermined vicinity width in the horizontal direction from the left end or the right end of the captured image.

  The separation region is a region having a predetermined second width that is separated with a central region in between. The entire region including the separation region, the left end vicinity region, and the right end vicinity region is a range in which the horizontal position (x coordinate) of the moving body may exceed the left end or the right end of the captured image.

  As illustrated in FIG. 18B, the lane information generation unit 31 determines that the vehicle is an adjacent lane when the detected x-coordinate (lateral position) of the moving body is within the left end vicinity region or the right end vicinity region. In this case, the moving body type determination unit 24 does not determine the moving body type. If the x-coordinate of the moving object is in the region near the left end or the region near the right end, the type of the moving object is not determined assuming that the moving object is not reflected in the captured image to the extent that the type of the moving object can be determined. . Thereby, it can prevent determining the classification of a moving body accidentally, and can determine with sufficient precision.

  As illustrated in FIG. 18B, the lane information generation unit 31 determines that the lane is in the field of view when the detected x coordinate of the moving body is within the central region. In this case, the moving body type determination unit 24 determines the type of the moving body. When the x coordinate of the moving object is in the center area, the vehicle head part or the tail part of the moving object (vehicle) such as the license plate, headlight, tail lamp, bumper, front grille, etc. is reflected near the center of the captured image. The type of the moving object can be determined with high accuracy.

  As illustrated in FIG. 18B, the lane information generation unit 31 determines that the detected lane is an adjacent lane when the detected moving object has an x coordinate that is not within the central region and the detected moving object has a width smaller than a predetermined width threshold. judge. In this case, the moving body type determination unit 24 does not determine the moving body type.

  In other words, if feature points near the vehicle head are distributed outside the field of view, it is determined that the vehicle width is within the field of view if it is greater than or equal to the width threshold of the vehicle width, and is outside the field of view if it is less than the width threshold. To do. Here, when the feature points are distributed outside the field of view, the start point (left end point) and end point (right end point) of the “vehicle head width” specified when moving objects are detected match the left and right ends of the captured image, respectively. It can be determined by whether or not to do so. Further, the vehicle width falling within the field of view can be determined by the length of the “vehicle head width”.

  The lane information generation unit 31 determines that the lane is in the field of view when the detected mobile object has an x coordinate that is not within the central region and the detected mobile object has a width greater than a predetermined width threshold. In this case, the moving body type determination unit 24 determines the type of the moving body.

  If the X coordinate of the moving object is not within the central area, the horizontal position of the moving object may not exist in the captured image beyond the left or right edge of the captured image, so the type of the moving object is accurately determined. You may not be able to. Therefore, for example, when the width of the moving body is larger than a predetermined width threshold, the moving body type is determined on the assumption that the moving body appears in the captured image to the extent that the moving body type can be determined. On the other hand, when the width of the moving body is smaller than the predetermined width threshold, the type of the moving body is not determined assuming that the moving body is not reflected in the captured image to the extent that the type of the moving body can be determined. Thereby, it can prevent determining the classification of a moving body accidentally, and can determine with sufficient precision.

  The lane information generation unit 31 determines that the lane is within the field of view when the detected X coordinate of the moving body is within the separation region. In this case, the moving body type determination unit 24 determines the type of the moving body. If the X coordinate of the moving object is not within the central area, the horizontal position of the moving object may not exist in the captured image beyond the left or right edge of the captured image, so the type of the moving object is accurately determined. You may not be able to. Therefore, for example, when there is a lateral position of the moving body in two separation areas separated by a central region and having a predetermined second width, the moving body can be determined to the extent that the type of the moving body can be determined. , The type of the moving object is determined. Thereby, it can prevent determining the classification of a moving body accidentally, and can determine with sufficient precision.

  Next, a method for generating position information will be described. FIG. 19 is an explanatory diagram illustrating an example of a method for generating position information by the moving object detection device 100 according to the present embodiment. In FIG. 19, the left diagram shows a lateral four-wheel feature projection histogram, and the right diagram shows a captured image.

  In order to determine that the vicinity of the license plate of the vehicle body has been shot, it is determined whether or not the shot position is correct. Since the license plate is mounted near the headlight of the vehicle body (in the case of the car tail, near the tail lamp), when the license plate is shot near the center of the captured image, the vehicle body exists below the captured image. I understand that I don't.

  The position information generation unit 32 sets a vehicle position determination region on the lower side of the lateral four-wheel feature projection histogram, and the histogram value (number of feature points) in the vehicle position determination region is set as shown in FIG. 19A. If it is less than the threshold value, it is determined that the image is taken at the correct position, and information indicating that the position of the moving object is correct is generated.

  Further, as shown in FIG. 19B, the position information generation unit 32 determines that the position is not the correct position when the histogram value (number of feature points) in the vehicle position determination area is greater than or equal to the threshold value, and the position of the moving object is correct. Generates no information. Note that the vehicle position determination region can be a value of about 10% of the number of pixels in the vertical direction of the captured image.

  Next, a method for generating tracking information will be described. FIG. 20 is an explanatory diagram illustrating an example of a tracking method of a moving object by the moving object detection device 100 of the present embodiment, and FIG. 21 is an explanatory diagram illustrating an example of tracking information by the moving object detection device 100 of the present embodiment. It is. In FIG. 20, the coordinates (x (0), y (0)) are the xy coordinates of the moving body at the start of tracking, and the coordinates (x (t−1), y (t−1)) are the latest (previous). ) The xy coordinates of the moving body in the frame (captured image), and the coordinates (x (t), y (t)) are the xy coordinates of the moving body in the current (now) frame (captured image).

  A tracking process is performed to determine whether or not a moving object that is continuous in time series from the trajectory of the xy coordinates of the moving object is the same object, and whether or not it is traveling linearly within the field of view. I do. In the tracking process, the xy coordinates of the moving object detected in the current frame (for example, the captured image at time t) are within a range that can be moved from the latest frame (for example, the captured image at time t−1). It is determined that the amount of movement from the start of tracking is greater than or equal to a threshold value.

  The determination as to whether or not it is a movable range is performed by setting a movable range with respect to the xy coordinates of the moving object in the most recent frame, and the xy coordinates of the moving object in the current frame being the movable range. Judgment is made based on whether or not it exists inside. In this case, the number of times determined to be a continuously movable range between frames is counted as the number of tracking times, and the higher the number of tracking times, the higher the reliability with which it is determined that the moving object is traveling linearly. .

  Also, whether or not the amount of movement from the start of tracking is greater than or equal to the threshold is determined by comparing the xy coordinates of the moving object at the start of tracking and the movement in the current frame for a tracking number of one or more. Judgment is based on the distance to the xy coordinates of the body. When the distance is small, it is determined that it is not a moving body and it is determined that tracking has failed. If the distance is large, it is determined that the object is a moving body and the tracking is successful.

  The movable range can be set as follows because the traveling direction differs between vehicle head measurement and vehicle tail measurement. In the case of vehicle head measurement, the movable range left end in the current frame = x (t−1) −constant d, the movable range right end = x (t−1) + constant d, and the movable range upper end = y (t− 1) —constant h, movable range lower end = lower end of captured image.

  Further, in the case of vehicle rear measurement, the movable range left end in the current frame = x (t−1) −constant d, the movable range right end = x (t−1) + constant d, the movable range upper end = imaging. The upper end of the image and the lower end of the movable range = y (t−1) + constant h.

  As illustrated in FIG. 21, the tracking information generation unit 33 determines whether the absolute value of the difference between the x coordinate at the start of tracking of the mobile object and the current x coordinate is greater than a threshold value Dx, or at the start of tracking of the mobile object. If the absolute value of the difference between the y-coordinate and the current y-coordinate is larger than the threshold value Dy, tracking information indicating that the tracking is successful and the mobile body is generated.

  In addition, the tracking information generation unit 33 has an absolute value of a difference between the x coordinate at the start of tracking of the moving object and the current x coordinate equal to or less than the threshold Dx, and the y coordinate at the start of tracking of the moving object and the current coordinate. If the absolute value of the difference from the y-coordinate is equal to or less than the threshold value Dy, tracking information indicating that the tracking has failed and that the object is not a moving object is generated.

  In the case of vehicle head measurement, the vehicle that has entered the field of view from the upper side of the captured image is tracked. In the case of vehicle tail measurement, the vehicle that has entered the field of view from the lower side of the captured image is tracked. Therefore, the tracking can be started when the xy coordinates of the moving object exist within the tracking start range designated in advance. For example, in the case of vehicle head measurement, the upper end of the tracking start range coincides with the upper end of the captured image, and the lower end of the tracking start range can be a predetermined number of pixels downward from the upper end of the captured image. Further, in the case of vehicle tail measurement, the upper end of the tracking start range can be a predetermined number of pixels upward from the lower end of the captured image, and the lower end of the tracking start range can be the lower end of the captured image.

  Next, the operation of the moving object detection apparatus 100 of the present embodiment will be described. FIG. 22 is a flowchart illustrating an example of a processing procedure for determining the type of a moving object by the moving object detection device 100 according to the present embodiment. Hereinafter, the main subject of the processing will be described as the control unit 10 for convenience. The control unit 10 acquires the coordinates (position) of the moving body (S31), acquires a lateral four-wheel feature projection histogram (S32), and acquires feature points extracted for each horizontal line (S33). The coordinates (position) of the moving object, the lateral four-wheel feature projection histogram, and the feature points are all obtained when detecting the position of the moving object.

  The control unit 10 sets a two-wheeled vehicle width range on the captured image (S34), and generates a lateral two-wheel feature projection histogram (S35). The control unit 10 calculates the average value V of the frequency in the horizontal direction two-wheel feature projection histogram (S36), calculates the total value N1 of the frequency (feature point) in the horizontal direction four-wheel feature projection histogram (S37), and then in the horizontal direction. A total value N2 of the frequencies (feature points) of the two-wheel feature projection histogram is calculated (S38).

  The control unit 10 determines the type of the moving body based on the calculated total values N1 and N2 and the average value V (S39), and ends the process. The determination conditions and determination results for the type of mobile object are as illustrated in FIG.

  The moving body detection apparatus 100 can also be realized using a general-purpose computer including a CPU, a RAM, and the like. That is, a computer program that defines each processing procedure as shown in FIG. 22 is recorded on a recording medium such as a DVD. Then, the computer program recorded on the recording medium is read by an optical disk device or the like, and loaded into a RAM provided in the computer, and the computer program is executed by the CPU, so that the mobile object detection device (mobile object) is executed on the computer. Determination device) can be realized.

  The xy coordinates and object information (vehicle type information, lane information, position information, tracking information) of the moving body detected by the moving body detection apparatus 100 of the present embodiment are used as the moving body information, and the moving body detection apparatus of the present embodiment. It is possible to output to a device (hereinafter referred to as an external device) that performs plate detection processing, vehicle determination processing, and the like, which is separate from 100.

  In the external device that performs the plate detection process, the license plate detection process is performed based on the original image captured by the imaging device. Therefore, in the moving body detection process of the mobile body detection device 100, a captured image obtained by reducing the original image is used. In such a case, the detected xy coordinates of the moving body may be converted to the coordinates of the original image.

  In addition, the external device that performs the vehicle determination process determines whether the vehicle is a vehicle based on the passing information, object information, and the like of the moving body for the moving body that failed to detect the license plate. Therefore, the object information output by the moving body detection apparatus 100 can represent two values, that is, a vehicle and a non-vehicle. Determination conditions in the external device are, for example, as follows. That is, (1) the vehicle determination process is effective (for example, when it is set to perform the vehicle determination process), the tracking result of the moving object is successful, and the tracking number of the moving object> the threshold value, and When the position of the moving body is a lane in the field of view, it is determined that the vehicle is a vehicle. (2) If the above condition (1) is not satisfied, it is determined that the vehicle is not a vehicle.

  In the above-described embodiment, one direction of the captured image is described as the horizontal direction of the captured image and the other direction is the vertical direction. However, one direction of the captured image is the vertical direction of the captured image, and the other direction is the horizontal direction. You can also.

  The embodiments and examples disclosed above should be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above embodiments and examples but by the scope of claims, and is intended to include all modifications and variations within the meaning and scope equivalent to the scope of claims. .

DESCRIPTION OF SYMBOLS 10 Control part 11 Difference image generation part 12 Feature point extraction part 13 Lateral direction feature point counting part 14 Vehicle head height area setting part 15 Vertical direction feature point counting part 16 Lateral position candidate range specification part 17 Vehicle width area setting part 18 Mobile body Coordinate detection unit 20 Vehicle type information generation unit 21 First type region setting unit 22 Second type region setting unit 23 Feature point distribution position specification unit 24 Moving body type determination unit 31 Lane information generation unit 32 Location information generation unit 33 Tracking information generation unit

Claims (9)

In a moving body detection apparatus that detects a moving body using a captured image in which a plurality of pixels are arranged in a matrix,
Extraction means for extracting feature points based on the pixel value of each pixel of the captured image;
First counting means for counting the number of feature points extracted by the extracting means for each one-directional line of the captured image;
Detecting means for detecting the vertical position of the moving body based on the position of the one-way line where the number of feature points counted by the first counting means is equal to or greater than a predetermined first feature point threshold;
A height region setting unit that sets a height region having a predetermined vertical dimension in the captured image with reference to the vertical position detected by the detection unit;
Second counting means for counting the number of feature points extracted by the extraction means for each other direction line in the height area set by the height area setting means,
The detection means includes
The lateral position of the moving body is detected based on the position of the other direction line where the feature points counted by the second counting means exist ,
A width region setting unit that has a predetermined lateral dimension based on the lateral position detected by the detection unit and sets a width region different from the height region in the captured image;
The first counting means includes
The number of feature points extracted by the extraction unit is counted for each one-way line of the width region set by the width region setting unit,
The detection means includes
Instead of the detected vertical position, the position of the one-way line where the number of feature points extracted for each one-way line of the width region set by the width region setting means is equal to or greater than a predetermined second feature point threshold value. Based on this, the moving body detection apparatus is configured to further detect the vertical position of the moving body. First pixel specifying means for specifying a first pixel in which a difference between pixel values of corresponding pixels of a first captured image and a second captured image at different imaging time points is greater than a first difference threshold;
Second pixel specifying means for specifying a second pixel having a pixel value difference between corresponding pixels of each of the first captured image and the second captured image that is larger than a second difference threshold smaller than the first difference threshold; ,
Differential image generation means for generating a differential image composed of the first pixel specified by the first pixel specifying means and a pixel adjacent to the first pixel and composed of the second pixel;
Processing means for filtering either the first captured image or the second captured image with the difference image generated by the difference image generating means;
The extraction means includes
The moving body detection device according to claim 1, wherein the feature point is extracted based on a pixel value of each pixel of a captured image obtained by filtering by the processing means. The detection means includes
Based on the position of the lowest unidirectional line among the unidirectional lines in which the number of characteristic points counted by the first counting means is equal to or greater than the first or second characteristic point threshold, the vertical position of the moving body moving object detection apparatus according to claim 1 or claim 2, characterized in that is arranged to detect. Among the other direction lines where the feature points counted by the second counting means exist, range specifying means for specifying a lateral position candidate range constituted by a plurality of continuously adjacent other direction lines,
The detection means includes
The horizontal position of the moving body is detected based on the position of another direction line representing the feature point existing in the horizontal position candidate range specified by the range specifying means. The moving body detection apparatus of any one of Claim 1 thru | or 3 . The range specifying means includes
The mobile object detection according to claim 4 , wherein when there are a plurality of lateral position candidate ranges apart from each other, the lateral position candidate range having the maximum number of other direction lines is specified. apparatus. The range specifying means includes
When there are a plurality of lateral position candidate ranges apart from each other, if the distance between adjacent lateral position candidate ranges is equal to or smaller than a predetermined separation threshold, the adjacent lateral position candidate ranges are specified as one lateral position candidate range. The moving body detection apparatus according to claim 5 , wherein the moving body detection apparatus is configured as described above. The moving body detection according to claim 6 , further comprising a separation threshold setting unit that sets the separation threshold according to a larger number of other direction lines in each of the adjacent lateral position candidate ranges. apparatus. In a computer program for causing a computer to detect a moving object using a captured image in which a plurality of pixels are arranged in a matrix,
On the computer,
Extracting feature points based on pixel values of each pixel of the captured image;
Counting the number of feature points extracted for each unidirectional line of the captured image;
Detecting the vertical position of the moving body based on the position of the one-way line where the number of counted feature points is equal to or greater than a predetermined first feature point threshold;
Setting a height region having a predetermined vertical dimension in the captured image with reference to the detected vertical position;
Counting the number of feature points extracted for each other direction line in the set height region;
Detecting a lateral position of the moving body based on the position of the other direction line where the counted feature points exist ;
Setting a width region in the captured image having a predetermined lateral dimension based on the detected lateral position and different from the height region;
Counting the number of feature points extracted for each one-way line of the set width region;
Instead of the detected vertical position, the moving body is based on the position of the one-way line where the number of feature points extracted for each one-way line in the set width region is equal to or greater than a predetermined second feature point threshold. And a step of further detecting a vertical position of the computer program. In a moving object detection method by a moving object detection device that detects a moving object using a captured image in which a plurality of pixels are arranged in a matrix,
Extracting feature points based on pixel values of each pixel of the captured image;
Counting the number of feature points extracted for each unidirectional line of the captured image;
Detecting the vertical position of the moving body based on the position of the one-way line where the counted number of feature points is equal to or greater than a predetermined first feature point threshold;
Setting a height region having a predetermined vertical dimension in the captured image with reference to the detected vertical position;
Counting the number of feature points extracted for each other direction line in the set height region;
Detecting the lateral position of the moving body based on the position of the other direction line where the counted feature points exist;
Setting a width region in the captured image having a predetermined lateral dimension based on the detected lateral position and different from the height region;
Counting the number of feature points extracted for each one-way line of the set width region;
Instead of the detected vertical position, the moving body is based on the position of the one-way line where the number of feature points extracted for each one-way line in the set width region is equal to or greater than a predetermined second feature point threshold. Further detecting the vertical position of the moving body.

Images