JP2018180945A - Object detection apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection apparatus and a program capable of accurately detecting a detection target object irrespective of the size of the detection target object on an image.SOLUTION: In an object detection apparatus 10, an object detection unit 22 outputs the position and the size of a detection frame representing a detection target object included in each detection processing frame set at each position from an input image by using CNN, an adjusting unit 30 adjusts a threshold to be used for integrating detection frames according to the properties of the output detection frames, and an integration unit 24 integrates the detection frames in which the degree of overlapping with other detection frame is equal to or greater than the threshold on the basis of the position and the size of the detection frame output for each detection processing frame and outputs a final object detection frame.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an object detection apparatus and program, and more particularly to an object detection apparatus and program for detecting an object to be detected from an image.

  Conventionally, using a convolutional neural network, a detection processing frame larger than a detection target object included in an image is parallel-shifted, and a plurality of detection frames and a certainty for the same object are output, and detection of an overlap of a predetermined size or more There is known a technique for performing high-accuracy object detection by repeating processing of adding a certainty with respect to a pair of frames and replacing it with one detection frame obtained by averaging coordinates of detection frames (Non-Patent Document 1).

  Also, using a convolutional neural network and a recursive neural network, a detection processing frame larger than the detection target object included in the image is parallelly moved, and a plurality of detection frames and certainty for the same object are output, and the positional relationship of the detection frames There is known a technique for performing high-accuracy object detection by integrating detection frames based on the magnitude relationship between accuracy and the like (Non-Patent Document 2).

  In the techniques of Non-Patent Documents 1 and 2 described above, many detection processes performed while translating the detection process frame share the processing result of the overlapping portion of the detection process frame according to the characteristics of the convolutional neural network. Because processing is easily parallelized, processing can be performed efficiently. In addition, by using a detection processing frame larger than the detection target object, it is possible to perform detection processing in consideration of an image around the detection target object, or in a method using a detection processing frame having the same size as the detection target object. It has the feature that a large number of detection processes become unnecessary by scanning detection processing frames of various shapes.

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun: Overfeat: Integrated Recognition, Localization and Detection, ICLR 2014 R. Stewart and M. Andriluka: End-to-end people detection in crowded scenes, arXiv, 2015

  However, in the technique described in Non-Patent Document 1, the amount of movement when translating the detection processing frame is fixed by the configuration of the convolutional neural network, and when the size of the detection target object is large, the number of detection frames is large. While the accuracy is added, while the accuracy of a small number of detection frames is added when the size of the detection target object is small, false detection is detected when the size of the detection target object is large using a threshold of certain accuracy. When the size of the target object is small, undetection tends to occur.

  In addition, in the technique described in the above Non-Patent Document 1, the accuracy of a large number of detection frames can be added even if the size of the object to be detected is small by performing the detection process again after enlarging the input image for this problem However, the processing speed is reduced because processing for the enlarged input image is required.

  In addition, in the technique described in the above non-patent document 2, accuracy addition is not performed in detection frame integration, but when a large detection frame is output by false detection, detection of other objects having an overlap in the detection frame is detected By deleting the frame by mistake, the undetected number increases and performance degradation is likely to occur.

  The present invention has been made to solve the above-mentioned problems, and provides an object detection apparatus and program that can accurately detect a detection target object regardless of the size of the detection target object on an image. The purpose is

  In order to achieve the above object, an object detection apparatus according to the present invention detects an object to be detected included in the detection processing frame for each detection processing frame set at each position using an input image and a detection model. Detection unit that outputs the position and size of the detection frame to be displayed, and the position and size of the detection frame output for each of the detection processing frames, detection that the overlapping degree overlapping with other detection frames is equal to or higher than a threshold Adjustment that adjusts the threshold used when integrating the detection frame according to the nature of the detection frame output by the detection unit, including an integration unit that integrates the frames and outputs the final object detection frame It further comprises a part.

  In the program according to the present invention, the position of the detection frame representing the detection target object included in the detection processing frame is set for each detection processing frame set at each position using the computer from the input image and using the detection model. And a detection unit that outputs a size, an adjustment unit that adjusts a threshold used when integrating detection frames according to the nature of the detection frame output by the detection unit, and the detection output for each of the detection processing frames This program is a program to function as an integration unit that integrates detection frames whose overlap degree with another detection frame is equal to or greater than a threshold based on the position and size of the frame and outputs the final detection frame as an object detection frame. .

  According to the present invention, the position and size of the detection frame representing the detection target object included in the detection processing frame for each detection processing frame set at each position using the detection model from the input image by the detection unit Output. The adjustment unit adjusts the threshold used when integrating the detection frames in accordance with the nature of the detection frame output by the detection unit. Then, based on the position and size of the detection frame output for each of the detection processing frames, the integration unit integrates detection frames in which the overlapping degree overlapping with another detection frame is equal to or higher than the threshold, Output as an object detection frame.

  As described above, it is possible to accurately detect the detection target object regardless of the size of the detection target object on the image by adjusting the threshold used when integrating the detection frames according to the nature of the detection frame. it can.

  As described above, according to the object detection apparatus and program of the present invention, the size of the detection target object on the image is adjusted by adjusting the threshold used when integrating the detection frames according to the nature of the detection frame. The effect of being able to detect the detection target object with high accuracy is obtained regardless of.

FIG. 1 is a block diagram showing an object detection device according to a first embodiment of the present invention. It is an image figure for demonstrating a detection process frame. It is a figure for demonstrating the method to integrate several detection frames. It is a flowchart which shows the content of the detection processing routine in the object detection apparatus which concerns on the 1st Embodiment of this invention. It is an image figure which shows the detection result in a prior art. It is a graph which shows an experimental result.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to an object detection apparatus for detecting a vehicle as a detection target object will be described as an example.

First Embodiment
As shown in FIG. 1, the object detection apparatus 10 according to the first embodiment detects an area from a detection target area, and an imaging apparatus 12 that generates a captured image and a captured image output from the imaging apparatus 12. A computer 14 storing a program for realizing a detection processing routine for detecting a vehicle as a target object and determining whether the vehicle is detected, and a display device 16 for displaying the processing result in the computer 14 Is equipped.

  The imaging device 12 captures an area to be detected and generates an image signal (not shown); and an A / D conversion unit (not shown) that A / D converts the image signal generated by the imaging unit. And an image memory (not shown) for temporarily storing an A / D converted image signal.

  The computer 14 is configured to include a CPU, a ROM storing a program of a detection processing routine to be described later, a RAM storing data and the like, and a bus connecting these. Describing this computer 14 by functional blocks divided into function realizing means determined based on hardware and software, as shown in FIG. 1, a captured image which is a gray-scale image output from the imaging device 12 is input An object input unit 20, an object detection unit 22 for outputting a processing frame representing a vehicle within the detection processing frame for each detection processing frame from the captured image input by the image input unit 20, an object detection The processing frame is integrated based on the adjusting unit 24 that adjusts the method of integrating the processing frame according to the nature of the processing frame output by the unit 22, and the processing frame is output by the object detection unit 22. And an integration unit 26 for outputting as a vehicle area representing

  The object detection unit 22 inputs, for each detection processing frame set at each position, an image cut out from the captured image by the detection processing frame into a CNN (Convolutional Neural Network, convolutional neural network) learned in advance, The position and size of the detection frame representing the detection target object included in the detection processing frame, and the accuracy indicating the detection object likeness are output.

  Here, as shown in FIG. 2, the size of the detection processing frame is set larger than the size on the image of the detection target object assumed, and the detection processing is performed at each position in a predetermined parallel movement unit. A frame is set.

  Specifically, the CNN centers on 20 × 15 = 300 grids as shown in FIG. 2 for one image while translating a detection processing frame larger than the detection target object included in the image by CNN. Perform 300 times of detection processing.

  The CNN takes an image as input, and is learned in advance so as to output the position, size (width, height) and accuracy of a detection frame representing an object to be detected included in the image. Specifically, the CNN is previously learned so that the detection frame is output when the detection frame surrounding the detection target object overlaps the detection target area (see FIG. 2) defined at the center of the detection processing frame. .

  The adjustment unit 24 adjusts the threshold used when integrating the detection frames in accordance with the nature of the detection frame output by the object detection unit 22. In the present embodiment, in accordance with the size of the detection frame output by the object detection unit 22, the threshold value regarding the overlapping degree is adjusted. Specifically, the larger the size of the detection frame, the larger the threshold value for the overlapping degree.

  The integration unit 26 determines the number of detection frames whose overlapping degree overlapping with another detection frame is equal to or higher than the adjusted threshold value, based on the position, size, and accuracy of the detection frame output for each detection processing frame. If there is a threshold value or more regarding the above, these detection frames are integrated into one detection frame with the highest accuracy, and then output as a final object detection frame.

  Specifically, as shown in FIG. 3, a final object detection frame is output. In this example, the center of the detection processing frame to which the detection frame which is the final object detection frame is output is connected to the centers of the overlapping detection frames, and 5 × 8 = 40 substantially overlapping detection frames are shown. It has been output. The size of these detection frames is about 90 × 160 pixels, and the threshold of the number of overlapping detection frames (the detection frame width 検 出 32) × (the detection frame height ÷ 32) is about 15, These detection frames are integrated into one detection frame and adopted as a final detection result. In the prior art, the number of overlapping detection frames is not considered in determining whether or not to use the detection processing result of the detection processing as the final detection result.

  In the present embodiment, the ratio of the area of the overlapping area to the area obtained by combining the two detection frames is used as the overlapping degree.

  Next, the operation of the object detection device 10 according to the first embodiment will be described.

  First, when the imaging device 12 is directed to the detection target area and an image representing the detection target area is captured by the imaging device 12, the computer 14 executes a detection processing routine shown in FIG.

  First, in step S100, a captured image obtained by imaging is acquired from the imaging device 12, and in step S102,

For each detection processing frame set at each position, an image extracted from the captured image by the detection processing frame is input to a CNN learned in advance, and the position of the detection frame representing the detection target object included in the detection processing frame , Size, and accuracy.

  In step S104, the integration method is adjusted according to the nature of the overlapping detection frame of the detection frames output in step S102. For example, in accordance with the size of the overlapping detection frame, the threshold regarding the overlapping degree is adjusted.

  In step S106, after combining the detection frames overlapped in the overlapping part equal to or more than the threshold adjusted in step S104 among the detection frames output in step S102 into one detection frame with the highest accuracy, Output as the final object detection frame.

  In step S108, it is determined whether there is another overlapping detection frame. If there is another overlapping detection frame, the process returns to step S104, the threshold is adjusted in accordance with the nature of the other overlapping detection frame, and the other overlapping detection frame is integrated in step S106.

  If it is determined in step S108 that there is no other overlapping detection frame, the process proceeds to step S110, and each of the final object detection frames output in step S106 is displayed as the detection result of the detection target object. It displays on the device 16 and ends the detection processing routine.

  As described above, according to the object detection apparatus according to the first embodiment, an image is obtained by adjusting the threshold value regarding the overlapping degree used when integrating detection frames according to the size of the detection frame. The detection target object can be detected with high accuracy regardless of the size of the upper detection target object.

  In addition, by adjusting the processing method when integrating according to the properties such as the size and shape of the detection frame, the image is resized to various resolutions and the object detection frame is not detected or detected even when the detection processing is not performed. Erroneous detection when the detection frame is large can be suppressed. That is, it becomes possible to integrate detection frames at high speed and with high accuracy.

  Also, processing is performed on the enlarged input image by adjusting the threshold used when integrating the detection frame based on the plurality of detection frames and the accuracy with respect to the original input image without enlargement according to the nature of the detection frame. Even if it does not exist, highly accurate detection is possible for various detection target sizes.

  Also, by adjusting the integration process according to the nature of the detection frame in the form of deleting detection frames with a large size if detection frames with a large size are not output by a fixed number, performance degradation due to false detection of large size objects It can be suppressed.

  In addition, when the detection frame with a large size is output as a false detection in the method of Non-Patent Document 2, the detection frame of another object having an overlap with the detection frame is erroneously deleted, thereby increasing undetection. The following two improvements have been made to the problem. As a first improvement, it is made difficult to be deleted even if it overlaps with other detection frames, in order to suppress non-detection. As a second improvement, in order to suppress false detection, if there is no certain number of overlapping detection frames, they are deleted from the integrated processing result.

Second Embodiment
Next, a second embodiment will be described. The object detection apparatus according to the second embodiment has the same configuration as that of the first embodiment, so the same reference numerals are given and the description is omitted.

  The second embodiment differs from the first embodiment in that the threshold for the number is adjusted according to the size of the detection frame.

  The adjustment unit 24 of the object detection device according to the second embodiment adjusts the threshold value related to the number when integrating detection frames according to the size of the detection frame output by the object detection unit 22.

  Specifically, the threshold for the number is adjusted to be larger as the size of the detection frame is larger. For example, adjustment based on the size of the detection frame is performed by setting the threshold for the number as (detection frame width の 32) × (detection frame height ÷ 32). Reference numeral 32 denotes the number of pixels to which a detection processing frame for cutting out an image to be input to the convolutional neural network used here moves in parallel.

  The integration unit 26 is configured to adjust the number of detection frames equal to or greater than the threshold, based on the position and size of the detection frames output for each detection processing frame, in which the detection frames having overlapping degrees overlapping with other detection frames are equal to or higher than the threshold In some cases, these detection frames are integrated into one detection frame with the highest accuracy, and then output as a final object detection frame.

  The other configuration and operation of the object detection device according to the second embodiment are the same as those of the first embodiment, and thus the description thereof is omitted.

  As described above, according to the object detection apparatus according to the second embodiment, the threshold value related to the number used when integrating the detection frames is adjusted according to the size of the detection frames, on the image. The detection target object can be detected with high accuracy regardless of the size of the detection target object.

  In the above embodiment, although the case of detecting a vehicle as a detection target object has been described as an example, the present invention is not limited to this, and another object such as a pedestrian may be used as a detection target object.

Third Embodiment
Next, a third embodiment will be described. The object detection apparatus according to the third embodiment has the same configuration as that of the first embodiment, so the same reference numerals are given and the description is omitted.

  In the third embodiment, the CNN further outputs the type of the detection target object included in the detection processing frame, and the integration unit integrates the detection frames in which the overlapping degree overlapping with other detection frames is equal to or higher than the threshold value. The adjustment unit is different from the first embodiment in that the adjustment unit adjusts the threshold value regarding the overlapping degree according to the type of the detection target object included in the detection processing frame.

  In the object detection unit 22 of the object detection apparatus according to the third embodiment, the CNN indicates the position and size of a detection frame representing the detection target object included in the detection processing frame, the accuracy indicating the detection object likeness, detection Output the type of target object. Examples of the type of object to be detected include vehicles and pedestrians.

  The adjustment unit 24 adjusts the threshold value regarding the overlapping degree according to the type of the detection target object of the detection frame output by the object detection unit 22.

  Specifically, when the type of the detection target object of the detection frame is a pedestrian, adjustment is performed so that the threshold value regarding the overlapping degree is reduced.

  The integration unit 26 determines that the overlapping degree overlapping with another detection frame of the same type is equal to or higher than the adjusted threshold based on the position, size, accuracy, and type of the detection frame output for each detection processing frame. When the number of detection frames is equal to or more than the threshold value, the detection frames are integrated into one detection frame with the highest accuracy, and then output as a final object detection frame.

  The other configuration and operation of the object detection device according to the third embodiment are the same as those of the first embodiment, and thus the description thereof is omitted.

  As described above, according to the object detection device according to the third embodiment, the threshold value regarding the overlapping degree used when integrating the detection frames is adjusted according to the type of the detection target object of the detection frame. Thus, regardless of the type of detection target object, the detection target object can be detected accurately.

Fourth Embodiment
Next, a fourth embodiment will be described. The same reference numerals are given to parts having the same configuration as the first embodiment and the description will be omitted.

  The fourth embodiment is different from the first embodiment in that the threshold regarding the accuracy is adjusted according to the size of the detection frame.

  The adjustment unit 24 of the object detection device according to the fourth embodiment adjusts the threshold regarding the accuracy in accordance with the size of the detection frame output by the object detection unit 22.

  Specifically, the threshold regarding the accuracy is adjusted to be larger as the size of the detection frame is larger.

  The integration unit 26 has an accuracy that is equal to or higher than the threshold regarding the accuracy adjusted based on the position, the size, and the accuracy of the detection frame output for each detection processing frame, and in the overlapping detection frames, When the number of detection frames whose overlapping degree overlapping with other detection frames is equal to or more than the threshold is equal to or more than the threshold for the number, these detection frames are integrated into one detection frame with the highest accuracy, and then the final object is obtained. Output as a detection frame.

  As described above, according to the object detection device according to the fourth embodiment, the threshold value related to the accuracy used when integrating the detection frames is adjusted on the image according to the size of the detection frames. The detection target object can be detected with high accuracy regardless of the size of the detection target object.

<Experimental result>
When the prior art is applied to the image shown in FIG. 5, an erroneous detection (a detection frame whose degree of coincidence with the vehicle is lower than the reference) generated in the processing for one detection processing frame overlaps the detection frame that is an erroneous detection. Detection frames of other vehicles possessed were deleted and two vehicles became undetected.

  In addition, when the method described in the second embodiment described above is used, the detection frames that are erroneously detected in the related art exceed the threshold of the number of overlapping detection frames by adjusting the threshold regarding the number. Because they are not present, vehicles that are not detected in the prior art are now detected.

  A part of the training data set of KITTI Object Detection Evaluation 2012, which is a data set typically used for evaluation of vehicle detection performance, was compared with the vehicle detection performance before and after improvement as test data, as shown in FIG. It was found that Average Precision, which is a performance index of KITTI, improved 10.4.5.9.7.5 points for each difficulty of Easy, Moderate and Hard, and improved for all difficulty.

  In the above embodiment, a CNN is used as a detection model for outputting a detection frame, but the present invention is not limited to this. A model other than CNN may be used as long as it can output the position, size, and accuracy of the detection frame included in the detection processing frame.

  In addition, although the case where the ratio of the area of the overlapping area to the combined area of the two detection frames is used is described as an example of the overlapping degree of the detection frames, the present invention is not limited thereto. The size of the overlapping area may be used as the overlapping degree.

DESCRIPTION OF SYMBOLS 10 object detection apparatus 12 imaging apparatus 14 computer 16 display apparatus 20 image input part 22 object detection part 24 adjustment part 26 integrated part

Claims (8)

A detection unit which outputs the position and size of a detection frame representing an object to be detected included in the detection processing frame for each detection processing frame set at each position from the input image using the detection model;
Based on the position and size of the detection frame output for each of the detection processing frames, the detection frames having an overlapping degree equal to or higher than the other detection frames are integrated and output as a final object detection frame Integration department,
Including
An adjustment unit which adjusts a threshold used when integrating a detection frame according to a property of a detection frame outputted by the detection unit. Object detection device.   The object detection apparatus according to claim 1, wherein the adjustment unit adjusts a threshold related to the degree of overlap in accordance with the size of the detection frame output by the detection unit. The integration unit integrates the detection frames when the number of detection frames to be integrated is equal to or more than a threshold value regarding the number,
The object detection apparatus according to claim 1, wherein the adjustment unit adjusts the threshold regarding the number according to the size of the detection frame output by the detection unit. The detection unit further outputs the type of the detection target object included in the detection processing frame;
The object detection apparatus according to claim 1, wherein the adjustment unit adjusts a threshold related to the overlapping degree according to the type of the detection target object output by the detection unit. The detection unit further outputs the accuracy of the detection target object represented by the detection frame,
The integration unit integrates a detection frame in which the overlapping degree overlapping with another detection frame is equal to or higher than a threshold, and the accuracy is equal to or higher than a threshold related to the accuracy.
The object detection apparatus according to claim 1, wherein the adjustment unit adjusts a threshold related to the accuracy in accordance with the size of the detection frame output by the detection unit. Let CNN (Convolutional Neural Network) be the detection model,
The detection unit inputs, to the CNN, an image extracted from the input image by the detection processing frame for each detection processing frame, and outputs the position and size of a detection frame representing a detection target object included in the detection processing frame. The object detection apparatus according to any one of claims 1 to 5.   The object detection apparatus according to any one of claims 1 to 6, wherein the detection processing frame is larger than a size on an image of a detection target object assumed. Computer,
A detection unit that outputs the position and size of a detection frame representing an object to be detected included in the detection processing frame for each detection processing frame set at each position using a detection model from an input image;
According to the nature of the detection frame output by the detection unit, an adjustment unit that adjusts a threshold used when integrating the detection frame, and the position and size of the detection frame output for each detection processing frame A program for functioning as an integration unit that integrates detection frames whose overlap degree overlapping with other detection frames is equal to or greater than a threshold and outputs the final detection frame as an object detection frame.