JP6891984B1 - Object detection device, object detection method, program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection device capable of performing highly accurate object detection regardless of the presence or absence of overlapping of objects. An image acquisition unit for acquiring an image, a detection candidate frame surrounding an area occupied by an object in the image, a visual recognition area including an area in which the object can be visually recognized in the detection candidate frame, and a detection candidate including an object identification class. An inference unit that infers and outputs information based on a learning model, a detection candidate frame consolidation unit that performs integrated processing that integrates one into the other when multiple detection candidate frames are output and overlaps with each other, and after integration processing It has an output unit that outputs detection candidate information of. In the integration process, the detection candidate frame consolidation unit specifies the front side of the overlapping detection candidate frames as the front detection candidate frame and the back side as the rear detection candidate frame, and is based on the size of the visible area in the rear detection candidate frame. The existence probability of the object in the rear detection candidate area is calculated, and it is determined whether or not to integrate the front detection candidate frame and the rear detection candidate frame. [Selection diagram] Fig. 1

Description

The present invention relates to an object detection device, an object detection method, a program and a recording medium.

Objects such as people and cars are detected and counted from images taken by surveillance cameras and the like. In recent years, many object detection methods that simultaneously detect and identify an object in an image by using deep learning have been proposed, and the accuracy of detection has been improved.

For example, in Non-Patent Document 1, feature maps of various sizes and default boxes (hereinafter referred to as detection candidate frames) that change the shape of a rectangle are prepared, and the loss function is calculated in the final layer to obtain various sizes. The position and frame size of the detection frame are estimated and the object identification class (for example, car, person, bicycle, etc.) is inferred at the same time. Since the detection candidate frames are prepared in proportion to the number of feature maps, a plurality of detection candidate frames corresponding to a plurality of feature maps are generated at the time of inference.

When trying to correspond to a multi-scale object by general object detection, it is necessary to prepare a plurality of detection candidate frames corresponding to the size. When multiple detection candidate frames occur, for example, using the non-maximum suppression (nms) method, the detection candidate frame with the maximum reliability (score) is selected from the detection candidate frames, and the final detection is performed. A frame is being used.

However, in the nms method, for example, when the rectangles indicating the detection frames have a large overlap, one of the two detection frames is selected and the other is deleted. For this reason, for example, when a plurality of objects are overlapped and displayed in the image, the overlap of the rectangles indicating the detection frames for each object becomes large, and these rectangles are the correct solutions for the object detection frames. Even so, one of them will be deleted. Therefore, when the overlap between a plurality of objects is large, it is difficult to deal with it by the nms method. It is possible to adjust the threshold value of nms and output without deleting the detection candidate frames with large overlap, but in that case, there is a trade-off that a large amount of false detections with large overlap is output, so adjustment of nms It is difficult to solve by itself.

Therefore, Non-Patent Document 2 proposes a method of learning the number of objects information that is useful when detecting overlapping objects, in addition to learning the position of the detection frame and the identification class of the objects. The learned object number information is output as the number of objects for each detection candidate frame at the time of inference. In the method of Non-Patent Document 2, the detection candidate frame is integrated so that the sum of the number of objects in the detection candidate frame is 1.

Wei Liu, et al., “SSD: Single Shot MultiBox Detector”, ECCV2016 Tetsuaki Suzuki, Hiromitsu Fuchigami, Hiroyoshi Miyano, "Sturdy Object Detection Based on Local Object Number Estimate", 21st Image Recognition and Understanding Symposium, June 2018

As described above, the method of combining nms with Non-Patent Document 1 has a problem that high-precision detection cannot be performed when a plurality of objects overlap each other.

On the other hand, although the method of Non-Patent Document 2 is useful when the objects overlap each other, the detection accuracy may decrease when the objects do not overlap each other. For example, when three detection candidate frames are generated for one object, the number of objects associated with each detection candidate frame is reduced to 1/3 so that the number of objects after integration of the frames is 1. Assuming that the threshold value for determining the number of objects as one is 0.8, if one of the three detection candidate frames occurs at a position outside the object, the sum of the number of objects in the remaining two detection candidate frames is used. A certain 2/3 does not meet the threshold (0.8), and the object is treated as undetected. Further, if the threshold value is set low in order to avoid such a situation, there is a problem that erroneous detection occurs.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an object detection device capable of performing highly accurate object detection regardless of the presence or absence of overlapping of objects.

The object detection device according to the present invention includes an image acquisition unit for acquiring an image, a detection candidate frame surrounding an area occupied by one object in the image, and a region in which the object 1 can be visually recognized in the detection candidate frame. The inference unit that infers and outputs the detection candidate information including the visual recognition area and the identification class of the object 1 for each object based on the learning model, and the detection candidate frames in which a plurality of the detection candidate frames are output and overlap each other If there is, the detection candidate frame consolidation unit that integrates one of the overlapping detection candidate frames into the other, and the detection candidate information for each of the detection candidate frames that exist after the integration process. The detection candidate frame consolidation unit has an output unit that outputs the above, and in the integration process, the detection candidate frame integration / abolition unit detects an object in which one of the detection candidate frames that overlap each other is located relatively in front of the image. It is specified as a front detection candidate frame which is a candidate frame, and the other of the detection candidate frames that overlap each other is specified as a rear detection candidate frame which is a detection candidate frame corresponding to an object located relatively behind in the image. The existence probability of the object in the rear detection candidate frame is calculated based on the size of the visible area in the rear detection candidate frame, and the front detection candidate frame and the rear are based on the calculated existence probability. It is characterized in that it is determined whether or not to integrate with the detection candidate frame.

Further, the object detection method according to the present invention is an object detection method executed by a detection device that detects an object in an image based on a learning model, in which a step of acquiring an image and one object in the image are included. The detection candidate frame surrounding the occupied area, the visible area including the area where the object 1 can be visually recognized in the detection candidate frame, and the detection candidate information including the identification class of the object 1 are obtained for each object based on the learning model. A step of inferring and outputting, and a step of performing an integration process of integrating one of the overlapping detection candidate frames into the other when a plurality of the detection candidate frames are output and there are detection candidate frames that overlap each other. And a step of outputting the detection candidate information for each of the detection candidate frames existing after the integration process, and in the step of performing the integration process, one of the detection candidate frames overlapping with each other is placed in the image. The step of specifying as the front detection candidate frame, which is the detection candidate frame corresponding to the object located on the relatively front side, and the other of the detection candidate frames overlapping with each other are set to the object located relatively on the back side in the image. A step of specifying the object as a rear detection candidate frame, which is a corresponding detection candidate frame, and a step of calculating the existence probability of the object in the rear detection candidate frame based on the size of the visible area in the rear detection candidate frame. It is characterized by including a step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame based on the calculated existence probability.

Further, the program according to the present invention provides a computer with a step of acquiring an image, a detection candidate frame surrounding an area occupied by one object in the image, and a region in which the object of one can be visually recognized in the detection candidate frame. A step of inferring and outputting the visible area including the visible area and the detection candidate information including the identification class of the object 1 for each object based on the learning model, and a detection candidate frame in which a plurality of the detection candidate frames are output and overlap each other are When present, one of the overlapping detection candidate frames overlaps with the step of specifying one of the overlapping detection candidate frames as a front detection candidate frame which is a detection candidate frame corresponding to an object located relatively in front of the image. A step of specifying the other side of the detection candidate frame as a rear detection candidate frame which is a detection candidate frame corresponding to an object located relatively in the back side of the image, and the size of the visible area in the rear detection candidate frame. based on, a step of calculating the existence probability of the object in the rear detection candidate frames, based on the calculated existence probability, whether integrated with the front detection candidate frame and the rear detection candidate frames It is characterized in that a step of determining and a step of outputting the detection candidate information for each of the detection candidate frames existing after the integration process performed in response to the determination are executed.

According to the present invention, it is possible to perform highly accurate object detection regardless of whether or not the objects overlap each other.

It is a block diagram which shows the structure of the object detection apparatus which concerns on Example 1 of this invention. It is a figure which shows typically the detection candidate frame and the visual area frame when a shield exists in front of a target. It is a figure which shows the example of the forward detection candidate frame which is selected when the target is displayed overlapping. It is a figure which shows the example of the rear region calculated when the targets are overlapped and displayed. It is a figure which shows the example of the rear view area calculated when the targets are overlapped and displayed. It is a figure which shows typically the flow of the integration of the frame when a false detection occurs. It is a figure which shows typically the case where the shield exists in front of a plurality of targets.

Hereinafter, examples of the present invention will be described with reference to the drawings. In the description and the accompanying drawings in each of the following examples, substantially the same or equivalent parts are designated by the same reference numerals.

FIG. 1 is a block diagram showing the configuration of the object detection device 100 of this embodiment. The object detection device 100 is a detection device that detects an object in an image using a convolutional neural network. The object detection device 100 includes an object learning unit 10 and an object estimation unit 20.

The object detection device 100 detects an object in an image based on the method of SSD (Single Shot Multibox Detector). In SSD, the detection candidate frame (default box) of the rectangular pattern corresponding to the size and aspect ratio of the object to be detected is superimposed and displayed on the image, and the position, size and identification class (for example, car, person, bicycle) of the object are displayed. Etc.). In other words, the detection candidate frame is a rectangular pattern frame that surrounds the area occupied by one object in the image.

Further, in this embodiment, apart from the detection candidate frame, the area range including the area where the object can be visually recognized in the detection candidate frame is displayed as the “visual area”. For example, in this embodiment, a rectangular area including an area in which the object to be detected is visually displayed without being shielded is displayed as a visible area. In the following description, the frame of the rectangular pattern indicating the visible area is referred to as the visible area frame.

FIG. 2 is a diagram schematically showing a detection candidate frame DF and a viewing area frame VF when the object to be detected is a human (person) and a shield exists in front of the object. The detection candidate frame DF indicates the position and size of the entire person including the area shielded by the shield SL. On the other hand, the viewing area frame VF indicates the position and range of the viewing area of the person in the image that is not shielded by the shield SL.

Referring to FIG. 1 again, the object learning unit 10 is a learning device that performs deep learning for object detection, and has a data storage unit 11, a learning unit 12, and a learning model 13.

The data storage unit 11 is composed of a semiconductor storage device such as a flash memory or an HDD (Hard Disk Drive), for example. The data storage unit 11 stores data (hereinafter, referred to as learning data) used for learning by the learning unit 12. For example, in this embodiment, data such as an image as a learning target, an object detection correct answer frame indicating the correct answer of the detection candidate frame for an object in the image, an object identification class, and a visual recognition area frame are stored as learning data. It is stored in part 11.

The learning unit 12 receives the input of the learning data stored in the data storage unit 11, performs deep learning of object detection based on the SSD method, and performs a learning model (that is, a mathematics constructed by using the deep learning method). Model) is generated. For example, the learning unit 12 uses three types of data, an identification class, an object detection correct answer frame, and a visual recognition area frame, for learning, and based on this learning, parameters such as a weighting coefficient of a convolutional neural network are generated as a learning model.

The learning model storage unit 13 is composed of a semiconductor storage device such as a flash memory or an HDD, and stores the learning model generated by the learning unit 12.

The object estimation unit 20 is composed of a processing control unit such as a CPU (Central Processing Unit), and estimates an object in an image (that is, object detection) based on a learning model generated by learning of the object learning unit 10. Do.

The object estimation unit 20 of the present embodiment performs integration processing of the detection candidate frames when a plurality of detection candidate frames overlap in the image and there is a possibility that the plurality of objects overlap. It is configured to be able to output an appropriate detection candidate frame.

The object estimation unit 20 includes an image acquisition unit 21, an inference unit 22, a detection candidate frame consolidation / abolition unit 23, and an output unit 24. These functional blocks are formed, for example, by the object estimation unit 20 executing a predetermined program.

The image acquisition unit 21 acquires an image including an object to be detected. The image acquisition unit 21 may be configured to be composed of an image pickup device such as a camera, or may be configured to acquire an image taken by another image pickup device from the outside.

The inference unit 22 infers the detection candidate frame, the object identification class, and the visual recognition area frame for the object in the image acquired by the image acquisition unit 21 based on the learning model stored in the learning model storage unit 13. To do. In the following description, the detection candidate frame, the viewing area frame, and the identification class for one object are also referred to as "detection candidate information".

The detection candidate frame consolidation / abolition unit 23 performs integration processing of a plurality of detection candidate frames inferred by the inference unit 22. In this embodiment, eliminating one of the pair of overlapping detection candidate frames and leaving only the other as the detection candidate frame is referred to as "integration". In addition, the "integration process" executed by the detection candidate frame consolidation unit 23 determines not to integrate the detection candidate frames in addition to the process when the detection candidate frames are actually integrated, and leaves both detection candidate frames as they are. Including case processing.

The detection candidate frame consolidation / abolition unit 23 includes a total overlap selection unit 231, a front candidate frame selection unit 232, a rear candidate frame overlap selection unit 233, a rear area calculation unit 234, a rear visibility area calculation unit 235, an object existence probability calculation unit 236, and a threshold value. It has a processing unit 237 and a frame integration determination unit 238.

The total weight selection unit 231 selects an overlapping detection candidate frame from the entire detection candidate frame inferred by the inference unit.

The front candidate frame selection unit 232 selects the detection candidate frame corresponding to the object located relatively in the foreground side in the image among the plurality of overlapping detection candidate frames as the “front detection candidate frame”.

The rear candidate frame overlap selection unit 233 is a detection candidate frame selected by the total overlap selection unit 231 (that is, overlapping detection candidate frames) other than the front detection candidate frame selected by the front candidate frame selection unit 232. Select the detection candidate frame as the "backward detection candidate frame". As a result, the detection candidate frame corresponding to the object located relatively in the back side in the image is selected as the rear detection candidate frame.

The rear area calculation unit 234 calculates the area in the rear detection candidate frame that does not overlap with the front detection candidate frame as the “rear area”.

The rear viewing area calculation unit 235 calculates the viewing area of the object in the rear detection candidate frame as the “rear viewing area”.

The object existence probability calculation unit 236 calculates the existence probability of the object in the rear detection candidate frame based on the rear area calculated by the rear area calculation unit 234 and the rear visual area calculated by the rear visual area calculation unit 235. To do.

The threshold value processing unit 237 performs a process of comparing the existence probability of the object calculated by the object existence probability calculation unit 236 with the threshold value.

The frame integration determination unit 238 determines whether or not to integrate the overlapping detection candidate frames based on the processing result of the threshold value processing unit 237.

The output unit 24 outputs the detection candidate information (detection candidate frame, visual recognition area, and object identification class) for the detection candidate frame existing after the integration process by the detection candidate frame consolidation / abolition unit 23 as the result of object estimation. For example, the output unit 24 outputs an image obtained by displaying a rectangular pattern frame indicating each of the detection candidate frame and the visual recognition area frame on the image acquired by the image acquisition unit 21. Further, the output unit 24 outputs the information of the object identification class by superimposing it on the image, for example, by providing a difference in the color of the detection candidate frame.

Next, the operation of object detection executed by the object detection device 100 of this embodiment will be described. Here, a case where the object to be detected is a person and two people are displayed overlapping in the image will be described as an example.

The inference unit 22 of the object estimation unit 20 acquires an image from the image acquisition unit 21 and makes inferences based on the learning model stored in the learning model storage unit 13. The inference unit 22 outputs a large number of detection candidate frames, a visible area frame, and information of the object identification class obtained by inference to the detection candidate frame consolidation / abolition unit 23.

The overall overlap selection unit 231 of the detection candidate frame consolidation / abolition unit 23 receives input of a large number of detection candidate frames from the inference unit 22, and selects a plurality of detection candidate frames in which the frames overlap each other.

The front candidate frame selection unit 232 is the ratio of the area of the visible area frame to the detection candidate frame (that is, the area of the visible area frame / the area of the detection candidate frame) for each of the detection candidate frames selected by the overall overlap selection unit 231. ) Is calculated as the visibility rate. Then, the front candidate frame selection unit 232 selects the detection candidate frame having the highest visibility rate as the front detection candidate frame.

The rear overlap selection unit 233 selects a plurality of detection candidate frames selected by the overall overlap selection unit 231 excluding the front detection candidate frame as the rear detection candidate frame.

FIG. 3 is a diagram showing an example of a front detection candidate frame FF selected by the front candidate frame selection unit 232 and a rear detection candidate frame BF selected by the rear overlap selection unit 233. Here, only the region where the object in the image exists is extracted and shown.

Since the person P1 is located in front of the person P2 in the image, that is, on the front side, the person P1 has a higher visibility rate than the person P2. Therefore, the detection candidate frame of the person P1 is selected as the front detection candidate frame FF, and the detection candidate frame of the person P2 is selected as the rear detection candidate frame BF.

Next, the rear region calculation unit 234 calculates the rear region, which is the frame region of the rear object, based on the front detection candidate frame FF and the rear detection candidate frame BF. For example, assuming that the region of the rear detection candidate frame BF is the first region A and the region of the front detection candidate frame FF is the second region B, the rear region is expressed by the following mathematical formula (1).

FIG. 4 is a diagram showing an example of the rear region RA calculated by the rear region calculation unit 234. As shown by diagonal lines, the region of the rear detection candidate frame BF that does not overlap with the front detection candidate frame FF is calculated as the rear region RA.

On the other hand, the rear viewing area calculation unit 235 calculates the viewing area of the rear object as the rear viewing area based on the viewing area of the rear detection candidate frame BF. For example, assuming that the visible area of the rear detection candidate frame BF is _VA , the rear visible area is expressed by the following mathematical formula (2).

FIG. 5 is a diagram showing an example of a rear view area BVA calculated by the rear view area calculation unit 235. As shown by the diagonal lines, the visible area of the person P2 behind is calculated as the rear visible area BVA.

The object existence probability calculation unit 236 is based on the rear view area RA calculated by the rear area calculation unit 234 and the rear view area BVA calculated by the rear view area calculation unit 235, and the rear view area BVA occupying the rear area RA. Is calculated as the object existence probability for the object behind. The object existence probability of the object behind is expressed by the following mathematical formula (3).

For example, when the ratio of the rear view region BVA is large, the probability that an object exists behind is high, so that the probability of existence of an object is high. On the other hand, when the ratio of the rear view region BVA is small, the probability that an object exists behind is low, so that the probability of existence of an object is low.

The threshold value processing unit 237 compares the object existence probability calculated by the object existence probability calculation unit with a predetermined threshold value, and determines whether the existence probability of the object behind is equal to or less than the threshold value.

The frame integration determination unit 238 determines whether or not to integrate the detection rear frames based on the determination result of the threshold value processing unit 237. For example, when it is determined that the object existence probability for the object behind is equal to or greater than the threshold value, it is determined that the object exists behind, and it is determined not to integrate the detection candidate frames. As shown in FIGS. 3 to 5, when the rear person P2 exists at a position overlapping the front person P1, the detection candidate frames are not integrated, and the front detection candidate frame FF is the detection candidate of the person P1. As a frame, the rear detection candidate frame BF is output as a detection candidate frame for the person P2, respectively.

On the other hand, when it is determined that the object existence probability for the rear object is less than the threshold value, the frame integration determination unit 238 determines that the object does not exist behind and decides to integrate the detection candidate frames. For example, when the detection candidate frame is erroneously detected as if the rear object exists even though there is no overlap between the objects, the detection candidate frames are integrated.

FIG. 6 is a diagram schematically showing a flow of integration of detection candidate frames when such erroneous detection of detection candidate frames occurs. Here, the black-painted area in the figure is erroneously detected as the visible area BVA of the rear object, and the detection candidate frame as shown by the solid line is erroneously detected as the rear detection candidate frame BF.

The front candidate frame selection unit 232 selects the detection candidate frame having the highest visibility rate among the overlapping detection candidate frames as the front detection candidate frame. Here, since the visibility rate of the front detection candidate frame FF is high, the person P1 is selected as the person located in front. As a result, the detection candidate frame as shown by the broken line in the figure is selected as the forward detection candidate frame FF. On the other hand, the rear candidate frame overlap selection unit 233 selects the detection candidate frame shown by the solid line as the rear detection candidate frame BF (STEP 1).

The rear region calculation unit 234 calculates the region in the rear detection candidate frame that does not overlap with the front detection candidate frame, that is, the region as shown by the diagonal line in the figure as the rear region RA. Further, the rear view area calculation unit 235 calculates the area shown in black in the figure as the rear view area BVA (STEP 2).

The object existence probability calculation unit 236 calculates the ratio of the rear visual region BVA to the rear region RA as the object existence probability for the rear object. The threshold value processing unit 237 compares the calculated object existence probability with the threshold value. For example, if the threshold value is 0.6, the object existence probability is less than the threshold because the ratio of the area of the rear visual region BVA to the area of the rear region RA is less than 0.6 in the example shown in FIG. It is determined (STEP3).

The frame integration determination unit 238 determines whether or not to integrate the detection candidate frames based on the processing result of the threshold value processing unit 237. Here, since the object existence probability is less than the threshold value, the detection frames are integrated. Specifically, the rear detection candidate frame BFF is abolished, and only the front detection candidate frame FF shown by the broken line in FIG. 6 is output as the detection candidate frame (STEP 4).

As described above, the object detection device 100 of the present embodiment calculates the object existence probability for the rear object based on the visibility rate, and determines whether or not to integrate the detection candidate frames. When the object existence probability is high, there is a high possibility that the objects overlap each other in the image, so the detection candidate frames are not integrated. On the other hand, when the object existence probability is low, there is no rear object and there is a high possibility that the objects do not overlap each other in the image. Therefore, the rear detection candidate frame is abolished and only the front detection candidate frame is detected. The detection candidate frames to be used as candidate frames are integrated.

Therefore, according to the object detection device 100 of the present embodiment, even when a large number of detection candidate frames are generated in the object detection corresponding to the multi-scale, the detection frames are appropriately integrated according to the object existence probability to obtain the object. It is possible to prevent deterioration in the accuracy of object detection when overlapping occurs. In addition, the accuracy when no overlap occurs can be ensured. That is, according to the object detection device 100 of this embodiment, it is possible to perform highly accurate object detection regardless of whether or not the objects overlap each other.

The embodiments of the present invention are not limited to those described in the above examples. For example, in the above embodiment, an example in which the threshold value processing unit 237 performs processing using a predetermined threshold value has been described, but the value of the threshold value can be dynamically changed.

For example, as shown in FIG. 7, when the shield SL exists so as to shield both the person P1 and the person P2, the calculated object existence probability is small, so that the threshold value is set to a high value (for example, 0. When it is set to 8 etc.), it is determined that the object existence probability is less than the threshold value. As a result, the detection candidate frames are integrated, and the rear detection candidate frame BF is not output even though the person P2 exists as a rear object. Therefore, for example, by changing the threshold value according to the visibility rate of the front detection candidate frame FF and reducing the threshold value when the visibility rate is low, it is possible to prevent the detection candidate frames from being erroneously integrated. be able to.

Further, in the above embodiment, a case where two people are displayed overlapping in the image has been described with the detection target as a person. However, the detection target is not limited to a person, and may be a non-human organism such as a dog or a cat, or an inanimate object such as a car or a bicycle. It can also be applied when the number of overlapping objects is three or more.

Further, in the above embodiment, an example in which the reasoning unit 22 infers the visual recognition area frame based on the learning model has been described. However, the visual recognition area for each pixel may be obtained by using a segmentation method that detects the identification class for each pixel in the image.

Further, the series of processes described in the above embodiment can be performed by computer processing according to a program stored in a recording medium such as a ROM.

100 Object detection device 10 Object learning unit 11 Data storage unit 12 Learning unit 13 Learning model storage unit 20 Object estimation unit 21 Image acquisition unit 22 Reasoning unit 24 Output unit 23 Detection candidate frame consolidation / abolition unit 231 Total weight selection unit 232 Forward candidate frame selection Unit 233 Rear candidate frame overlap selection unit 234 Rear area calculation unit 235 Rear visual area calculation unit 236 Object existence probability calculation unit 237 Threshold processing unit 238 Frame integration determination unit

Claims (12)

The image acquisition unit that acquires images and
The detection candidate frame surrounding the area occupied by the object 1 in the image, the visible area including the area in which the object 1 can be visually recognized in the detection candidate frame, and the detection candidate information including the identification class of the object 1. An inference unit that infers and outputs each object based on the learning model,
When a plurality of detection candidate frames are output and there are detection candidate frames that overlap each other, a detection candidate frame consolidation unit that performs an integration process for integrating one of the overlapping detection candidate frames into the other, and a detection candidate frame consolidation unit.
An output unit that outputs the detection candidate information for each of the detection candidate frames existing after the integration process, and
Have,
In the integrated process, the detection candidate frame consolidation unit
One of the detection candidate frames that overlap each other is specified as a front detection candidate frame that is a detection candidate frame corresponding to an object located relatively in front of the image.
The other of the detection candidate frames that overlap each other is specified as a rear detection candidate frame that is a detection candidate frame corresponding to an object located relatively far in the image.
Based on the size of the visible area in the rear detection candidate frame , the existence probability of the object in the rear detection candidate frame is calculated.
An object detection device for determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame based on the calculated existence probability. The first aspect of the present invention is that the detection candidate frame consolidation unit identifies the front detection candidate frame and the rear detection candidate frame based on the area of the visible area for each of the overlapping detection candidate frames. The object detection device described. The detection candidate frame consolidation unit compares the existence probability calculated for the rear detection candidate frame with the threshold value, and based on the comparison result, whether or not to integrate the rear detection candidate frame into the front detection candidate frame. The object detection device according to claim 1 or 2, wherein the object detection device is characterized in that. When the existence probability calculated for the rear detection candidate frame is lower than the threshold value, the detection candidate frame consolidation unit integrates the rear detection candidate frame with the front detection candidate frame.
The object detection according to claim 3, wherein when the existence probability calculated for the rear detection candidate frame is higher than the threshold value, the rear detection candidate frame and the front detection candidate frame are not integrated. apparatus. An object detection method executed by a detection device that detects an object in an image based on a learning model.
Steps to get the image and
The detection candidate frame surrounding the area occupied by the object 1 in the image, the visible area including the area in which the object 1 can be visually recognized in the detection candidate frame, and the detection candidate information including the identification class of the object 1. Steps to infer and output each object based on the learning model,
When a plurality of detection candidate frames are output and there are detection candidate frames that overlap each other, a step of performing an integration process of integrating one of the detection candidate frames that overlap each other into the other.
A step of outputting the detection candidate information for each of the detection candidate frames existing after the integration process, and
Have,
The step of performing the integration process is
A step of specifying one of the detection candidate frames that overlap each other as a front detection candidate frame that is a detection candidate frame corresponding to an object located relatively in front of the image.
A step of specifying the other of the detection candidate frames that overlap each other as a rear detection candidate frame that is a detection candidate frame corresponding to an object located relatively in the back side of the image.
A step of calculating the existence probability of the object in the rear detection candidate frame based on the size of the visible area in the rear detection candidate frame, and a step of calculating the existence probability of the object in the rear detection candidate frame.
A step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame based on the calculated existence probability, and
An object detection method comprising. In the step of specifying the front detection candidate frame and the step of specifying the rear detection candidate frame, the front detection candidate frame and the rear detection candidate frame are based on the area of the visible area for each of the overlapping detection candidate frames. The object detection method according to claim 5, wherein the frame is specified. The step of performing the integration process includes a step of comparing the existence probability calculated for the backward detection candidate frame with the threshold value.
In the step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame, the rear detection candidate frame is used as the front detection candidate frame based on the comparison result between the existence probability and the threshold value. The object detection method according to claim 5 or 6, wherein it is determined whether or not to integrate. In the step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame, if the existence probability calculated for the rear detection candidate frame is lower than the threshold value, the rear detection candidate frame It was decided to integrate the frame into the forward detection candidate frame,
The seventh aspect of claim 7, wherein when the existence probability calculated for the rear detection candidate frame is higher than the threshold value, it is determined that the rear detection candidate frame and the front detection candidate frame are not integrated. Object detection method. On the computer
Steps to get the image and
The detection candidate frame surrounding the area occupied by the object 1 in the image, the visible area including the area in which the object 1 can be visually recognized in the detection candidate frame, and the detection candidate information including the identification class of the object 1. Steps to infer and output each object based on the learning model,
When a plurality of detection candidate frames are output and there are detection candidate frames that overlap each other, the detection candidate frame corresponding to an object in which one of the detection candidate frames that overlap each other is located relatively in front of the image. The step to identify as a forward detection candidate frame, which is
A step of specifying the other of the detection candidate frames that overlap each other as a rear detection candidate frame that is a detection candidate frame corresponding to an object located relatively in the back side of the image.
A step of calculating the existence probability of the object in the rear detection candidate frame based on the size of the visible area in the rear detection candidate frame, and a step of calculating the existence probability of the object in the rear detection candidate frame.
A step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame based on the calculated existence probability, and
A step of outputting the detection candidate information for each of the detection candidate frames existing after the integration process performed according to the determination, and a step of outputting the detection candidate information.
A program characterized by executing. In the step of specifying the front detection candidate frame and the step of specifying the rear detection candidate frame, the front detection candidate frame and the rear detection candidate frame are based on the area of the visible area for each of the overlapping detection candidate frames. The program according to claim 9, wherein the frame is specified. Including a step of comparing the existence probability calculated in the step of calculating the existence probability of the object in the rear detection candidate frame with the threshold value.
In the step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame, the rear detection candidate frame is used as the front detection candidate frame based on the comparison result between the existence probability and the threshold value. The program according to claim 9 or 10, wherein it determines whether or not to integrate. In the step of determining whether or not to integrate the front detection candidate frame and the rear detection candidate frame, if the existence probability calculated for the rear detection candidate frame is lower than the threshold value, the rear detection candidate frame It was decided to integrate the frame into the forward detection candidate frame,
The eleventh aspect of claim 11, wherein when the existence probability calculated for the rear detection candidate frame is higher than the threshold value, it is determined that the rear detection candidate frame and the front detection candidate frame are not integrated. Program.

Images