KR20220079464A - Method for detecting object and device performing the same - Google Patents

Abstract

Disclosed are an object detection method and an apparatus therefor. A detection method according to various embodiments includes an operation of generating a first image in which one or more objects are detected from an object image and a false positive rate (FSR) than the first image based on a penalty score and generating the reduced second image, wherein the penalty score may be generated based on a false positive rate of each of one or more classes corresponding to the one or more objects.

Description

Object detection method and device therefor

Various embodiments of the present invention relate to an object detection method and an apparatus therefor.

In the field of computer vision, advances in deep learning, including convolutional neural networks and backpropagation learning methods, have enabled many researches and advancements. Among the technologies in the computer vision field, deep learning is most actively applied to object classification technology and object detection technology. The object classification technology is a technology for classifying the types of objects present in an image, and the object detection technology is a process of object classification that distinguishes the types of objects and the object location area recognition process that outputs the location information of the objects This means the technology that is integrated and performed.

Since object detection technology is closely related to humans, such as autonomous driving and imaging medicine, it is also used in fields where the decision of a deep learning system is closely related to human life, so it is very important to improve the performance of object detection.

The background art described above is possessed or acquired by the inventor in the process of deriving the disclosure of the present application, and cannot necessarily be said to be a known technology disclosed to the general public prior to the present application.

In a deep learning system, a false positive may be a case in which what the deep learning system predicts as the correct answer is false. Since the deep learning system gives a high confidence score to the unlearned data, it may output a false positive result for the unlearned data. In fields where the decisions of deep learning systems are closely related to human life, such as autonomous driving and imaging medicine, false detections of deep learning systems may lead to serious accidents. For example, if there is no vehicle in front of the autonomous vehicle, but the object detection system in the autonomous vehicle falsely detects that the vehicle is present, the autonomous vehicle may suddenly stop and endanger the life of the driver. Accordingly, a technique for robustly detecting an object against false positives may be required.

Various embodiments may provide a technique for robustly detecting an object in false positives based on a penalty score according to a false positive rate.

However, the technical tasks are not limited to the above-described technical tasks, and other technical tasks may exist.

A detection method according to various embodiments includes an operation of generating a first image in which one or more objects are detected from an object image and a false positive rate (FSR) than the first image based on a penalty score and generating the reduced second image, wherein the penalty score may be generated based on a false positive rate of each of one or more classes corresponding to the one or more objects.

The operation of generating the first image includes generating a feature map by inputting the object image to a classification model that classifies the one or more objects from the object image and detecting the one or more objects in the characteristic map and generating the first image and one or more first inference indices corresponding to the first image by inputting the characteristic map to a detection model.

The classification model may be learned based on first learning data related to the object to be detected by the detection model and second learning data irrelevant to the object to be detected.

The second learning data may be labeled based on one or more classes included in the first learning data.

The generating of the second image may include generating the penalty score based on verification data and generating the second image and one or more second inference indices corresponding to the second image based on the penalty score. Including, the verification data may be a combination of the first learning data and the second learning data.

The second inference index may be a value obtained by multiplying a difference between a false positive rate of one class and an average of the false positive rate by the first inference index.

The penalty score may be generated for a class having a false positive rate exceeding an average of the false positive rate.

An apparatus according to various embodiments includes a memory including instructions and a processor electrically connected to the memory and configured to execute the instructions, wherein when the instructions are executed by the processor, the processor is configured to generate one or more objects in an object image. generating a first image in which an object is detected, and generating a second image having a reduced false positive rate (FSR) than the first image based on a penalty score, wherein the penalty score is The one or more classes corresponding to the one or more objects may be generated based on a false detection rate of each.

The processor generates a feature map by inputting the object image to a classification model for classifying the one or more objects from the object image, and the feature map to a detection model for detecting the one or more objects in the feature map may be input to generate the first image and one or more first inference indices corresponding to the first image.

The classification model may be learned based on first learning data related to the object to be detected by the detection model and second learning data irrelevant to the object to be detected.

The second learning data may be labeled based on one or more classes included in the first learning data.

The processor generates the penalty score based on verification data, and generates the second image and one or more second inference indices corresponding to the second image based on the penalty score, wherein the verification data includes the first It may be a combination of the learning data and the second learning data.

The second inference index may be a value obtained by multiplying a difference between a false positive rate of one class and an average of the false positive rate by the first inference index.

The penalty score may be generated for a class having a false positive rate exceeding an average of the false positive rate.

1 is a diagram for describing a detection apparatus according to various embodiments of the present disclosure;
2 is a diagram for explaining an operation of generating a penalty score by a detection apparatus according to various embodiments of the present disclosure;
3 is a diagram for describing an example of false detection according to various embodiments of the present disclosure;
4 is a diagram for describing an operation of robustly detecting an object by a detection apparatus according to various embodiments of the present disclosure;
5 is another example of a detection apparatus according to various embodiments.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit described in the embodiments.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected to” another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the existence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted.

1 is a diagram for describing a detection apparatus according to various embodiments of the present disclosure;

Referring to FIG. 1 , according to various embodiments, the detection apparatus 100 may include an object detection model 110 and a penalty model 150 . For example, the object detection model 110 may be YOLOv3, and the classification model 111 may be Darknet53. The detection apparatus 100 may detect one or more objects in the object image to be robust against false positives by utilizing each configuration (the object detection model 110 and the penalty model 150 ).

In FIG. 1 , a case in which the object detection model 110 and the penalty model 150 are implemented on the same hardware is described, but the present invention is not limited thereto, and the object detection model 110 and the penalty model 150 according to an embodiment. can also be implemented in separate hardware capable of communicating.

According to various embodiments, the detection apparatus 100 may generate a first image in which one or more objects are detected from an object image, and generate a second image with a reduced false positive rate than the first image based on the penalty score. have. The detection apparatus 100 may generate a penalty score based on a false detection rate of each of one or more classes corresponding to one or more objects.

According to various embodiments, the object detection model 110 may include a classification model 111 and a detection model 113 . The object detection model 110 may primarily detect one or more objects in the object image.

According to various embodiments, the classification model 111 may classify one or more objects in an object image to generate (eg, transform) a feature map. The classification model 111 includes first training data related to the object to be detected by the detection model 113 (eg, in-distribution data), and second training data independent of the object to be detected (eg, distribution). It may be generated (eg, learned) to output a first image obtained by classifying one or more objects from an object image by receiving out-of-distribution data. For example, the classification model 111 may be additionally fine-tuned by the second training data after being trained by the first training data. The second training data may be labeled based on one or more classes included in the first training data. For example, the second training data may be labeled such that one or more classes included in the second training data are uniformly distributed in one or more classes included in the first training data.

According to various embodiments, the detection model 113 may generate a first image in which one or more objects are detected in the characteristic map and one or more first inference indices. The first inference index is an index for determining a class corresponding to the object included in the characteristic map, and the detection model 113 determines that the object of the corresponding class has a characteristic when the first inference index exceeds a predetermined threshold. By determining to be included in the map, the first image may be generated. The first inference index may be a value obtained by multiplying an object score by a class score. The object score may mean a probability that a specific object exists at a specific position in the first image, and the class score may be a probability that a class to which the specific object belongs is a specific class.

According to various embodiments, the penalty model 150 may generate a second image and a second inference index having a reduced false positive rate compared to the first image based on the penalty score. The second inference index is an index for determining a class corresponding to the object included in the first image, and the penalty model 150 indicates that when the second inference index exceeds a predetermined threshold, the object of the corresponding class is By determining that the second image is included in the second image, the second image may be generated. The second inference index may be a value obtained by multiplying a difference between a false positive rate of one class and an average of the false positive rate by the first inference index. The penalty model 150 may calculate a false positive rate, an average of false positive rates, and a penalty score by an operation described later with reference to FIG. 2 .

According to various embodiments, the penalty model 150 may generate a penalty score for a class having a false positive rate that exceeds an average of the false positive rate. The penalty model 150 may effectively reduce the false positive rate with respect to a class having a high probability of false positive by applying a penalty score only to a class having a false positive rate exceeding the average.

2 is a diagram for explaining an operation of generating a penalty score by a detection apparatus according to various embodiments of the present disclosure;

Referring to FIG. 2 , according to various embodiments, the detection apparatus 100 may receive verification data and generate a penalty score for each class included in the verification data. The verification data may be a combination of the first learning data and the second learning data in a ratio of 1:5.

According to various embodiments, the classification model 111 may generate (eg, transform) a feature map by classifying one or more objects in the verification data. The classification model 111 includes first training data related to the object to be detected by the detection model 113 (eg, in-distribution data), and second training data independent of the object to be detected (eg, distribution). It may be generated (eg, learned) to output a first image obtained by classifying one or more objects from an object image by receiving out-of-distribution data. For example, after the classification model 111 is trained by the first training data, it may be additionally fine-tuned by the second training data. The second training data may be labeled based on one or more classes included in the first training data. For example, the second training data may be labeled such that one or more classes included in the second training data are uniformly distributed in one or more classes included in the first training data.

According to various embodiments, the detection model 113 may generate result data in which one or more objects are detected in the characteristic map and one or more third inference indices. The third inference index is an index for determining a class corresponding to the object included in the characteristic map of the verification data, and the detection model 113 determines that the third inference index exceeds a predetermined threshold value of the corresponding class. By determining that the object is included in the characteristic map, result data can be generated. The third inference index may be a value obtained by multiplying an object score by a class score. The object score may mean a probability that a specific object exists at a specific location of the result data, and the class score may be a probability that a class to which the specific object belongs is a specific class.

According to various embodiments, the penalty model 150 may generate a penalty score for each class based on a false positive rate of each of one or more classes included in the result data. One or more classes may be classes to which one or more objects belong. The penalty model 150 may generate (eg, calculate) a false positive rate for each class by calculating a ratio of false positives that the object detection model 110 predicts as a correct answer for a ground truth that is not an actual correct answer. The penalty model 150 may calculate an average of the false positive rates by dividing the sum of the false positive rates of each class by the number of all classes. The penalty model 150 may generate a penalty score for a class having a false positive rate that exceeds an average of the false positive rate.

3 is a diagram for explaining an example of false detection.

Referring to FIG. 3 , a deep learning model for detecting an object may falsely detect multiple objects. The red box 300 may indicate that the deep learning model erroneously detects that the obstacle vehicle does not exist in front of the driving vehicle, but that the obstacle vehicle is present. As a result of a false detection, the driving vehicle may make an abrupt stop during an autonomous driving situation, which may lead to an accident and endanger the life of the driver.

The error matrix for the performance evaluation index of the deep learning model can be organized as shown in Table 1 below. Among the elements in Table 1, a false positive is a case in which the deep learning model predicts that the correct answer is false, and may correspond to the above example of false positive.

[Table 1]

4 is a diagram for describing an operation of robustly detecting an object by a detection apparatus according to various embodiments of the present disclosure;

Operations 410 and 420 may be used to describe an operation in which the detection apparatus 100 robustly detects an object based on a penalty score of each class included in the object image.

In operation 410, the detection apparatus 100 may generate a first image in which one or more objects are detected from the object image. The detection apparatus 100 may classify one or more objects in an object image to generate a characteristic map, and detect one or more objects in the characteristic map.

In operation 420 , the detection apparatus 100 may generate a second image having a reduced false positive rate compared to the first image based on the penalty score. The detection apparatus 100 may generate a penalty score based on a false detection rate of each of one or more classes corresponding to one or more objects.

5 is another example of a detection apparatus according to various embodiments of the present disclosure;

Referring to FIG. 5 , according to various embodiments, the detection apparatus 500 may include a memory 510 and a processor 530 .

According to various embodiments, the memory 510 may store instructions (eg, a program) executable by the processor 530 . For example, the instructions may include instructions for executing an operation of the processor 530 and/or an operation of each component of the processor 530 .

According to various embodiments, the memory 510 may be implemented as a volatile memory device or a nonvolatile memory device. The volatile memory device may be implemented as dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM). Nonvolatile memory devices include EEPROM (Electrically Erasable Programmable Read-Only Memory), Flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM (Spin-Transfer Torque (STT)-MRAM), Conductive Bridging RAM (CBRAM) , FeRAM(Ferroelectric RAM), PRAM(Phase change RAM), Resistive RAM(RRAM), Nanotube RRAM(Nanotube RRAM), Polymer RAM(Polymer RAM(PoRAM)), Nano Floating Gate Memory (NFGM)), a holographic memory, a Molecular Electronic Memory Device, and/or an Insulator Resistance Change Memory.

According to various embodiments, the processor 530 may execute computer readable code (eg, software) stored in the memory 510 and instructions induced by the processor 530 . The processor 530 may be a hardware-implemented data processing device having a circuit having a physical structure for executing desired operations. Target operations may include, for example, code or instructions included in a program. A data processing device implemented in hardware includes, for example, a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

According to various embodiments, the operation performed by the processor 530 may be substantially the same as the operation of the detection apparatus 100 described with reference to FIGS. 1 to 4 . Each configuration (eg, the object detection model 110 and the penalty model 150 ) of the detection apparatus 100 described with reference to FIGS. 1 to 4 may be executed by the processor 530 . Accordingly, detailed description will be omitted.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA) array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using a general purpose computer or special purpose computer. The processing device may execute an operating system (OS) and a software application running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. have. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, those of ordinary skill in the art may apply various technical modifications and variations based thereon. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (15)

generating a first image in which one or more objects are detected in the object image; and
Generating a second image with a reduced false positive rate (FSR) than the first image based on a penalty score
including,
The penalty score is
One or more classes corresponding to the one or more objects will be generated based on the false positive rate of each, the detection method.
According to claim 1,
The operation of generating the first image comprises:
generating a feature map by inputting the object image into a classification model for classifying the one or more objects from the object image; and
generating the first image and one or more first inference indices corresponding to the first image by inputting the characteristic map to a detection model for detecting the one or more objects in the characteristic map
A detection method comprising:
3. The method of claim 2,
The classification model is
The detection method, wherein the detection model is learned based on first learning data related to the object to be detected and second learning data irrelevant to the object to be detected.
4. The method of claim 3,
The second learning data,
The detection method, which is labeled based on one or more classes included in the first training data.
4. The method of claim 3,
The operation of generating the second image comprises:
generating the penalty score based on verification data; and
generating the second image and one or more second inference indices corresponding to the second image based on the penalty score
including,
The verification data is
The detection method, which is a combination of the first learning data and the second learning data.
6. The method of claim 5,
The second inference index is,
A value obtained by multiplying a difference between a false positive rate of one class and an average of the false positive rate by the first inference index.
According to claim 1,
The penalty score is
and a class having a false positive rate that exceeds the average of the false positive rate.
According to claim 1,
A computer program stored in a computer-readable recording medium in combination with hardware to execute the method of any one of claims 1 to 7.
a memory containing instructions; and
a processor electrically connected to the memory and configured to execute the instructions
including,
When the instructions are executed by the processor, the processor
generating a first image in which one or more objects are detected in the object image;
Generates a second image with a reduced false positive rate (FSR) than the first image based on a penalty score,
The penalty score is
One or more classes corresponding to the one or more objects will be generated based on the false positive rate of each, the device.
10. The method of claim 9,
The processor is
generating a feature map by inputting the object image to a classification model that classifies the one or more objects in the object image;
The apparatus of generating the first image and one or more first inference indices corresponding to the first image by inputting the characteristic map to a detection model for detecting the one or more objects in the characteristic map.
11. The method of claim 10,
The classification model is
The apparatus of claim 1, wherein the detection model is learned based on first training data related to the object to be detected and second training data irrelevant to the object to be detected.
12. The method of claim 11,
The second learning data,
The apparatus of claim 1, which is labeled based on one or more classes included in the first training data.
12. The method of claim 11,
The processor is
generate the penalty score based on validation data;
generating the second image and one or more second inference indices corresponding to the second image based on the penalty score;
The verification data is
which is a combination of the first learning data and the second learning data.
14. The method of claim 13,
The second inference index is,
A value obtained by multiplying a difference between a false positive rate of one class and an average of the false positive rate by the first inference index.
10. The method of claim 9,
The penalty score is
and for a class having a false positive rate that exceeds the average of the false positive rate.

Images