KR20220083613A - Method and Apparatus for Detecting Object in Image - Google Patents

Abstract

Object recognition method that can reduce the problem of difficult recognition due to the small size of the object included in the image, can quickly zoom in the camera, and thus greatly improve the accuracy of the classification task of the object in the image and devices. The object recognition method includes: receiving an input image; obtaining a first object class (c), a first reliability (p), and a bounding box (b) by performing object prediction from the input image; A second binarized image obtained by calculating a first object area ratio from a first binarized image obtained by binarizing an input image portion inside a bounding box based on a first threshold, and binarizing an input image portion inside a bounding box based on a second threshold calculating a second object area ratio in ; determining whether the input image is focused based on a difference between the first and second object area ratios; When it is determined that the input image is clearly focused, the input image is cropped and the cropped portion is enlarged to obtain an enlarged cropped image, and object prediction is performed from the enlarged cropped image to obtain a second object class (c _m ) and obtaining a second reliability p _m ; and determining a final object class based on the first reliability p and the second reliability p _m .

Description

Method and Apparatus for Detecting Object in Image

The present invention relates to a method and apparatus for recognizing an object, and more particularly, to a method and apparatus suitable for recognizing a small object from an image received from a camera or a wide area surveillance image stored in a storage device.

Machine learning to be applied to the determination of the type and location of objects in the image is made based on various learning image data, and it can be said that the performance of the learned model is greatly influenced by the qualitative method of the learning level. Machine learning through image analysis is time-consuming and requires a large amount of data sets for machine learning. At the time of filing of the present application, technologies for recognizing terrestrial objects, such as cars, people, or animals, included in images acquired by a camera have shown a lot of performance improvement along with the development of graphics processor speed and the evolution of machine learning. .

Meanwhile, in recent years, social unrest is increasing due to the invasion of airports, public places, or other protected areas by small drones. In particular, various technologies are being discussed to protect life and property from attacks by small unmanned aerial vehicles that can be used for military purposes. Detection through radar, image signal analysis-based detection, noise characteristic-based detection, etc. have been proposed as technologies applicable to the detection of small UAVs.

In particular, even in the case of detection by image analysis, unlike the case of detecting an object on the ground, it is not an effective solution because the size of the small drone is small and the distance from the camera is far, so the size of the small drone object included in the image is very small. . Therefore, there is a need for a method capable of detecting all invading small unmanned aerial vehicles and accurately recognizing each small unmanned aerial vehicle. It is possible to think of ways to enlarge the area occupied by a small drone within the image by using a zoom lens and to narrow the surveillance area covered by each camera by arranging multiple cameras, but the economic burden increases as the number of required cameras increases rapidly. A problem arises that In addition, since the time required for the camera to enter the zoom-in state is relatively long, the zoom-in may not be successfully performed within a required time.

The present invention is intended to solve such a problem, and it is possible to reduce the problem of difficult recognition due to the small size of the object included in the image, and the camera can be quickly zoomed in and driven, thereby classifying the object in the image Provided are an object recognition method and apparatus capable of accurately detecting a small unmanned aerial vehicle by greatly improving work accuracy.

An object recognition method according to an aspect of the present invention includes the steps of receiving an input image; obtaining a first object class (c), a first reliability (p), and a bounding box (b) by performing object prediction from the input image; A first object area ratio is calculated from a first binarized image obtained by binarizing an input image portion inside the bounding box based on a first threshold, and a second binarization of an input image portion inside the bounding box based on a second threshold. calculating a second object area ratio in the binarized image; determining whether the input image is focused based on a difference between the ratios of the first and second object areas; When it is determined that the input image is clearly focused, the input image is cropped and the cropped portion is enlarged to obtain an enlarged cropped image, and object prediction is performed from the enlarged cropped image to obtain a second object class (c) _m ) and a second confidence level p _m ; and determining a final object class based on the first reliability p and the second reliability p _m .

The determining whether the input image is focused may include: determining whether a difference between the first and second object area ratios is greater than a predetermined reference value; determining that the input image is not focused when the difference is greater than the reference value; and determining that the input image is focused if the difference is not greater than the reference value.

In an embodiment, the input image may be received from a camera. In this case, the step of determining whether the input image is focused may further include supplying a control signal for focus adjustment to the camera when it is determined that the input image is not focused.

The control signal may include a zoom-in control command or an auto-focus command.

The determining of the final object class further includes the step of assigning first and second weights w1 and w2 to the first reliability p and the second reliability p _m , respectively, and comparing them, The final object class may be determined according to the result.

In the determining of the final object class, if the value obtained by multiplying the second reliability p _m by the second weight w2 is greater than the value obtained by multiplying the first reliability p by the first weight w1 , determining the final object class for the second object class (c _m ); and if the value obtained by multiplying the second reliability p _m by the second weight w2 is not greater than the value obtained by multiplying the first reliability p by the first weight w1, the first object class c ) to determine the final object class; may include.

An object recognition apparatus according to an aspect of the present invention comprises: a memory for storing program instructions; and a processor connected to the memory and executing the program instructions stored in the memory. The program instructions, when executed by the processor, cause the processor to: accept an input image; performing object prediction from the input image to obtain a first object class (c), a first reliability (p), and a bounding box (b); A first object area ratio is calculated from a first binarized image obtained by binarizing an input image portion inside the bounding box based on a first threshold, and a second binarization of an input image portion inside the bounding box based on a second threshold. calculating a second object area ratio in the binarized image; determining whether the input image is focused based on a difference between the ratio of the first and second object areas; When it is determined that the input image is clearly focused, the input image is cropped and the cropped portion is enlarged to obtain an enlarged cropped image, and object prediction is performed from the enlarged cropped image to obtain a second object class (c) _m ) and a second confidence level (p _m ); An operation of determining a final object class may be performed based on the first reliability p and the second reliability p _m .

According to an embodiment of the present invention, it is possible to reduce the problem of difficulty in recognition due to the small size of the object included in the image, and to quickly zoom in the camera, and thus the accuracy of the classification operation of the object in the image can be greatly improved. Therefore, it is possible to detect all the small unmanned aerial vehicles invading, and it is possible to accurately detect each small unmanned aerial vehicle.

In particular, according to an embodiment, various environmental requirements such as illuminance, image color, contrast, and object size are well reflected through image binarization, and a clearly focused image is secured based on the ratio of objects in the bounding box. . In this process, by determining whether to focus based on the binarized images with different thresholds and controlling the focusing of the camera, the physical camera zoom-in time can be reduced. On the other hand, a final object class is determined by selecting an object class extracted from an image reproduced through cropping and magnification of the screen and an object class having a higher accuracy. Accordingly, classification accuracy can be significantly improved, so that it is possible to detect all intruding small UAVs, and it is possible to accurately recognize each small UAV.

1 is a screen shot showing an example of a small unmanned aerial vehicle detection failure in an image.
2 is a screen shot showing another example of a small unmanned aerial vehicle detection failure case in an image.
3 is a view exemplarily showing various shapes of the small UAV according to the altitude, the distance from the camera, the shooting direction, and the tilt angle of the small UAV.
4 is a schematic diagram of an object recognition system according to an embodiment of the present invention.
5 is a functional block diagram of an object recognition apparatus according to an embodiment of the present invention.
6 is a diagram illustrating an example of setting a bounding box.
7 is a diagram illustrating an example of image binarization.
8 is a diagram illustrating an example of an enlarged image by cropping an input image.
9 is a physical block diagram of an object recognition apparatus according to an embodiment of the present invention.
10 is a flowchart illustrating an object recognition method according to an embodiment of the present invention.
11 is a detailed flowchart showing the final classification determination step shown in FIG. 10 in more detail.
12 is a diagram exemplarily showing a difference in a ratio of an object area between a first binarized image and a second binarized image with respect to an out-of-focus image.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each drawing, like reference numerals are used for similar components.

Ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term “and/or” includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

When detecting a small UAV by image analysis, since the size of the small UAV is small and the distance from the camera is far, the size of the small UAV object displayed in the image is small, so the accuracy of object recognition and classification is significantly reduced. 1 shows an example of an object classification error when the object size is very small. In the illustrated example, the reliability of object classification is 0.482, which is very low. In addition, the detected object is classified incorrectly even though it is not an IRONMAN model.

Although the object can be enlarged by zooming in the camera and applied as an input to the classification model, the zoom-in may not be successfully performed within the required time because it takes a short time to zoom in the camera. In addition, since the focus of the detection target object is blurred, the accuracy of object recognition and classification may not increase. 2 shows an example of a classification error for an enlarged object. In the illustrated example, the object classification reliability is 0.987, which is a fairly high value, but a classification error occurs as the small UAV is a DJI model even though it is a different model. In particular, as shown in FIG. 3 , the shape of a small UAV can vary greatly depending on the height of the small UAV, the distance from the camera, the shooting direction, and the tilt angle. It's not easy to do.

4 is a schematic diagram of an object recognition system according to an embodiment of the present invention. The illustrated object recognition system includes a surveillance camera 10 (hereinafter abbreviated as “camera”) and an object recognition device 100 that receives an image acquired by the camera 10 and detects a small UAV object included in the image. to provide The object recognition apparatus 100 may be installed to be located adjacent to the camera 10 . However, the present invention is not limited thereto, and the object recognition apparatus 100 may be installed at a location far away from the camera 10 . In this case, the object recognition apparatus 100 may be connected to the camera 10 through an IP network. Meanwhile, the object recognition apparatus 100 may detect an object based on images stored in internal and external storage devices in addition to the images acquired by the camera.

5 is a functional block diagram of the object recognition apparatus 100 according to an embodiment of the present invention. For convenience of description, the camera 10 is also illustrated in FIG. 5 . The object recognition apparatus 100 includes a first object prediction unit 110 , a focusing determination unit 120 , a camera control unit 140 , a crop/enlargement unit 150 , a second object prediction unit 160 , and a final classification determination A portion 170 is provided.

The first object predictor 110 may obtain the object class c, the reliability p, and the bounding box b from the input image from the camera 10 . As is well known, the bounding box is a rectangular object set to surround the area where the object to be detected is most likely to exist in order to narrow the object search area. An example of a bounding box is shown in FIG. 6 . According to an embodiment, the operation of the first object prediction unit 110 may be performed based on a predetermined prediction model learned in advance. The prediction model may be built, for example, based on a convolutional neural network (CNN), but the present invention is not limited thereto.

The focusing determination unit 120 binarizes the input image and determines whether the input image is clearly focused, that is, whether the camera 10 that provided the input image is well focused. And, when it is determined that the input image is not clearly focused, the focusing determination unit 120 may cause the camera control unit 140 to transmit a control signal to the camera 10 to change the focusing.

In an embodiment, the focusing determination unit 120 includes the first image binarization unit 122 , the first object ratio calculation unit 124 , the second image binarization unit 126 , and the second object ratio calculation unit 128 . , an object ratio difference calculation unit 130 , and a determination unit 132 .

The first image binarization unit 122 binarizes the input image to generate a first binarized image. At this time, FIG. 7 shows an example of image binarization. The first image binarizer 122 may classify the level of each pixel into '0' (black) and '1' (white) based on the first threshold value T ₁ . That is, when the luminance level of the pixel is greater than the first threshold (T ₁ ), the pixel value of the corresponding pixel in the first image binarized image has a value of '0', indicating that the corresponding pixel is a pixel (pixel On) within the object. , when the luminance level is smaller than the first threshold value T ₁ , the pixel value of the corresponding pixel in the first image binarized image has a value of '1', indicating that the corresponding pixel is a pixel outside the object (pixel Off). The first object ratio calculator 124 calculates an object area ratio Ratio ₁ representing a ratio of the number of object pixels to the total number of pixels in a bounding box in the first binarized image according to Equation 1 below.

)만큼 다르게 설정될 수 있다. 제2 객체 비율 계산부(128)는 제2 이진화 영상에서 상기 수학식 1에 의하여 바운딩 박스 내의 전체 픽셀 수에 대한 객체 픽셀의 수의 비율을 나타내는 객체 영역 비율(Ratio₂)을 계산한다.The second image binarization unit 126 binarizes the input image to generate a second binarized image. In this case, the second image binarizer 126 may classify the level of each pixel into '0' and '1' based on the second threshold value T ₂ . The second threshold (T ₂ ) is a constant value than the first threshold (T ₁ ) as in Equation 2 (

) can be set differently. The second object ratio calculator 128 calculates an object area ratio Ratio ₂ representing a ratio of the number of object pixels to the total number of pixels in a bounding box according to Equation 1 in the second binarized image.

)보다 크면 즉, 다음 수학식 3을 만족하면, 카메라(10)의 포커싱이 되어 있지 않다고 판단할 수 있다.The object ratio difference calculation unit 130 calculates a difference (Ratio ₁ -Ratio ₂ ) between the ratios of the object areas output from the first and second object ratio calculation units 124 and 128 , and the determination unit 132 is the object area Based on the ratio difference (Ratio ₁ -Ratio ₂ ), it is determined whether the input image is clearly focused. In an embodiment, the determination unit 132 determines that the object area ratio difference (Ratio ₁ -Ratio ₂ ) is a specific reference value (

), that is, if the following Equation 3 is satisfied, it can be determined that the camera 10 is not focused.

When the camera 10 is not focused, when the second threshold T ₂ is applied, the fade-out tendency is greater than when the focus is achieved, so that the ratio of pixels outside the object is increased. Therefore, when the focus is not performed, the object area derived by applying the second threshold value T ₂ is significantly reduced compared to the object area when the first threshold value T ₁ is applied. Using this phenomenon, it is possible to numerically classify a situation in which a focus is not made as shown in Equation (3).

When the camera 10 is not focused, the determination unit 132 causes the camera control unit 140 to transmit a control signal for focus adjustment, and a difference in object area ratio with respect to an image newly input from the camera 10 . (Ratio ₁ -Ratio ₂ ) may be recalculated and wait until Equation 4 is satisfied. In this way, the camera control unit 140 may transmit a control signal for focus adjustment to the camera 10 under the control of the focusing determination unit 120 . The control signal may include a zoom-in control command or an auto-focus command.

로 나타낼 수 있다. 제2 객체 예측부(160)는 크롭 및 확대된 영상에서 객체 클래스(c_m) 및 신뢰도(p_m)를 획득한다. 제2 객체 예측부(160)의 동작은 제1 객체 예측부(110)와 동일한 예측 모델을 토대로 수행될 수 있다. 최종 분류 판단부(170)는 제1 객체 예측부(110)에 의해 생성된 신뢰도(p)와 제2 객체 예측부(160)에 의해 생성된 신뢰도(p_m)에 각각의 가중치(w1, w2)를 부여하여, 최종 객체 클래스를 결정한다. 일 실시예에 있어서, 최종 객체 클래스는 제1 객체 예측부(110)에 의해 결정된 객체 클래스(c)와 제2 객체 예측부(160)에 의해 결정된 객체 클래스(c_m) 중에서 어느 하나로 정해질 수 있다.On the other hand, when the determination unit 132 determines that the camera 10 is well focused based on the input image, the crop/enlargement unit 150 is located in a larger area than the bounding box or the bounding box as shown in FIG. 8 . The input image is cropped and the cropped image is enlarged. the magnification ratio

can be expressed as The second object prediction unit 160 obtains an object class (c _m ) and reliability (p _m ) from the cropped and enlarged image. The operation of the second object predictor 160 may be performed based on the same prediction model as that of the first object predictor 110 . The final classification determining unit 170 assigns weights w1 and w2 to the reliability p generated by the first object prediction unit 110 and the reliability p _m generated by the second object prediction unit 160, respectively. ) to determine the final object class. In an embodiment, the final object class may be determined as any one of the object class c determined by the first object predictor 110 and the object class c _m determined by the second object predictor 160 have.

As described above, a binarized image of an enlarged image in a state in which sharp focusing is secured is highly likely to provide the level of sharpness before enlargement, and thus the reliability of final class determination may be increased. In particular, by adjusting the respective weights w1 and w2 while operating based on the weights w1 and w2 as described above, it becomes possible to secure stable reliability according to the performance of the learning model. In addition, the image processing by binarization, cropping, and magnification as described above is temporally advantageous over physical magnification by zoom-in of the camera, and can provide rapid detection of a moving object.

The object recognition apparatus 100 illustrated in FIG. 5 may be implemented by a general-purpose data processing apparatus including a processor and a memory. 9 shows an example of the physical configuration of the object recognition apparatus 100 according to an embodiment of the present invention. The object recognition apparatus 100 may include at least one processor 220 , a memory 240 , and a storage device 260 . Components of the object recognition apparatus 100 may be connected by a bus to exchange data.

The processor 220 may execute program instructions stored in the memory 240 and/or the storage device 260 . Processor 220 may include at least one central processing unit (CPU), graphics processing unit (GPU), or other processor capable of performing the method according to the present invention. The memory 240 may include, for example, a volatile memory such as a random access memory (RAM) and a non-volatile memory such as a read only memory (ROM). The memory 240 loads program instructions stored in the storage device 260 and provides them to the processor 220 so that the processor 220 can execute them. The storage device 260 is a recording medium suitable for storing program instructions and data, for example, a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, a compact disk read only memory (CD-ROM), and a DVD. (Optical Media) such as (Digital Video Disk), magneto-optical media such as floppy disk, flash memory or EPROM (Erasable Programmable ROM), or manufactured based on them It may include a semiconductor memory such as an SSD.

When the program instructions are executed by the processor 220 , the processor 220 may perform an operation necessary to implement an object recognition method to be described later. For example, the program instructions, when executed by the processor 220 , cause the processor 220 to: receive an input image; performing object prediction from the input image to obtain a first object class (c), a first reliability (p), and a bounding box (b); A first object area ratio is calculated from a first binarized image obtained by binarizing an input image portion inside the bounding box based on a first threshold, and a second binarization of an input image portion inside the bounding box based on a second threshold. calculating a second object area ratio in the binarized image; determining whether the input image is focused based on a difference between the ratios of the first and second object areas; When it is determined that the input image is clearly focused, the input image is cropped and the cropped portion is enlarged to obtain an enlarged cropped image, and object prediction is performed from the enlarged cropped image to obtain a second object class (c) obtaining _m ) and a second confidence level p _m ; and determining a final object class based on the first reliability p and the second reliability p _m .

10 is a flowchart illustrating an object recognition method according to an embodiment of the present invention.

When an image is input (step 300), the first object prediction unit 110 predicts an object from the input image (step 302). In the object prediction process, the first object prediction unit 110 may obtain an object class (c), a reliability (p), and a bounding box (b).

Next, the first image binarization unit 122 binarizes the input image to generate a first binarized image, and the first object ratio calculator 124 calculates an object area ratio Ratio ₁ from the first binarized image ( step 304). Meanwhile, the second image binarization unit 126 binarizes the input image to generate a second binarized image, and the second object ratio calculator 128 calculates an object area ratio Ratio ₂ from the second binarized image ( step 306).

Next, the object ratio difference calculation unit 130 calculates the difference between the object area ratios (Ratio ₁ -Ratio ₂ ) (step 308), and the focus determination unit 120 determines the object area ratio difference (Ratio ₁ -Ratio ₂ ). ), it is determined whether the input image is clearly focused (step 310). If it is determined based on the input image that the camera 10 is not focused, the focus determination unit 120 causes the camera control unit 140 to transmit a focus adjustment command or an auto-focus command to the camera 10 , Let the focusing of (10) be adjusted (step 312). Based on the image newly input from the camera 10 , the process may be maintained in a standby state until the focus determination unit 120 determines that the camera 10 is well focused.

If it is determined in step 210 that the camera 10 is well focused based on the input image, the crop/enlarger 150 crops the input image with respect to a bounding box or a larger area than the bounding box, and the cropped image is enlarged (step 314). The second object prediction unit 160 may predict an object from the cropped and enlarged image to obtain an object class (c _m ) and reliability (p _m ) (operation 316 ). The final classification determining unit 170 assigns weights w1 and w2 to the reliability p generated by the first object prediction unit 110 and the reliability p _m generated by the second object prediction unit 160, respectively. ) to determine the final object class (step 318).

11 is a detailed flowchart showing an embodiment of the final classification determination step (step 318) in more detail. According to the embodiment shown in FIG. 11 , the final classification determiner 170 multiplies the reliability p generated by the first object predictor 110 by the first weight w1 and predicts the second object. A value obtained by multiplying the reliability p _m generated by the unit 160 by the second weight w2 may be compared (operation 320). If the value obtained by multiplying the reliability p _m generated by the second object prediction unit 160 by the second weight w2 is the reliability p generated by the first object prediction unit 110, the first weight ( w1), the final classification determiner 170 may determine the final object class from the object class c _m predicted by the second object predictor 160 (step 322). If the value obtained by multiplying the reliability p _m generated by the second object prediction unit 160 by the second weight w2 is the reliability p generated by the first object prediction unit 110, the first weight ( w1), the final classification determiner 170 may determine the final object class from the object class c predicted by the first object predictor 110 (operation 324).

=-40)인 55로 설정되었다. 도면에서 볼 수 있는 바와 같이, 포커스가 맞지 않은 영상의 경우, 제2 임계치를 토대로 정해지는 제2 이진화 영상 내에서 객체 영역의 비율이 줄어든 것을 확인할 수 있다. 따라서, 본 발명의 메커니즘이 정상적으로 작용한다고 볼 수 있다.12 exemplarily shows the difference in the ratio of the object area between the first binarized image obtained based on the first threshold value T ₁ and the second binarized image obtained based on the second threshold value T ₂ from the out-of-focus image. . In this example, the first threshold T ₁ is set to 95, and the second threshold T ₂ is 40 less (ie,

=-40) was set to 55. As can be seen in the drawing, in the case of an out-of-focus image, it can be confirmed that the ratio of the object area is reduced in the second binarized image determined based on the second threshold. Therefore, it can be seen that the mechanism of the present invention operates normally.

As mentioned above, the apparatus and method according to the embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. In addition, the computer-readable recording medium may be distributed in a network-connected computer system to store and execute computer-readable programs or codes in a distributed manner.

The computer-readable recording medium may include a hardware device specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. The program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although some aspects of the invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, wherein a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also represent a corresponding block or item or a corresponding device feature. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, programmable computer or electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, the field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

Although the above has been described with reference to the preferred embodiment of the present invention, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

Claims (12)

A method of recognizing an object in an image, comprising:
receiving an input image;
obtaining a first object class (c), a first reliability (p), and a bounding box (b) by performing object prediction from the input image;
A first object area ratio is calculated from a first binarized image obtained by binarizing an input image portion inside the bounding box based on a first threshold, and a second binarization of an input image portion inside the bounding box based on a second threshold. calculating a second object area ratio in the binarized image;
determining whether the input image is focused based on a difference between the ratios of the first and second object areas;
When it is determined that the input image is clearly focused, the input image is cropped and the cropped portion is enlarged to obtain an enlarged cropped image, and object prediction is performed from the enlarged cropped image to obtain a second object class (c) _m ) and a second confidence level p _m ; and
and determining a final object class based on the first reliability (p) and the second reliability (p _m ). The method according to claim 1, wherein the step of determining whether the input image is focused
determining whether a difference between the first and second object area ratios is greater than a predetermined reference value;
determining that the input image is not focused when the difference is greater than the reference value; and
determining that the input image is focused if the difference is not greater than the reference value;
An object recognition method comprising The method according to claim 2, wherein the input image is received from a camera,
The step of determining whether the input image is focused
supplying a control signal for focus adjustment to the camera when it is determined that the input image is not focused;
An object recognition method further comprising a. The method according to claim 3, wherein the control signal includes a zoom-in control command or an auto-focus command. The method according to claim 1, wherein determining the final object class comprises:
assigning first and second weights w1 and w2 to the first reliability p and the second reliability p _m , respectively, and comparing them;
The object recognition method further comprising: determining the final object class according to the comparison result. The method of claim 5, wherein determining the final object class comprises:
If the value obtained by multiplying the second reliability p _m by the second weight w2 is greater than the value obtained by multiplying the first reliability p by the first weight w1, the second object class c _m ) to determine the final object class; and
If the value obtained by multiplying the second reliability p _m by the second weight w2 is not greater than the value obtained by multiplying the first reliability p by the first weight w1, the first object class c determining the final object class;
An object recognition method comprising An apparatus for recognizing an object in an image, comprising:
a memory storing program instructions; a processor connected to the memory and executing the program instructions stored in the memory;
The program instructions, when executed by the processor, cause the processor to:
accept an input image;
performing object prediction from the input image to obtain a first object class (c), a first reliability (p), and a bounding box (b);
A first object area ratio is calculated from a first binarized image obtained by binarizing an input image portion inside the bounding box based on a first threshold, and a second binarization of an input image portion inside the bounding box based on a second threshold. calculating a second object area ratio in the binarized image;
determining whether the input image is focused based on a difference between the ratio of the first and second object areas;
When it is determined that the input image is clearly focused, the input image is cropped and the cropped portion is enlarged to obtain an enlarged cropped image, and object prediction is performed from the enlarged cropped image to obtain a second object class (c) _m ) and a second confidence level (p _m );
An object recognition apparatus configured to determine a final object class based on the first reliability (p) and the second reliability (p _m ). 8. The method of claim 7, wherein the program instructions that cause the processor to determine whether the input image is in focus cause the processor to:
determine whether a difference between the first and second object area ratios is greater than a predetermined reference value;
if the difference is greater than the reference value, it is determined that the input image is not focused; and
If the difference is not greater than the reference value, the object recognition apparatus determines that the input image is focused. The method according to claim 8, wherein the object recognition device is interlocked with a camera to receive the input image from the camera,
Program instructions that cause the processor to determine whether the input image is in focus cause the processor to:
An object recognition apparatus configured to supply a control signal for focus adjustment to the camera when it is determined that the input image is not focused. The apparatus of claim 9 , wherein the control signal includes a zoom-in control command or an auto-focus command. 8. The method of claim 7, wherein program instructions causing the processor to determine the final object class cause the processor to:
First and second weights w1 and w2 are respectively assigned to the first reliability p and the second reliability p _m and compared, and the final object class is determined according to the comparison result. object recognition device. 12. The method of claim 11, wherein program instructions causing the processor to determine the final object class cause the processor to:
If the value obtained by multiplying the second reliability p _m by the second weight w2 is greater than the value obtained by multiplying the first reliability p by the first weight w1, the second object class c _m ) to determine the final object class;
If the value obtained by multiplying the second reliability p _m by the second weight w2 is not greater than the value obtained by multiplying the first reliability p by the first weight w1, the first object class c An object recognition device for determining the final object class.

Images