KR20230090255A - Apparatus and method for object recognition - Google Patents

Abstract

The present invention relates to an object recognition device, and includes an object inference model for generating an image of a size to be input to the machine learning inference model by processing an original image captured by a camera module, wherein the object inference model comprises: It includes a machine learning inference model, and outputs results of recognition and classification of an object inferred through the machine learning inference model, wherein the machine learning inference model processes an input image to identify an object included in the input image. It is characterized by reasoning.

Description

Object recognition device and method {APPARATUS AND METHOD FOR OBJECT RECOGNITION}

The present invention relates to an object recognition apparatus and method, and more particularly, to an object recognition apparatus and method capable of improving inference accuracy and inference speed for object recognition and classification.

In general, machine learning is called machine learning, and it can be said that a computer learns and predicts an algorithm based on data input by a user. In other words, machine learning is a technology that recognizes the hierarchical structure and certain patterns of related things, judges and decides information that has not been input by itself, and predicts future situations.

One of the types of machine learning learning is supervised learning, which means an algorithm that classifies a model by weighing a result value for each data used for training. In addition, another learning type of machine learning includes unsupervised learning, which refers to an algorithm that divides into groups by finding commonalities in training data to which result values are not separately assigned. In addition, another type of learning of machine learning includes reinforcement learning, which means an algorithm that provides compensation according to how a user behaves in different situations without separately preparing training data.

The machine learning is used in various fields such as games, automobiles, and robots.

The background art of the present invention is disclosed in Korean Patent Registration No. 10-2261187 (registered on May 31, 2021, surveillance video analysis system and method based on machine learning).

According to one aspect of the present invention, the present invention was created to solve the above problems, to provide an object recognition apparatus and method that can improve the accuracy and reasoning speed for object recognition and classification. There is a purpose.

An object recognition apparatus according to an aspect of the present invention includes an object inference model that processes an original image captured by a camera module and generates an image of a size to be input to the machine learning inference model, wherein the object inference model , Includes the machine learning inference model, and outputs a result of recognition and classification of an object inferred through the machine learning inference model, wherein the machine learning inference model processes an input image to obtain information included in the input image. It is characterized by inferring an object.

In the present invention, the object inference module extracts, as an image to be input to the machine learning inference model, an image of object-containing regions clustered in the original image according to a size to be input to the machine learning inference model to be characterized

In the present invention, in order to extract the object-containing region, the object inference module calculates a background line from a binarized image in which the original image is reduced, and then calculates a boundary line by adding an offset to the sky space above the background line, , It is characterized in that the object-containing region is extracted only from the sky space excluding the region below based on the boundary line.

In the present invention, the object inference module converts the color image, which is the original image, into a gray image having the same values of R, G, and B, and generates a reduced binarized image by downscaling the gray image; The binarized image is characterized in that it is an image expressed as 1 when the pixel value is greater than or equal to a specified threshold value and expressed as 0 when the pixel value is smaller than the specified threshold value.

)를 적용한 후 가중 평균함으로써, 최종 배경선(B_i)을 아래의 수학식 3과 같이 산출하는 것을 특징으로 한다.In the present invention, the object inference module calculates N background lines as shown in Equation 2 below by applying different threshold values, and each of the N background lines has a different weight (

) is applied and then a weighted average is performed to calculate the final background line (B _i ) as shown in Equation 3 below.

(Equation 2)

(Equation 3)

In the present invention, the object inference module calculates an object line to detect an object in the reduced binarized image, and calculates the object line calculation threshold value (THRESHOLD_O) specified in the following equation to calculate the object line. 5, the object line is detected by repeating the process of finding the first vertical pixel index (OBJ _i,O ) where the value of the vertical pixel for the horizontal pixel i of the binarized image is the minimum.

(Equation 5)

In the present invention, the object inference module independently detects an object in a plurality of partitioned regions obtained by dividing the reduced binary image in the vertical direction, but the slope of the object line exceeds the object detection threshold (THRESHOLD_OBJECT). It is characterized in that the point of greatest increase is detected as the position of the object.

In the present invention, the object inference module clusters object-containing regions according to a size that can be input to a machine learning inference model when the positions of objects are detected in the reduced binarized image, but can be combined within a corresponding image It is characterized by clustering with the smallest number.

In the present invention, the object inference module is characterized in that by applying a specified ratio (RATIO) to the clustered object-containing area in the reduced binarized image, mapping to the object-containing area in the original image.

An object recognition method according to another aspect of the present invention includes receiving an original image by an object inference module; extracting, by the object inference module, an area including at least one object according to a size to be input to a machine learning inference model through processing of the original image; and performing, by the object inference module, recognition and classification of an object included in the at least one object-containing region by the machine learning inference model and outputting the result.

In the present invention, the step of extracting at least one object-containing region according to the size to be input to the machine learning inference model includes: generating, by the object inference module, a reduced binarized image by transforming the original image; clustering a region including at least one object corresponding to an object detected in the reduced binary image; and extracting the object-containing region to be input to the machine learning inference model by mapping the object-containing region clustered in the downsized binarized image to the object-containing region in the original image.

In the present invention, in order to extract at least one object-containing region from the reduced binary image, the object inference module calculates a background line from the reduced binary image and adds an offset to the sky space above the background line. It is characterized in that the boundary line is calculated, and the object-containing region is extracted only from the sky space excluding the area below based on the boundary line.

In the present invention, in order to extract at least one object-containing region from the reduced binarized image, the object inference module divides the reduced binarized image into a plurality of divided regions in a vertical direction, and the plurality of divided regions It is characterized in that the overall inference time is shortened by independently detecting objects in .

An object recognition method according to another aspect of the present invention includes generating a reduced image by reducing an original image by an object inference module; generating, by the object inference module, a reduced binarized image by converting the reduced image into a binarized image; the object inference module detecting an object in the reduced binarized image and clustering object-containing regions according to a size that can be input to a machine learning inference model; mapping, by the object inference module, an object-containing region clustered in the reduced binarized image to an object-containing region in the original image; and inferring the object by inputting the object-containing regions mapped to the original image into a machine learning inference model.

In the present invention, in order to extract the object-containing region, the object inference module calculates a boundary line by calculating a background line from the reduced binarized image and then adding an offset to a sky space above the background line, and calculating the boundary line. It is characterized in that the object-containing area is extracted only from the sky space excluding the area below based on .

In the present invention, in the step of generating the reduced binarized image, the object inference module converts the color image, which is the original image, into a gray image having the same R, G, and B values, and the gray image After generating a reduced image by downscaling, a binary image is generated by expressing 1 when the pixel value is greater than or equal to a designated threshold value and expressing 0 when the pixel value is smaller than the designated threshold value.

)를 적용한 후 가중 평균함으로써, 최종 배경선(B_i)을 아래의 수학식 3과 같이 산출하는 것을 특징으로 한다.In the present invention, in order to calculate the background line, the object inference module calculates N background lines as shown in Equation 2 below by applying different threshold values, and has different weights for each of the N background lines. (

) is applied and then a weighted average is performed to calculate the final background line (B _i ) as shown in Equation 3 below.

(Equation 2)

(Equation 3)

In the present invention, in order to detect an object in the reduced binary image, the object inference module calculates an object line to detect an object in the reduced binary image, and the slope of the object line is a threshold value for object detection. (THRESHOLD_OBJECT) is characterized by detecting the point of greatest increase as the location of the object.

In the present invention, in order to calculate the object line, the object inference module applies the designated object line calculation threshold value (THRESHOLD_O) to Equation 5 below to calculate the horizontal pixel i of the reduced binarized image. It is characterized in that the object line is detected by repeating a process of finding a first vertical pixel index (OBJ _i,O ) having a minimum vertical pixel value.

(Equation 5)

In the present invention, in the step of mapping the object-containing region clustered in the reduced binarized image to the object-containing region in the original image, the object inference module includes a clustered object-containing region in the reduced binary image ((( Applying the specified ratio (RATIO) to cx, cy), (cx+w, cy+h)), the area containing the object in the original image (((cx, cy), (cx+w, cy+h))× RATIO).

According to one aspect of the present invention, the present invention does not simply downsize the entire original image and inputs it to the machine learning inference model, but calculates the object-containing region from the original image and inputs it to the machine learning inference model, thereby improving object recognition and classification. To improve inference accuracy and reasoning speed.

In addition, according to another aspect of the present invention, the present invention can be directly applied as an input method of a machine learning inference model for detecting objects appearing in sky space and confirming their types, and in particular, inference accuracy and inference speed in high-resolution images. to be able to improve

According to another aspect of the present invention, the present invention can be directly applied to a system for tracking the physical location of an inferred object.

1 is an exemplary view showing a schematic configuration of an object recognition apparatus according to an embodiment of the present invention.
FIG. 2 is an exemplary diagram shown to explain problems that may occur in a process of classifying objects included in an image captured through a camera module in FIG. 1 using machine learning.
FIG. 3 is an exemplary diagram showing an object recognition error image when the size of the object is small in the process of classifying the object in FIG. 2 .
4 is an exemplary view schematically illustrating a method of increasing the size of an object to be input to a machine learning model in order to improve object recognition and classification accuracy and inference speed according to an embodiment of the present invention. am.
FIG. 5 is an exemplary view showing a more specific configuration of the object inference module in FIG. 1 .
FIG. 6 is an exemplary diagram for explaining a method of extracting an object-containing region from an original image by an object inference module in FIG. 1 .
FIG. 7 is an exemplary diagram shown in FIG. 6 to explain the principle of detecting a background line in an original image.
FIG. 8 is an exemplary diagram shown in FIG. 7 to explain an image binary-coding method as a step prior to detecting a background line in an original image.
FIG. 9 is an exemplary diagram shown to explain a method of calculating a background line from a reduced binarized image in FIG. 8 .
FIG. 10 is an exemplary diagram shown to explain the method of detecting an object in a reduced binarized image in FIG. 8 .
FIG. 11 is an exemplary diagram shown to explain the method of detecting an object using a segmented region in a reduced binarized image in FIG. 8 .
FIG. 12 is an exemplary diagram shown to explain a method of clustering object-containing regions in a reduced binarized image in FIG. 11 .
FIG. 13 is an exemplary diagram for explaining the process of calculating an object-containing region from an original image and inputting the object-containing region to a machine learning inference model in FIG. 6 .

Hereinafter, an embodiment of an object recognition apparatus and method according to the present invention will be described with reference to the accompanying drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of explanation. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

Recently, a technology for recognizing an object (eg, a car, a person, an animal, etc.) in an image captured by a camera has been greatly improved along with the processing speed of a graphics card and the evolution of machine learning.

On the other hand, recently, in the field of defense, a technology for recognizing the appearance of drones at an early stage has emerged in order to respond to drone attacks by applying the technology for recognizing objects in the image. It is emerging as a very important technology in terms of privacy protection.

At this time, the technology for recognizing the object in the image is implemented through machine learning (machine learning) to determine what the object (ie, object) to be recognized on the screen is and in what area the object is located. Machine learning (machine learning) for object recognition is performed through image learning on various screens, and its performance is greatly influenced by high-quality learning methods.

1 is an exemplary view showing a schematic configuration of an object recognition device according to an embodiment of the present invention, and a method of detecting a drone (unmanned aerial vehicle) and determining the type will be exemplarily described below.

As shown in FIG. 1 , the object recognition apparatus according to the present embodiment includes a camera module 101 , an object inference module 102 , and an image output module 104 .

The camera module 101 photographs an object 10 in space.

The object inference module 102 determines the type of object 10 included in the image captured by the camera module 101 (ie, classifies the object), and outputs the image when the object 10 is present. Determine where it is located on the module 104 (ie, on the screen) (eg, display a bounding box).

The image output module 104 displays the type of object 10 classified by the object inference module 102 and the location (ie, bounding box) 105 of the object on the screen (see FIG. 3 ). That is, the image output module 104 is a concept including a screen.

At this time, the object 103 (ie, the object image) displayed on the image output module 104 (ie, on the screen) is a real object 10 in the space captured by the camera module 101 It means an object 103 (ie, an object image) in the form of an image output on the screen.

In addition, a bounding box 105 is displayed around the object 103 (ie, object image) displayed on the screen of the image output module 104 .

FIG. 2 is an exemplary diagram shown in FIG. 1 to explain problems that may occur in the process of classifying an object included in an image captured through a camera module using machine learning, and FIG. 2 is an exemplary view showing an object recognition error image when the size of the object is small in the process of classifying the object.

Referring to FIG. 2, since the size of an image input to a machine learning model (or machine learning inference model) is limited, the object inference module 102 determines the size that can be input to a machine learning model (or machine learning inference model). After reducing the size of the original image 201 (ie, according to the specified ratio), the reduced image 203 of the original image 201 is input to the machine learning inference model 205.

Here, the machine learning inference model 205 means a model for inferring an object using machine learning, and the object inference module 102 may include the machine learning inference model 205 .

However, as described above, when the size of the original video 201 is reduced, the amount of information is lost as a result. There is a problem.

For example, when the size of an image is reduced as described above, the size of an object in the corresponding image is also reduced, so there is a problem in that the accuracy of object detection and classification is significantly reduced. For example, referring to FIG. 3, the type of object (e.g., drone model) displayed as a bounding box in the image is described as “IRONMAN (i.e., the model name of a known drone)”, but it is actually an incorrectly determined model , the accuracy is displayed as “0.482”, indicating that the accuracy is very low. In addition, since the size of the minimum learning object of the learning model must be reduced in order to recognize a small-sized object by machine learning, it takes a relatively long learning time compared to learning a large-sized object, resulting in final inference (inference). ), there is a problem that the time is relatively long compared to large objects.

In other words, in order to recognize an object in real time from an image captured by the camera module 101, inference time as well as accuracy is a very important factor. That is, fast reasoning is required to detect the appearance of an object within a short period of time and to rapidly identify the type of the detected object.

However, as described above, if you want to classify an object using machine learning, the accuracy of object recognition and classification in the process of reducing the size of the image to match the size that can be input to the machine learning model (or machine learning inference model) A problem may occur in that the inference time is decreased and the inference time is also increased (ie, the inference speed is slowed down).

Therefore, the object recognition apparatus according to the present embodiment provides a method of improving the accuracy of object recognition and classification, and also reducing inference time (ie, inference speed is increased).

4 is a schematic diagram of a method of increasing the size of an object to be input to a machine learning model (or machine learning inference model) in order to improve object recognition and classification accuracy and inference speed according to an embodiment of the present invention. , and FIG. 5 is an exemplary diagram showing a more specific configuration of the object inference module in FIG. 1 .

Referring to FIG. 4, a background line corresponding to the boundary formed by buildings and trees on the ground is formed on the lower side of the image, and an object of a very small size compared to the size of the image is formed in the sky space on the upper side of the image (e.g. : Unmanned aerial vehicle) is filmed.

Therefore, as described above, when the entire original image is reduced and input to the machine learning inference model 205, the size of an object (eg, a drone) becomes smaller enough to make it difficult to classify the type of the object. A problem occurs in which the accuracy of object recognition and classification is further deteriorated.

Therefore, in this embodiment, the object inference module 102 does not reduce the size of the entire image, but inputs the image 305 extracted from only a part of the region including the object to a machine learning model (or machine learning inference model), As a result, through the object classification processor 102a and the object prediction processor 102b (see FIG. 5 ), the image input to the machine learning model (or machine learning inference model) (that is, the image extracted from only a partial region including the object) ) by increasing the size of the object relative to the size of the object, the accuracy of object recognition and classification relative to the method of reducing the size of the entire image to the size to be input to the machine learning model (or machine learning inference model) Offers ways to improve.

Meanwhile, the object classification processor 102a and the object prediction processor 102b may be integrated into one processor and included in the object inference module 102 .

FIG. 6 is an exemplary diagram for explaining a method of extracting an object-containing region from an original image by an object inference module in FIG. 1 .

Referring to FIG. 6 , the object inference module 102 divides the original image 201 in the vertical direction into a designated number (N) of regions. Hereinafter, a region divided by the designated number (N) in the vertical direction is referred to as a divided region 301 .

The object inference module 102 searches for objects 103 existing in each segmented area 301 of the original image 201, and then selects an object-containing area 305 including one or more objects 103. After calculation, the image information of the object-containing region 305 is input to the machine learning inference model 205. That is, the object inference module 102 according to the present embodiment calculates the object-containing region 305 in which the objects searched for in the original image 201 are clustered, and the original image 201 (ie, the entire The image information of the object-containing region 305, not the image), is input to the machine learning inference model 205.

Accordingly, the machine learning inference model 205 receives the image information of the object-containing region 305 as an input and calculates a final classification result.

Meanwhile, in the original image 201 shown in FIG. 6, the background line 304 represents the actual boundary between ground objects such as buildings or trees and the sky space in the entire image, and the boundary line 302 represents the background line 304 It means a line formed by adding an offset 303 of a predetermined interval from to the upper side (ie, sky space).

At this time, the reason for setting the boundary line 302 is to exclude the area below the boundary line 302 from the object search range.

In other words, the reason for placing the boundary line 302 is to compensate for errors that may occur in estimating the background line 304. This is to reduce errors in judgment (e.g., the possibility of mistakenly judging a tree branch extending into the sky space as an object).

Hereinafter, a method of detecting the background line 304 and the boundary line 302 will be described.

FIG. 7 is an exemplary diagram shown in FIG. 6 to explain the principle of detecting a background line in an original image.

Referring to FIG. 7 , in order for the object inference module 102 to detect a background line 304 in the original image 201, the reduced image 203 according to a specified ratio (RATIO) of the original image 201 ) is reduced to

The reason for reducing the original image 201 to the reduced image 203 as described above is to reduce the load of calculation (ie, calculation for detecting the background line).

That is, in order to detect the background line 304, calculation in units of pixels must be performed. After detecting the background line 304 using the reduced image 203, the object position of the reduced image 203 is used again. This is to reduce the calculation load in the process of restoring (or reducing) the position of the object in the original image 201.

For example, in order for the object inference module 102 to detect the background line 304 in the original image 201, the original image 201 (eg, vertical × horizontal pixels = H × W) is selected at a specified ratio (RATIO = h). It is assumed that a reduced image 203 (eg, vertical × horizontal pixels = h × w) is generated by scaling down according to /H = w / W. At this time, when the position (ie, coordinates) of any one object in the reduced image 203 is (cx, cy), the position (ie, coordinates) of the object in the reduced image 203 is the original image ( 201), the position of the object in the original image 201 becomes (cx/RATIO, cy/RATIO).

FIG. 8 is an exemplary diagram shown in FIG. 7 to explain an image binary-coding method as a step prior to detecting a background line in an original image.

Referring to FIG. 8 , the object inference module 102 converts a color image 500 into a gray image 501 having the same R, G, and B values.

Thereafter, the object inference module 102 generates a reduced image 502 by reducing the gray image 501 according to a specified ratio (eg, reduced RATIO), and then generates a binary image 503 from the reduced image 502. ) to create

Here, the binarized image 503 is an image in which pixel values are expressed as 0 or 1, and can be calculated by Equation 1 below.

That is, referring to Equation 1, when each pixel value of the reduced image 502 is equal to or greater than the designated threshold value (THRESHOLD), it is expressed as 1, and when each pixel value is less than the designated threshold value (THRESHOLD), it is expressed as 0. is expressed as At this time, the threshold value THRESHOLD may be set in plurality.

FIG. 9 is an exemplary diagram shown to explain a method of calculating a background line from a reduced binarized image in FIG. 8 .

The object inference module 102 calculates a first background line by applying a first threshold value (THRESHOLD_1) (600) and calculates a second background line by applying a second threshold value (THRESHOLD_2) (601), In the same way, the Nth background line is calculated by applying the Nth threshold (THRESHOLD_N) 602 (see Equation 2).

The reason for calculating a plurality of background lines as described above is to detect the most optimal background line among the background lines (see Equation 3).

)는 각기 다르게 적용되며, 상기 가중치(

)들의 합은 1이다(수학식 3 참조).For reference, the weight applied to calculate the N background lines (

) is applied differently, and the weight (

) is 1 (see Equation 3).

That is, referring to the vertical pixel enlarged image shown in FIG. 9 , the position of the background line is calculated by adding the values of the corresponding vertical pixels, which is expressed as Equation 2.

For example, when L _i is concatenated in Equation 2, a background line at the first threshold value THRESHOLD_1, a background line at the second threshold value THRESHOLD_2, and a background line at the Nth threshold value THRESHOLD_N are calculated. , The final background line (that is, the optimal background line) among the plurality of background lines is calculated by weighting the average of the values from the first threshold to the Nth threshold (THRESHOLD_1 to THRESHOLD_N), which is expressed by Equation 3 below is expressed as

That is, in FIG. 9, the final background line (304 in FIG. 6) for the horizontal pixel i becomes B _i as shown in Equation 3 below.

Equation 4 below represents C _i , which is the final boundary line 302 having a gap as much as OFFSET from the background line 304 .

FIG. 10 is an exemplary diagram shown to explain the method of detecting an object in a reduced binarized image in FIG. 8 .

Referring to FIG. 10 , the object inference module 102 applies an object line calculation threshold (THRESHOLD_O) designated for object detection in the reduced binarized image 503 to Equation 5 below to obtain an object line 703. ) is detected (calculated).

That is, Equation 5 below is the first vertical pixel index (OBJ _i,O ) at which the vertical pixel value for the horizontal pixel i is the minimum in the reduced binarized image 503 to which the object line calculation threshold (THRESHOLD_O) is applied. is to find Accordingly, as shown in FIG. 10, the object line 703 is a line that touches the boundary line 302 where there is no object (position), and is perpendicular to the boundary line 302 where there is an object (position). A straight line is created.

As a result, as shown in FIG. 10, with respect to all horizontal pixels, the place (position) where the object is located protrudes (the height of the protrusion varies depending on the location of the object), and the place where there is no object (position) has a boundary line 302. An object line 703 in the form of following is created.

FIG. 11 is an exemplary diagram shown to explain the method of detecting an object using a segmented region in a reduced binarized image in FIG. 8 .

Referring to FIG. 11 , the object inference module 102 detects at least one object 103 in each divided area 301 divided into a plurality of parts.

At this time, as the number of the divided regions 301 increases, the interval between one divided region 301 narrows, so that the possibility of two or more objects 103 existing in one divided region 301 gradually decreases.

Accordingly, by independently (or in parallel) detecting objects in the plurality of partition regions 301, the overall inference time can be shortened.

Hereinafter, detection of one object 103 in each partition area 301 will be described in more detail.

For example, as one example of a condition for a point of the object line 703 to become the object 103 within each segmentation area 301, the increase of the object line 703 is greater than the designated object detection threshold value (THRESHOLD_OBJECT). case can be cited.

For example, assuming that the object line 703 is a function, as shown in Equation 6 below, a y value satisfying the case of “dy / dx > THRESHOLD_OBJECT”, that is, a point having a vertical pixel position value (x, y) becomes the location where the object 103 exists. That is, the point where the gradient increases the most becomes the point where the object is located.

However, the illustrative description above is not intended to limit the method of detecting an object in the segmented area 301, and may be applied to detect an object in the entire image even if the area is not segmented.

On the other hand, when the positions of objects are detected in the reduced binarized image 503 as described above, the object-containing regions 305 are clustered (ie, the objects are grouped according to the size that can be input to the machine learning model (or machine learning inference model)). are combined to be included in the object-containing region), and the clustered object-containing regions 305 are restored (reduced) to the original image 201.

In this case, it is preferable to reduce the object inference time by minimizing the number of object-containing regions to be calculated (or created).

For example, referring to FIG. 12 , when object-containing regions 305 each including one object are created according to the clustering method, up to four object-containing regions 305 are created, each containing two objects. When the object-containing area 305 is created, two object-containing areas 305 are created.

If the object containing area 305 including all four objects is created, the size that can be input to the machine learning model (or machine learning inference model) is exceeded, so in FIG. ) to cluster 906 the objects.

Hereinafter, a method of clustering the object-containing regions will be described.

FIG. 12 is an exemplary diagram shown to explain a method of clustering object-containing regions in a reduced binarized image in FIG. 11 .

Referring to FIG. 12, when the total number of objects is F, in a combination using a designated clustering function ( _F C _f ) (ie, f = F, F-1, ..., 1), the object containing area ( 305), clustering is performed so that the number of objects included is maximized. That is, starting from f = F, clustering is performed while excluding clustered objects.

The above process is performed first from clustering that includes a large number of objects.

Meanwhile, the object-containing region ((cx, cy), (cx + w, cy + h)) 305 in the reduced binarized image 503 shown in FIG. 12 is reduced as shown in Equation 7 below. By applying a specified ratio (RATIO) to the object-containing area in the binarized image 503, the object-containing area in the original image 201 (((cx, cy), (cx + w, cy + h)) × RATIO).

FIG. 13 is an exemplary diagram for explaining the process of calculating an object-containing region from an original image and inputting the object-containing region to a machine learning inference model in FIG. 6 .

As described with reference to FIG. 12, in the original image 201 using the object-containing region ((cx, cy), (cx + w, cy + h)) detected in the reduced binarized image 503 When the object-containing area (((cx, cy), (cx + w, cy + h)) × RATIO) is calculated (extracted), the object-containing area ((( cx, cy), (cx+w, cy+h))×RATIO) may be input to the machine learning inference model 205 to calculate a final result.

As described above, in this embodiment, rather than simply downscaling the entire original image 201 and inputting it to the machine learning inference model 205, one or more object-containing regions 305 are calculated in the original image 201 and machine learning Input to the inference model 205 has an effect of improving inference accuracy and inference speed for object recognition and classification.

In addition, this embodiment can be directly applied as an input method of a machine learning inference model for detecting an object (e.g., UAV) appearing in the sky space and confirming its type, especially in high-resolution (e.g., 4K, FHD) images. It has excellent inference performance and has an effect that can be directly applied to a system that tracks the physical location of an inferred object.

The present invention has been described above with reference to the embodiments shown in the drawings, but this is only exemplary, and those skilled in the art can make various modifications and equivalent other embodiments. you will understand the point. Therefore, the technical protection scope of the present invention should be determined by the claims below. Implementations described herein may also be embodied in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (eg, discussed only as a method), the implementation of features discussed may also be implemented in other forms (eg, an apparatus or program). The device may be implemented in suitable hardware, software and firmware. The method may be implemented in an apparatus such as a processor, which is generally referred to as a processing device including, for example, a computer, microprocessor, integrated circuit, programmable logic device, or the like. Processors also include communication devices such as computers, cell phones, personal digital assistants ("PDAs") and other devices that facilitate communication of information between end-users.

101: camera module 102: object inference module
102a: object classification processor 102b: object prediction processor
103: object or object image 104: image output module
105: bounding box 205: machine learning inference model

Claims (20)

An object inference model for processing an original image captured by a camera module and generating an image of a size to be input to the machine learning inference model;
The object inference model,
It includes the machine learning inference model, and outputs a result of recognition and classification of an object inferred through the machine learning inference model,
The machine learning inference model,
An object recognition device characterized in that it infers an object included in the input image by processing the input image.
The method of claim 1, wherein the object inference module,
As an image to be input to the machine learning inference model,
The object recognition device characterized in that for extracting images of object-containing regions clustered in the original image according to a size to be input to the machine learning inference model.
The method of claim 2, wherein the object inference module,
In order to extract the object-containing region,
A background line is calculated from the binarized image in which the original image is reduced, and a boundary line is calculated by adding an offset to a sky space above the background line;
An object recognition device characterized in that for extracting an object-containing area only from the sky space excluding areas below based on the boundary line.
The method of claim 3, wherein the object inference module,
Converting the color image, which is the original image, into a gray image having the same R, G, and B values, and generating a reduced binary image by downscaling the gray image;
The binarized image,
If the pixel value is greater than or equal to the specified threshold value, it is expressed as 1.
An object recognition device characterized in that the image is expressed as 0 when the pixel value is smaller than a designated threshold.
The method of claim 3, wherein the object inference module,
N background lines are calculated as in Equation 2 below by applying different threshold values, and each of the N background lines has a different weight (

) is applied and then a weighted average is performed to calculate the final background line (B _i ) as shown in Equation 3 below.
(Equation 2)

(Equation 3)

The method of claim 3, wherein the object inference module,
Calculating an object line to detect an object in the reduced binary image,
In order to calculate the object line, the first vertical pixel index (OBJ) at which the vertical pixel value for the horizontal pixel i of the binarized image is minimized by applying the designated object line calculation threshold value (THRESHOLD_O) to Equation 5 below. _{i, O} ) object recognition device characterized in that for detecting the object line by repeatedly performing a process of finding.
(Equation 5)

The method of claim 6, wherein the object inference module,
Independently detecting objects in a plurality of divided regions obtained by dividing the reduced binary image in the vertical direction,
The object recognition device, characterized in that for detecting, as the position of the object, a point where the inclination of the object line increases the most beyond an object detection threshold (THRESHOLD_OBJECT).
The method of claim 7, wherein the object inference module,
When the positions of objects are detected in the reduced binary image,
An object recognition device characterized by clustering object-containing regions according to a size that can be input to a machine learning inference model, but clustering in a minimum number that can be combined in a corresponding image.
The method of claim 8, wherein the object inference module,
The apparatus for recognizing an object, characterized in that by applying a specified ratio (RATIO) to the clustered object-containing area in the reduced binarized image, mapping to an object-containing area in the original image. object inference module receiving an original image;
extracting, by the object inference module, an area including at least one object according to a size to be input to a machine learning inference model through processing of the original image; and
and performing, by the object inference module, recognition and classification of an object included in the at least one object-containing region by the machine learning inference model and outputting the result.
11. The method of claim 10, wherein extracting at least one object-containing region according to a size to be input to the machine learning inference model comprises:
The object inference module,
converting the original image to generate a reduced binary image;
clustering a region including at least one object corresponding to an object detected in the reduced binary image; and
and extracting the object-containing region to be input to the machine learning inference model by mapping the object-containing region clustered in the reduced binarized image to the object-containing region in the original image.
The method of claim 11, in order to extract at least one object-containing region from the reduced binarized image,
The object inference module,
Calculating a background line in the reduced binary image,
Calculate a boundary line by adding an offset to the sky space above the background line;
An object recognition method characterized by extracting an object-containing region only from the sky space excluding the region below based on the boundary line.
The method of claim 11, in order to extract at least one object-containing region from the reduced binarized image,
The object inference module,
Dividing the reduced binarized image into a plurality of partitioned regions in a vertical direction;
The object recognition method, characterized in that for reducing the overall inference time by independently detecting the object in the plurality of partitioned areas.
generating a reduced image by reducing the original image by an object inference module;
generating, by the object inference module, a reduced binarized image by converting the reduced image into a binarized image;
the object inference module detecting an object in the reduced binarized image and clustering object-containing regions according to a size that can be input to a machine learning inference model;
mapping, by the object inference module, an object-containing region clustered in the reduced binarized image to an object-containing region in the original image; and
and inferring an object by inputting object-containing regions mapped to the original image to a machine learning inference model.
The method of claim 14, in order to extract the object-containing region,
The object inference module,
After calculating a background line from the reduced binary image, calculating a boundary line by adding an offset to the sky space above the background line;
An object recognition method characterized by extracting an object-containing region only from the sky space excluding the region below based on the boundary line.
The method of claim 13, wherein in the generating of the reduced binarized image,
The object inference module,
After converting the color image, which is the original image, into a gray image having the same R, G, and B values, and reducing the gray image to generate a reduced image,
An object recognition method characterized by generating a binarized image by expressing 1 when a pixel value is greater than or equal to a specified threshold value and expressing it as 0 when the pixel value is smaller than a specified threshold value.
16. The method of claim 15, in order to calculate the background line,
The object inference module
N background lines are calculated as in Equation 2 below by applying different threshold values, and each of the N background lines has a different weight (

), and then performing a weighted average to calculate the final background line (B _i ) as shown in Equation 3 below.
(Equation 2)

(Equation 3)

The method of claim 14, in order to detect an object in the reduced binarized image,
The object inference module,
Calculating an object line to detect an object in the reduced binary image,
The object recognition method characterized by detecting, as the position of the object, a point where the slope of the object line increases the most beyond a threshold for object detection (THRESHOLD_OBJECT).
The method of claim 18, in order to calculate the object line,
The object inference module,
By applying the designated object line calculation threshold (THRESHOLD_O) to Equation 5 below, the first vertical pixel index (OBJ _i,O ) at which the vertical pixel value of the horizontal pixel i of the reduced binary image is minimum is obtained. An object recognition method characterized by detecting the object line by repeatedly performing a finding process.
(Equation 5)

15. The method of claim 14, wherein in the step of mapping an object-containing region clustered in the reduced binarized image to an object-containing region in the original image,
The object inference module,
The object-containing area (((cx, cy ), (cx + w, cy + h)) × RATIO).

Images