KR20200021398A - Image processing apparatus and method - Google Patents

Abstract

Provided are an image processing method and apparatus for determining whether an object corresponds to a search target object. The image processing method of the present disclosure comprises: a step of receiving an analysis target image including at least one object; a step of acquiring numerical information regarding whether the at least one object included in the analysis target image corresponds to a detection target object by using an artificial intelligence model; a primary detection step of determining the at least one object as the detection target object when the acquired numerical information is greater than or equal to a first threshold value; and a step of outputting the determination result, wherein sensitivity in the primary detection step is adjusted by adjusting a parameter in the artificial intelligence model or the first threshold value related to the acquisition of the numerical information.

Description

Image processing apparatus and method {IMAGE PROCESSING APPARATUS AND METHOD}

The present disclosure relates to an image processing apparatus and a method for detecting an object. More specifically, the present disclosure provides an image processing apparatus, a method for determining whether an object included in an analysis target image corresponds to a detection target object using an artificial intelligence model, and a computer for recording a program for executing the image processing method of the present disclosure. A readable recording medium.

Object recognition is a process of recognizing an area recognized as an object in an image as one of a plurality of preset classes. An object may mean a specific object in an image.

Recently, researches for recognizing or classifying objects included in images using artificial intelligence models are increasing.

On the other hand, deep learning is an artificial intelligence model that learns a very large amount of data and selects the highest answer probability based on the learning result when new data is input. It can automatically find feature factors in the course of training the model based on the data.

An object of the present disclosure is to provide an image processing apparatus and method for determining whether an object included in an image corresponds to a detection target object.

Another technical problem of the present disclosure is to provide an image processing apparatus and method for determining whether a detection object corresponds to a detection object by adjusting a sensitivity of object detection based on an artificial intelligence model.

Technical problems to be achieved in the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

According to an aspect of the present disclosure, a method of receiving an analysis target image including one or more objects; Obtaining numerical information about whether the one or more objects included in the analysis target image correspond to the detection target object using an artificial intelligence model; A first detection step of determining the at least one object as the detection target object when the obtained numerical information is equal to or greater than a first threshold value; And outputting the determined result, and adjusting the sensitivity of the first detection step by adjusting a parameter or the first threshold value in the artificial intelligence model related to the acquisition of the numerical information. An image processing method may be provided.

According to another aspect of the present disclosure, an image receiving unit receiving an analysis target image including one or more objects; Acquiring numerical information about whether the one or more objects included in the analysis target image corresponds to a detection object using an artificial intelligence model, and when the acquired numerical information is equal to or greater than a first threshold value, the one or more objects are selected. An object detector configured to perform a first detection step of determining the detection target object; And an output unit for outputting the determined result, wherein the sensitivity of the first detection step is adjusted by adjusting a parameter in the artificial intelligence model related to the acquisition of the numerical information or the first threshold value. An image processing apparatus may be provided.

According to another aspect of the present disclosure, a computer-readable recording medium having recorded a program for executing the image processing method of the present disclosure may be provided.

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, an image processing apparatus and method for determining whether an object included in an image corresponds to a detection object may be provided.

According to the present disclosure, an image processing apparatus and method for determining whether a detection object corresponds to a detection target object by adjusting the sensitivity of object detection based on an artificial intelligence model may be provided.

Effects obtained in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment.
2 is a diagram illustrating a process of determining whether an object included in an image corresponds to a detection object according to an exemplary embodiment.
3 is a diagram illustrating a process of generating and analyzing context information of an image according to an exemplary embodiment.
4 is a diagram illustrating a process of detecting an object by analyzing an image according to an embodiment of the present disclosure.
5 is a diagram illustrating an image in which colors are expressed based on physical properties of an object according to an exemplary embodiment.
6 is a diagram illustrating an image processing method according to an exemplary embodiment.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present disclosure. However, the present disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

In describing the embodiments of the present disclosure, when it is determined that a detailed description of a known structure or function may obscure the gist of the present disclosure, a detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals designate like parts.

In the present disclosure, when a component is "connected", "coupled" or "connected" with another component, it is not only a direct connection, but also an indirect connection in which another component exists in the middle of the connection. It may also include. In addition, when a component "includes" or "having" another component, it means that it may further include another component, without excluding the other component unless otherwise stated. .

In the present disclosure, the terms "first" and "second" are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance between the components unless specifically mentioned. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and likewise, a second component in one embodiment may be referred to as a first component in another embodiment. It may also be called.

In the present disclosure, the components distinguished from each other are for clearly describing each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated into one hardware or software unit, or one component may be distributed into a plurality of hardware or software units. Therefore, even if not mentioned otherwise, such integrated or distributed embodiments are included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments are not necessarily required components, and some may be optional components. Therefore, an embodiment composed of a subset of components described in an embodiment is also included in the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment.

Referring to FIG. 1, the image processing apparatus 100 may include an image receiver 110, an object detector 120, an object analyzer 130, and / or an output unit (not shown). However, this only illustrates some components necessary to describe the present embodiment, and the components included in the image processing apparatus 100 are not limited to the above-described example. For example, two or more components may be implemented in one component, or an operation executed in one component may be divided and implemented to be executed in two or more components. In addition, some components may be omitted or additional components may be added.

The image processing apparatus 100 according to an embodiment receives the analysis target image 140, detects one or more objects included in the received analysis target image 140 using an artificial intelligence model, and detects the detected object. It may be determined whether the corresponds to the detection target object.

The image receiver 110 may receive the analysis target image 140. The analysis target image 140 may include at least one object. For example, the analysis target image 140 may be an image regarding an article including at least one object. Also, for example, the analysis target image 140 may be an X-ray image of an article photographed by the X-ray reader. The image may be a raw image captured by an X-ray imaging device or an image of any form (format) for storing or transmitting the raw image. The image may be obtained by capturing and data-forming image information that is taken by an X-ray reading device and transmitted to an output device such as a monitor.

The object detector 120 may detect an object included in the analysis target image received by the image receiver 110. For example, the object detector 120 may detect an object included in the analysis target image by using artificial intelligence technology. In detail, the artificial intelligence model may be a deep learning model for object detection.

AI technology allows computers to learn data and make decisions as if they were humans. Artificial neural networks are mathematical models inspired by biological neural networks. Neurons can refer to models that have problem-solving abilities by changing the strength of synapses through learning. Artificial neural networks generally consist of an input layer, a hidden layer, and an output layer, and neurons included in each layer are connected by weight, and a linear combination of the weight and neuron values and a nonlinear Through the activation function, the artificial neural network can be shaped to approximate complex functions. The purpose of artificial neural network learning is to find weights that minimize the difference between the output computed at the output layer and the actual output.

Deep neural networks are artificial neural networks made up of multiple hidden layers between the input and output layers, and many hidden layers can model complex nonlinear relationships. The structure is called deep learning. Deep learning learns a very large amount of data, and when new data is input, it can select adaptively the highest answer based on the learning results, so it can operate adaptively according to the image. As you learn, you can automatically find the feature factors.

Deep learning-based models of the present disclosure can be used for a full convolutional neural network (fully convolutional neural network), a convolutional neural network (convolutional neural network), a cyclic neural network (recurrent neural network). ), But is not limited to at least one of a restricted Boltzmann machine (RBM) and a deep belief neural network (DBN). Alternatively, the method may include machine learning methods other than deep learning. Or it could include a hybrid model that combines deep learning and machine learning. For example, a feature of an image may be extracted by applying a deep learning based model, and a machine learning based model may be applied when classifying or recognizing an image based on the extracted feature. The machine learning based model may include a support vector machine (SVM), AdaBoost, and the like, but is not limited thereto.

In addition, the method of learning a deep learning-based model of the present disclosure may include at least one of supervised learning, unsupervised learning, or reinforcement learning, but is not limited thereto. Supervised learning is trained using a series of training data and a corresponding label (target output value). A neural network model based on supervised learning is a model that infers a function from training data. Can be. Supervised learning receives a series of training data and corresponding target output values, and learns to find errors by modifying the model based on the results by learning to compare the actual output values with the target output values. do. Supervised learning can be further divided into regression, classification, detection, semantic segmentation, etc., depending on the type of the result. Functions derived from supervised learning can then be used to predict new outcomes. As such, the neural network model based on supervised learning optimizes the parameters of the neural network model through learning a large number of training data.

The object detector 120 may acquire numerical information regarding whether one or more objects included in the analysis target image correspond to the detection target object by using the artificial intelligence model. The object to be detected may be set in advance, and the artificial intelligence model may be pre-learned with respect to the object to be detected. Also, the object to be detected may vary depending on the field of application. For example, the present invention may be applied to a field for processing and analyzing various kinds of images, such as a field for determining whether a product is prohibited from entering, a field for determining whether a product is infringing on a company security, or a medical field. In detail, the object to be detected may include a storage medium such as a USB, a notebook, a hard drive, a knife, a lighter, a gun, and the like. Accordingly, the image processing apparatus and / or image processing method of the present disclosure may detect a specific object by analyzing an input image and may be applied to various fields such as an enterprise security system.

The numerical information may mean a probability, score, or distribution indicating whether the detected object corresponds to the object to be detected when the object is detected by applying an artificial intelligence model to an image including one or more objects. . For example, if the detection target object is a USB, a knife, a lighter, the numerical information may include a probability that the object included in the analysis image corresponds to a USB, a probability corresponding to a knife, and / or a probability corresponding to a lighter.

The numerical information may provide a reliability or a criterion for determining which detection target object the object detected using the artificial intelligence model corresponds to. For example, as a result of applying the artificial intelligence model, the object detector 120 may determine that the detection object corresponds to the detection object in order of the detection object having the high numerical information about the object. In addition, the object detector 120 may determine the at least one object as the detection target object when the obtained numerical information is equal to or greater than a first threshold value. The first threshold value may vary depending on the object to be detected. The first threshold may be preset or input by a user. By using the first threshold value, the object determining apparatus of the present disclosure can secure more reliability. Also, for example, the first threshold may be a parameter related to object detection.

For example, when the numerical information includes the probability that the object included in the analysis target image corresponds to the USB, the probability corresponding to the knife, and / or the probability corresponding to the lighter, the probability that the detected object corresponds to the USB is the first threshold value. If abnormal, the detected object may be determined to correspond to USB. Further, for example, when the probability that the detection object corresponds to the USB and the probability that the knife corresponds in the numerical information is greater than or equal to the first threshold value and the probability that the detection object corresponds to the USB is greater than the probability that the knife corresponds, respectively, the detected object May be determined to correspond to USB first.

Also, for example, when there are a plurality of detection target objects, at least some of the detection target objects may have different thresholds than the other detection target objects. In this case, however, as the number of objects to be detected increases, the number of thresholds increases accordingly. In addition, the threshold value applied to at least some of the detection target objects may be different from the threshold value applied to the other detection target objects. As such, when the number of thresholds increases or the difference between the thresholds is large, it may be difficult to adjust or manage the value of each of the thresholds. Therefore, when there are a plurality of objects to be detected, for example, one representative threshold value may be set, and the sensitivity coefficients related to object detection may be set by the number of objects to be detected, respectively. The final threshold for each object to be detected may be set by multiplying the set threshold by the sensitivity coefficient. The final threshold may be a first threshold for each object to be detected. As a result, the image processing apparatus and / or the image processing method of the present disclosure may individually adjust the sensitivity of object detection with respect to multiple objects, that is, a plurality of detection target objects. As another embodiment, when acquiring the numerical information on whether one or more objects included in the analysis target image correspond to the detection target object, an internal parameter (eg, coefficient) related to sensitivity may be differently set for each detection target object. . For example, an internal parameter can be set to 2 for USB and an internal parameter can be set to 1 for a knife. In addition, by applying the internal parameter to the actually obtained numerical information, the obtained numerical information may be corrected (eg, by multiplying). In this way, even if the same numerical information is obtained, the higher correction numerical information can be finally obtained for the USB than the knife, so that even if the same threshold value is used for the USB and the knife, the detection sensitivity for the USB can be increased.

In addition, the object detector 120 may output a determination result regarding whether the detected object corresponds to a detection target object.

In addition, the object detector 120 may adjust the first threshold value. The adjustment may be preset or entered by the user. For example, by adjusting the first threshold value, it is possible to adjust the possibility that the object included in the image corresponds to the detection target object, that is, the sensitivity of the object detection.

Alternatively, the object detector 120 may adjust the sensitivity of the object detection by adjusting a parameter inside the artificial intelligence model related to the acquisition of the numerical information. By adjusting parameters inside the artificial intelligence model, a plurality of different numerical information may be obtained with respect to one input image. The adjustment may be preset or entered by the user. For example, the parameter inside the artificial intelligence model may be a parameter inside filter kernels of the convolutional layer and / or the pooling layer in the convolutional neural network.

As described above, it is possible to adjust the degree of determination (sensitivity) as to whether the detected object corresponds to the object to be detected by adjusting numerical information by adjusting parameters inside the artificial intelligence model or adjusting the first threshold value. As a result, the image processing apparatus of the present disclosure may adjust a parameter inside the artificial intelligence model or adjust the first threshold in order to adjust the detection sensitivity in the object detection process.

On the other hand, in the medical imaging field, doctors generally diagnose patients based on various features shown in X-ray medical images. For example, the patient may be diagnosed as a lung lesion based on various features of internal organs shown in the lung image. If you can clearly distinguish the areas on the medical image, you will be able to make a more accurate diagnosis. Therefore, it is necessary to distinguish and detect regions in the medical image. In other words, medical images do not generally include separate objects such as USB and knife, but internal organs, tissues, and organs are mixed, so it is necessary to clearly distinguish them. For example, in the case of lung imaging, it is necessary to more clearly detect organs such as lungs, air, pleura, and diaphragm. Alternatively, lung images may include objects in various images that can be determined as consolidation, interstitial, nodule, mass, pulmonary dilatation, and the like. Accordingly, the sensitivity of object detection in the lung image may be adjusted.

In the above, the lung image is taken as an example, but is not limited thereto. That is, it is also possible to adjust the sensitivity of object detection in the medical image by applying the invention of the present disclosure by inputting various medical images for diagnosis or detection of lesions based on the image. Furthermore, the invention of the present disclosure may be applied to any field in which a plurality of objects are to be detected from an input image, regardless of the field and use of the image.

The image processing apparatus and / or the image processing method of the present disclosure may acquire numerical information regarding whether one or more regions included in the medical image correspond to a detection target region by using an artificial intelligence model. The detection target region may be set in advance, and the artificial intelligence model may be trained on the detection target region in advance. Also, the image processing apparatus and / or the image processing method of the present disclosure may determine whether the detected region corresponds to the detection target region by adjusting numerical information by adjusting a parameter inside the artificial intelligence model or by adjusting a first threshold value. You can adjust the degree of sensitivity. The detection target area may vary depending on the type of medical image. The detection subject area may be an area related to determining whether a lesion has occurred. For example, in the case of a lung image, the detection target area may include organs such as lung, air, pleura, and diaphragm.

The operation of the object detector may be applied even when the input image is a medical image. The object analyzer 130 may secondarily determine whether the corresponding object corresponds to the detection target object with respect to one or more objects determined by the object detection unit 120 as the detection target object.

For example, the output result when the first threshold is relatively large may be determined to be a more reliable detection result than the output result when the first threshold is smaller. Therefore, whether the detected object when the first threshold value is relatively small corresponds to the object to be detected or is detected because of a form similarity to the object to be detected or does not correspond to the object to be detected. We need to make a second judgment about that. That is, it may be required to determine whether or not overdetect the output result.

For example, the secondary detection step may be performed only on an object having one or more objects determined as the detection target object by the object detection unit 120 and having numerical information less than or equal to a second threshold. The second threshold may vary depending on the object to be detected. The second threshold may be preset or input by the user.

The object analyzer 130 may use an image matching technique and / or an image recognition technique. The image matching technique may be a method of determining similarity between images by extracting feature points between the input image and the reference image. For example, when an image of the object to be detected is secured in advance, such as when there is an X-ray image of an item forbidden to enter the company security, the previously obtained image of the object to be detected may be used as a reference image. Meanwhile, the image recognition technique may use various methods generally used in the field of recognition or classification.

The object analyzer 130 may apply an application technique differently according to the presence or absence of a reference image. For example, if there is a reference image, an image matching technique or an image recognition technique may be applied. On the other hand, in the absence of a reference picture, it may be difficult to apply an image matching technique because feature points between the input picture and the reference picture cannot be extracted and compared. In addition, when there is no reference image, in order to apply an image recognition technique, a trained artificial intelligence model capable of recognizing and classifying an input image may be required. For example, the AI model may be a different model from the model used in the object detector 120.

If the result of the secondary determination by the object analyzer 130 is different from the result determined by the object detector 120, the image processing apparatus of the present disclosure may determine whether the detected object corresponds to another detected object based on the numerical information. You can judge again.

For example, when a detection object has a probability corresponding to USB, a probability corresponding to a knife, and / or a probability corresponding to a writer in numerical information, and has a probability value in the order described above, the object detection unit 120 may detect the detected object as USB. It can be determined primarily that In addition, the object analyzer 130 may secondarily determine whether the detected object corresponds to the USB. As a result of the secondary determination by the object analyzer 130, when it is determined that the detected object does not correspond to the USB, the object analyzer 130 may output the secondary determination result as a final result, Based on this, it may be further determined whether the detection object corresponds to a knife having the next highest probability value after the USB (that is, having the highest probability value except USB). By performing the above additional determination, it may be determined more accurately whether the detection object corresponds to the detection object.

In addition, the operation of the object analyzer may be applied when the input image is a medical image. In this case, the detection target object may mean a detection target region. For example, the image processing apparatus and / or the image processing method of the present disclosure may secondly determine whether the corresponding area corresponds to the detection target area with respect to one or more areas determined as the detection target area.

2 is a diagram illustrating a process of determining whether an object included in an image corresponds to a detection object according to an exemplary embodiment.

The input image 210 may be an X-ray image obtained by passing an arbitrary object through the X-ray search apparatus. For example, the input image 210 may be an image including at least one or more detection target objects. For example, the object to be detected may include a prohibited item for corporate security, such as a knife, a lighter, a USB, a gun, or the like.

The image processing apparatus of the present disclosure may perform object detection on an input image 210 using an artificial intelligence model. Also, the image processing apparatus of the present disclosure may determine whether the detected object corresponds to a detection object. The image processing apparatus of the present disclosure may determine the detection object as the detection target object when the numerical information of the object detected in the input image is equal to or greater than a first threshold value. Also, the image processing apparatus of the present disclosure may output an object determined as the detection target object.

For example, the image processing apparatus of the present disclosure may display an object region determined as the detection target object on an input image, and generate a resultant image. The first result image 220, the second result image 230, and the third result image 240 may be images in which object areas having a first threshold value or more are displayed on the input image 210 in a rectangular box. . Here, the first threshold values applied to the first result image 220, the second result image 230, and the third result image 240 may be different. For example, the magnitude of the first threshold value may be in order of the first result image 220, the second result image 230, and the third result image 240. Referring to FIG. 2, the first result image 220 may display a first area 222 detected as a knife. In addition, the second result image 230 may additionally display a second area 224 detected to correspond to the USB in addition to the first area 222. In addition, in addition to the first area 222 and the second area 224, the third result image 240 may further display a third area 226 detected as being corresponding to the USB.

In addition, the image processing apparatus of the present disclosure may first determine the position of the detection object in order to display the object area on the image. In detail, the image processing apparatus of the present disclosure may specify a bounding box surrounding the object area and generate location information of the detected object based on the specified rectangular box. The location information of the object may be location information of four vertices forming a rectangular box, but is not limited thereto. For example, the location information may be represented by the coordinates (x, y) of one vertex of the rectangle box and the width and height of the rectangle box. The coordinate (x, y) of the one vertex may be the coordinate of the upper left corner of the rectangular box 262. The coordinate (x, y) of the vertex may be specified based on the coordinate (0, 0) of the upper left corner of the input image.

As described above, the numerical information may be a probability indicating whether the detection object corresponds to the detection target object. When the performance of the artificial intelligence model is guaranteed, it can be seen that the object determined as the detection target object in the image processing apparatus of the present disclosure is correctly determined as the first threshold value increases. That is, in the case of an object detected when the first threshold is relatively small in terms of reliability regarding object detection, it is necessary to secondarily determine whether the object corresponds to the object to be detected.

For example, referring to FIG. 2, the first region 222, the second region 224, and the third region displayed on the third result image 240, which is the result image when the first threshold value is the smallest. 226 needs to be secondarily determined whether the object corresponds to the object to be detected in view of the reliability regarding object detection. However, the first area 222 and the second area 224 are areas displayed on the first result image 220 and / or the second result image 230, and the third area 226 is the first result image ( 220 or the second result image 230 is not displayed. Accordingly, the first region 222 and the second region 224 may be regarded as relatively correct regions compared to the third region 226.

Therefore, the image processing apparatus of the present disclosure may perform additional analysis on the third region 226 as to whether the object on the third region 226 corresponds to the detection target object. For example, the image processing apparatus of the present disclosure crops the third region 226 that requires further analysis in the third result image 240, and recognizes various types of image matching techniques or images for the cropped region 250. The techniques can be applied. Meanwhile, a pre-learned AI model capable of analyzing the detected object may be needed according to an analysis technique. For example, when there is no image to be detected, a pre-learned deep learning model capable of recognizing or classifying the specific object may be required to recognize or classify the detected object as a specific object using the deep learning model. have.

The image processing apparatus of the present disclosure may finally determine that an object on the region is not USB through additional analysis of the third region 226. In addition, the image processing apparatus of the present disclosure may output the determination result. For example, the image processing apparatus of the present disclosure may display an object region determined as the detection target object on the input image as a result of the final determination. The fourth resultant image 260 is an image in which the first region 222 and the second region 224 are displayed in a rectangular box on the input image 210 except for the third region 226.

As another embodiment, the image processing apparatus of the present disclosure may adjust a parameter inside the artificial intelligence model related to the acquisition of the numerical information in order to adjust the sensitivity of the object detection. By adjusting parameters inside the artificial intelligence model, a plurality of different numerical information may be obtained with respect to one input image. The adjustment may be preset or entered by the user. Meanwhile, in FIG. 2, the first result image 220, the second result image 230, and the third result image 240 may be obtained by adjusting the numerical information instead of the first threshold value.

The image processing apparatus of the present disclosure may adjust a parameter inside the artificial intelligence model related to the acquisition of the first threshold value and / or numerical information when the object is detected. In addition, the parameters within the AI model related to the acquisition of the first threshold and / or numerical information of the present disclosure may be automatically adjusted. The adjustment may be preset or entered by the user. As such, by automatically adjusting a parameter in the artificial intelligence model related to the acquisition of the first threshold value and / or numerical information, a detection result corresponding thereto may be generated, and further object analysis may be performed on the generated detection result. can do. Therefore, the image processing apparatus of the present disclosure does not simply complete the object analysis based on the primary object detection result, but selects the result and applies an additional object recognition technique to the selected result to provide more stable performance of artificial intelligence. I can guarantee it.

In addition, the image processing apparatus of the present disclosure may provide an artificial intelligence model that performs detection and / or secondary verification for multiple classes from an image including an object. For example, detection of multiple classes may mean adjusting a parameter inside the AI model or adjusting a first threshold to detect a plurality of detection target objects.

Meanwhile, the image processing apparatus of the present disclosure extracts a feature of an analysis target image to detect, classify, or find a location of an object of interest, generates context information based on the extracted feature, and extracts an object included in the image. Features and generated contextual information can be used.

3 is a diagram illustrating a process of generating and analyzing context information of an image according to an embodiment of the present disclosure. The image processing apparatus of the present disclosure may include a feature extractor 310, a context generator 320, and / or a feature. And a context analyzer 330.

In detail, the feature extractor 310 may extract a feature from the input image 312 using the input image 312 and generate a feature image 314 including the extracted feature information. The extracted feature may be a feature of a local region of the input image. The input image 312 may include a feature map of each layer in the input image or the composite product neural network model of the image analyzing apparatus. Also, the feature image 314 may include a feature map and / or a feature vector obtained by applying a convolutional neural network technique and / or a pooling technique to the input image 312. The context generator 320 may generate context information by applying a convolutional neural network technique and / or a pooling technique to the feature image 314 extracted by the feature extractor 310. For example, the context information may be a representative value representing all or a partial region of the analysis target image. In addition, the context information may be global context information of the input image. For example, the context generator 320 may generate context information of various scales, such as the entire image, the quadrant region, and the ninth region, by variously adjusting the pooling interval. For example, a full contextual information image 322 including contextual information about an image of an image full size, a quadrant contextual information image 324 and an image including contextual information about a quadrant image having a size that is divided into quarters of the entire image. A ninth contextual information image 326 including contextual information about a ninth-segment image of a total size divided by nine may be obtained. The feature and context analyzer 330 may more accurately analyze a specific region of the analysis target image using both the feature image 314 and the context information image 322, 324, and 326.

For example, when an image including a boat having a form similar to a car is an input image, the identified object is selected from the feature image 314 including a local feature extracted by the feature extractor 310. I can't tell if it's a car or a boat. That is, the feature extractor 310 may recognize the shape of the object based on the local feature, but there may be cases in which the object may not be correctly identified and classified only with the shape of the object. The context generator 320 generates context information 322, 324, and 326 based on the analysis target image or the feature image 214, thereby more accurately identifying and classifying objects. For example, the feature extracted for the whole image is recognized or classified as "natural landscape", the feature extracted for the quadrant image is recognized or classified as "lake", and the feature extracted for the ninth image is "water". If it is recognized or classified as, the extracted features "natural landscape", "lake", "water" can be generated and used as the context information. The feature and context analyzer 330 may identify an object having the shape of the boat or automobile as a boat by using the context information.

In the embodiment described with reference to FIG. 3, the context information for the entire image, the context information for the quadrant image, and the context information for the ninth segment have been described. However, the size of the image from which the context information is extracted is described. It is not limited to this. For example, context information about an image having a size other than the above-described image may be generated and utilized.

4 is a diagram illustrating a process of detecting an object by analyzing an image according to an embodiment of the present disclosure.

For example, the image processing apparatus 400 may correctly receive and / or classify an object included in the image 410 by receiving the image 410 and generating information on image areas of various sizes. The input image 410 may be, for example, an X-ray image including a bag. The image processing apparatus 400 analyzes the input image 410 as described above, extracts the feature of the entire image and the feature of the partial region of the image, and accurately identifies the object included in the image 410 by using the same. can do. The feature 422 for the entire image may be, for example, a feature of a bag shape. Features for some regions of the image may include, for example, features 424 for handles, features 426 for zippers, features 428 for rings, etc. The image processing apparatus 400 may generate the generated image. By utilizing features 422, 424, 426, 428 as contextual information, one can accurately identify that an object included in the image 410 is a “bag.” If some of the generated features are “bags” The image processing apparatus 400 cannot identify that an object included in the image 410 is a “bag” or identify an object included in the image 410 as a “bag”. Can provide analytical results.

In addition, the image processing apparatus of the present disclosure may use a feature that the range of color expression is different according to the physical properties of the object in order to detect one or more objects included in the input image or generate the final output image.

FIG. 5 is a view illustrating an image in which colors are expressed based on physical properties of an object according to an exemplary embodiment of the present disclosure, wherein a bag image 500, a medicine container image 510, and a carrier photographed by an X-ray reading apparatus are illustrated. A bag image 520 is shown. In the case of the bag loop 502, the bag zipper 504, the medicine 512, and the bottle 522, the range of color expression may be different depending on the physical properties of the object. Meanwhile, the bag loop 502, the bag zipper 504, the medicine 512, and the bottle 522 are relatively vividly colored so that they can be distinguished from other objects, while any contents in the carrier bag ( In the case of 524, it may be difficult to identify what the arbitrary contents 524 are in the carrier bag image 520 and may not be easily distinguished from other objects. Therefore, the image processing apparatus of the present disclosure may vary the degree of enhancement of the image according to the color expression range. To this end, the color distribution of each of the divided regions may be analyzed and weighted to at least some of the regions.

The one or more weights may include weights for at least some of the n color representation ranges. For example, if one region has n color ranges, the number of weights in the region may range from 1 to n.

For example, when one weight is determined for one region, the determined weight may be applied to all color expression ranges included in the one region. Alternatively, the determined weight may be applied to at least a portion of all color expression ranges included in the one region. For example, the determined weight may be applied only to a predetermined color expression range that is an object of image enhancement.

Or, for example, a weight may be determined for each of the n color representation ranges. That is, the number of weights for one region may be n. In this case, a weight corresponding to each of all the color expression ranges included in the area may be applied to the corresponding color expression range. The weight may be given a relatively high weight for a predetermined color expression range that is an object of image enhancement.

Or, for example, a weight may be determined for each of m color representation ranges greater than 1 and less than n. That is, the number of weights for one region may be m. In this case, the weighted weight may be applied only to the color weighted range of the color color ranges included in the area. As described above, a relatively high weight is given to a predetermined color expression range that is an object of image enhancement.

As described above, the weight may be determined relatively high for the predetermined color expression range among the n color expression ranges. For example, when the object included in the X-Ray image is an organic material, the boundary is often expressed more clearly than the object having other physical properties (metal, inorganic, etc.). Therefore, a relatively high weight may be given to a portion of the divided region corresponding to the color expression range representing the organic material.

The predetermined color representation range to which the relatively high weight is given may be one or more. For example, when the total color representation range is n, the predetermined color representation range to which a relatively high weight is given may be 1 to n-1. When there are a plurality of predetermined color representation ranges, the degree of image enhancement required for each may be different, and thus weighted. For example, when an image is clearly displayed in the order of metal-> inorganic-> organic, a relatively high weight may be given only to the range of color expression for organic matter, but a weight that is relatively higher than metal for inorganic and organic matter may be given. You can also grant. In this case, a relatively high weight may be given to the organic material rather than the inorganic material.

For example, the color may be expressed by a combination of red, green, and blue having values of 0 to 255, respectively. In this case, for example, the color expression range represented by Red may be (R, G, B) = (10 to 255, 0 to 10, 0 to 10). Also, for example, the color expression range represented by Green may be (R, G, B) = (0-10, 10-255, 0-10). Also, for example, the color expression range represented by Blue may be (R, G, B) = (0-10, 0-10, 10-255). For example, a color representation range that is red may be an object of enhancement, in which case, a high weight may be given to the color representation range of red. For example, when a color of a predetermined region is represented by (R, G, B) = (50, 8, 3), the color is included in the red color expression range, and thus is subject to enhancement. If the weight assigned to the red color representation range is 2, the color of the corresponding region may be enhanced by (R, G, B) = (50x2, 8x2, 3x2) by enhancement. Alternatively, only specific element colors may be weighted and enhanced to (R, G, B) = (50x2, 8, 3). The weight may be determined differently according to the degree of strengthening.

The enhancement of the color may be adaptively applied to the color expression range represented by Green and / or the color expression range represented by Blue.

The same or different weights may be applied to each color representation range. Alternatively, the weight may not be applied to some color representation ranges.

6 is a diagram for describing an image processing method according to an exemplary embodiment.

In operation S600, an analysis target image including one or more objects may be received.

The analysis target image may include at least one object. For example, the image may be a raw image captured by an X-ray imaging device or an image of any form (format) for storing or transmitting the raw image. The image may be obtained by capturing and data-forming image information that is taken by an X-ray reading device and transmitted to an output device such as a monitor.

In operation S610, numerical information regarding whether the one or more objects included in the analysis target image correspond to the detection target object may be obtained using the artificial intelligence model.

For example, the artificial intelligence model may be a deep learning model for object detection. The object to be detected may be set in advance, and the artificial intelligence model may be pre-learned with respect to the object to be detected. Also, the object to be detected may vary depending on the field of application. For example, the storage medium may include a storage medium such as a USB, a notebook, a hard drive, a knife, a lighter, a gun, and the like. Accordingly, the image processing apparatus and / or image processing method of the present disclosure may detect a specific object by analyzing an input image and may be applied to various fields such as an enterprise security system.

The numerical information may mean a probability, a score, or a distribution indicating whether the detected object corresponds to the detection target object when the object is detected by applying an artificial intelligence model to the analysis target image including one or more objects. For example, if the detection target object is a USB, a knife, a lighter, the numerical information may include a probability that the object included in the analysis image corresponds to a USB, a probability that corresponds to a knife, and / or a probability that corresponds to a lighter.

When the numerical information obtained in step S620 is equal to or greater than the first threshold value, one or more objects may be determined as the detection target object as the first detection step.

The first threshold value may vary depending on the object to be detected. The first threshold may be preset or input by a user. By using the first threshold value, the object determination method of the present disclosure can ensure more reliability. Also, for example, the first threshold may be a parameter related to object detection.

Meanwhile, the sensitivity of the first detection step may be adjusted by using a first threshold value or a parameter inside an artificial intelligence model related to acquisition of numerical information. By adjusting parameters inside the artificial intelligence model, a plurality of different numerical information may be obtained with respect to one input image. The adjustment may be preset or entered by the user. For example, the parameter inside the artificial intelligence model may include filter kernels of the convolutional layer and / or the pooling layer in the convolutional neural network. Accordingly, the degree of determination as to whether the detected object corresponds to the object to be detected may be adjusted by adjusting numerical information by adjusting parameters in the artificial intelligence model or adjusting the first threshold value.

In operation S630, whether the at least one object corresponds to the detection target object may be determined by applying an image matching technique or an image recognition technique to the at least one object determined as the detection target object in operation S620. .

For example, the secondary detection step may be performed only on an object having one or more objects determined as the detection target object in operation S620 and having numerical information less than or equal to a second threshold. The second threshold may vary depending on the object to be detected. The second threshold may be preset or input by the user.

On the other hand, if there is an image of the object to be detected (for example, if there is an X-ray image of the prohibited items to enter the company security), an image matching technique using the image may be applied. The image matching technique may be, for example, a method of determining similarity between images by extracting feature points between input images. In addition, when there is no image of the object to be detected, various image recognition methods generally used in the field of recognition or classification may be applied.

In addition, when the second determination result is different from the result determined in operation S620, the method of the present disclosure may again determine whether the detection object corresponds to another detection object based on the numerical information. For example, based on the numerical information, it may be determined whether the object corresponds to a search target object having a next probability value except the detection target object determined in operation S630.

Exemplary methods of the present disclosure are represented as a series of operations for clarity of description, but are not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order as necessary. In order to implement the method according to the present disclosure, the illustrated step may further include other steps, may include remaining steps except for some steps, or may include additional other steps except for some steps.

The various embodiments of the present disclosure are not an exhaustive list of all possible combinations and are intended to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In hardware implementations, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), General Purpose It may be implemented by a general processor, a controller, a microcontroller, a microprocessor, and the like.

It is intended that the scope of the disclosure include software or machine-executable instructions (eg, an operating system, an application, firmware, a program, etc.) to cause an operation in accordance with various embodiments of the method to be executed on an apparatus or a computer, and such software or Instructions, and the like, including non-transitory computer-readable media that are stored and executable on a device or computer.

Claims (23)

Receiving an analysis target image including an object;
Obtaining numerical information on whether the object included in the analysis target image corresponds to a detection target object using an artificial intelligence model;
A primary detection step of determining whether the numerical information is equal to or greater than a first threshold value;
A second detection step of determining whether the numerical information is equal to or greater than a second threshold value; And
Determining whether the object corresponds to the object to be detected based on results of the first detection step and the second detection step,
The second threshold is greater than the first threshold,
And the second detecting step is performed only when the numerical information is equal to or greater than a first threshold value. The method of claim 1,
And adjusting the sensitivity of the first detection step or the second detection step by adjusting the first threshold value or the second threshold value. The method of claim 2,
The first threshold value or the second threshold value, characterized in that determined based on the learning of the artificial intelligence model, image processing method. The method of claim 1,
And adjusting the sensitivity of the first detection step or the second detection step by adjusting a parameter inside the artificial intelligence model. The method of claim 1,
The first threshold value and the second threshold value is determined differently according to the type of the object to be detected, image processing method. The method of claim 1,
And when the numerical information is equal to or greater than a first threshold and less than or equal to the second threshold, the object is determined not to be the detection target object. The method of claim 6,
If it is determined that the object is not the detection target object, changing the type of the detection target object. The method of claim 1,
Acquiring the numerical information,
A feature extraction step of extracting a feature of the object;
Generating context information based on the extracted feature; And
And obtaining the numerical information based on the extracted feature and the generated context information. The method of claim 8,
The generating of the context information may include generating the context information on all or a portion of the analysis target image.
The context information may be an entire or partial region of the analysis target image. The method of claim 9,
The generating of the context information may include generating the context information by applying at least one of a convolutional neural network technique and a pooling technique to the extracted features. The method of claim 10,
The generating of the context information may include generating the context information on all or a portion of the analysis target image by adjusting the pooling interval. An image receiver configured to receive an analysis target image including an object;
An output unit configured to output an analysis result of the analysis target image; And
A processor for controlling the receiving unit and the output unit,
The processor,
Acquires numerical information about whether the object included in the analysis target image corresponds to a detection target object using an artificial intelligence model,
Perform first detection to determine whether the numerical information is equal to or greater than a first threshold value,
Perform second detection to determine whether the numerical information is equal to or greater than a second threshold value,
It is determined whether the object corresponds to the object to be detected based on the results of the first detection and the second detection,
The second threshold is greater than the first threshold,
And the second detection is performed only when the numerical information is equal to or greater than a first threshold value. The method of claim 12,
The processor,
And adjusting the sensitivity of the first detection or the second detection by adjusting the first threshold value or the second threshold value. The method of claim 13,
The first threshold value or the second threshold value, characterized in that determined based on the learning of the artificial intelligence model, the image processing apparatus. The method of claim 1,
The processor is
And adjusting the sensitivity of the first detection or the second detection by adjusting a parameter inside the artificial intelligence model. The method of claim 12,
And the first threshold value and the second threshold value are differently determined according to the type of the object to be detected. The method of claim 12,
And when the numerical information is equal to or greater than a first threshold and less than or equal to the second threshold, the object is determined not to be the detection target object. The method of claim 17,
The processor,
And when it is determined that the object is not the detection target object, changing the type of the detection target object. The method of claim 12,
The processor,
Extract features of the object,
Generate context information based on the extracted feature,
And the numerical information is obtained based on the extracted feature and the generated context information. The method of claim 19,
The context information is generated for all or part of the region of the analysis target image.
The context information may be an entire or partial region of the analysis target image. The method of claim 20,
The contextual information is generated by applying at least one of a convolutional neural network technique and a pooling technique to the extracted feature. The method of claim 21,
And the context information is generated for all or a portion of the analysis target image by adjusting the pooling interval. A computer-readable recording medium having recorded a program,
The program,
Receiving an analysis target image including an object;
Obtaining numerical information on whether the object included in the analysis target image corresponds to a detection target object using an artificial intelligence model;
A primary detection step of determining whether the numerical information is equal to or greater than a first threshold value;
A second detection step of determining whether the numerical information is equal to or greater than a second threshold value; And
Determining whether the object corresponds to the object to be detected based on results of the first detection step and the second detection step,
The second threshold is greater than the first threshold,
And said secondary detecting step is performed only when said numerical information is equal to or greater than a first threshold value.

Images