KR102045223B1 - Apparatus, method and computer program for analyzing bone age - Google Patents

Abstract

The present invention relates to a method, an apparatus, and a computer program for analyzing bone age. A method for analyzing bone age based on a hand bone image, comprising the following steps of: receiving the hand bone image; extracting a plurality of regions of interest from the hand bone image; calculating a first feature for each of the regions of interest by applying, respectively, the regions of interest to a plurality of region of interest feature extraction models that have been independently learned to extract a feature of each of the regions of interest; and calculating an analysis value of the hand bone image for identifying the hand bone image by applying the first feature for each of the regions of interest to a pre-learned first integrated analysis model as an input vector. The first integrated analysis model generates one input vector with a plurality of features calculated by applying the regions of interest extracted in learning images to the region of interest feature extraction models, respectively, and is a machine learning model learned by using a plurality of input vectors corresponding to the number of the learning images as learning data, and the analysis value includes a vector value representing a bone maturation grade or a probability for each bone age, or bone age corresponding to a regression analysis model. According to the present invention, it is possible to reduce effect of an error on an analysis model that training data generated for directing and learning the entire image may have and to improve learning accuracy and objectivity of a deep neural network.

Description

Bone age analysis method, device and computer program {APPARATUS, METHOD AND COMPUTER PROGRAM FOR ANALYZING BONE AGE}

The present invention relates to a bone age analysis method, apparatus and computer program, and more particularly, to a method, apparatus and computer program for analyzing bone age based on the bone image using machine learning.

Image analysis technology is widely used in various fields, and the use of machine analysis algorithms is particularly active as machine learning algorithms increase the accuracy of object identification.

In the image analysis, when a query image is received, the computer classifies a feature of the query image (or a feature of an object included in the query image) through a model trained based on machine learning. The analysis result of the query image shows the user which object is included in the query image-for example, what product is included in the image-or what characteristics each object has-for example, what color is the product , A pattern, a person's face included in the image, etc. can be easily identified, and the user can quickly classify and understand a large number of image data.

In order to identify an object, it is necessary to learn the characteristics of the object to be identified. A supervised learning method is used to train a machine learning model with a large amount of identified data.

Conventionally, an analysis model may be generated using a machine learning method, and a method of analyzing a query image through a trained analysis model may be classified into two types.

Firstly, the analysis model is generated by analyzing the entire image and the image is analyzed. When the entire image is input into the machine learning model and trained, the analysis model learns the features in the image itself and uses it for classification.

Second, the ROI (Region of Interest) is extracted from the image to train an analysis model on the region of interest, and the image is analyzed based on the region of interest. At this time, the area of interest is generally specified by experts in the relevant field. By using the accumulated experience and knowledge of experts in the field, the main area of interest is extracted and the image is analyzed based on the area of interest. It can increase.

However, both methods have their own limitations. In analyzing the entire image as a learning target, when there are many complex or characteristic parts in the image, the classification result for the same query image (analysis target) may be different. This is because the identification value may vary depending on the subject generating the supervised learning data. In the whole image-based classification method, if the accuracy and objectivity of the training data used to generate the analysis model are not secured, the reliability of the classification result may be greatly deteriorated, and the whole image-based classification method may depend on an intuitive determination of the entire image.

In addition, since the whole image-based classification method analyzes even the parts that do not have an important meaning in classifying the image, noise that is not related to the query image identification may negatively affect the automatic analysis result of the corresponding image.

On the other hand, the method of analyzing the feature for each region of interest has a limitation in that the features of the remaining regions other than the region of interest are not reflected in the entire image or the importance of the selected region of interest may not be large.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an image analysis method, an apparatus, and a computer program that can improve the accuracy of image analysis.

In addition, the present invention is to provide an image analysis method that can reduce the effect of the error of the training data generated for supervised learning of the entire image on the analysis model, and improve the learning accuracy and objectivity of the deep neural network The purpose.

In addition, the present invention improves objectivity and accuracy of image classification by separately detecting a feature in a main region of interest in the image and reflecting it in the final image classification, when the image includes noise elements that are likely to cause a decision error. For other purposes.

Another object of the present invention is to provide a basis for and determination of image classification results of a deep neural network, referred to as a black box, by extracting important features for each region of interest and outputting identification values in identifying a corresponding query image. It is done.

According to an aspect of the present invention, there is provided a method of analyzing bone age based on a bone image, including receiving the bone image, extracting a plurality of regions of interest from the bone image, and each feature of the region of interest. Calculating a first feature for each of the plurality of ROIs by applying the ROIs to a plurality of ROI feature extraction models that are independently trained to extract the first ROI, and inputting the first features of the plurality of ROIs as one input vector. Calculating an analysis value of the bone image for identifying the bone image by applying the trained first integrated analysis model, wherein the first integrated analysis model includes a plurality of ROIs extracted from a training image. Generates an input vector with a plurality of features calculated by applying to the ROI feature extraction model, respectively, A machine learning model trained using a plurality of input vectors corresponding to the number of learning images as learning data, and the analysis value is a vector value representing a bone maturity level or probability for each bone age, or bone age corresponding to a regression analysis model. It is characterized by including one.

According to the present invention as described above, it is possible to increase the accuracy of image analysis compared to the conventional method.

In addition, according to the present invention, the effect of the training data generated for supervised learning of the entire image may be reduced on the analysis model, and the learning accuracy and objectivity of the deep neural network may be improved.

In addition, according to the present invention, if the image contains noise elements that are likely to cause a decision error, the feature is detected separately in the main region of interest in the image and then reflected in the final image classification to increase the objectivity and accuracy of the image classification. Can be.

In addition, according to the present invention, it is possible to provide a judgment basis and an interpretation on the image classification result of the deep neural network referred to as a black box.

1 is a block diagram illustrating an image analyzing apparatus according to an embodiment of the present invention;
2 is a view for explaining a feature extraction model according to an embodiment of the present invention;
3 is a bone image for explaining a region of interest extraction according to an embodiment of the present invention;
4 is a flowchart illustrating an image analysis method according to an embodiment of the present invention;
5 is a flowchart illustrating an image analysis method according to another exemplary embodiment.

The above objects, features, and advantages will be described in detail with reference to the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. In describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings are used to indicate the same or similar components, all combinations described in the specification and claims may be combined in any way. And unless specified otherwise, reference to the singular may include one or more, and reference to the singular may also include the plural expression.

The present invention can be applied to various fields, but can be effectively used for bone image analysis, particularly for determining bone age. Bone age plays an important role in estimating the progress of physical growth in children and adolescents and when growth stops. Therefore, it is used to determine the degree of growth potential. The bone age determines the state of fusion between the epiphysis and the epiphyseal as the epiphyseal plate (growth plate) progresses through ossification.

The bone age analysis is mainly used for bone age analysis. In general, bone age is determined by looking at bone maturity using the atlas-based GP method or the TW method.

The GP method is a method of intuitively viewing and discriminating the entire image of the hand bone used for the bone age reading, and the TW method separates the regions of interest for each of the major sub-articular regions from the bone image, and then determines the maturity level for each region of interest. It is a method of calculating bone age by combining.

When the GP method is used, the specialist compares the representative image by the bone age and the query image, and determines the bone age of the representative image that is most similar to the query image as the bone age of the query image. However, many studies have reported that the GP method may have a large deviation depending on the reader, and that the same reader may make a different evaluation according to time even if the same reader makes a judgment.

Compared with the GP method, the TW method has less research results and the difference in the time difference classification results between the same readers, and it is evaluated that it is more accurate and less error probable. However, although the TW method is smaller than the GP method in classifying the maturity grade for each region of interest, an error may occur. In addition, there is a limitation that portions other than the ROI detected as the evaluation target are not included in the bone age analysis.

These problems are the same when the GP method and the TW method are implemented using machine learning. According to the present invention, the above problems can be overcome and the accuracy of the bone image analysis can be greatly improved.

Hereinafter, an image analyzing apparatus according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

Referring to FIG. 1, the image analyzing apparatus 100 according to an exemplary embodiment of the present disclosure includes an input unit 110, a detector 130, an image analyzer of interest 150, and an integrated analyzer 170. The apparatus may further include an image analyzer 160.

The input unit 110 may be an input module that receives a query image from a user, or may be a communication module that receives a query image from another electronic device (not shown) through a wired / wireless communication network. In another embodiment, the input unit 110 may be an imaging module that acquires an image of an object. The query image may be a two-dimensional color or black and white image, may be obtained through an imaging device such as a camera, a radiographic apparatus, a computed tomography apparatus, or may be obtained through a capture function of an electronic device. The input unit 110 may transmit the received query image to the entire image analyzer 160 and / or the detector 130.

The detector 130 extracts one or more regions of interest from the query image. Herein, the region of interest refers to a specific region having importance in analyzing the query image. In machine learning analysis, the importance of a particular region is often based on the domain knowledge to which the image belongs. In other words, the analysis based on the region of interest is intended to improve accuracy by removing noise in regions other than the region of interest that may cause errors in the analysis. For example, when receiving a bone image as shown in FIG. 3 as a query image, the image analysis apparatus 100 may analyze the bone image to measure bone age. The detector 130 may extract regions R1 to R5, which are mainly considered for bone age measurement, as regions of interest.

The detector 130 includes a manual detector 135 and an automatic detector 137, and the ROI may be manually or automatically extracted. The manual detector 135 may extract an area corresponding to the location information set by the user as the ROI. That is, in the example of FIG. 3, the user may directly set one region R1 to R5 that is the region of interest in the bone image, and the manual detector 135 extracts the region of interest by extracting the region designated by the user as the region of interest. It may be delivered to the unit 160.

The automatic detector 137 may apply a trained automatic detection model to the query image to extract a region corresponding to the specific region as the region of interest. Herein, the specific area means an area that is mainly considered for measuring bone age in the above-described example. The automatic detector 137 may include an automatic detection model that is a deep neural network trained (trained and evaluated) using a plurality of specific region images to extract a specific image from the entire image. As the automatic detection model, a deep neural network-based Faster-RCNN, Yolo detection technique, etc. may be used, but is not limited thereto.

For example, the automatic detection unit 137 may train the automatic detection model using a plurality of radial images. In this case, when an arbitrary bone image (full bone image) is input, the automatic detection unit 137 corresponds to the radial bone. R1 can be extracted to the region of interest.

The detector 130 may extract one or more ROIs (interest regions 1, 2,..., N) from the query image and transfer the extracted ROIs to the ROI 150.

The learner 140 may include an ROI learner 143, an entire region learner 145, and an integrated learner 147.

The ROI learner 143 independently learns one or more ROI feature extraction models and provides the ROI to the ROI 150. Each region of interest feature extraction model is a machine learning model, which may be a deep neural network or a regression analysis model composed of one or more layers.

The ROI learner 143 may train the ROI feature extraction model by using a plurality of ROI images previously identified for each ROI, that is, classified according to a predetermined identification value, as training data. The identification value may be the same kind of value as the analysis value of the query image or may be another kind of value correlated with the analysis value of the query image.

For example, when finally determining bone age of the query image, the ROI learning unit 143 may use the bone age-specific radial image as training data to train the first region of interest (radius) feature extraction model. In addition, radial image by bone maturity level (A to I) correlated with bone age may be used as training data. That is, the bone maturity grade of the radial (R1) in the bone image shown in Figure 4 affects the determination of the bone age of the whole bone image, the first interest using a plurality of radial images grouped by bone maturity grade We can train the domain feature extraction model.

Therefore, when a specific ROI is applied to the ROI feature extraction model thus learned, an analysis value of the ROI may be obtained. In other words, when the radial image is input to the first region of interest feature extraction model, bone age or bone maturity level of the input radial image may be acquired as an analysis value of the ROI.

Like the ROI learner 143, the whole region learner 145 trains the entire region feature extraction model and provides the same to the entire image analyzer 160. The full region feature extraction model is a machine learning model, which may be a deep neural network or a regression analysis model composed of one or more layers.

The full region learner 145 may train a full region feature extraction model by using a plurality of full region images previously identified for the entire image, that is, classified according to a predetermined identification value, as training data. The identification value may be the same kind of value as the analysis value of the query image. For example, when the bone age of the bone image is to be determined as described above, the entire bone image determined as the specific bone age may be used as training data.

The integrated learning unit 147 may train the integrated analysis model by using the feature (or feature value) extracted through the ROI-specific learning and / or the whole region feature value extracted by the entire domain learner. The integrated learning unit 147 will be described in more detail in the description of the integrated analyzing unit 170.

The ROI analyzer 150 may calculate the first feature for each ROI by applying one or more ROIs to one or more ROI feature extraction models that are independently trained to extract the features of each ROI. The first feature for each ROI calculated through the ROI feature extraction model is used as an input of the integrated analyzer 170.

For example, if the ROI region (radius region) in the bone image is a region of interest 1 and the ROI feature extraction model learned from the plurality of radial images is called the first ROI feature extraction model 155, the ROI analyzer 150 ) May input the ROI 1 detected by the detector 130 to the first ROI feature extraction model to calculate a first feature of the ROI.

An embodiment of a method of calculating a feature of the ROI by using the ROI feature extraction model will be described in detail with reference to FIG. 2.

A region of interest feature extraction model is a deep neural network composed of one or more layers, and subsampling between multiple convolutional layers and multiple convolutional layers that produce a feature map of the features of the region of interest. It may be a convolutional neural network (CNN) including a pooling layer that performs the operation. The convolutional neural network may extract features from the input image by alternately performing the convolutional product and subsampling on the input image. A feature or feature value refers to a vector value representing key features that are important for classifying or identifying an image. According to an embodiment of the present invention, the first characteristic input to the integrated analyzer 170 may be a value output from one layer or a value input from one layer among one or more layers constituting the deep neural network.

A convolutional neural network includes multiple convolution layers, multiple subsampling layers, max-pooling layers, and pooling layers, and a global average pooling layer and a fully-connected layer. Connected layer), Softmax layer (Softmax layer) and the like. The composite product layer is a layer that performs a composite product on the input image, and the subsampling layer is a layer that extracts the maximum or average value of the input image locally and maps it to the 2D image. Can be done.

In the product multiplication layer, information such as the size of the kernel, the number of kernels to be used (the number of maps to be generated), and the weight table to be applied in the product calculation is needed. In the subsampling layer, information about the size of the kernel to be subsampled and information about whether to select the maximum value or the minimum value among the values in the kernel region is needed.

In FIG. 2 illustrating an example of a convolutional neural network (CNN), the convolutional neural network includes an integrated layer (Layer 2) in which subsampling is performed between convolutional layers (Layer 1 and Layer 3), and a GAP layer, It can include a fully connected layer and a softmax layer. This is for convenience of description, it is possible to configure a multiplicative neural network by combining a variety of layers having a different characteristic than that applied to this example, the present invention is not limited by the configuration and structure of the deep neural network.

As described above, the first feature calculated by the image analyzer 150 of interest may be a value output from one layer or a value input from one layer among one or more layers constituting the deep neural network. An embodiment when using the first feature is as follows.

As shown in FIG. 2, when the average value of each feature map is obtained from the GAP layer of the ROI feature extraction model and transferred to the SoftMax layer through the fully connected layer, the image analysis unit 150 calculates the GAP layer through the GAP. An average value vector of each feature map may be set as a feature of the corresponding ROI. In this case, the average values A ¹ , A ² ,..., A ^K of the K channel feature maps calculated through the GAP are input to the integrated analysis unit 170 as feature values of the region of interest n.

라 하면, 채널 k의 특징맵의 평균값인

는 다음과 같이 나타낼 수 있다. The feature value at position (i, j) in the feature map of channel k

In this case, the average value of the feature map of channel k

Can be expressed as:

[Equation 1]

(K= 채널의 총 개수)이 되며, 통합분석부(170)로는 관심 영역별 특징값

(N: 관심 영역의 총 개수)이 입력될 수 있다. That is, the feature value of the region of interest n

(K = total number of channels), and the integrated analysis unit 170 characterizes each region of interest.

(N: total number of regions of interest) may be input.

)으로 사용한다고 가정하자. 관심 영상 분석부(150)는 최종 클래스 분류를 위한 각 클래스의 스코어 또는 정규화된 확률값을 관심 영역의 특징값으로 산출하기 위하여 소프트맥스의 입력인 각 클래스 스코어 값을 통합분석부 입력에 넣을 수 있다. 물론 소프트맥스 결과로 나온 정규화된 확률값을 해당 관심 영역의 특징값으로 설정하여 통합분석부 입력에 넣을 수도 있다. In another embodiment, the image of interest analyzer 150 may use a value input to one layer among various layers constituting the deep neural network as the first feature. For example, in the exemplary embodiment of FIG. 2, a value of the image of the ROI 150 input to the softmax layer may be a first feature (

Suppose you use The image analysis unit 150 may insert each class score value, which is an input of the softmax, into the integrated analysis unit input to calculate the score or normalized probability value of each class for final class classification as a feature value of the ROI. Of course, the normalized probability value resulting from the softmax result can be set as a feature value of the region of interest and put into the integrated analysis unit input.

라고 하면,

는 다음과 같이 나타낼 수 있다. In this example, the score of class C

Speaking of

Can be expressed as:

[Equation 2]

는 i채널 특징맵의 평균값,

는 클래스 c에 대한

의 가중치이다. Where K is the total number of channels,

Is the mean value of the i-channel feature map,

For class c

Is the weight of.

(M= 클래스의 총 개수)이 되며, 통합분석부(170)로는 관심 영역별 특징값

이 입력될 수 있다. In this case, the feature value of the region of interest n

(M = total number of classes), and the integrated analysis unit 170 characterizes each region of interest.

Can be input.

)을 산출할 수 있다. 이는 통합분석부(170)에 특징값을 입력하는 것과는 별개로, 관심 영역 그 자체의 분석결과를 사용자에게 제공하기 위함이다. 관심 영역분석부(150)에서 산출된 관심 영역별 분석값은 출력부(190)로 전달될 수 있다.The ROI analysis unit 150 separates the above-described feature value for each ROI, and analyzes the ROIs.

) Can be calculated. This is to provide the analysis result of the ROI itself to the user, apart from inputting the feature value into the integrated analysis unit 170. The ROI analysis value calculated by the ROI 150 may be transmitted to the output unit 190.

The entire image analyzer 160 may or may not be included in the image analysis apparatus 100 according to an exemplary embodiment. The query may be applied to a full region feature extraction model trained as a plurality of images. Compute a second feature of the image. That is, the full region feature extraction model is a machine learning framework trained on the basis of the entire image for feature extraction of the entire image. According to an embodiment of the bone image analysis, the whole region feature extraction model is obtained by learning and evaluating the entire image of the bone by bone age. It is a deep neural network trained to determine bone age based on the distribution of feature values included in the entire image of the bone.

The full region feature extraction model used by the whole image analyzer 160 is different from the ROI feature extraction model in that the input image is a whole image instead of a part of the query image, and is a neural network trained based on the whole image. The configuration and learning method may be the same as the ROI feature extraction model.

The query image itself is input to the entire image analyzer 160. For example, when the query image is a bone image, a hand bone image, and when the query image is a face image, an image including the shape of the entire face may be input to the entire image analyzer 160. Since only the ROI extracted through the detector 130 is input, the ROI image analyzer 150 may input only a partial region such as a radial image and a third joint image of a third finger bone when the query image is a bone image. When the query image is a face image, an image of one region, such as an eye region, a nose region, and a mouth region, may be input to the ROI 150.

As described above, the whole region feature extraction model is also a deep neural network composed of one or more layers as shown in FIG. 2, and in the case of a compound multiplication neural network, a feature map is generated by arranging a kernel on an input query image and performing a synthesis product. Through subsampling, the process of sampling the values in the kernel region may be repeated. Through this process, the entire image analyzer 160 may extract a feature of the query image (second feature).

Like the first feature input from the image analyzer 150 to the integrated analyzer 170, the second feature may be an input value input to a specific layer or an output value of a specific layer among one or more layers constituting the deep neural network. . Of course, this may be the result of passing through the entire deep neural network. Since the method of calculating the second feature is the same as that described in the image analysis unit 150 of interest, overlapping description thereof will be omitted.

In general, when a feature value is calculated using a machine learning model, the image is classified by referring to a probability distribution of the feature value. However, the present invention improves the accuracy of image analysis by not only performing image learning and classification, but also learning and characterizing a feature value calculated in each deep neural network learned for each region of interest for image classification. It has different technical characteristics from the prior art.

The integrated analysis unit 170 calculates an analysis value of the query image by applying the first feature to a previously learned integrated analysis model. If the entire image analyzer 160 and the image analysis unit 150 of interest are both used for image analysis, the integrated analyzer 170 may apply the first and second features to the previously learned integrated analysis model. The analysis value of the query image may be calculated.

In this case, the integrated analysis model is generated by the integrated learning unit 147, such as a deep neural network or a regression model trained with one or more features calculated by applying one or more regions of interest extracted from an image to one or more regions of interest feature extraction. It may be a machine learning model. In another embodiment, the integrated analysis model is a neural network trained with a feature calculated by applying an image to a full region feature extraction model and one or more features calculated by applying one or more ROIs extracted from the image to one or more ROI feature extraction models. , Deep neural networks or regression models.

는

로 나타낼 수 있으며, 여기서 H는 관심 영역의 개수(전체 영상 포함)이며

는 관심 영역 h의 특징값을 의미한다. In other words, the integrated analysis model is a machine learning model trained with feature values, and the feature (first feature) calculated by applying the ROIs extracted from the plurality of learning images to the ROI feature extraction model is used as training data, or the learning image. It may be trained using the feature (second feature) and the first feature calculated by applying to the full-domain feature extraction model as the training data. Therefore, the input vector of the integrated analysis unit 170

Is

Where H is the number of regions of interest (including the entire image)

Denotes a feature value of the region of interest h.

For example, in the bone image analysis, when the radial region is referred to as the region of interest 1, the integrated learning unit 147 is calculated by applying a plurality of radial images grouped by bone age or bone maturity level to the first region of interest feature extraction model. The integrated analysis model can be trained using the features of the radial image. That is, the integrated analysis model may learn features of feature values of the radial image for each bone age through supervised learning.

을 질의 영상의 관심 영역 1의 특징 벡터로 산출하며,

을 입력받은 통합 분석부(170)는

의 특징을 산출하여

의 특징에 대응되는 골 연령을 식별할 수 있다. 이와 같은 방식으로, 통합 분석부(170)는 각 관심 영역별 특징(

) 및/또는 전체 영상의 특징(

)를 학습 대상으로 하여 골연령별 영상 특징을 학습할 수 있다. Therefore, when the query image is input later, when the detector 130 extracts the radial image from the query image, the ROI analyzer 150 applies the radial image to the first ROI feature extraction model.

Is calculated as the feature vector of the region of interest 1 of the query image,

Integrated analysis unit 170 received the

By calculating the features of

Bone age corresponding to the characteristics of can be identified. In this way, the integrated analysis unit 170 may be characterized by each region of interest (

) And / or features of the entire image (

) Can be used to learn the image characteristics of each bone age.

In summary, the integrated analysis model used by the integrated analysis unit 170 of the present disclosure may learn only the features of the ROI or learn a combination of the features of the ROI and the features of the entire image. Subsequently, in analyzing the query image, the integrated analysis unit 170 may classify by receiving the feature value of the query image as an input value instead of receiving the image as an input value. In the present specification, the output of the integrated analysis unit 170 is referred to as an analysis value of a query image, which is actually a classification result of analyzing the query image, that is, a machine learning model (full region feature extraction model and / or region of interest) of the query image. It can be understood that it is a feature vector of the result value or output value of the feature extraction model).

The integrated analysis unit 170 identifies the query image by using the analysis value. The analysis value may be a vector value representing a category to which the query image belongs or a probability value for each category, or a result value of a regression analysis model.

는 일종의 카테고리에 해당하는 "10세"로 산출될 수도 있고, 회귀분석모델이 사용된 경우에는 "10.253"과 같이 골 연령으로 특정한 값이 산출될 수도 있다.For example, when analyzing the bone image, the analysis value may be a vector value representing the probability of each bone age, and the bone age corresponding to the highest probability value in the analysis value may be identified as the bone age of the query image, or the regression model It may be a corresponding bone age value. Therefore, if the age of the query image is calculated as the age of 10, the analysis value output from the integrated analysis unit as shown in FIG.

May be calculated as "10 years old" corresponding to a category, or when a regression analysis model is used, a specific value may be calculated as bone age, such as "10.253".

뿐 아니라, 제1관심 영역의 분석값

, 제2관심 영역의 분석값

, 제3관심 영역의 분석값

, 제4관심 영역의 분석값

을 모두 포함할 수 있다. 도 1의 195는 질의 영상의 골 연령에 대한 분석 결과로, 제1관심 영역의 골 연령은 11세, 제2관심 영역의 골 연령은 9세, 제3 관심 영역의 골 연령은 12세로 분석된 경우의 일 예이다.The output unit 190 may provide the user with the analysis value of the query image calculated by the integrated analyzer 170 and the analysis value for each ROI calculated by the ROI analyzer 150. For example, as shown in 195 of FIG. 1, the analysis result provided to the user may include an analysis value of a query image.

In addition, the analysis value of the first region of interest

, Analytical values in the second region of interest

, Analytical values in the third region of interest

, Analytical values in the fourth region of interest

It can include both. 195 of FIG. 1 is a result of analyzing the bone age of the query image. The bone age of the first region of interest is 11 years old, the bone age of the second region of interest is 9 years old, and the bone age of the third region of interest is 12 years old. One example of the case.

The analysis value of the ROI may be another kind of value that may affect the analysis value of the query image, that is, correlated with the analysis value of the query image. When the region of interest is trained using the bone maturity level rather than bone age as an identification value, the analysis value derived by applying the region of interest to the ROI feature extraction model may be the bone maturity level. In this case, the user may be provided with a bone maturity grade for each region of interest as an analysis value of the region of interest. For example, the bone maturity grade of the first region of interest may be analyzed as C, the bone maturity grade of the second region of interest is D, and the bone maturity grade of the third region of interest is C. Therefore, the image analysis apparatus 100 of the present invention is effective in providing the basis and interpretation of the image analysis result by providing the bone age or the bone maturity level of each region of interest together with the bone age of the query image. .

4 and 5 are flowcharts illustrating an image analyzing method according to an exemplary embodiment.

Referring to FIG. 4, an electronic device according to an embodiment of the present disclosure may receive a query image (S100) and extract one or more regions of interest from the query image (S200).

In operation 200, the electronic device may extract an area corresponding to the location information set by the user as the ROI, and apply the automatic detection model trained on the specific region to the query image to search for the ROI corresponding to the specific area. You can also extract

Next, the electronic device may calculate the first feature for each ROI by applying one or more ROIs to one or more ROI feature extraction models that are independently trained to extract the features of each ROI (S300). ). The ROI feature extraction model is a deep neural network composed of one or more layers, and the first feature may be a value output from one of the one or more layers or a value input into any one layer.

The electronic device may calculate an analysis value for identifying the query image by applying the first feature for each region of interest to the previously trained integrated analysis model (S400). That is, an analysis value indicating which category a query image belongs to, what probability value it has, and what value it corresponds to may be calculated. The integrated analysis model is a machine learning model trained with one or more features calculated by applying one or more ROIs extracted from a plurality of learning images to one or more ROI feature extraction models, respectively, and the analysis value is obtained by integrating the first features. It may be understood as being a feature vector or meaning a category value.

In operation 450, the electronic device may calculate an analysis value for each ROI. The analysis value calculated in operation 450 may be the same kind of value as the analysis value of the query image, or may be another kind of value correlated with the analysis value of the query image. For example, when the bone age of the query image is derived as an analysis value, the analysis value of the ROI calculated in step 450 may be the bone age of the ROI or may be a bone maturity grade correlated with the bone age measurement. . This may vary depending on which value is used as an identification value when each ROI feature extraction model is trained.

In operation 500, the electronic device may provide an analysis value of the ROI and an analysis value of the query image to the user. For example, the analysis value may be displayed through a display included in the electronic device, or the analysis value may be output through a speaker attached to the electronic device.

According to the embodiment of FIG. 5, the electronic device extracts a region of interest to calculate a first feature for each region of interest, and applies the query image to the entire region feature extraction model trained as a plurality of learning images to generate the query image. Two features may be calculated (S350). In this case, in operation 400, the electronic device may calculate an analysis value of the query image by applying the first feature and the second feature to the integrated analysis model. In this case, the integrated analysis model is a neural network trained with a feature calculated by applying a training image to a full region feature extraction model and one or more features calculated by applying one or more regions of interest extracted from the training image to one or more region of interest feature extraction models. Or a deep neural network, and the analysis value calculated in step 400 may be understood as a feature value of a combination of the first and second features.

Some embodiments omitted in the present specification may be equally applicable to the same subject matter. In addition, the above-described present invention can be variously substituted, modified, and changed within the scope without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains to the above-described embodiments and attached It is not limited by the drawings.

100: video analysis device
110: input unit
130: detector
140: learning unit
150: the image analysis unit of interest
160: the entire image analysis unit
170: integrated analysis unit

Claims (11)

In the method of analyzing bone age based on the bone image,
Receiving the bone image;
Extracting a plurality of ROIs from the bone image;
Calculating a first feature for each of the plurality of ROIs by applying the ROI to each of a plurality of ROI feature extraction models trained independently to extract the features of each ROI;
Calculating an analysis value of the bone image for identifying the bone image by applying the first features of the plurality of ROIs to a first integrated analysis model previously learned as an input vector,
The first integrated analysis model
A single input vector is generated from a plurality of features calculated by applying a plurality of ROIs extracted from a training image to the ROI feature extraction model, and a plurality of input vectors corresponding to the number of the training images is used as training data. Is a trained machine learning model,
The analysis value is
A bone age analysis method comprising a bone age corresponding to a vector value representing a bone maturity grade or bone age probability, or a regression analysis model.
The method of claim 1,
Calculating the second feature of the bone image by applying the bone image to the entire region feature extraction model trained as the training image;
The calculating of the analysis value
Calculating an analysis value of the bone image by applying the first and second features of the plurality of ROIs to a second integrated analysis model as one input vector,
The second integrated analysis model
A single input vector is generated with a feature calculated by applying the training image to the full region feature extraction model and a plurality of features calculated by applying the plurality of ROIs extracted from the training image to the ROI feature extraction model, respectively. And a machine learning model trained using a plurality of input vectors corresponding to the number of learning images as learning data.
The method of claim 1,
The region of interest feature extraction model is a deep neural network composed of one or more layers,
The first characteristic is a bone age analysis method is a value output from any one of the one or more layers or a value input to the one of the layers.
The method of claim 2,
The ROI feature extraction model and the ROI feature extraction model are deep neural networks composed of one or more layers,
The first feature and the second feature is a bone age analysis method is a value output from any one of the one or more layers or a value input to the one of the layers.
The method of claim 1,
The ROI feature extraction model is trained using a plurality of ROIs classified according to a predetermined identification value as training data.
The identification value is bone age analysis method, characterized in that the bone age or bone maturity grade.
The method of claim 1,
Calculating an analysis value for each ROI by applying the ROIs to the ROI feature extraction models, respectively;
Bone age analysis method comprising the step of providing the analysis value of the bone image and the analysis value for each region of interest to the user.
The method of claim 1,
Extracting the region of interest
And extracting a region corresponding to the specific region as the region of interest by applying an automatic detection model trained on the specific region to the bone image.
The method of claim 1,
The region of interest is
Bone age analysis method comprising the area corresponding to the radius.
In the device for analyzing bone age based on the bone image,
An input unit to receive the bone image;
A detector extracting a plurality of ROIs from the bone image;
An interest image analyzer configured to calculate first features for each of the plurality of ROIs by applying the ROIs to a plurality of ROI feature extraction models trained independently to extract the features of each ROI;
The first feature of each region of interest is applied to a first integrated analysis model previously learned as an input vector to calculate an analysis value of the bone image, and to integrate the bone image using the analysis value. Includes an analysis unit,
The first integrated analysis model
A plurality of ROIs extracted from a training image are applied to the ROI feature extraction model to generate one input vector, and a plurality of input vectors corresponding to the number of the training images are used as training data. Machine learning model,
The analysis value is
Bone age analysis apparatus comprising a bone value corresponding to a vector value representing the bone maturity grade or the probability of each bone age, or a regression analysis model.
The method of claim 9,
And a full image analyzing unit configured to calculate a second feature of the bone image by applying the bone image to the whole region feature extraction model trained as the learning image.
The integrated analysis unit
Calculating the analysis value of the bone image by applying the first and second features of the plurality of ROIs to a second integrated analysis model previously learned as an input vector,
The second integrated analysis model
A single input vector includes a feature calculated by applying the training image to the full region feature extraction model and a plurality of features calculated by applying the plurality of ROIs extracted from the training image to the ROI feature extraction model. A bone age analysis device, which is a machine learning model generated and trained using a plurality of input vectors corresponding to the number of learning images as learning data.
A bone age analysis program stored on a computer readable medium for carrying out any one of the methods of claims 1-8.

Images