KR20180064863A - SMI automatic analysis method of hand-wrist radiation images using deep learning - Google Patents

Abstract

The present invention relates to a method for automatically analyzing a skeleton maturity indicator (SMI) through a predetermined deep neural network (DNN) from a plurality of hand and wrist radiation images. According to an embodiment of the present invention, the method for automatically analyzing an SMI from hand and wrist radiation images using deep learning, comprises: a first step of dividing a plurality of patch images including regions of interest having a predetermined same size and shape from a plurality of input hand and wrist radiation images for learning; a second step of determining a weight to determine a region of interest to be an analysis target of an SMI among the patch images using the divided patch images in a predetermined first DNN model as learning data; a third step of applying the determined weight to a hand and wrist radiation image for analysis to be analyzed in a predetermined second DNN model, to extract the region of interest from the hand and wrist radiation image for analysis, and extracting feature information of the region of interest; and a fourth step of determining the SMI from an extracted feature point of the region of interest.

Description

[0001] The present invention relates to a method for automatically analyzing bone maturity (SMI) from a hand-woven radiograph using deep-

The present invention relates to a method of automatically analyzing skeletal maturity (SMI) from a hand-held radiographic image using deep running, and more particularly, to a method of automatically analyzing skeletal maturity (SMI) through a deep neural network (DNN) skeletal maturity indicators. < / RTI >

Bone Maturation (SMI) is an index to determine the biological age of a person and is used as an important criterion in performing medical diagnosis. Many studies have been carried out on the analysis of bone maturity using hand and arm radiographs.

In the past, the physician checked the radiographic image of the hand and arm and analyzed bone maturity based on accumulated personal experience and know-how. However, in this case, there was a problem in that there was a deviation in the analysis result for each doctor individual and it was difficult to make an accurate judgment.

In order to solve such a problem, a technique for analyzing bone maturity from a hand-held image using a computer program has been disclosed. For example, U.S. Patent No. 7,848,893 discloses a technique in which the intensity of an image corresponding to the shape of a bone is sampled from a bone radiographic image of a hand and set as a Principal Component Analysis (PCA) A method for evaluating the maturity value of an image is disclosed.

As another example, European Patent Publication No. 0626656 discloses a technique for extracting a feature that is relatively distinct from a source image and determining the maturity of the bone from a change in the pixel value of the extracted image, and Korean Patent Application Publication No. 2007- 0077319 describes the factors that include the anatomical features, structure, size, concentration, sharpness and shape of the growth plate of the bones that make up the wrist of the X-ray photographs. The calculated RUS score A method for predicting growth is disclosed.

However, in the above-described conventional techniques, the hand-held radiation image is analyzed using a known image processing technique to extract the feature points from the image intensity, structure, size, and pixel values, It is very difficult to set the criteria for bone maturity by mapping and analyzing bone maturity. In addition, since the inconsistency in the size and shape of the hand-arm part is inferior for each person, the difficulty of reading is very high, and the complicated reading is required for a large number of human radiation images in a short time. And the diagnosis period is prolonged.

United States Patent Application Publication No. 7848893 European Patent Publication No. EP0626656 Korean Patent Publication No. 10-2007-0077319

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is an object of the present invention to provide a dip analyzer capable of automatically analyzing skeletal maturity indicators (SMI) through a deep neural network (DNN) And to provide a method for automatically analyzing bone maturity from a human skull radiographic image.

A method for automatically analyzing skeletal maturity from a human skeleton radiographic image using deep learning according to an embodiment of the present invention includes a step of extracting a plurality of patch images including a region of interest having the same size and shape, A first step of distinguishing between; Determining a weight for determining a region of interest to be analyzed for bone maturity (SMI) among the patch images using a plurality of the patch images divided as a learning data in a predetermined Deep Neural Network (DNN) model ; Extracting a region of interest from the analytical weapon radiation image by applying the determined weight to the analytical weapon radiation image to be analyzed in a predetermined second deep neural network model and extracting the feature information of the ROI A third step; And a fourth step of determining bone maturity (SMI) from the extracted ROI feature points.

In the present invention, the region of interest includes, in the handpiece, a sesamoid of the thumb, a proximal phalanx of the stump, a middle phalanx of the stump, a distal phalanx of the stump, And is a middle node bone of the base.

In the present invention, the weights are determined according to the width of the ossification, the fusion, the golfer's capping, and the golfer's neck in the region of interest of the thumb bone, the first node bone of the thumb, the middle node bone, the end node bone, It is determined to determine a different level of region of interest.

According to an embodiment of the present invention, the first step may include: extracting images of the same size and shape predetermined from the input plurality of learning human-machine-vision radiographic images; And separating the extracted image into a patch image using a HOG feature and an SVM classifier as a region of interest of a seed bone, a bones, a radius, a background, and a handwarmer.

In the present invention, the first deep neural network model has a FCN (Fully Convolutional Network) structure.

In the present invention, the second step may include receiving a plurality of patch images through the first deep neural network model of the FCN structure, determining a plurality of ROIs, and determining a weight for each ROI to determine the ROIs do.

In the present invention, the third step may include: extracting the region of interest by assigning the determined weight to the inputted analyzing weapon radiation image in the reinforcement learning module; Extracting feature points from the analysis weapon radiation image in the feature extraction module; Extracting feature points for the region of interest by mapping the extracted regions of interest and feature points in a combiner; And determining bone maturity using the minutiae of the region of interest in the decision module.

According to the present invention, by using deep learning, it is possible to automatically perform radiological imaging diagnosis of high confidence in bone maturity.

In addition, according to the present invention, a deep neural network model is used to extract a region of interest that is an object of bone maturity determination in a hand-held radiological image, and the accuracy of bone maturity is improved from the feature information of the region of interest.

1 is a view illustrating an automatic analysis process of bone maturity from a hand-and-mouth radiographic image using the deep running according to the present invention.
FIG. 2 is a diagram illustrating an example of dividing a plurality of human-machine-vision radiation images into patch images according to an embodiment of the present invention.
3 is a structural diagram of a first deep neural network (DNN) model according to the present invention.
FIG. 4 is a structural diagram of a second deep neural network (DNN) model according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

FIG. 1 is a view illustrating a process of automatically analyzing skeletal maturity from a hand-held radiographic image using deep learning according to the present invention. FIG. 2 is a diagram illustrating an example of dividing a plurality of hand- to be.

Referring to FIG. 1, the process of automatically analyzing bone maturity according to the present invention includes a patch image classification process 110, a learning process 120 in a first deep neural network model, and a classification process 130 in a second deep neural network module. Lt; / RTI >

In the patch image segmentation process 110, a plurality of image data of a plurality of training images of the human and / or female parts of the human body, which are inputted as shown in FIG. 2, using various image processing techniques, Identify patch images. Specifically, the patch image segmentation process includes extracting images of the same size and shape predetermined from the plurality of input learning skull radiographic images, and extracting the extracted images using a Histogram of Oriented Gradient (HOG) feature and SVM (Support Vector Machine) The classifier is used to distinguish the patch image consisting of the region of interest of the seed bone, bones, radius, background, and handpiece.

In the present invention, the region of interest includes, for example, a sesamoid of a thumb, a proximal phalanx of a stop, a middle phalanx of a stop, a distal phalanx of a stop, It is set as the middle bones of the base. This is because these five sites are closely related to bone maturity and thus are considered as a key consideration in determining bone maturity. Of course, another region may be set as a region of interest.

The learning process 120 of the first deep neural network (DNN) model uses the first deep neural network (DNN) model to classify the patch images as the learning data, SMI) to determine the region of interest to be analyzed.

In a plurality of patch images classified as described above, an unnecessary region is included in the determination of bone maturity such as a background, so that the region of interest, which is a part to be considered when determining bone maturity, is determined. In this case, (weight). For example, such a weight may be determined by assigning a weight to a region of interest of a patch image input to a first deep neural network (DNN) model to determine a region of interest. If the region of interest is not a region of interest, .

In particular, in the present invention, as shown in FIG. 2, ossification, fusing, capping of a golfer's bone, and the like are performed on a seed bone of a thumb, a first node bone of a stop, a middle node bone, an end node bone, Weights are determined to determine different levels of ROI depending on the width of the goal.

Through the learning process 120 in the first deep neural network model, the weight w for determining the region of interest substantially in the handwritten region from a plurality of patch images is determined. This is because the input of the first deeper neural network is i and the output is y, the relation of y = f (i, w) using the function f of the first deep neural network model is determined. The output y is the region of interest output from the first deep neural network model, and the weight w must be determined such that it is substantially equal to the region of interest y 'identified in the incoming handwoven radiographic image. Thus, in the first deep neural network model, the weight w is determined to be y = y '.

The classification process 130 in the second deep neural network module receives the analytical arsenic radiation image to be analyzed and applies the weight w determined as described above to the second deep neural network model, And extracts feature information of the extracted region of interest. Here, the feature information is set in advance as information serving as a criterion for determining the bone maturity in the region of interest. The bone maturity (SMI) is determined from the extracted region feature information.

FIG. 3 is a structural diagram of a first deep neural network (DNN) model according to the present invention, and FIG. 4 is a structural diagram of a second deep neural network (DNN) model according to the present invention.

Referring to FIG. 3, the first deep neural network (DNN) model according to the present invention has a FCN (Fully Convolutional Network) structure in which vectorization is not performed in the middle layer. As described above, if the first deep neural network of the FCN structure is trained by an error backpropagation algorithm, the network can perform semantic segmentation on the handwarming radiation image. This can distinguish which part of the image space is the background and which part is the node bone, thereby extracting the region of interest from the packaged image. At this time, the weight w applied to extract the ROI is extracted. This weight w is also applied to the second deep neural network model.

Referring to FIG. 4, the second deep neural network (DNN) model according to the present invention includes a reinforcement learning model 131, a feature extraction module 132, a combination module 133, and a decision module 134.

The reinforcement learning module 131 receives the analytical arsenic radiation image to be analyzed and applies the weight w extracted from the first deep neural network model to the input analytical arsenic radiation image, We extract the region of interest to analyze bone maturity (SMI) for radiographic images. In the embodiment of the present invention, preferably, the reinforcement learning module 131 may have the same structure as the first deep neural network, and more preferably, it may have an FCN structure.

The feature extraction module 132 is a module that is connected in parallel with the reinforcement learning module 131 and performs functions independently. The feature extraction module 132 receives the analytical handheld radiation image to be analyzed and extracts the feature information from the handheld radiation image do. The feature extraction module 132 extracts the feature information from the entire association between the hand and the hand as the reference information for determining the bone maturity in the region of interest.

The combining module 133 maps the extracted results in the enhanced learning module 131 and the feature extraction module 132 to each other. That is, the feature information extracted from the enhancement learning module 131 is mapped to the feature information extracted from the feature extraction module 132 to extract feature information on the region of interest. This may be implemented using an operation that combines the activation maps of the reinforcement learning module 131 and the output of the feature extraction module 132 and may be implemented as a matrix concatenation But is not limited thereto.

The decision module 134 uses the feature information of the region of interest to determine bone maturity (SMI). This is to determine the bone maturity from the feature information, which is an index of bone maturity judgment, in the area of interest closely related to bone maturity in the hand - wrist.

 While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. Furthermore, the terms "comprises", "comprising", or "having" described above mean that a component can be implanted unless otherwise specifically stated, But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

131: Reinforcement Learning Model 132: Feature Extraction Module
133: Coupling module 134: Decision module

Claims (7)

A first step of dividing a plurality of patch images including a region of interest having the same size and shape, which are set in advance, from a plurality of input learning hand and arm radiographic images;
Determining a weight for determining a region of interest to be analyzed for bone maturity (SMI) among the patch images using a plurality of the patch images divided as a learning data in a predetermined Deep Neural Network (DNN) model ;
Extracting a region of interest from the analytical weapon radiation image by applying the determined weight to an analytical weapon radiation image to be analyzed in a predetermined second deep neural network model and extracting feature information of the ROI A third step; And
And a fourth step of determining a bone maturity level (SMI) from the extracted ROI minutiae points. 2. The apparatus of claim 1,
It is characterized by a sesamoid of the thumb, a proximal phalanx, a middle phalanx, a distal phalanx, and a middle node bone of the hand in the hand. A Method of Automatic Analysis of Bone Maturity from Human Skeletal Radiographic Images Using Deep Learning. 3. The method of claim 2,
Determine regions of interest at different levels depending on ossification, fusion, capping of the epiphysis, and width of the epiphysis in the region of interest of the bone of the thumb, the first node bone of the stop, the middle node bone, the tip node bone and the middle node of the base. A Method for Automatic Analysis of Bone Maturation from Human Brain Radiographic Images Using Deep Learning Determined. 2. The method according to claim 1,
Extracting images of the same size and shape as predetermined from the plurality of input learning and learning arms radiographic images; And
Dividing the extracted image into regions of interest of the seed bone, bone, radius, background, and handpiece using the HOG feature and the SVM classifier; A method for automatically analyzing bone maturity from a hand - arm radiographic image using deep - 2. The method of claim 1, wherein the first deep-
A Method of Automatic Analysis of Bone Maturation from Human Brain Radiographic Images Using Deep Learning with FCN (Fully Convolutional Network) Structure. 6. The method according to claim 5,
A plurality of patch images are received through the first deep neural network model of the FCN structure to determine a plurality of ROIs and a weight is determined for each ROI to determine the ROIs, Automatic analysis of bone maturity. 2. The method according to claim 1,
Extracting the region of interest by assigning the determined weight to the input analysis humanoid radiation image in the reinforcement learning module;
Extracting feature points from the analysis weapon radiation image in the feature extraction module;
Extracting feature points for the region of interest by mapping the extracted regions of interest and feature points in a combiner; And
Determining bone maturity using a feature point of the ROI in a decision module; A method for automatically analyzing bone maturity from a hand - arm radiographic image using deep -

Images