WO2019214052A1

WO2019214052A1 - Method for assessing bone age using x-ray image of hand, device, computer apparatus, and storage medium

Info

Publication number: WO2019214052A1
Application number: PCT/CN2018/095385
Authority: WO
Inventors: 王健宗; 吴天博; 刘新卉; 刘莉红; 马进; 肖京
Original assignee: 平安科技（深圳）有限公司
Priority date: 2018-05-08
Filing date: 2018-07-12
Publication date: 2019-11-14
Also published as: CN108968991A; CN108968991B

Abstract

The present invention provides a method of assessing a bone age using an X-ray image of a hand, a device, a computer apparatus, and a storage medium The method comprises: processing an X-ray image of a hand undergoing bone age assessment to obtain a hand bone image meeting a specified pixel requirement; inputting the hand bone image into a bone age assessment model based on a convolutional neural network, and performing computation; and acquiring a computation result output by the bone age assessment model, the result being a bone age of hand bones.

Description

Hand bone X-ray bone age assessment method, device, computer equipment and storage medium

This application claims priority to Chinese Patent Application No. 2018104331162, filed on May 8, 2018, entitled "Hand bone X-ray bone age assessment method, device, computer equipment and storage medium", The entire contents are incorporated herein by reference.

Technical field

The present invention relates to the field of computer technology, and in particular, to a bone age X-ray bone age assessment method, device, computer device and storage medium.

Background technique

Bone age assessment is widely used in the medical field to study and measure the growth and development of the human body and to diagnose diseases.

The existing bone age assessment method generally involves X-ray filming of the subject's hand and wrist, and then the doctor interprets it according to the taken X-ray film. Because of the different characteristics of the left hand bone at different ages, doctors can estimate bone age through these characteristics. Doctors generally use G-P mapping and TW3 scoring when diagnosing X-ray films. However, when the G-P map method is evaluated, there are inaccurate problems; and the TW3 scoring method requires the doctor to subjectively judge with empirical knowledge, and the evaluation result is easily affected by other factors, resulting in inaccurate evaluation.

Therefore, providing a new bone age assessment method has become an urgent problem to be solved.

technical problem

The main object of the present invention is to provide a bone age X-ray slice bone age assessment method, apparatus, computer device and storage medium that automatically perform bone age assessment and has high evaluation accuracy.

Technical solution

The invention provides a bone bone X-ray bone age assessment method, which comprises:

Hand bone x-ray film of the bone age to be predicted is processed into a photo of the hand bone required by the specified pixel;

Inputting the hand bone photograph into a preset bone age assessment model based on a convolutional neural network for calculation;

Obtaining a calculation result output by the bone age assessment model, the result being the bone age of the hand bone.

The hand bone X-ray bone age estimating device proposed by the invention comprises:

a first processing unit, configured to process a hand bone X-ray film of a bone age to be predicted into a photo of a hand bone required by a specified pixel;

a calculating unit, configured to input the photo of the hand bone into a preset bone age estimation model based on a convolutional neural network for calculation;

And an output unit, configured to obtain a calculation result output by the bone age evaluation model, where the result is a bone age of the hand bone.

The computer device of the present invention comprises a memory and a processor, the memory storing computer readable instructions, wherein the processor implements the steps of the method when the computer readable instructions are executed.

The computer non-volatile readable storage medium of the present invention has stored thereon computer readable instructions, wherein the computer readable instructions are executed by a processor.

Beneficial effect

The beneficial effects of the present invention are: processing a hand bone x-ray of a bone age to be predicted into a photo of a hand bone required by a specified pixel; and inputting the hand bone photo into a preset bone age estimation model based on a convolutional neural network for calculation Obtaining a calculation result outputted by the bone age evaluation model, the result is a bone age of the hand bone; and a bone age assessment model based on a convolutional neural network can automatically perform bone age assessment, and the evaluation accuracy is high.

DRAWINGS

1 is a schematic diagram showing the steps of a bone age X-ray film bone age evaluation method according to an embodiment of the present invention;

2 is a schematic diagram showing the steps of a method for assessing the bone age of a hand bone X-ray film according to another embodiment of the present invention;

3 is a structural block diagram of a bone bone X-ray bone age estimating device according to an embodiment of the present invention;

4 is a structural block diagram of a calculation unit of a bone bone X-ray bone age estimating device according to an embodiment of the present invention;

FIG. 5 is a structural block diagram of a bone bone X-ray bone age estimating apparatus according to another embodiment of the present invention; FIG.

FIG. 6 is a schematic block diagram showing the structure of a computer device according to an embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Referring to FIG. 1, the bone age X-ray slice bone age estimation method in the embodiment includes:

Step S1, processing the hand bone X-ray film of the bone age to be predicted into a photo of the hand bone required by the specified pixel;

Step S2, inputting the photo of the hand bone into a preset bone age estimation model based on a convolutional neural network for calculation;

Step S3: Acquire a calculation result output by the bone age assessment model, and the result is a bone age of the hand bone.

In the step S1, the bone bone X-ray evaluation method of the hand bone in the embodiment needs to obtain the hand bone X-ray film of the bone age to be predicted, and specifically, the left hand bone X-ray film needs to be acquired, because the age is different. The left hand bone has different characteristics, so the age can be accurately estimated according to the different characteristics of the left hand bone X-ray film. When it is necessary to accurately assess the age through the hand bone X-ray film, for example, the insurance company needs to estimate the insurance coverage according to the age of the insured, that is, the preset bone age assessment model based on the convolutional neural network can be used according to the left hand bone X-ray film. Quickly calculate the bone age of the hand bone. The preset bone age assessment model based on convolutional neural network needs to be trained through a large amount of hand bone X-ray data. The trained bone age assessment model can output the calculation result of the input hand bone photo, and the result is the above hand bone. Bone age. The bone age assessment model based on convolutional neural network requires a hand-size X-ray film of a specified size. Therefore, before the hand bone X-ray film is input into a preset bone age estimation model based on the convolutional neural network, it needs to be treated. The hand bone X-ray film for predicting bone age is processed into a photo of the hand bone required by the specified pixel; wherein the specific treatment method is to treat the hand bone X-ray film of the bone age to be predicted while maintaining the aspect ratio, firstly the hand bone X The maximum dimension of the light sheet is adjusted to 256 pixels. It should be pointed out that when the hand bone X-ray film is rectangular, the length of the long side is first adjusted to 256 pixels, and then the shorter side of the X-ray film of the bone is edge-added, so that the hand bone X-ray film becomes 256*. A photo of the hand bone of 256-size pixels, the hand bone photograph will be taken as a hand bone photograph calculated into a preset bone age estimation model based on a convolutional neural network. Preferably, the above-mentioned hand bone photograph can also be normalized and then input into a bone age estimation model based on a convolutional neural network. The normalization method can be processed by the normalize function in opencv, and the above-mentioned hand bone photographs are normalized to a mean value of 0, and the variance is 1. The purpose is to make the hand bone photographs have similar statistical distribution, which is convenient for volume-based The bone photo of the bone age assessment model of the neural network is processed, and the convergence of the bone age assessment model based on the convolutional neural network can be accelerated.

In step S2, the hand bone x-ray film of the bone age to be predicted is processed into a hand bone photo required by the specified pixel, and then input into a preset bone age estimation model based on a convolutional neural network for calculation, wherein the preset is based on convolution The bone age assessment model of the neural network needs to be trained through a large amount of hand bone X-ray data. The trained bone age assessment model based on the convolutional neural network can output the calculation result of the input hand bone photograph, and the result is the above hand bone. Bone age. After the bone age assessment model based on the convolutional neural network is successfully trained, after inputting the photo of the hand bone required by the specified pixel, the bone age of the hand bone is calculated based on the bone age evaluation model of the convolutional neural network.

In step S3, the display device obtains the calculation result output by the bone age evaluation model, and the calculation result is the bone age of the hand bone, and the bone age of the hand bone is displayed by the display device or printed by the printing device.

In a specific embodiment, the insurance company needs to assess the amount of insurance coverage based on the age of the policyholder. First, the staff of the insurance company asks the insured to first fill in the personal information of the insured, including personal information such as the age, occupation, income and address of the insured, because the insured amount needs to be assessed according to the age of the insured, therefore, The accuracy of the insured's age is very important. In order to verify the accuracy of the age filled by the insured, the insurance company's staff will guide the insured to first collect the X-ray film of the insured's left hand bone through the X-ray machine equipment, and input the X-ray film of the insured's left hand bone into the left hand bone. X-ray films were evaluated for age in devices that were evaluated for age. The pre-stored program in the device processes the input left-hand bone X-ray film into a photo of the hand bone required by the specified pixel; and calculates the image of the opponent bone in the bone age estimation model based on the convolutional neural network; and obtains the output of the bone age evaluation model Calculating the result, the result is the bone age of the hand bone, and comparing the bone age of the hand bone with the age filled by the insured person, so as to obtain whether the age filled in by the insured is true and accurate, it is necessary to point out that when the insured The age and the age error detected by the device are less than 0.8 years, which means that the age filled by the policyholder is true and accurate.

The method for assessing the bone age of the hand bone X-ray in the embodiment, the step S2 of inputting the photo of the hand bone into a preset bone age estimation model based on a convolutional neural network, comprising:

Step S21, performing convolution calculation on the photo of the hand bone to obtain a first picture feature;

Step S22, performing multiple convolution calculation on the first picture feature to obtain a second picture feature;

Step S23, performing spatial transformation and alignment processing on the second picture feature by using a spatial transformation network to obtain a third picture feature;

Step S24, performing convolution calculation on the third picture feature to obtain a fourth picture feature;

Step S25, the fourth picture features are combined by a fully connected layer to form a global picture feature, thereby outputting a calculation result.

In step S21, in the bone age estimation model based on the convolutional neural network in the embodiment, for the input hand bone photo, a convolution calculation of the bone photo is required to obtain a low-dimensional image feature as the first picture. Features, specifically, the Overfeat network is used as a convolutional layer of the bone image for image feature extraction; when the features of the hand bone photo are extracted through the Overfeat network, a large dimension of the image feature is obtained, in order to facilitate multiple convolution The calculation is performed by performing a dimensionality reduction on the extracted image features by using a pooling layer to obtain a first picture feature, where the pooling layer can be processed by using a maximum pool or an average pool, in this embodiment. The processing is performed by taking the maximum pooling method.

In step S22, the bone age estimation model based on the convolutional neural network in the embodiment performs multiple iterative convolution calculations on the first picture feature to obtain a second picture feature. Specifically, it is necessary to perform three iterative convolution calculations on the first picture feature to obtain a high-dimensional picture feature, wherein each convolution calculation processes the first picture feature through a convolution layer, and then passes through a pooling The layer performs the dimension reduction processing on the extracted image features, where the pooling layer can be processed by using the maximum pooling or the average pooling method. In this embodiment, the processing is performed by taking the maximum pooling method. The picture features obtained by one convolution calculation are iteratively convoluted, thereby obtaining a high-dimensional second picture feature by performing three iterative convolution calculations on the first picture feature.

In step S23, since the hand bone X-ray film is affected by factors such as exposure time and shooting angle during the shooting process, the spatial difference of the hand bone X-ray film is large, and the bone age is based on the convolutional neural network. When the evaluation model is calculated, when the space difference corresponding to the second picture feature outputted by the convolution calculation is large, the second picture feature needs to be spatially transformed and aligned, so the space of the second picture feature is spatially transformed by the spatial transformation network. Transformation and alignment can make the calculation results of the bone age assessment model based on convolutional neural network more accurate. The spatial transformation network in this embodiment needs to first estimate six transformation parameters in the spatial transformation network, and can adaptively transform and align the second image features according to the six parameters, and the specific operations include translation, scaling, and rotation. And other geometric transformations, etc. The above six parameters can be estimated by the Backpropagation algorithm (backpropagation algorithm). According to the estimated six parameters, the second picture feature can be spatially transformed and aligned through the spatial transformation network to obtain the third picture feature. The spatial transformation network is added to the bone age assessment model based on convolutional neural network to reduce the influence of the spatial difference of the hand bone X-rays on the evaluation results of the bone age assessment model based on the convolutional neural network, so that the convolutional neural network is based on the convolutional neural network. The calculation results of the bone age assessment model output are more accurate.

In step S24, the bone age estimation model based on the convolutional neural network in the embodiment performs spatial transformation and alignment on the second picture feature through the spatial transformation network to obtain a third picture feature, before processing through the fully connected layer. Convolution calculation is needed. Specifically, a convolution calculation is performed on the third picture feature to extract picture features, and then the extracted picture features are subjected to dimensionality reduction through a pooling layer, wherein the pooling layer can The processing is performed by means of the maximum pooling or the average pooling. In this embodiment, the processing is performed by taking the maximum pooling. The fourth picture feature is obtained through a convolution calculation, thereby facilitating inputting the fourth picture feature into the fully connected layer for processing.

In step S25, since the fourth picture feature obtained by the convolution calculation is a partial picture feature, the fourth picture feature needs to be combined by the fully connected layer to form a global picture feature, and finally the hand bone is calculated according to the global picture feature. Bone age.

The method for training a bone age assessment model based on a convolutional neural network in this embodiment includes:

Obtaining a specified amount of sample data, and dividing the sample data into a training set and a test set, wherein the sample data includes a photo of a hand bone of a known bone age, and bone age data corresponding to the hand bone photo of the known bone age;

The sample data of the training set is input into a preset convolutional neural network for training, and a result training model is obtained;

Verifying the result training model using sample data of the test set;

If the verification passes, the result training model is recorded as the bone age estimation model based on the convolutional neural network.

For the bone age assessment model based on convolutional neural network, it can only be used to calculate the bone age of the hand bone after the training is completed. In the training of the bone age assessment model based on the convolutional neural network, a large amount of sample data needs to be acquired, and the sample data is divided into a training set and a test set, wherein the sample data includes a photo of the hand bone of a known bone age, and The bone age data corresponding to the hand bone photograph of the above known bone age. The sample data of the above training set is input into a preset convolutional neural network for training, and a result training model for performing bone age assessment is obtained.

For the training result model obtained by the training, the hand bone photograph of the known bone age in the sample data of the test set is input to the bone age prediction result of the hand bone photograph predicted by the result training model, and the hand bone in the sample data of the test set is passed The actual bone age result of the photograph is compared with the predicted bone age of the hand bone photograph predicted by the result training model to verify whether it is within the preset error range, specifically, the bone age of the hand bone photograph predicted by the result training model. The predicted value is the difference between the predicted bone age of the hand bone photo calculated by the Euclidean loss layer and the true value of the bone age of the hand bone photograph. The calculation formula is

In the formula, pred is the predicted bone age of the hand bone photograph, and the true value of the bone age of the hand bone photograph is taken. The difference between the predicted bone age of the hand bone photograph and the true value of the bone age of the hand bone photograph is measured by the Euclidean loss layer. When the value calculated by the Euclidean loss layer is less than the preset value, the verification is passed, and the result training model can be used as the above-mentioned bone age estimation model based on the convolutional neural network. In the bone age estimation model based on the convolutional neural network in this embodiment, when the bone age prediction value of the hand bone photograph predicted by the result training model and the bone age true value error of the hand bone photograph are less than 0.8 years, the determination is based on the convolutional nerve The training of the bone age assessment model of the network is completed.

Referring to FIG. 2, a method for assessing the skeletal age of the hand bone X-ray in another embodiment, before the step S21 of performing convolution calculation on the hand bone photograph to obtain the first picture feature, further includes:

Step S201, performing data augmentation processing on the hand bone photograph.

In step S201, since the bone age estimation model based on the convolutional neural network is trained, data enrichment can be performed on the input hand bone photograph, that is, m equally spaced n is uniformly extracted in each input hand bone photograph. The block area of *n adds all the extracted block areas to the training data, thereby increasing the size of the training set, effectively avoiding the occurrence of over-fitting in the training process and improving the training effect. When the bone age assessment model based on the convolutional neural network is completed, when the bone age assessment model is performed by the convolutional neural network, the data can be augmented by inputting the hand bone photograph, and the training set can be increased. The size is not only effective to avoid the occurrence of over-fitting in the calculation process, but also improve the accuracy of the calculation.

The method for assessing the bone age of the hand bone X-ray film in the embodiment, before the step S1 of processing the hand bone x-ray film of the bone age to be predicted into the hand bone photo required by the specified pixel, includes:

In step S103, the bones of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film are selected as the hand bone X-ray film of the bone age to be predicted.

In step S103, since the bones in the hand bone, the metaphysis, and the bone at the wrist are the most characteristically differentiated bone portions for age assessment, only the hand bone x-rays of the candidate bone age to be predicted may be selected. These characteristic bone parts serve as input pictures through a bone age assessment model based on convolutional neural networks. Specifically, the feature area model based on the deep network can be used to select the bone age feature regions such as the skeleton, the dry segment and the wrist at the wrist by means of marking, and the calculation amount can be reduced and the efficiency can be improved without affecting the bone age prediction result. The specific steps are: firstly, the hand bone X-ray film of the bone age to be predicted is uniformly scaled to a fixed size, for example, to a size of 1024*1024 pixels; secondly, according to the bone age evaluation TW3 method, the hand bone X-ray film to be predicted is marked. The coordinates of the bone-age feature area such as the epiphysis, the dry segment and the bone at the wrist are saved and corresponding to the bounding box; and the labeled X-ray film to be predicted corresponding to the marked bounding box coordinates Data enhancement is performed, and the data enhancement manner includes graphic operations such as rotation, mirror flipping, scaling, and panning. Accordingly, the marked bounding box coordinates also need to undergo the same processing. Finally, the data-enhanced bounding box coordinates and the labeled hand bone X-rays to be predicted are input into the deep network as training data to train the deep network-based feature region model. Specifically, the bounding box is Coordinates (4 values) are used as training tags to train the feature region model based on the deep network. After the training, the feature region model based on the depth network can automatically select the bone part of the hand bone X-ray feature to be predicted. The coordinates of the X-ray image of the characteristic bone portion of the hand bone X-ray film to be predicted can be selected according to the coordinates of the characteristic bone portion.

In the method for assessing the skeletal age of the hand bone X-ray film in the embodiment, the step S103 of selecting the bone of the epiphysis, the metaphyseal end and the wrist at the wrist of the candidate hand bone X-ray film as the hand bone X-ray film of the bone age to be predicted includes:

Step S102, adjusting the contrast of the candidate hand bone X-ray film.

Before the adjusting the contrast S102 of the candidate hand bone X-ray film, the method includes:

In step S101, the background portion of the candidate hand bone X-ray film is unified into black.

In step S101, before the bones of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film are selected as the hand bone x-rays of the bone age to be predicted, the candidate hand bone X-ray film can also be adjusted. The contrast is such that the picture features in the hand bone X-ray film are more obvious, so that the training model based on the bone age estimation model based on the convolutional neural network is more efficient and the evaluation result is more accurate. Before adjusting the contrast of the optional hand bone X-ray film, since the background portion of the alternative hand bone X-ray film may contain a little other color in addition to black, it is necessary to first unify the background of the hand bone X-ray film to be predicted. Into black. The specific method is to first select a pixel block of a certain size at four angular positions of the hand bone X-ray film to be predicted, for example, a pixel block of 10*10 pixels, calculate the mean value of the four pixel blocks, and then calculate The obtained mean value is compared with half of the maximum pixel value that can be achieved by the hand bone X-ray film of the bone age to be predicted, and the X-ray film is normalized to 0 to the maximum pixel value, thereby realizing the hand bone X to be predicted. The background of the light sheet is unified into black.

In step S102, after the background portion of the hand bone radiograph of the bone age to be predicted is processed to be black, the step of adjusting the contrast of the hand bone x-ray film of the candidate bone age to be predicted may be performed. It should be pointed out that when the hand bone X-ray film of the bone age to be predicted is a three-channel picture, the three-channel picture needs to be grayed out first, and the component method, the maximum value method, and the average can be used. The method of the value method and the weighted average method performs grayscale processing on the X-ray film. After the gray scale processing of the hand bone X-ray film for predicting the bone age, the contrast of the hand bone X-ray film of the candidate bone age to be predicted is adjusted; the specific method is to adopt the contrast contrast adaptive histogram equalization algorithm (CLAHE algorithm). To adjust the contrast of the hand bone X-ray film of the candidate bone age to be predicted, wherein the contrast adaptive histogram equalization algorithm (CLAHE algorithm) adopts a histogram of the adaptive trim image, and then uses the trimmed histogram to predict the bone age. The balance adjustment of the hand bone radiograph has the advantage of making the contrast of the hand bone x-ray of the candidate bone age to be predicted more natural.

In summary, the hand bone X-ray film of the bone age to be predicted is processed into a photo of the hand bone required by the specified pixel; the hand bone photo is input into a preset bone age estimation model based on the convolutional neural network for calculation; The calculation result output by the bone age evaluation model is the bone age of the hand bone; the bone age assessment model based on the convolutional neural network can automatically perform bone age assessment, and the evaluation accuracy is high; and the feature area based on the depth network is utilized The model selects the bone age feature areas such as the epiphysis, the dry segment and the bone at the wrist, and reduces the calculation amount and improves the efficiency without affecting the bone age prediction result.

Referring to FIG. 3, the bone bone X-ray aging apparatus of the present embodiment includes:

a first processing unit 10, configured to process a hand bone x-ray film of a bone age to be predicted into a photo of a hand bone required by a specified pixel;

The calculating unit 20 is configured to input the photo of the hand bone into a preset bone age estimation model based on a convolutional neural network for calculation;

The output unit 30 is configured to obtain a calculation result output by the bone age evaluation model, and the result is a bone age of the hand bone.

In the embodiment, the bone bone X-ray aging apparatus of the hand bone needs to obtain the hand bone X-ray film of the bone age to be predicted, and specifically, the left hand bone X-ray film needs to be obtained, because the left hand bone is different at different age stages. Features, so the age can be accurately assessed based on the different characteristics of the left-handed X-ray film taken. When it is necessary to accurately assess the age through the hand bone X-ray film, for example, the insurance company needs to estimate the insurance coverage according to the age of the insured, that is, the preset bone age assessment model based on the convolutional neural network can be used according to the left hand bone X-ray film. Quickly calculate the bone age of the hand bone. The preset bone age assessment model based on convolutional neural network needs to be trained through a large amount of hand bone X-ray data. The trained bone age assessment model can output the calculation result of the input hand bone photo, and the result is the above hand bone. Bone age. The bone age assessment model based on convolutional neural network requires a hand-size X-ray film of a specified size. Therefore, before the hand bone X-ray film is input into a preset bone age estimation model based on the convolutional neural network for calculation, the first process is performed. The unit 10 processes the hand bone x-ray film of the bone age to be predicted into a photo of the hand bone required by the specified pixel; wherein the specific processing method is that the hand bone X-ray film of the bone age to be predicted is kept unchanged in the aspect ratio, Adjust the maximum dimension of the hand bone X-ray to 256 pixels. It should be pointed out that when the hand bone X-ray film is rectangular, the length of the long side is first adjusted to 256 pixels, and then the shorter side of the X-ray film of the bone is edge-added, so that the hand bone X-ray film becomes 256*. A photo of the hand bone of 256-size pixels, the hand bone photograph will be taken as a hand bone photograph calculated into a preset bone age estimation model based on a convolutional neural network. Preferably, the above-mentioned hand bone photograph can also be normalized and then input into a bone age estimation model based on a convolutional neural network. The normalization method can be processed by the normalize function in opencv, and the above-mentioned hand bone photographs are normalized to a mean value of 0, and the variance is 1. The purpose is to make the hand bone photographs have similar statistical distribution, which is convenient for volume-based The bone photo of the bone age assessment model of the neural network is processed, and the convergence of the bone age assessment model based on the convolutional neural network can be accelerated.

The calculating unit 20 processes the hand bone x-ray film of the bone age to be predicted into a hand bone photo required by the specified pixel, and inputs it into a preset bone age estimation model based on a convolutional neural network for calculation, wherein the preset convolutional neural network is used. The bone age assessment model needs to be trained through a large amount of hand bone X-ray data. The trained bone age assessment model based on convolutional neural network can output the calculation result of the input hand bone photograph, which is the bone age of the above hand bone. After the bone age assessment model based on the convolutional neural network is successfully trained, after inputting the photo of the hand bone required by the specified pixel, the bone age of the hand bone is calculated based on the bone age evaluation model of the convolutional neural network.

The output unit 30 obtains the calculation result output by the bone age evaluation model, and the calculation result is the bone age of the hand bone, and controls the bone age of the hand bone to be displayed through the display device or printed by the printing device.

Referring to FIG. 4, the hand bone X-ray bone age estimating device in the embodiment, the calculating unit 20 includes:

The first processing module 210 is configured to perform convolution calculation on the hand bone photo to obtain a first picture feature;

The second processing module 220 is configured to perform multiple iterative convolution calculations on the first picture feature to obtain a second picture feature.

The transform module 230 is configured to perform spatial transformation and alignment processing on the second picture feature by using a spatial transformation network to obtain a third picture feature;

a third processing module 240, configured to perform convolution calculation on the third picture feature to obtain a fourth picture feature;

The executing module 250 is configured to combine the fourth picture features together by the fully connected layer to form a global picture feature, thereby outputting a calculation result.

In the skeletal age estimation model based on the convolutional neural network in the embodiment, for the input hand bone photo, the first processing module 210 is configured to perform a convolution calculation on the first bone photo to obtain a low-dimensional image feature as the first Picture features, specifically, use the Overfeat network as a convolutional layer of the bone image for image feature extraction; when extracting the features of the hand bone photo through the Overfeat network, a large dimensional image feature will be obtained, in order to facilitate multiple volumes. The product is calculated by using a pooling layer to perform the dimension reduction processing on the extracted image features. The pooling layer can be processed by using the maximum pooling or the average pooling method. In this embodiment, Specifically, the method of taking the maximum pool is adopted.

In the skeletal age estimation model based on the convolutional neural network in the embodiment, the second processing module 220 is configured to perform a plurality of iterative convolution calculations on the first picture feature to obtain a second picture feature. Specifically, it is necessary to perform three iterative convolution calculations on the first picture feature to obtain a high-dimensional picture feature, wherein each convolution calculation processes the first picture feature through a convolution layer, and then passes through a pooling The layer performs the dimension reduction processing on the extracted image features, where the pooling layer can be processed by using the maximum pooling or the average pooling method. In this embodiment, the processing is performed by taking the maximum pooling method. The picture features obtained by one convolution calculation are iteratively convoluted, thereby obtaining a high-dimensional second picture feature by performing three iterative convolution calculations on the first picture feature.

Since the X-ray film of the hand bone is affected by factors such as exposure time and shooting angle during the shooting process, the spatial difference of the hand bone X-ray film is large, and is calculated by the bone age estimation model based on the convolutional neural network. If the spatial difference corresponding to the second picture feature outputted by the convolution calculation is large, the second picture feature needs to be spatially transformed and aligned, and the transformation module 230 spatially transforms and aligns the second picture feature through the spatial transformation network. It can make the calculation result of the bone age assessment model based on convolutional neural network more accurate. The spatial transformation network in this embodiment needs to first estimate six transformation parameters in the spatial transformation network, and can adaptively transform and align the second image features according to the six parameters, and the specific operations include translation, scaling, and rotation. And other geometric transformations, etc. The above six parameters can be estimated by the Backpropagation algorithm (backpropagation algorithm). According to the estimated six parameters, the second picture feature can be spatially transformed and aligned through the spatial transformation network to obtain the third picture feature. The spatial transformation network is added to the bone age assessment model based on convolutional neural network to reduce the influence of the spatial difference of the hand bone X-rays on the evaluation results of the bone age assessment model based on the convolutional neural network, so that the convolutional neural network is based on the convolutional neural network. The calculation results of the bone age assessment model output are more accurate.

The bone age estimation model based on the convolutional neural network in this embodiment performs spatial transformation and alignment on the second picture feature through the spatial transformation network to obtain a third picture feature. Before processing through the fully connected layer, the third processing module 240 Performing convolution calculation on the third picture feature, in particular, performing a convolution calculation on the third picture feature to extract the picture feature, and then performing dimension reduction processing on the extracted picture feature through a pooling layer, where The pooling layer can be processed in the manner of the maximum pooling or the average pooling. In this embodiment, the processing is performed by taking the maximum pooling method. The fourth picture feature is obtained through a convolution calculation, thereby facilitating inputting the fourth picture feature into the fully connected layer for processing.

The fourth picture feature obtained by the convolution calculation is a partial picture feature, and the executing module 250 is configured to combine the fourth picture features to form a global picture feature through the fully connected layer, and finally calculate the hand bone according to the global picture feature. Bone age.

Verifying the result training model using sample data of the test set;

In the embodiment, the hand bone X-ray bone age estimating device, the calculating unit 20, further includes:

The augmentation module 260 is configured to perform data augmentation processing on the hand bone photo.

Since the augmentation module 260 is used for training the bone model based on the convolutional neural network, the augmentation module 260 can be used for data augmentation of the input hand bone photograph, that is, uniformly extracting m equal intervals in each input hand bone photograph. The n*n block area adds all the extracted block areas to the training data, thereby increasing the size of the training set, effectively avoiding the occurrence of over-fitting in the training process and improving the training effect. When the bone age assessment model based on the convolutional neural network is completed, when the bone age assessment model is performed by the convolutional neural network, the data can be augmented by inputting the hand bone photograph, and the training set can be increased. The size is not only effective to avoid the occurrence of over-fitting in the calculation process, but also improve the accuracy of the calculation.

Referring to FIG. 5, the bone bone X-ray aging apparatus of the present embodiment further includes:

The selecting unit 103 is configured to select the bones of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film as the hand bone X-ray of the bone age to be predicted.

Since the bones in the hand bone, the metaphyseal end and the bone at the wrist are the most characteristically differentiated bone parts for age assessment, only the characteristic bone parts can be selected in the hand bone x-ray of the candidate bone age to be predicted. Input pictures through a bone age assessment model based on convolutional neural networks. Specifically, the selection unit 103 can select the bone age feature regions such as the skeleton, the dry segment and the skeleton at the wrist through the feature region model of the depth network by means of marking, and reduce the calculation amount without affecting the bone age prediction result. ,Improve efficiency. The specific steps are: firstly, the hand bone X-ray film of the bone age to be predicted is uniformly scaled to a fixed size, for example, to a size of 1024*1024 pixels; secondly, according to the bone age evaluation TW3 method, the hand bone X-ray film to be predicted is marked. The coordinates of the bone-age feature area such as the epiphysis, the dry segment and the bone at the wrist are saved and corresponding to the bounding box; and the labeled X-ray film to be predicted corresponding to the marked bounding box coordinates Data enhancement is performed, and the data enhancement manner includes graphic operations such as rotation, mirror flipping, scaling, and panning. Accordingly, the marked bounding box coordinates also need to undergo the same processing. Finally, the data-enhanced bounding box coordinates and the labeled hand bone X-rays to be predicted are input into the deep network as training data to train the deep network-based feature region model. Specifically, the bounding box is Coordinates (4 values) are used as training tags to train the feature region model based on the deep network. After the training, the feature region model based on the depth network can automatically select the bone part of the hand bone X-ray feature to be predicted. The coordinates of the X-ray image of the characteristic bone portion of the hand bone X-ray film to be predicted can be selected according to the coordinates of the characteristic bone portion.

The bone bone X-ray aging apparatus of the present embodiment includes:

The unifying unit 101 is configured to unify the background portion of the candidate hand bone X-ray film into black.

The second processing unit 102 is configured to adjust a contrast of the candidate hand bone X-ray film.

The second processing unit 102 is configured to adjust the candidate hand bone X-ray before selecting the bone of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray as the hand bone x-ray of the bone age to be predicted. Contrast, to make the picture features in the hand bone X-ray film more obvious, so that the training results by the convolutional neural network-based bone age assessment model are more efficient and the evaluation results are more accurate. Before adjusting the contrast of the alternative hand bone X-ray film, since the background portion of the alternative hand bone X-ray film may contain a little other color in addition to black, the unified unit 101 is used to unify the hand bone X-ray to be predicted. The background of the piece is black. The specific method is to first select a pixel block of a certain size at four angular positions of the hand bone X-ray film to be predicted, for example, a pixel block of 10*10 pixels, calculate the mean value of the four pixel blocks, and then calculate The obtained mean value is compared with half of the maximum pixel value that can be achieved by the hand bone X-ray film of the bone age to be predicted, and the X-ray film is normalized to 0 to the maximum pixel value, thereby realizing the hand bone X to be predicted. The background of the light sheet is unified into black.

After processing the background portion of the hand bone radiograph of the bone age to be predicted as black, the second processing unit 102 adjusts the contrast of the hand bone x-ray film of the candidate bone age to be predicted. It should be pointed out that when the hand bone X-ray film of the bone age to be predicted is a three-channel picture, the three-channel picture needs to be grayed out first, and the component method, the maximum value method, and the average can be used. The method of the value method and the weighted average method performs grayscale processing on the X-ray film. After the gray scale processing of the hand bone X-ray film for predicting the bone age, the contrast of the hand bone X-ray film of the candidate bone age to be predicted is adjusted; the specific method is to adopt the contrast contrast adaptive histogram equalization algorithm (CLAHE algorithm). To adjust the contrast of the hand bone X-ray film of the candidate bone age to be predicted, wherein the contrast adaptive histogram equalization algorithm (CLAHE algorithm) adopts a histogram of the adaptive trim image, and then uses the trimmed histogram to predict the bone age. The balance adjustment of the hand bone radiograph has the advantage of making the contrast of the hand bone x-ray of the candidate bone age to be predicted more natural.

Referring to FIG. 6, a computer device is also provided in the embodiment of the present invention. The computer device may be a server, and its internal structure may be as shown in FIG. 6. The computer device includes a processor, memory, network interface, and database connected by a system bus. Among them, the computer designed processor is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The memory provides an environment for the operation of operating systems and computer readable instructions in a non-volatile storage medium. The database of the computer device is used for data such as a preset convolutional neural network based X-ray bone age assessment model. The network interface of the computer device is used to communicate with an external terminal via a network connection. The computer readable instructions are executed by the processor to implement a hand bone X-ray bone age assessment method.

The processor performs the above steps of the skeletal age evaluation method of the hand bone X-ray: processing the hand bone x-ray film of the bone age to be predicted into a photo of the hand bone required by the specified pixel; and inputting the hand bone photo into the preset convolution-based convolution Calculating in the bone age assessment model of the neural network; obtaining a calculation result output by the bone age assessment model, the result being the bone age of the hand bone.

The above computer equipment, based on a convolutional neural network, establishes a bone age assessment model based on a convolutional neural network, and processes the hand bone photo of the hand bone to be predicted into a specified pixel requirement, and inputs the hand bone photo to the preset. Calculating in the bone age estimation model based on the convolutional neural network, obtaining the calculation result outputted by the bone age evaluation model, the result is the bone age of the hand bone, and the bone age estimation model based on the convolutional neural network can automatically perform the bone age Evaluation and high accuracy of evaluation.

In one embodiment, the step of inputting the hand bone photo into a preset convolutional neural network-based bone age assessment model comprises: performing convolution calculation on the hand bone photo to obtain a first picture feature. Performing a plurality of iterative convolution calculations on the first picture feature to obtain a second picture feature; spatially transforming and aligning the second picture feature by a spatial transform network to obtain a third picture feature. Performing convolution calculation on the third picture feature to obtain a fourth picture feature; combining the fourth picture features by a fully connected layer to form a global picture feature, thereby outputting a calculation result.

In one embodiment, the processor, before performing the convolution calculation on the hand bone photo to obtain the first picture feature, includes: performing data augmentation processing on the hand bone photo to increase the size of the training set. Effectively avoid the occurrence of over-fitting in the calculation process, and improve the accuracy of the calculation.

In one embodiment, before the step of processing the hand bone x-ray of the bone age to be predicted into the photo of the hand bone required by the specified pixel, the method comprises: selecting the bone of the hand bone, the metaphysis, and the bone at the wrist as the X-ray of the hand bone The hand bone X-ray film of the bone age to be predicted is used to select the bone age feature regions such as the epiphysis, the dry segment and the wrist at the wrist by using the feature area model based on the depth network, and reduce the bone age prediction result without affecting the bone age prediction result. Calculate the amount and increase the efficiency.

In one embodiment, prior to the step of selecting the bones of the bones, the metaphyseal ends, and the wrists in the hand bone X-ray film as the hand bone x-ray film of the bone age to be predicted, the method comprises: adjusting the contrast of the hand bone X-ray film.

In one embodiment, the adjusting the contrast of the hand bone X-ray film comprises: unifying the background portion of the hand bone X-ray film into black.

It will be understood by those skilled in the art that the structure shown in FIG. 6 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.

An embodiment of the present invention further provides a computer non-volatile readable storage medium having stored thereon computer readable instructions, and when the computer readable instructions are executed by the processor, a bone bone X-ray bone age assessment method is implemented. The hand bone x-ray film of the bone age to be predicted is processed into a photo of the hand bone required by the specified pixel; the hand bone photo is input into a preset bone age estimation model based on a convolutional neural network for calculation; and the bone age is obtained. The calculation result of the model output is evaluated, and the result is the bone age of the hand bone.

The above computer non-volatile readable storage medium is based on a convolutional neural network to establish a bone age estimation model based on a convolutional neural network, and the hand bone photo of the hand bone to be predicted to be processed into a specified pixel is required to be The hand bone photograph is input into a preset bone age estimation model based on a convolutional neural network, and the calculation result outputted by the bone age evaluation model is obtained, and the result is the bone age of the hand bone, and the bone age based on the convolutional neural network The evaluation model can automatically perform bone age assessment and has high evaluation accuracy.

One of ordinary skill in the art can understand that all or part of the process of implementing the above embodiments can be completed by computer readable instructions, which can be stored with a non-volatile computer. The readable storage medium, which when executed, may include the flow of an embodiment of the methods as described above. Any reference to a memory, storage, database or other medium used in the present application and embodiments may include non-volatile and/or volatile memory. The non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual speed rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronization. Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a series of elements includes those elements. It also includes other elements not explicitly listed, or elements that are inherent to such a process, device, item, or method. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, the device, the item, or the method that comprises the element. The above is only the preferred embodiment of the present invention, and is not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the invention and the drawings are directly or indirectly applied to other related The technical field is equally included in the scope of patent protection of the present invention.

Claims

A method for assessing bone age of a hand bone X-ray film, comprising:

Hand bone x-ray film of the bone age to be predicted is processed into a photo of the hand bone required by the specified pixel;

Inputting the hand bone photograph into a preset bone age assessment model based on a convolutional neural network for calculation;

Obtaining a calculation result output by the bone age assessment model, the result being the bone age of the hand bone.
The method for assessing bone age of a hand bone X-ray according to claim 1, wherein the step of inputting the photo of the hand bone into a preset bone age estimation model based on a convolutional neural network comprises:

Convolution calculation of the hand bone photograph to obtain a first picture feature;

Performing a plurality of iterative convolution calculations on the first picture feature to obtain a second picture feature;

Performing spatial transformation and alignment processing on the second picture feature by using a spatial transformation network to obtain a third picture feature;

Performing a convolution calculation on the third picture feature to obtain a fourth picture feature;

The fourth picture features are combined by a fully connected layer to form a global picture feature, thereby outputting a calculation result.
The skeletal skeletal age estimation method for a hand bone according to claim 2, wherein the step of convoluting the hand bone photograph to obtain a first picture feature comprises:

Data augmentation processing is performed on the hand bone photograph.
The skeletal skeletal age estimation method for a hand bone according to claim 1, wherein the step of processing the hand bone x-ray film of the bone age to be predicted into the photo of the hand bone required by the specified pixel comprises:

The bones of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film are selected as the hand bone X-ray film of the bone age to be predicted.
The skeletal age evaluation method for a hand bone X-ray according to claim 4, wherein the skeleton of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film is selected as the hand bone X-ray of the bone age to be predicted. Before the steps, include:

Adjust the contrast of the candidate hand bone X-ray film.
The skeletal skeletal age estimation method for a hand bone according to claim 5, wherein the step of adjusting the contrast of the candidate hand bone X-ray film comprises:

The background portion of the candidate hand bone X-ray film is unified into black.
The skeletal skeletal age estimation method for a hand bone according to claim 1, wherein the method for training a bone age assessment model based on a convolutional neural network comprises:

Obtaining a specified amount of sample data, and dividing the sample data into a training set and a test set, wherein the sample data includes a photo of a hand bone of a known bone age, and bone age data corresponding to the hand bone photo of the known bone age;

The sample data of the training set is input into a preset convolutional neural network for training, and a result training model is obtained;

Verifying the result training model using sample data of the test set;

If the verification passes, the result training model is recorded as the bone age estimation model based on the convolutional neural network.
A hand bone X-ray bone age estimating device, comprising:

a processing unit, configured to process a hand bone x-ray film of a bone age to be predicted into a photo of a hand bone required by a specified pixel;

a calculating unit, configured to input the photo of the hand bone into a preset bone age estimation model based on a convolutional neural network for calculation;

And an output unit, configured to obtain a calculation result output by the bone age evaluation model, where the result is a bone age of the hand bone.
The hand bone x-ray skeletal age estimating device according to claim 8, wherein the calculating unit comprises:

a first processing module, configured to perform convolution calculation on the hand bone photo to obtain a first picture feature;

a second processing module, configured to perform a plurality of iterative convolution calculations on the first picture feature to obtain a second picture feature;

a transform module, configured to perform spatial transformation and alignment processing on the second picture feature by using a spatial transformation network to obtain a third picture feature;

a third processing module, configured to perform convolution calculation on the third picture feature to obtain a fourth picture feature;

And an execution module, configured to combine the fourth picture features by a fully connected layer to form a global picture feature, thereby outputting a calculation result.
The hand bone x-ray skeletal age estimating device according to claim 8, wherein the calculating unit further comprises:

The augmentation module is configured to perform data augmentation processing on the hand bone photo.
The hand bone x-ray skeletal age estimating device according to claim 8, wherein the hand bone X-ray skeletal age estimating device further comprises:

The selection unit is configured to select bones of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film as the hand bone X-ray of the bone age to be predicted.
The hand bone x-ray skeletal age estimating device according to claim 11, wherein the hand bone X-ray bone age estimating device comprises:

And a second processing unit, configured to adjust a contrast of the candidate hand bone X-ray film.
The hand bone X-ray bone age estimating device according to claim 12, wherein the hand bone X-ray bone age estimating device comprises:

a unit for unifying the background portion of the candidate hand bone X-ray into black.
The hand bone X-ray skeletal age estimating apparatus according to claim 8, wherein the calculating unit is further configured to acquire a specified amount of sample data, and divide the sample data into a training set and a test set, wherein the The sample data includes a photo of the hand bone of a known bone age, and bone age data corresponding to the hand bone photograph of the known bone age; the sample data of the training set is input into a preset convolutional neural network for training, and the result training model is obtained. And validating the result training model by using the sample data of the test set; if the verification is passed, the result training model is recorded as the bone age estimation model based on a convolutional neural network.
A computer device comprising a memory and a processor, the memory storing computer readable instructions, wherein the processor executes the computer readable instructions to implement a hand bone X-ray bone age assessment method, the hand bone X-ray bone age assessment methods include:

Hand bone x-ray film of the bone age to be predicted is processed into a photo of the hand bone required by the specified pixel;

Inputting the hand bone photograph into a preset bone age assessment model based on a convolutional neural network for calculation;

Obtaining a calculation result output by the bone age assessment model, the result being the bone age of the hand bone.
The computer device according to claim 15, wherein the step of inputting the hand bone photo into a preset convolutional neural network based bone age assessment model comprises:

Convolution calculation of the hand bone photograph to obtain a first picture feature;

Performing a plurality of iterative convolution calculations on the first picture feature to obtain a second picture feature;

Performing spatial transformation and alignment processing on the second picture feature by using a spatial transformation network to obtain a third picture feature;

Performing a convolution calculation on the third picture feature to obtain a fourth picture feature;

The fourth picture features are combined by a fully connected layer to form a global picture feature, thereby outputting a calculation result.
The computer device according to claim 16, wherein the step of convoluting the hand bone photo to obtain a first picture feature comprises:

Data augmentation processing is performed on the hand bone photograph.
The computer device according to claim 15, wherein the step of processing the hand bone x-ray film of the bone age to be predicted into the photo of the hand bone required by the specified pixel comprises:

The bones of the epiphysis, the metaphysis, and the wrist in the candidate hand bone X-ray film are selected as the hand bone X-ray film of the bone age to be predicted.
A computer non-transparent readable storage medium having stored thereon computer readable instructions, wherein the computer readable instructions are executed by a processor to implement a bone bone X-ray bone age assessment method, the hand bone X Light bone age assessment methods include:

Hand bone x-ray film of the bone age to be predicted is processed into a photo of the hand bone required by the specified pixel;

Inputting the hand bone photograph into a preset bone age assessment model based on a convolutional neural network for calculation;

Obtaining a calculation result output by the bone age assessment model, the result being the bone age of the hand bone.
The computer non-volatile readable storage medium according to claim 19, wherein said step of inputting said hand bone photograph into a predetermined convolutional neural network based bone age assessment model comprises: :

Convolution calculation of the hand bone photograph to obtain a first picture feature;

Performing a plurality of iterative convolution calculations on the first picture feature to obtain a second picture feature;

Performing spatial transformation and alignment processing on the second picture feature by using a spatial transformation network to obtain a third picture feature;

Performing a convolution calculation on the third picture feature to obtain a fourth picture feature;

The fourth picture features are combined by a fully connected layer to form a global picture feature, thereby outputting a calculation result.