CN117252881B - Bone age prediction method, system, equipment and medium based on hand X-ray image - Google Patents
Bone age prediction method, system, equipment and medium based on hand X-ray image Download PDFInfo
- Publication number
- CN117252881B CN117252881B CN202311541168.9A CN202311541168A CN117252881B CN 117252881 B CN117252881 B CN 117252881B CN 202311541168 A CN202311541168 A CN 202311541168A CN 117252881 B CN117252881 B CN 117252881B
- Authority
- CN
- China
- Prior art keywords
- image
- age
- relative rank
- rank
- input image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 210000000988 bone and bone Anatomy 0.000 title claims abstract description 56
- 239000013598 vector Substances 0.000 claims abstract description 61
- 238000012935 Averaging Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 30
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 abstract description 11
- 238000012545 processing Methods 0.000 abstract description 9
- 230000007547 defect Effects 0.000 abstract description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 6
- 230000004913 activation Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 4
- 230000002146 bilateral effect Effects 0.000 description 3
- 238000003708 edge detection Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 210000000707 wrist Anatomy 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 210000002411 hand bone Anatomy 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 241000135164 Timea Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000014461 bone development Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 210000000236 metacarpal bone Anatomy 0.000 description 1
- 239000012466 permeate Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000012488 skeletal system development Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000003813 thumb Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
Abstract
The invention discloses a bone age prediction method, a system, equipment and a medium based on hand X-ray images, relates to bone age prediction in the field of artificial intelligence, and aims to solve the technical problem that multiple steps based on ROI (region of interest) areas in the existing deep learning are not suitable for being realized by an automatic method. The method comprises the steps of obtaining corresponding feature vectors through images, key points and central lines, selecting the first k nearest feature vectors, and obtaining iterative values of the images by averaging according to ages corresponding to the feature vectors; inputting the related data into a relative rank regressor, and outputting a relative rank by the relative rank regressor; and carrying out iterative computation by adopting the initial value and the relative rank, and outputting the absolute rank at the moment as the predicted age of the input image after the iteration is stopped. The method overcomes the defects of strong subjectivity, high labor and time cost, inapplicability to automatic processing and the like of a clinical method, and solves the technical problem that multiple steps based on the ROI area are inapplicable to the realization of an automatic method.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence, relates to bone age prediction, and in particular relates to a bone age prediction method, system, equipment and medium based on hand X-ray images.
Background
Bone age prediction is a medical task for assessing the growth and development of children and adolescents. Bone age refers to the degree of maturity of a person's skeleton, and is generally expressed in terms of skeletal development stages corresponding to age. In the bone age prediction task, a doctor or a medical expert can determine the bone development condition of a patient through clinical examination and bone age assessment, and the prediction can help the doctor to more accurately evaluate the growth development state of the patient, detect and monitor abnormal growth and make a personalized treatment plan.
There are two main diagnostic methods clinically available at present, one is the GP method and the other is the TW method. Both methods are to take the left hand X of the patient. The light sheet is compared with the bone age measurement map. The GP method directly obtains the predicted age of the patient by comparing the whole hand area, the TW method compares the designated ROI key areas, scores each ROI, and finally performs statistics and conversion to corresponding bone ages. Meanwhile, as deep learning technology gradually permeates various fields of medical images and a series of advanced results are achieved, a plurality of bone age prediction methods based on deep learning are also developed in the market at present. Bone age prediction methods based on deep learning can be broadly divided into two categories: one is to take the whole picture directly as input of convolutional neural network without based on ROI, and to predict bone age by adopting end-to-end single-stage structure. Another is the ROI-based approach, which typically uses a priori knowledge of the human being to perform additional image processing on the original image. The whole process generally comprises a plurality of image processing steps, firstly, hands are segmented from an original image, interference of the background on experimental results is reduced, then, the corresponding ROI key areas are detected and extracted by using priori knowledge of human beings, and finally, the corresponding age results are generated.
The invention patent application with the application number of 201910693283.5 discloses a method for evaluating the bone age of a hand bone X-ray film based on a heterogeneous data fusion network, which comprises the following steps of: preprocessing an X-ray film image, and extracting a wrist bone part in the image; step two, constructing a convolutional neural network to extract image features, wherein the process of constructing the convolutional neural network is as follows: step 2.1: inputting a group of hand bone X-ray films processed in the step one; step 2.2: performing a convolution operation with a size of 7*7, and then performing batch normalization, and performing a Relu activation function operation; step 2.3: extracting main features through MaxPooling operation; step 2.4: the extracted features go through a residual convolution module that includes 2 sets 3*3 of convolution operations and a catch normalization; step 2.5: repeating the step 2.4 for three times to obtain the characteristic Fcoarse of the X-ray film; step 2.6: inputting Fcoarse into an attention mechanism module to obtain fine features Ffine; step 2.7: inputting Ffine into a spatial pyramid pooling module to obtain a one-dimensional vector Vimage with a fixed size of 512 x 21; step three, constructing a text feature extraction model; step four, constructing a fusion layer, and combining the image features and the text features; and fifthly, training the model, and after full convergence, storing and deriving a model structure and weight parameters. The method for analyzing the X-ray film through the convolutional neural network automatically assists in evaluating the bone age of the X-ray film, and has high efficiency and high speed compared with the traditional method; the attention mechanism module is adopted in the X-ray film image feature extraction, so that the network features are refined, and important image information is effectively utilized; two multi-source heterogeneous data of X-ray film images and text information are used, and the two information are fused and complemented, so that the evaluation effect is improved.
The invention patent application with application number 202211076541.3 discloses a method for predicting the bone age of children, which comprises the following steps: randomly selecting part of hand X-ray films from the hand X-ray film public data set with the bone age label to form a data set, and adjusting all pictures to a specified size; establishing a TENet model and simultaneously establishing a training set; taking the training set as input of a TENet model, training the TENet model by using an Adam optimizer, and obtaining a trained TENet model when the maximum iteration number is reached; the TENet model comprises a hand topology module, an edge feature enhancement module and a deep learning network improvement model D, wherein the edge feature enhancement module is an improved Canny edge detection algorithm, and the improved Canny edge detection algorithm uses a bilateral filtering denoising algorithm and an Ojin algorithm. In the conventional Canny edge detection algorithm, linear Gaussian filtering is adopted to remove noise, convolution with a weighting coefficient is used to remove noise from an image, and the method is basically mean blurring, namely, the Gaussian blurring is carried out according to weighted average, the closer the distance is, the larger the point weight is, the smaller the point weight is, and therefore the edge information and the characteristic information in the image are "blurred" by calculating the mean value, and many characteristics are lost. The bilateral filtering is a nonlinear filtering method, is a compromise process combining the spatial proximity of the image and the similarity of pixel values, and simultaneously considers the spatial domain information and the gray level similarity, namely, the purpose is achieved by introducing a spatial domain kernel and a value domain kernel. The bilateral filter can well preserve image edge features and filter out noise of low frequency components.
In addition, the invention patent application with the application number of 202310173447.8 also discloses an intelligent bone age assessment method based on multi-region combination, which comprises the following steps of: step 1: performing image enhancement on the X-ray film image of the wrist part to be detected; step 2: extracting a plurality of interest areas in the wrist bone X-ray film, and improving the accuracy of bone age assessment by utilizing the interest areas; step 3: sending the pictures and the interested areas processed in the step 1 and the step 2 to a bone age evaluation network for bone age evaluation; in the step 3, 1) performing feature extraction on the whole X-ray film and the images of three interest areas by using a convolutional neural network acceptance-ResNet-V2 to generate 46080-dimensional image feature vectors; 2) Sex information corresponding to the X-ray film passes through a full-connection layer with the number of neurons being 32, and a 32-dimensional sex characteristic vector is generated; 3) Calculating the whole X-ray film image and the images of the metacarpal bone region, the thumb region and the phalangeal bone region through an acceptance-ResNet-V2 network, and generating 46080-dimensional image feature vectors; 4) And splicing the image feature vector and the sex feature vector to form a new feature vector. And finally, calculating by using two full-connection layers with the number of the neurons being 1000, and obtaining a corresponding estimated bone age value by regression.
As in the above patent applications, most of the existing deep learning is implemented by ROI (region of interest) based methods, and the ROI based methods in deep learning improve the accuracy of prediction, but bring a series of limitations at the same time: 1) The strongly supervised ROI attention identified by domain experts may not be well suited to be implemented with automated methods; 2) The method requires a large number of ROI labeling frames labeled by experts, and generates great labor cost; 3) It is difficult to train through end-to-end methods, as it introduces additional processing, while also facing extremely high complexity and time costs.
Disclosure of Invention
The invention aims at: the technical problem that the existing deep learning is based on the defect that multiple steps of the ROI area are not suitable for being realized by an automatic method is solved, and a bone age prediction method, a bone age prediction system, bone age prediction equipment and a bone age prediction medium based on hand X-ray images are provided.
The invention adopts the following technical scheme for realizing the purposes:
a bone age prediction method based on hand X-ray images comprises the following steps:
s1, predicting an initial value;
obtaining a feature vector corresponding to the X-ray image according to the obtained X-ray image, key points and central lines, selecting the first k feature vectors closest to the feature vector of the X-ray image from all training sets with age labels based on knn algorithm, and averaging according to ages corresponding to the k feature vectors to obtain an initial value of the first iteration of the X-ray image;
step S2, outputting a relative rank;
x-ray image as input image of relative rank regressorAnd selecting and inputting the image +_ according to the size of the sliding window>Corresponding reference image->Reference image->The method comprises the steps of carrying out a first treatment on the surface of the Input image +.>Reference image->Reference image->Input image->Key points and center lines of (2) reference image +.>Key points and center lines of (2) reference image +.>Key points and center lines of (2) and input image +.>Sex->And actual age->All input relative rank regressors, output input image +.>Relative rank of->；
S3, outputting a predicted age;
the initial value obtained in the step S1 and the relative rank output in the step S2 are adoptedPerforming iterative calculation, and outputting absolute rank ++at this time after iteration stop>As input image +.>Is a predicted age of (a).
Further, in step S1, the obtained X-ray image, the key point and the center line are input into an encoder to perform feature extraction, so as to obtain feature vectors corresponding to the X-ray image;
the backbone network of the encoder is a ResNet-50 network.
Further, in step S2, the relative rankThe calculation mode of (a) is as follows:
wherein,representing input image +.>Corresponding feature vector, ">Representing reference image +.>Corresponding feature vector, ">Representing reference image +.>Corresponding feature vector, ">、/>Respectively indicate sex, age, and->、/>Respectively indicate the sexLearning multiplier, age->Is a learning multiplier of (a); />Representing regression calculation, outputting a [ -1,1 [ -1 ]]A number in between.
Further, the relative rank regressor is trained, and the loss function during training is as follows:
wherein,representing +_ according to the input image>Calculated relative rank, ++>Representing a relative rank calculated from images in the training set with age tags; />Representing input image +.>Absolute rank corresponding to the current iteration number.
Further, in step S3, when performing iterative computation, the iterative computation formula is:
when (when)Or stopping iteration if the iteration number reaches a preset upper limit; outputting the absolute rank at that timeAs input image +.>Is a predicted age of (2);
wherein,、/>respectively represent the input image at time t>Corresponding two reference images.
A bone age prediction system based on hand X-ray images, comprising:
the initial value prediction module is used for obtaining feature vectors corresponding to the X-ray image according to the obtained X-ray image, key points and central lines, selecting the first k feature vectors closest to the feature vectors of the X-ray image from all training sets with age labels based on knn algorithm, and averaging according to ages corresponding to the k feature vectors to obtain an initial value of the first iteration of the X-ray image;
a relative rank output module for taking the X-ray image as the input image of the relative rank regressorAnd selecting and inputting the image +_ according to the size of the sliding window>Corresponding reference image->Reference image->The method comprises the steps of carrying out a first treatment on the surface of the Input image +.>Reference image->Reference image->Input image->Key points and center lines of (2) reference image +.>Key points and center lines of (2) reference image +.>Key points and center lines of (2) and input image +.>Sex->And actual age->All input relative rank regressors, output input image +.>Relative rank of->；
A predicted age output module for outputting the initial value obtained by the initial value prediction module and the relative rank outputted by the relative rank output modulePerforming iterative calculation, and outputting absolute rank ++at this time after iteration stop>As input image +.>Is a predicted age of (a).
A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method described above.
A computer-readable storage medium, characterized by: a computer program is stored which, when executed by a processor, causes the processor to perform the steps of the above method.
The beneficial effects of the invention are as follows:
1. compared with the current clinical GP and TW bone age prediction method, the method can realize automation, and overcomes the defects of strong subjectivity, high labor and time cost, inapplicability to automatic processing and the like of the clinical method.
2. Compared with a bone age prediction model which is not based on the ROI in the current deep learning, the method introduces the corresponding key points and central line information of the patient, and solves the problems that the accuracy and the interpretability are relatively poor due to the lack of attention to a specific ROI region in the former.
3. Compared with the bone age prediction model based on the ROI in the current deep learning, the method abandons a multi-step processing mode of the original image, reduces training complexity and labor cost, and ensures accuracy and interpretability of the model.
4. Compared with the existing bone age prediction model based on deep learning, the method combines the ordered regression idea with the bone age prediction task for the first time, adopts a sliding window mechanism to continuously iterate the bone age prediction, and makes a certain innovation on the method.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic flow chart of knn prediction in the present invention;
FIG. 3 is a flow chart of a relative rank regressor of the present invention;
fig. 4 is a schematic view of a sliding window structure according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention.
Thus, all other embodiments, which can be made by one of ordinary skill in the art without undue burden from the invention, are intended to be within the scope of the invention.
Example 1
The embodiment provides a bone age prediction method based on a hand X-ray image, which takes the hand X-ray image and the corresponding key points and central line as input and finally outputs the predicted age. As shown in fig. 1, the specific prediction method includes the following steps:
s1, predicting an initial value;
and obtaining a feature vector corresponding to the X-ray image according to the obtained X-ray image, the key points and the central line, selecting the first k feature vectors closest to the feature vector of the X-ray image from all training sets with age labels based on knn algorithm, and averaging according to ages corresponding to the k feature vectors to obtain an initial value of the first iteration of the X-ray image.
Acquiring an X-ray image, a key point and a central line corresponding to a patient, inputting the X-ray image, the key point and the central line into an encoder for feature extraction, and obtaining a feature vector corresponding to the X-ray image; the backbone network of the encoder is a ResNet-50 network. As shown in fig. 2, the knn algorithm is then used to select the first k (in this embodiment, k=5) feature vectors closest to the feature vector of the X-ray image from the feature space corresponding to all the age-labeled training sets (in this embodiment, the training set uses the existing public data set, i.e., the RSNA data set), and average the ages corresponding to the k feature vectors to obtain the initial value of the first iteration of the X-ray image.
Step S2, outputting a relative rank;
x-ray image as input image of relative rank regressorAnd selecting and inputting the image +_ according to the size of the sliding window>Corresponding reference image->Reference image->The method comprises the steps of carrying out a first treatment on the surface of the Input image +.>Reference image->Reference image->Input image->Key points and center lines of (2) reference image +.>Key points and center lines of (2) reference image +.>Key points and center lines of (2) and input image +.>Sex->And actual age->All input relative rank regressors, output input image +.>Relative rank of->。
Firstly, the embodiment designs a relative rank regressor for outputting relative rank, which is shown in fig. 3, and consists of a feature extractor and a regression module, wherein the feature extractor also adopts a ResNet-50 network, and the output of the last pooling layer of the ResNet-50 network is adopted as an input imageFeature vector +.>The method comprises the steps of carrying out a first treatment on the surface of the The regressor comprises three full-connection layers, wherein the first two full-connection layers adopt a ReLU activation layer, and the last layer adopts a tanh activation layer to output [ -1,1 [ -1 ]]Number between as input image +.>Relative rank of->. Because the relative rank describes the input image +.>Relationship with the relative size of the age between two reference pictures, thus reference picture +.>Reference image->It is also necessary to obtain the corresponding feature vector +.>Feature vector->And the weights among the three are shared.
In order to fully utilize the patient identity information to improve the final prediction accuracy, the embodiment also introduces the sex of the patientAnd age->And the dimension is not increased by manual embedding, but learning multiplier +.>、/>To balance the importance of each input. Finally, relative rank->The calculation mode of (a) is as follows:
wherein,representing input image +.>Corresponding feature vector, ">Representing reference image +.>Corresponding feature vector, ">Representing reference image +.>Corresponding feature vector, ">、/>Respectively indicate sex, age, and->、/>Respectively indicate the sexLearning multiplier, age->Is a learning multiplier of (a); />Representing regression calculation, outputting a [ -1,1 [ -1 ]]A number in between.
Since regressors are trained in an end-to-end fashion, it is necessary to rely on the size of the sliding windowTo select the input image +.>Corresponding reference image->Reference image->. This->Fixing to a constant will also reduce the training difficulty of the relative rank regressor.
When the relative rank regressor takes training set images with age labels as trainingAnd the corresponding relative rank is calculated to be the group-trunk, which is marked as +.>. When the relative rank regressor is trained, the loss function adopted is a square error, and specifically:
wherein,representing +_ according to the input image>Calculated relative rank, ++>Representing a relative rank calculated from images in the training set with age tags; />Representing input image +.>Absolute rank corresponding to the current iteration number.
The training essence of the relative rank regressor is to reduce the relative rank output value in the regressor as far as possible through trainingAnd (3) withErrors between the two, thereby improving the accuracy of bone age prediction as much as possible.
S3, outputting a predicted age;
the initial value obtained in the step S1 and the relative rank output in the step S2 are adoptedPerforming iterative calculation, and outputting absolute rank ++at this time after iteration stop>As input image +.>Is a predicted age of (a).
The essence of the sliding window algorithm is the relative rank output by the relative rank regressorTo describe an input image of a patientAnd two reference pictures->、/>Is based on the relative rank +.>To move to the left or right to a certain extent and acquire the patient input image in the current iteration state +.>Absolute rank>I.e. the corresponding age. Then selecting two reference images corresponding to the moment according to the length of the sliding window size, repeating the output relative rank, calculating the absolute rank according to the relative rank, and if and only if the iteration frequency upper limit is reached or the absolute rank of the current moment and the absolute rank of the last iteration moment are reached->When the same, the corresponding absolute rank is output>As the predicted age of the current patient.
First, an input image is obtained by knn algorithmInitial value +.>Wherein the superscript corresponds to the number of iterations. Then, according to the relative rank of the relative rank regressor output +.>Absolute rank->To->The iteration, the specific operation is shown in figure 4. In each iteration process, taking the result of the t-1 th iteration as the center of the t-th iteration, selecting the result according to the length of the sliding windowAnd window sliding is carried out on the reference image of the iteration number through the relative rank output by the relative rank regressor. In the iterative process, the relation between the relative rank and the absolute rank (i.e., the iterative calculation formula) is expressed as:
the above formula can be simplified because the sliding window length is fixed, so the t-th iteration will have the result of t-1 st as the center of this iteration, i.e。
If and only ifOr stopping iteration if the iteration number reaches a preset upper limit; outputting the absolute rank +.>As input image +.>Is a predicted age of (2);
wherein,、/>respectively represent the input image at time t>Corresponding two reference images.
Example 2
The embodiment provides a bone age prediction system based on a hand X-ray image, which takes the hand X-ray image and the corresponding key points and central lines thereof as input, and finally outputs the predicted age. As shown in fig. 1, the bone age prediction system comprises the following steps:
the initial value prediction module is used for obtaining feature vectors corresponding to the X-ray image according to the obtained X-ray image, key points and central lines, selecting the first k feature vectors closest to the feature vectors of the X-ray image from all training sets with age labels based on knn algorithm, and averaging according to ages corresponding to the k feature vectors to obtain the initial value of the first iteration of the X-ray image.
Acquiring an X-ray image, a key point and a central line corresponding to a patient, inputting the X-ray image, the key point and the central line into an encoder for feature extraction, and obtaining a feature vector corresponding to the X-ray image; the backbone network of the encoder is a ResNet-50 network. As shown in fig. 2, the knn algorithm is then used to select the first k (in this embodiment, k=5) feature vectors closest to the feature vector of the X-ray image from the feature space corresponding to all the age-labeled training sets (in this embodiment, the training set uses the existing public data set, i.e., the RSNA data set), and average the ages corresponding to the k feature vectors to obtain the initial value of the first iteration of the X-ray image.
A relative rank output module for taking the X-ray image as the input image of the relative rank regressorAnd selecting and inputting the image +_ according to the size of the sliding window>Corresponding reference image->Reference image->The method comprises the steps of carrying out a first treatment on the surface of the Input image +.>Reference image->Reference image->Input image->Key points and center lines of (2) reference image +.>Key points and center lines of (2) reference image +.>Key points and center lines of (2) and input image +.>Sex->And actual age->All input relative rank regressors, output input image +.>Relative rank of->。
Firstly, the embodiment designs a relative rank regressor for outputting relative rank, which is shown in fig. 3, and consists of a feature extractor and a regression module, wherein the feature extractor also adopts a ResNet-50 network, and the output of the last pooling layer of the ResNet-50 network is adopted as an input imageFeature vector +.>The method comprises the steps of carrying out a first treatment on the surface of the The regressor comprises three full-connection layers, wherein the first two full-connection layers adopt a ReLU activation layer, and the last layer adopts a tanh activation layer to output [ -1,1 [ -1 ]]Number between as input image +.>Relative rank of->. Because the relative rank describes the input image +.>Relationship with the relative size of the age between two reference pictures, thus reference picture +.>Reference image->It is also necessary to obtain the corresponding feature vector +.>Feature vector->And the weights among the three are shared.
In order to fully utilize the patient identity information to improve the final prediction accuracy, the embodiment also introduces the sex of the patientAnd age->And the dimension is not increased by manual embedding, but learning multiplier +.>、/>To balance the importance of each input. Finally, relative rank->The calculation mode of (a) is as follows:
wherein,representing input image +.>Corresponding feature vector, ">Representing reference image +.>Corresponding feature vector, ">Representing reference image +.>Corresponding feature vector, ">、/>Respectively indicate sex, age, and->、/>Respectively indicate the sexLearning multiplier, age->Is a learning multiplier of (a); />Representing regression calculation, outputting a [ -1,1 [ -1 ]]A number in between.
Since regressors are trained in an end-to-end fashion, it is necessary to rely on the size of the sliding windowTo select the input image +.>Corresponding reference image->Reference image->. This->Fixing to a constant will also reduce the training difficulty of the relative rank regressor.
When the relative rank regressor takes training set images with age labels as trainingAnd the corresponding relative rank is calculated to be the group-trunk, which is marked as +.>. When the relative rank regressor is trained, the loss function adopted is a square error, and specifically:
wherein,representing +_ according to the input image>Calculated relative rank, ++>Representing a relative rank calculated from images in the training set with age tags; />Representing input image +.>Absolute rank corresponding to the current iteration number.
The training essence of the relative rank regressor is to reduce the relative rank output value in the regressor as far as possible through trainingAnd (3) withErrors between the two, thereby improving the accuracy of bone age prediction as much as possible.
A predicted age output module for outputting the initial value obtained by the initial value prediction module and the relative rank outputted by the relative rank output modulePerforming iterative calculation, and outputting absolute rank ++at this time after iteration stop>As input image +.>Is a predicted age of (a).
The essence of the sliding window algorithm is the relative rank output by the relative rank regressorTo describe an input image of a patientAnd two reference pictures->、/>Is based on the relative rank +.>To move to the left or right to a certain extent and acquire the patient input image in the current iteration state +.>Absolute rank>I.e. the corresponding age. Then selecting two reference images corresponding to the moment according to the length of the sliding window size, repeating the output relative rank, calculating the absolute rank according to the relative rank, and if and only if the iteration frequency upper limit is reached or the absolute rank of the current moment and the absolute rank of the last iteration moment are reached->When the same, the corresponding absolute rank is output>As the predicted age of the current patient.
First, an input image is obtained by knn algorithmInitial value +.>Wherein the superscript corresponds to the number of iterations. Then, according to the relative rank of the relative rank regressor output +.>Absolute rank->To->The iteration, the specific operation is shown in figure 4. In each iteration process, taking the result of the t-1 th iteration as the center of the t iteration, selecting the reference image of the iteration according to the length of the sliding window, and performing window sliding through the relative rank output by the relative rank regressor. In the iterative process, the relation between the relative rank and the absolute rank (i.e., the iterative calculation formula) is expressed as:
the above formula can be simplified because the sliding window length is fixed, so the t-th iteration will have the result of t-1 st as the center of this iteration, i.e。
If and only ifOr stopping iteration if the iteration number reaches a preset upper limit; outputting the absolute rank +.>As input image +.>Prediction of (2)Age, age;
wherein,、/>respectively represent the input image at time t>Corresponding two reference images.
Example 3
A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of a bone age prediction method based on hand X-ray images.
The computer equipment can be computing equipment such as a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or D interface display memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like. Of course, the memory may also include both internal storage units of the computer device and external storage devices. In this embodiment, the memory is often used to store an operating system and various application software installed on the computer device, such as program codes of the bone age prediction method based on the hand X-ray image. In addition, the memory may be used to temporarily store various types of data that have been output or are to be output.
The processor may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to execute the program code stored in the memory or process data, such as the program code of the bone age prediction method based on the hand X-ray image.
Example 4
A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of a bone age prediction method based on hand X-ray images.
Wherein the computer readable storage medium stores an interface display program executable by at least one processor to cause the at least one processor to perform the steps of the bone age prediction method based on hand X-ray images as described above.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server or a network device, etc.) to perform the bone age prediction method based on the hand X-ray image according to the embodiments of the present application.
Claims (8)
1. The bone age prediction method based on the hand X-ray image is characterized by comprising the following steps of:
s1, predicting an initial value;
obtaining a feature vector corresponding to the X-ray image according to the obtained X-ray image, key points and central lines, selecting the first k feature vectors closest to the feature vector of the X-ray image from all training sets with age labels based on knn algorithm, and averaging according to ages corresponding to the k feature vectors to obtain an initial value of the first iteration of the X-ray image;
step S2, outputting a relative rank;
x-ray image as input image of relative rank regressorAnd selecting and inputting images according to the size of the sliding windowCorresponding reference image->Reference image->The method comprises the steps of carrying out a first treatment on the surface of the Input image +.>Reference image->Reference image->Input image->Key points and center lines of (2) reference image +.>Key points and center lines of (2) reference image +.>Key points and center lines of (2) and input image +.>Sex->And actual age->All input relative rank regressors, output input image +.>Relative rank of->；
S3, outputting a predicted age;
the initial value obtained in the step S1 and the relative rank output in the step S2 are adoptedPerforming iterative calculation, and outputting absolute rank ++at this time after iteration stop>As input image +.>Is a predicted age of (a).
2. The bone age prediction method based on hand X-ray image according to claim 1, wherein in step S1, the obtained X-ray image, key points and center line are input into an encoder for feature extraction to obtain feature vectors corresponding to the X-ray image;
the backbone network of the encoder is a ResNet-50 network.
3. The bone age prediction method based on hand X-ray image as claimed in claim 1, wherein in step S2, the relative rank isThe calculation mode of (a) is as follows:
wherein,representing input image +.>Corresponding feature vector, ">Representing reference image +.>The corresponding feature vector is used to determine the feature vector,representing reference image +.>Corresponding feature vector, ">、/>Respectively indicate sex, age, and->、/>Respectively indicate sex->Learning multiplier, age->Is a learning multiplier of (a); />Representing regression calculation, outputting a [ -1,1 [ -1 ]]A number in between.
4. A method for predicting bone age based on hand X-ray images as defined in claim 3, wherein the relative rank regressor is trained with a loss function of:
wherein,representing +_ according to the input image>Calculated relative rank, ++>Representing a relative rank calculated from images in the training set with age tags; />Representing input image +.>Absolute rank corresponding to the current iteration number.
5. The bone age prediction method based on hand X-ray image according to claim 1, wherein in step S3, when performing iterative calculation, an iterative calculation formula is:
when (when)Or stopping iteration if the iteration number reaches a preset upper limit; outputting the absolute rank +.>As input image +.>Is a predicted age of (2);
wherein, 、/>respectively represent the input image at time t>Corresponding two reference images.
6. A bone age prediction system based on hand X-ray images, comprising:
the initial value prediction module is used for obtaining feature vectors corresponding to the X-ray image according to the obtained X-ray image, key points and central lines, selecting the first k feature vectors closest to the feature vectors of the X-ray image from all training sets with age labels based on knn algorithm, and averaging according to ages corresponding to the k feature vectors to obtain an initial value of the first iteration of the X-ray image;
a relative rank output module for taking the X-ray image as the input image of the relative rank regressorAnd selecting and inputting the image +_ according to the size of the sliding window>Corresponding reference image->Reference image->The method comprises the steps of carrying out a first treatment on the surface of the Input image +.>Reference image->Reference image->Input image->Key points and center lines of (2) reference image +.>Key points and center lines of (2) reference image +.>Key points and center lines of (2) and input image +.>Sex->And actual age->All input relative rank regressors, output input image +.>Relative rank of->；
A predicted age output module for outputting the initial value obtained by the initial value prediction module and the relative rank outputted by the relative rank output modulePerforming iterative calculation, and outputting absolute rank ++at this time after iteration stop>As input image +.>Is a predicted age of (a).
7. A computer device, characterized by: comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized by: a computer program is stored which, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311541168.9A CN117252881B (en) | 2023-11-20 | 2023-11-20 | Bone age prediction method, system, equipment and medium based on hand X-ray image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311541168.9A CN117252881B (en) | 2023-11-20 | 2023-11-20 | Bone age prediction method, system, equipment and medium based on hand X-ray image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117252881A CN117252881A (en) | 2023-12-19 |
| CN117252881B true CN117252881B (en) | 2024-01-26 |
Family
ID=89126872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311541168.9A Active CN117252881B (en) | 2023-11-20 | 2023-11-20 | Bone age prediction method, system, equipment and medium based on hand X-ray image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117252881B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0503236D0 (en) * | 2005-02-16 | 2005-03-23 | Ccbr As | Vertebral fracture quantification |
| CN107591200A (en) * | 2017-08-25 | 2018-01-16 | 卫宁健康科技集团股份有限公司 | Stone age marker recognition appraisal procedure and system based on deep learning and image group |
| CN109215013A (en) * | 2018-06-04 | 2019-01-15 | 平安科技（深圳）有限公司 | Automatic stone age prediction technique, system, computer equipment and storage medium |
| CN109377484A (en) * | 2018-09-30 | 2019-02-22 | 杭州依图医疗技术有限公司 | A kind of method and device detecting the stone age |
| CN109741309A (en) * | 2018-12-27 | 2019-05-10 | 北京深睿博联科技有限责任公司 | A kind of stone age prediction technique and device based on depth Recurrent networks |
| CN110503624A (en) * | 2019-07-02 | 2019-11-26 | 平安科技（深圳）有限公司 | Stone age detection method, system, equipment and readable storage medium storing program for executing |
| CN112132788A (en) * | 2020-08-29 | 2020-12-25 | 浙江工业大学 | Bone age assessment method based on characteristic region grade identification |
| CN115131323A (en) * | 2022-07-04 | 2022-09-30 | 浙江工业大学 | Bone age assessment method based on image enhancement |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200064444A1 (en) * | 2015-07-17 | 2020-02-27 | Origin Wireless, Inc. | Method, apparatus, and system for human identification based on human radio biometric information |
-
2023
- 2023-11-20 CN CN202311541168.9A patent/CN117252881B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0503236D0 (en) * | 2005-02-16 | 2005-03-23 | Ccbr As | Vertebral fracture quantification |
| CN107591200A (en) * | 2017-08-25 | 2018-01-16 | 卫宁健康科技集团股份有限公司 | Stone age marker recognition appraisal procedure and system based on deep learning and image group |
| CN109215013A (en) * | 2018-06-04 | 2019-01-15 | 平安科技（深圳）有限公司 | Automatic stone age prediction technique, system, computer equipment and storage medium |
| CN109377484A (en) * | 2018-09-30 | 2019-02-22 | 杭州依图医疗技术有限公司 | A kind of method and device detecting the stone age |
| CN109741309A (en) * | 2018-12-27 | 2019-05-10 | 北京深睿博联科技有限责任公司 | A kind of stone age prediction technique and device based on depth Recurrent networks |
| CN110503624A (en) * | 2019-07-02 | 2019-11-26 | 平安科技（深圳）有限公司 | Stone age detection method, system, equipment and readable storage medium storing program for executing |
| CN112132788A (en) * | 2020-08-29 | 2020-12-25 | 浙江工业大学 | Bone age assessment method based on characteristic region grade identification |
| CN115131323A (en) * | 2022-07-04 | 2022-09-30 | 浙江工业大学 | Bone age assessment method based on image enhancement |
Non-Patent Citations (6)
| Title |
|---|
| An Automated TW3-RUS Bone Age Assessment Method with Ordinal Regression-Based Determination of Skeletal Maturity;Dongxu Zhang 等;《Journal of Digital Imaging》;1001-1015 * |
| Bone Age Assessment Based on Rank-Monotonicity Enhanced Ranking CNN;BO LIU 等;《IEEE Access》;120976-120983 * |
| BONE AGE ASSESSMENT USING MACHINE LEARNING AND IMAGE PROCESSING;Tushar Jain 等;《International Journal of Advanced Research in Computer and Communication Engineering》;第11卷(第4期);287-296 * |
| Deep focus approach for accurate bone age estimation from lateral cephalogram;Hyejun Seo 等;《Journal of Dental Sciences 》;34-43 * |
| 不同人工智能模型对基于手腕部 DR 影像的骨龄预测比较;李莉红 等;《临床放射学杂志》;1-4 * |
| 基于改进ResNet和注意力机制的骨龄评估方法研究;乔嘉昕;《万方数据知识服务平台》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117252881A (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109345508B (en) | Bone age evaluation method based on two-stage neural network | |
| CN110399929B (en) | Fundus image classification method, fundus image classification apparatus, and computer-readable storage medium | |
| Talebi et al. | Learned perceptual image enhancement | |
| CN109215013B (en) | Automatic bone age prediction method, system, computer device and storage medium | |
| CN111080675B (en) | Target tracking method based on space-time constraint correlation filtering | |
| Xu et al. | Robust contour tracking in ultrasound tongue image sequences | |
| Wang et al. | Deep convolutional architecture for natural image denoising | |
| CN111680757A (en) | Zero sample image recognition algorithm and system based on self-encoder | |
| CN113255557A (en) | Video crowd emotion analysis method and system based on deep learning | |
| Tripathi | Facial image noise classification and denoising using neural network | |
| CN110197721B (en) | Tendon condition assessment method, device and storage medium based on deep learning | |
| Zhou et al. | Adaptive weighted locality-constrained sparse coding for glaucoma diagnosis | |
| Hong et al. | A novel evolutionary approach to image enhancement filter design: method and applications | |
| CN114708301A (en) | Motion artifact identification method and device, storage medium and electronic equipment | |
| CN109190505A (en) | The image-recognizing method that view-based access control model understands | |
| CN117252881B (en) | Bone age prediction method, system, equipment and medium based on hand X-ray image | |
| CN116129417A (en) | Digital instrument reading detection method based on low-quality image | |
| US20220138573A1 (en) | Methods and systems for training convolutional neural networks | |
| CN113284122B (en) | Roll paper packaging defect detection method and device based on deep learning and storage medium | |
| CN111368602A (en) | Face image blurring degree evaluation method and device, readable storage medium and equipment | |
| Sharma et al. | Solving image processing critical problems using machine learning | |
| CN114841887A (en) | Image restoration quality evaluation method based on multi-level difference learning | |
| Gao et al. | DRL-FVRestore: An Adaptive Selection and Restoration Method for Finger Vein Images Based on Deep Reinforcement | |
| Dong et al. | An improved deep neural network method for an athlete's human motion posture recognition | |
| Wu et al. | Q-DNN: A quality-aware deep neural network for blind assessment of enhanced images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |