WO2022007972A1

WO2022007972A1 - Total hip joint image processing method and apparatus

Info

Publication number: WO2022007972A1
Application number: PCT/CN2021/107720
Authority: WO
Inventors: 张逸凌; 刘星宇
Original assignee: 北京长木谷医疗科技有限公司
Priority date: 2020-07-06
Filing date: 2021-07-21
Publication date: 2022-01-13
Also published as: CN111888059A; CN111888059B

Abstract

A total hip joint image processing method, comprising: acquiring an X-ray image of a hip joint, the X-ray image of the hip joint containing an image of a reference object, and the reference object being a reference object of a known size (S101); according to the scale of the image size of the reference object to the actual size thereof, restoring the X-ray image of the hip joint in size (S102); on the basis of a depth learning model, recognizing the restored X-ray image of the hip joint, and determining a leg length difference, an acetabular cup position and the size and model of a femoral stem prosthesis (S103); and according to a rotation center of the femoral stem prosthesis and a rotation center of the acetabular cup determined in the process of recognizing the X-ray image of the hip joint, determining an osteotomy line position (S104). The total hip joint image processing method provides better preoperative support for total hip replacement surgeries.

Description

A method and device for processing a total hip image

This application claims the priority of the Chinese patent application with the application number CN202010643713.5 and the invention titled "Method and Device for Image Processing of Total Hip Joint Based on Deep Learning and X-ray" filed with the Chinese Patent Office on July 6, 2020, The entire contents of which are incorporated herein by reference.

technical field

The present application relates to the technical field of data processing, and in particular, to a method and device for processing a total hip joint image.

Background technique

In the medical field, the preoperative planning of total hip replacement surgery mainly includes calculating the required prosthesis size and the position of the osteotomy line. The preoperative planning of total hip replacement surgery plays a very important role in the success rate of the surgery. Therefore, how to It is very important to provide accurate preoperative planning. At present, the main preoperative planning method is to measure manually through various tools, which is inefficient and cannot guarantee the accuracy. Therefore, it is urgent to provide a more convenient and accurate preoperative planning method to provide a better solution for total hip replacement surgery. Preoperative support.

SUMMARY OF THE INVENTION

The present application proposes a method and device for processing a total hip joint image, so as to provide a more convenient and more accurate preoperative planning manner to provide better preoperative support for total hip replacement surgery.

In order to achieve the above object, according to the first aspect of the present application, a total hip joint image processing method based on deep learning and X-ray is provided.

The total hip image processing method based on deep learning and X-ray according to the present application includes:

Obtain an X-ray image of the hip joint, the X-ray image of the hip joint includes an image of a reference object, and the reference object is a reference object with a known size; according to the ratio of the image size of the reference object and its actual size, the hip The size of the X-ray image of the joint is restored; based on the deep learning model, the restored X-ray image of the hip joint is identified, and the identification result is obtained, wherein the identification result includes the position of the key point used to determine the leg length difference, the use of The position of the femoral head used to determine the position of the acetabular cup, and the area of the femoral head, the cortical bone area, and the base of the femoral neck used to determine the size of the femoral stem component; according to the center of rotation of the femoral stem component and the X at the hip joint The center of rotation of the acetabular cup determined during line image recognition determines the position of the osteotomy line.

Optionally, identifying the restored X-ray image of the hip joint based on the deep learning model to determine the leg length difference includes: converting the X-ray image of the hip joint into a grayscale image; Each pixel value of the degree map is predicted to determine the key point of the teardrop and the key point of the lesser trochanter; according to the key point of the teardrop and the key point of the lesser trochanter, the leg length difference is determined.

Optionally, the identifying and determining the position of the acetabular cup based on the restored X-ray image of the hip joint based on the deep learning model includes: converting the X-ray image of the hip joint into a grayscale image; Each pixel value of the degree map is predicted to determine the position of the femoral head; the center of rotation of the femoral head is calculated according to the centroid formula of the plane image; the diameter of the acetabular cup is calculated according to the diameter of the femoral head; the acetabulum is determined according to the center of rotation of the bone and the diameter of the acetabular cup cup position.

Optionally, identifying the X-ray image of the restored hip joint based on the deep learning model to determine the specification and model of the femoral stem prosthesis includes: converting the X-ray image of the hip joint into a grayscale image; based on a third neural network The model identifies the grayscale image to determine the anatomical axis of the medullary canal; the grayscale image is identified based on the fourth neural network model to determine the central axis of the femoral neck; the femoral neck shaft angle is determined according to the anatomical axis of the medullary canal and the central axis of the femoral neck; The femoral neck shaft angle, the medullary canal area determined during the process of determining the anatomical axis of the medullary canal, and the center of rotation of the femoral head determine the size of the femoral stem prosthesis.

Optionally, the determining the position of the osteotomy line according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition of the hip joint includes: comparing the rotation center of the femoral stem prosthesis with the hip The position of the rotation center of the acetabular cup coincides to determine the actual position of the femoral stem prosthesis; the position of the osteotomy line is determined along the coating position of the femoral stem prosthesis.

Optionally, the identifying the grayscale map based on the third neural network model and determining the anatomical axis of the medullary cavity include: predicting each pixel value of the grayscale map based on the third neural network model, and determining the femoral head region and the bone. Cortical area; determine the medullary cavity area according to the femoral head area and the bone cortex area; perform linear fitting on the coordinates of multiple center points of the medullary cavity area to determine the medullary cavity anatomical axis.

Optionally, identifying the grayscale image based on the fourth neural network model and determining the central axis of the femoral neck includes: predicting each pixel value of the grayscale image based on the fourth neural network model, and determining the femoral head region and the femoral head. Neck base area; calculate the femoral head center coordinates and femoral neck base center coordinates corresponding to the femoral head area and the femoral neck base area according to the centroid formula of the plane image; determine the femoral neck center axis according to the femoral head center coordinates and the femoral neck base center coordinates.

Optionally, the deep learning and X-ray-based total hip image processing method of the present application further includes: calculating the postoperative leg length difference and offset distance according to the position of the osteotomy line.

In order to achieve the above object, according to the second aspect of the present application, a method for processing a total hip joint image is provided, comprising: inputting the original X-ray image of the target hip joint with information to be determined into a pre-trained first neural network model , identify and obtain at least one key point position of a tear drop and at least one key point position of the lesser trochanter in the X-ray image of the target hip; The straight line determined by the position of the key point of the lesser trochanter determines the leg length difference corresponding to the X-ray image of the target hip joint; wherein, the training process of the first neural network model includes: combining the original X-ray image of the hip joint with the marked teardrop key point and The position of the key point of the lesser trochanter of the femur is input into the convolutional neural network as a sample set, the input original image is fitted with the Gaussian distribution function of the feature points, and the convolutional pooling sampling is performed to iteratively learn and train to obtain the first neural network. model, wherein the original image is a grayscale image corresponding to an unlabeled X-ray sample image of the hip joint.

In order to achieve the above object, according to a third aspect of the present application, a method for processing a total hip joint image is provided, comprising: inputting the target hip X-ray original image of the information to be determined into a pre-trained second neural network model , identify the position of the femoral head; calculate the center of rotation of the femoral head based on the centroid formula of the plane image; determine the diameter of the acetabular cup according to the diameter of the femoral head; determine the center of rotation of the femoral head and the diameter of the acetabular cup The position of the acetabular cup; wherein, the training process of the second neural network model includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, and performing convolution pooling sampling until iterative The second neural network model is obtained by learning and training, wherein the pixel attribute value includes 0 representing the background pixel and 1 representing the femoral head pixel.

In order to achieve the above object, according to the fourth aspect of the present application, a method for processing a total hip joint image is provided, comprising: inputting the original X-ray image of the target hip joint with information to be determined into a pre-trained neural network model, identifying Obtain the femoral head area and the cortical bone area; determine the medullary cavity area according to the femoral head area and the bone cortex area; perform linear fitting on the coordinates of multiple center points of the medullary cavity area to determine the anatomical axis of the medullary canal; The original X-ray image is input into the pre-trained neural network model, and the femoral head area and the femoral neck base area are identified; based on the femoral head area and the femoral neck base area, the center coordinates of the femoral head and the center coordinate of the femoral neck base are determined; and determine the central axis of the femoral neck according to the center coordinate of the femoral head and the center coordinate of the base of the femoral neck; determine the femoral neck shaft angle based on the medullary canal anatomical axis and the femoral neck central axis; and determine the femoral stem prosthesis according to the femoral neck shaft angle model; wherein, the training process of the neural network model includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, performing convolution pooling sampling until iterative learning and training to obtain a third A neural network model, wherein the pixel attribute values include a value of 0 representing a background pixel, a value of 1 representing a pixel of the femoral head, and a value of 2 representing a pixel of the cortical bone; the training process of the neural network model further includes: The original X-ray image and the marked pixel attribute value are passed into the convolutional neural network, and the row convolution pooling sampling has been iteratively learned and trained to obtain a fourth neural network model, wherein the pixel attribute value includes the value representing the background pixel 0, the value of 1 for the pixels of the femoral head, and the value of 2 for the pixels of the base of the femoral neck.

In order to achieve the above object, according to the fifth aspect of the present application, a total hip image processing device based on deep learning and X-ray is provided.

The total hip image processing device based on deep learning and X-ray according to the present application includes: a scale calibration unit, which is configured to perform a real-size X-ray image of the hip joint according to the ratio of the image size of the reference object and its actual size. restoration; the leg length difference determination unit is configured to identify the X-ray image of the restored hip joint based on the first neural network model, and determine the leg length difference; the acetabular cup position determination unit is configured to be based on the second neural network The model recognizes the X-ray image of the restored hip joint, and determines the position of the acetabular cup; the femoral stem prosthesis specification determination unit is configured to determine the restoration of the hip joint based on the third neural network model and the fourth neural network model. The X-ray image is identified to determine the size of the femoral stem prosthesis; the osteotomy line determination unit is configured to determine the center of rotation of the femoral stem prosthesis and the center of rotation of the acetabular cup determined in the process of identifying the X-ray image of the hip joint Osteotomy line location.

In order to achieve the above object, according to a sixth aspect of the present application, a computer-readable storage medium is provided, where the computer-readable storage medium stores computer instructions, and the computer instructions are configured to cause the computer to execute the above-mentioned first The deep learning and X-ray-based total hip image processing method according to any one of the aspects.

In order to achieve the above object, according to a seventh aspect of the present application, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores data that can be A computer program executed by the at least one processor, where the computer program is executed by the at least one processor, so that the at least one processor executes the deep learning and X-ray-based method according to any one of the first aspects above. Total hip image processing method.

In the embodiments of the present application, in the method and device for processing a total hip joint image, an X-ray image of the hip joint is acquired, and the X-ray image of the hip joint includes an image of a reference object, and the reference object is a known size. Reference object; according to the ratio of the image size of the reference object and its actual size, restore the size of the X-ray image of the hip joint; identify the X-ray image of the restored hip joint based on the deep learning model, determine the leg length difference, The position of the acetabular cup and the specification and model of the femoral stem prosthesis; the position of the osteotomy line is determined according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition of the hip joint. It can be seen that, in the preoperative planning method for total hip arthroplasty in this embodiment, the X-ray image of the hip joint is restored to its real size, and subsequent position recognition is more accurate with the actual size; In the process of image recognition, the recognition is based on the deep learning model, which ensures the accuracy and speed of the leg length difference, acetabular cup position, femoral stem prosthesis size, and osteotomy line position determined according to the recognition results. This provides better preoperative support for total hip replacement surgery.

Description of drawings

The accompanying drawings, which form a part of this application, are used to provide an understanding of the application and make other features, objects and advantages of the application more apparent. The accompanying drawings and descriptions of the exemplary embodiments of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the attached image:

1 is a flowchart of a method for processing a total hip joint image based on deep learning and X-ray provided according to an embodiment of the present application;

2 is a schematic diagram of an X-ray image of a hip joint provided according to an embodiment of the present application;

3-4 are schematic diagrams of determining the actual position of the osteotomy line in the clinic according to an embodiment of the present application;

5 is a flowchart of a method for determining a leg length difference provided according to an embodiment of the present application;

6 is a schematic diagram of automatically identifying the key points of teardrops and the key points of the lesser trochanter according to an embodiment of the present application;

7 is a schematic diagram of a leg length difference determination provided according to an embodiment of the present application;

8 is a flowchart of a method for determining the position of an acetabular cup provided according to an embodiment of the present application;

9 is a schematic diagram of identifying a femoral head according to an embodiment of the present application;

10 is a schematic diagram of a center of rotation of a femoral head provided according to an embodiment of the present application;

11 is a schematic diagram of the position of an acetabular cup provided according to an embodiment of the present application;

12 is a flowchart of a method for determining the specification and model of a femoral stem prosthesis provided according to an embodiment of the present application;

13 is a schematic diagram of identifying a femoral head region and a cortical bone region provided according to an embodiment of the present application;

14 is a schematic diagram of a medullary cavity region provided according to an embodiment of the present application;

Fig. 15 is a schematic diagram of determining the anatomical axis of the medullary cavity according to an embodiment of the present application;

16 is a schematic diagram of identifying a femoral head region and a femoral neck base region according to an embodiment of the present application;

17 is a schematic diagram of a central axis of a femoral neck provided according to an embodiment of the present application;

FIG. 18 is a block diagram of a total hip joint image processing apparatus based on deep learning and X-ray provided according to an embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are Examples of some, but not all, examples of this application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application.

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances for the embodiments of the application described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

The embodiments in this application and the features in the embodiments may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

According to an embodiment of the present application, a method for processing a total hip joint image based on deep learning and X-ray is provided. As shown in FIG. 1 , the method includes the following steps:

S101. Acquire an X-ray image of the hip joint, and the X-ray image of the hip joint includes an image of a reference object.

An X-ray image of the hip joint is obtained by taking an X-ray of the hip joint while taking an object of known size, the reference object, in the same photo. The X-ray image of the hip joint thus obtained includes the image of the reference object. As shown in FIG. 2 , it is an X-ray image of the hip joint, wherein the image of the standard size at the bottom center of the image is the image of the reference object. In practical applications, the selection of the reference object and the discharge position during shooting can be adjusted according to the adaptability, which is not limited in this embodiment.

S102. According to the ratio of the image size of the reference object and its actual size, restore the size of the X-ray image of the hip joint.

The size of the reference object is known, and the image size of the reference object can also be obtained by measurement. According to the ratio of the image size of the reference object and its actual size, the difference between the X-ray image of the hip joint and the actual size of the hip joint can be determined. Scale (the two are the same scale), and then restore the true size of the X-ray image of the hip joint according to the scale. The restoration of the real size of the X-ray image of the hip joint is the basis for the subsequent image recognition, so that the leg length difference, the position of the acetabular cup, the size of the femoral stem prosthesis, and the position of the osteotomy line can be determined according to the recognition results. The gap with the actual corresponding position is smaller to ensure the accuracy of recognition.

Optionally, the restoration operation may be to select the size of a key part of an object of known size. By calculating the distance between two points between pixels in the image, and converting the ratio with the actual size of the object, the ratio is determined, and then the ratio of the X-ray image of the hip joint is corrected according to the ratio.

S103. Recognize the restored X-ray image of the hip joint based on the deep learning model, and obtain a recognition result, wherein the recognition result includes a key point position for determining the leg length difference, a thigh for determining the position of the acetabular cup Bone location, and the femoral head area, cortical bone area, and femoral neck base area used to determine the size of the femoral stem prosthesis.

The deep learning model is a neural network model. The input and output of the model that may be used to determine the leg length difference, the position of the acetabular cup, and the size of the femoral stem prosthesis may be different, but the principle of model training is the same. The principle of neural network model training is: convert the X-ray image of the hip joint into a 0-255 grayscale image, then manually select and label the image, and divide each pixel label of the image into several attribute values (according to the actual Requirements, the types of attribute values are different, such as two, three, etc.) and name them respectively, and then input them into the neural network model for convolution pooling sampling and iterative learning and training to obtain the neural network model.

The neural network model in this step is a classification neural network, which is to classify different areas in the image. For example, when determining the leg length difference, the neural network model is mainly used to identify key points of teardrops and lesser trochanter; For another example, when determining the position of the acetabular cup, the neural network model is mainly used to identify the femoral head area; for example, when determining the size of the femoral stem prosthesis, the neural network model is mainly used to identify the femoral head and bone cortex. area as well as the femoral head and base of the femoral neck.

The neural network in this embodiment may be a convolutional neural network LeNet, a convolutional neural network AlexNet, a visualization convolutional neural network ZF-Net, a convolutional neural network GoogleNet, a convolutional neural network VGG, a convolutional neural network Inception, a convolutional neural network Neural Network ResNet, Convolutional Neural Network DensNet, Convolutional Neural Network Inception ResNet, etc.

To determine the leg length difference, the position of the acetabular cup, and the size and model of the femoral stem prosthesis, some coordinates and fitting are calculated based on the image recognition results.

S104. Determine the position of the osteotomy line according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition process of the hip joint.

"Determine the position of the osteotomy line according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition of the hip joint" The rotation center of the femoral stem prosthesis can be adjusted by moving the femoral stem prosthesis Coinciding with the previously calculated center of rotation of the acetabular cup, the actual position of the femoral stem prosthesis is obtained. The location of the coating along the femoral stem component can determine the actual position of the osteotomy line in clinical practice, as shown in Figure 3-4. Figure 3 shows moving the femoral stem prosthesis to a predetermined position so that the rotation center of the femoral stem prosthesis coincides with the previously calculated acetabular cup rotation center position, and Figure 4 shows the position of the osteotomy line determined according to the shape of the femoral stem prosthesis.

From the above description, it can be seen that in the deep learning and X-ray-based total hip image processing method according to the embodiment of the present application, an X-ray image of the hip joint is obtained, and the X-ray image of the hip joint includes a reference object The image of the reference object is a reference object with a known size; according to the ratio of the image size of the reference object and its actual size, the X-ray image of the hip joint is restored in size; based on the deep learning model, the restored hip joint is The X-ray image is used to identify the leg length difference, the position of the acetabular cup and the size of the femoral stem prosthesis. The center of rotation determines the position of the osteotomy line. It can be seen that, in the preoperative planning method for total hip arthroplasty in this embodiment, the X-ray image of the hip joint is restored to its real size, and subsequent position recognition is more accurate with the actual size; In the process of image recognition, the recognition is based on the deep learning model, which ensures the accuracy and speed of the leg length difference, acetabular cup position, femoral stem prosthesis size, and osteotomy line position determined according to the recognition results. This provides better preoperative support for total hip replacement surgery.

As a refinement of the above-mentioned embodiment, step S103 separately describes the detailed steps of determining the leg length difference, the position of the acetabular cup, and the specification and model of the femoral stem prosthesis.

FIG. 5 shows a flowchart of an embodiment of determining the leg length difference, which may include the following steps:

S201. Convert the X-ray image of the hip joint into a grayscale image.

Convert the X-ray image of the hip joint to a 0-255 grayscale image.

S202. Predict the value of each pixel of the grayscale image based on the first neural network model, and determine the position of the key point of the teardrop and the key point of the lesser trochanter of the femur.

Before making predictions, the first neural network model must be obtained by training the samples. The unlabeled original image (the grayscale image corresponding to the X-ray sample image of the hip joint), as well as the manually identified and marked teardrop key points and the labels of the key points of the lesser trochanter of the femur can be passed into the convolutional neural network, and the input The original image is fitted with the Gaussian distribution function of the feature points, and the convolution pooling sampling is performed to iteratively learn and train to obtain the first neural network model. It should be noted that the convolutional neural network in this step can be the convolutional neural network LeNet, the convolutional neural network AlexNet, the visualization convolutional neural network ZF-Net, the convolutional neural network GoogleNet, the convolutional neural network VGG, the convolutional neural network. Neural Network Inception, Convolutional Neural Network ResNet, Convolutional Neural Network DensNet, Convolutional Neural Network Inception ResNet, etc.

After the first neural network model is obtained, the grayscale image corresponding to the X-ray image of the hip joint is input into the first neural network model, and the key points of the teardrop and the lesser trochanter can be automatically identified. As shown in FIG. 6 , FIG. 6 is a schematic diagram of automatically identifying the key points of the tear drop and the key points of the lesser trochanter of the femur.

S203. Determine the leg length difference according to the key point of the tear drop and the position of the key point of the lesser trochanter.

As shown in Figure 7, the horizontal straight line is determined by two key points of teardrop, which is the connection line of the two key points of teardrop, and the two vertical line segments are determined by the key point of the lesser trochanter and the horizontal straight line , the two vertical lines are denoted as A and B respectively, and the difference between A and B is the leg length difference.

Figure 8 shows an embodiment flow chart of determining the position of the acetabular cup, which may include the following steps:

S301. Convert the X-ray image of the hip joint into a grayscale image.

Convert the X-ray image of the hip joint to a 0-255 grayscale image.

S302. Predict the value of each pixel of the grayscale image based on the second neural network model, and determine the position of the femoral head.

Before making predictions, the second neural network model must first be obtained by training the samples. The unlabeled original image (the grayscale image corresponding to the X-ray sample image of the hip joint) and the labeled image that manually identifies the attribute value of the labeled pixel can be passed into the convolutional neural network, including two attribute values, named 0, 1. The value 0 represents the background pixel, and 1 represents the femoral head pixel; it is passed into the convolutional neural network, and the convolution pooling sampling is performed to iteratively learn and train to obtain the second neural network model. It should be noted that the convolutional neural network in this step can be the convolutional neural network LeNet, the convolutional neural network AlexNet, the visualization convolutional neural network ZF-Net, the convolutional neural network GoogleNet, the convolutional neural network VGG, the convolutional neural network. Neural Network Inception, Convolutional Neural Network ResNet, Convolutional Neural Network DensNet, Convolutional Neural Network Inception ResNet, etc.

After the second neural network model is obtained, the grayscale image corresponding to the X-ray image of the hip joint is input into the second neural network model, and each pixel value can be predicted. Each pixel value of the X-ray image is automatically classified into an attribute: 0-background, 1-femoral head, and the automatic identification of the femoral head area (ie, the position of the femoral head) is completed, as shown in Figure 9. Figure 9 is a schematic diagram of identifying the femoral head.

S303. Calculate the center of rotation of the femoral head according to the centroid formula of the plane image.

Because the obtained image of the femoral head area is a binary image, its mass distribution is uniform, so the centroid and centroid coincide, and the center point coordinates of the femoral head can be calculated according to the centroid formula of the plane image, that is, the rotation center of the femoral head. Assuming that the binary image is B[i,j], the coordinates of the center point of the femoral head can be obtained according to the following formula:

in:

What is obtained here is the pixel coordinates of the center point of the femoral head, and the pixel coordinates need to be converted into image coordinates.

The coordinates of the center of the image plane coordinates are:

then the pixel coordinates

The transformation formula to image coordinates (x', y') is:

Wherein, are the row and column spacing of the image array, respectively. Finally, through the output display module, the position of the rotation center of the femoral head is obtained, as shown in Figure 9. The center point of the circle in Figure 10 is the center of rotation of the femoral head.

S304. Calculate the diameter of the acetabular cup according to the diameter of the femoral head.

The diameter of the femoral head was determined from the area of the femoral head and the center of rotation of the femoral head, and the diameter of the acetabular cup was calculated from the diameter of the femoral head. To calculate the diameter of the acetabular cup according to the diameter of the femoral head, the diameter of the acetabular cup can be determined by referring to any of the existing calculation methods.

S305. Determine the acetabular cup position according to the bone rotation center and the acetabular cup diameter.

The position of the acetabular cup is automatically determined based on the diameter of the femoral head and the position of the center of rotation of the femoral head, as shown in Figure 11. The area delineated by the lines in Figure 11 is the position of the acetabular cup.

Figure 12 shows an embodiment flow chart of determining the specification and model of a femoral stem prosthesis, which may include the following steps:

S401. Convert the X-ray image of the hip joint into a grayscale image.

Convert the X-ray image of the hip joint to a 0-255 grayscale image.

S402. Identify the grayscale image based on the third neural network model, and determine the anatomical axis of the medullary cavity.

Determining the anatomical axis of the medullary canal can include the following steps:

First, predict each pixel value of the grayscale image based on the third neural network model to determine the femoral head area and the bone cortex area;

Before making predictions, a third neural network model must be obtained by training samples. The unlabeled original image (the grayscale image corresponding to the X-ray sample image of the hip joint) and the labeled image with the attribute value of the labeled pixel can be passed into the convolutional neural network, including three attribute values, named 0, 1, 2. The value 0 represents the background pixel, 1 represents the femoral head pixel, and 2 represents the bone cortex; it is passed into the convolutional neural network, and the convolutional pooling sampling is performed to iteratively learn and train to obtain the third neural network model. It should be noted that the convolutional neural network in this step can be the convolutional neural network LeNet, the convolutional neural network AlexNet, the visual convolutional neural network ZF-Net, the convolutional neural network GoogleNet, the convolutional neural network VGG, the convolutional neural network. Neural Network Inception, Convolutional Neural Network ResNet, Convolutional Neural Network DensNet, Convolutional Neural Network Inception ResNet, etc.

After the third neural network model is obtained, the grayscale image corresponding to the X-ray image of the hip joint is input into the third neural network model, and each pixel value can be predicted. Each pixel value of the X-ray image is automatically classified into an attribute: 0-background, 1-femoral head, 2-cortical bone, to complete the automatic identification of femoral head area and cortical bone area, as shown in Figure 13. 13 is a schematic diagram of identifying the femoral head region and the cortical bone region.

Secondly, determine the medullary cavity area according to the femoral head area and the bone cortex area;

The end of the lesser trochanter can be intercepted to the end of the femur, and the medullary canal region is obtained by subtracting the cortical bone region from the femoral region in the image, as shown in Figure 14.

Finally, the anatomical axis of the medullary canal is determined by linear fitting on the coordinates of multiple center points in the medullary canal region.

As shown in Figure 15, from the end position of the lesser trochanter, the intersection of each horizontal row and the medullary cavity is four coordinates, named A1, A2, B1, B2 from left to right; the midpoint can be obtained from the two points, A1(X1 , Y1), the midpoint coordinates of A2 (X2, Y2): B1, B2 can be calculated in the same way. Each row calculates the coordinates of the midpoint of the medullary canal in turn

Fitting these points into a straight line is the anatomical axis of the medullary canal (also the femoral anatomical axis).

S403. Identify the grayscale image based on the fourth neural network model, and determine the central axis of the femoral neck.

Determining the central axis of the femoral neck may include the following steps:

First, predict each pixel value of the grayscale image based on the fourth neural network model to determine the femoral head area and the femoral neck base area;

Before making predictions, a fourth neural network model must be obtained by training samples. The unlabeled original image (the grayscale image corresponding to the X-ray sample image of the hip joint) and the labeled image with the attribute value of the labeled pixel can be passed into the convolutional neural network, including three attribute values, named 0, 1, 2. The value 0 represents the background pixel, 1 represents the femoral head pixel, and 2 represents the femoral neck base pixel; it is passed into the convolutional neural network, and the convolutional pooling sampling is performed to iteratively learn and train to obtain the fourth neural network model. It should be noted that the convolutional neural network in this step can be the convolutional neural network LeNet, the convolutional neural network AlexNet, the visual convolutional neural network ZF-Net, the convolutional neural network GoogleNet, the convolutional neural network VGG, the convolutional neural network. Neural Network Inception, Convolutional Neural Network ResNet, Convolutional Neural Network DensNet, Convolutional Neural Network Inception ResNet, etc.

After the fourth neural network model is obtained, the grayscale image corresponding to the X-ray image of the hip joint is input into the fourth neural network model, and each pixel value can be predicted. Automatically classify each pixel value of the X-ray image into an attribute: 0-background, 1-femoral head, 2-femoral neck base pixel, complete the automatic identification of femoral head area and femoral neck base area, as shown in Figure 16 . Fig. 16 is a schematic diagram of identifying the femoral head region and the femoral neck base region.

Secondly, the center coordinates of the femoral head and the base of the femoral neck corresponding to the femoral head area and the femoral neck base area are calculated according to the centroid formula of the plane image;

The calculation methods of the center coordinates of the femoral head and the center coordinates of the base of the femoral neck are similar, and reference may be made to the implementation method of calculating the coordinates of the center point of the femoral head in step S303, which is not repeated here.

Finally, the center axis of the femoral neck is determined according to the center coordinates of the femoral head and the center coordinates of the base of the femoral neck.

The line connecting the center coordinates of the femoral head and the center coordinates of the base of the femoral neck is the center axis of the femoral neck, as shown in Figure 17. The two diagonally downward line segments in Figure 17 are the central axis of the femoral neck.

S404. Determine the femoral neck shaft angle according to the anatomical axis of the medullary cavity and the central axis of the femoral neck.

The angle formed by the anatomical axis of the medullary cavity and the central axis of the femoral neck is the femoral neck shaft angle.

S405. Determine the specification and model of the femoral stem prosthesis according to the femoral neck shaft angle, the medullary canal area determined in the process of determining the anatomical axis of the medullary canal, and the center of rotation of the femoral head.

According to the angle value of the femoral neck shaft angle, combined with the shape of the medullary cavity, the position of the center of rotation of the femoral head can give a recommendation for the selection of the femoral stem prosthesis model. Femoral stem prosthesis models are distinguished by the shape and size of the femoral stem prosthesis.

As a supplementary description of the embodiment of FIG. 1 , after the position of the osteotomy line is determined, it also includes calculating the post-operative leg length difference and offset according to the position of the osteotomy line. Offset includes femoral eccentricity: refers to the vertical distance from the center of rotation of the femoral head to the long axis of the femoral shaft. Also includes the joint offset, which can be the cumulative sum of the femoral and acetabular offsets.

It should be noted that the steps shown in the flowcharts of the accompanying drawings may be executed in a computer system, such as a set of computer-executable instructions, and, although a logical sequence is shown in the flowcharts, in some cases, Steps shown or described may be performed in an order different from that herein.

According to the embodiment of the present application, there is also provided a total hip joint image processing method based on deep learning and X-ray, comprising inputting the original X-ray image of the target hip joint with the information to be determined into the pre-trained first neural network model, Identify and obtain at least one key point position of a tear drop and at least one key point position of the lesser trochanter in the X-ray image of the target hip; The straight line determined by the position of the trochanter key point determines the leg length difference corresponding to the X-ray image of the target hip joint; wherein, the training process of the first neural network model includes: the original image of the X-ray of the hip joint, and the marked teardrop key point and the femur. The position of the key point of the lesser trochanter is input into the convolutional neural network as a sample set, and the input original image is fitted with the Gaussian distribution function of the feature points, and the convolution pooling sampling is performed to iteratively learn and train to obtain the first neural network model. , wherein the original image is a grayscale image corresponding to an unlabeled X-ray sample image of the hip joint.

In this embodiment, the optional implementation manner of determining the leg length difference is the same as the implementation manner of the first embodiment, and details are not described herein again.

According to the embodiment of the present application, there is also provided a total hip joint image processing method based on deep learning and X-ray, comprising inputting the target hip X-ray original image of the information to be determined into a pre-trained second neural network model, Identify the position of the femoral head; calculate the rotation center of the femoral head based on the center of mass formula of the plane image; determine the diameter of the acetabular cup according to the diameter of the femoral head; determine the hip rotation center according to the femoral head and the diameter of the acetabular cup cup position; wherein, the training process of the second neural network model includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, and performing convolution pooling sampling until iterative learning A second neural network model is obtained by training, wherein the pixel attribute value includes 0 representing the background pixel and 1 representing the femoral head pixel.

In this embodiment, the optional implementation manner of determining the position of the acetabular cup is the same as the implementation manner in the first embodiment, which will not be repeated here.

According to the embodiment of the present application, there is also provided a total hip joint image processing method based on deep learning and X-ray, including: inputting the original X-ray image of the target hip joint with information to be determined into a pre-trained neural network model, identifying Obtain the femoral head area and the cortical bone area; determine the medullary cavity area according to the femoral head area and the bone cortex area; perform linear fitting on the coordinates of multiple center points of the medullary cavity area to determine the anatomical axis of the medullary canal; The original X-ray image is input into the pre-trained neural network model, and the femoral head area and the femoral neck base area are identified; based on the femoral head area and the femoral neck base area, the center coordinates of the femoral head and the center coordinate of the femoral neck base are determined; and determine the center axis of the femoral neck according to the center coordinate of the femoral head and the center coordinate of the base of the femoral neck; determine the femoral neck shaft angle based on the anatomical axis of the medullary cavity and the femoral neck center axis; and determine the femoral stem prosthesis according to the femoral neck shaft angle model; wherein, the training process of the neural network model includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, performing convolution pooling sampling until iterative learning and training to obtain a third A neural network model, wherein the pixel attribute values include a value of 0 representing a background pixel, a value of 1 representing a pixel of the femoral head, and a value of 2 representing a pixel of the cortical bone; the training process of the neural network model further includes: The original X-ray image and the marked pixel attribute value are passed into the convolutional neural network, and the row convolution pooling sampling has been iteratively learned and trained to obtain a fourth neural network model, wherein the pixel attribute value includes the value representing the background pixel 0, the value of 1 for the pixels of the femoral head, and the value of 2 for the pixels of the base of the femoral neck.

In this embodiment, the optional implementation manner of determining the model of the femoral stem prosthesis is the same as the implementation manner in the first embodiment, which will not be repeated here.

According to the embodiment of the present application, a deep learning and X-ray-based total hip image processing device for implementing the methods described in Figures 1-17 is also provided. As shown in Figure 18, the device includes:

The scale calibration unit 51 is configured to perform a true restoration of the size of the X-ray image of the hip joint according to the ratio of the image size of the reference object and its actual size;

The leg length difference determining unit 52 is configured to identify the restored X-ray image of the hip joint based on the first neural network model, and determine the leg length difference;

The acetabular cup position determining unit 53 is configured to identify the restored X-ray image of the hip joint based on the second neural network model, and determine the acetabular cup position;

The femoral stem prosthesis specification determining unit 54 is configured to recognize the restored X-ray image of the hip joint based on the third neural network model and the fourth neural network model, and determine the femoral stem prosthesis specification;

The osteotomy line determining unit 55 is configured to determine the position of the osteotomy line according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition process of the hip joint.

For the process of implementing the functions of each unit and module in the apparatus of the embodiment of the present application, reference may be made to the relevant description in the method embodiment, and details are not repeated here.

From the above description, it can be seen that in the total hip image processing device based on deep learning and X-ray in the embodiment of the present application, an X-ray image of the hip joint is obtained, and the X-ray image of the hip joint includes a reference object The image of the reference object is a reference object with a known size; according to the ratio of the image size of the reference object and its actual size, the X-ray image of the hip joint is restored in size; based on the deep learning model, the restored hip joint is The X-ray image is used to identify the leg length difference, the position of the acetabular cup and the size of the femoral stem prosthesis. The center of rotation determines the position of the osteotomy line. It can be seen that, in the preoperative planning method for total hip arthroplasty in this embodiment, the X-ray image of the hip joint is restored to its real size, and subsequent position recognition is more accurate with the actual size; In the process of image recognition, the recognition is based on the deep learning model, which ensures the accuracy and speed of the leg length difference, acetabular cup position, femoral stem prosthesis size, and osteotomy line position determined according to the recognition results. This provides better preoperative support for total hip replacement surgery.

According to an embodiment of the present application, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are configured to cause the computer to perform the above method embodiments. A total hip image processing method based on deep learning and X-ray.

According to an embodiment of the present application, an electronic device is further provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores data executable by the at least one processor. The computer program is executed by the at least one processor, so that the at least one processor executes the deep learning and X-ray-based total hip joint image processing method in the above method embodiments.

Obviously, the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be It is implemented with program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately made into individual integrated circuit modules, or a plurality of modules or steps in them can be made into A single integrated circuit module is implemented. As such, the present application is not limited to any particular combination of hardware and software.

The above-mentioned embodiments are not intended to limit the present application, and for those skilled in the art, the present application may have various modifications and changes.

Claims

A method for processing a total hip joint image, the method comprising:

acquiring an X-ray image of the hip joint, the X-ray image of the hip joint includes an image of a reference object, and the reference object is a reference object of known size;

According to the ratio of the image size of the reference object and its actual size, the size of the X-ray image of the hip joint is restored;

Identify the restored X-ray image of the hip joint based on the deep learning model, and obtain the identification result, wherein the identification result includes the position of the key point used to determine the leg length difference, the position of the femoral head used to determine the position of the acetabular cup , and the femoral head area, cortical bone area and femoral neck base area used to determine the size of the femoral stem prosthesis;

The position of the osteotomy line is determined according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition of the hip joint.
The method for processing a total hip joint image according to claim 1, wherein the identification of the restored X-ray image of the hip joint based on the deep learning model to determine the leg length difference comprises:

Convert the X-ray image of the hip joint to a grayscale image;

Predict the value of each pixel of the grayscale image based on the first neural network model, and determine the key point of the teardrop and the position of the key point of the lesser trochanter;

The leg length difference was determined according to the key point of the teardrop and the key point of the lesser trochanter of the femur.
The method for processing a total hip joint image according to claim 1, wherein the identifying and determining the position of the acetabular cup based on the deep learning model of the restored X-ray image of the hip joint comprises:

Convert the X-ray image of the hip joint to a grayscale image;

Predict each pixel value of the grayscale image based on the second neural network model, and determine the position of the femoral head;

Calculate the center of rotation of the femoral head according to the centroid formula of the plane image;

Calculate the diameter of the acetabular cup from the diameter of the femoral head;

The position of the acetabular cup is determined according to the center of rotation of the bone and the diameter of the acetabular cup.
The method for processing a total hip joint image according to claim 1, wherein the recognizing the X-ray image of the restored hip joint based on the deep learning model to determine the specification and model of the femoral stem prosthesis comprises:

Convert the X-ray image of the hip joint to a grayscale image;

Identify the grayscale image based on the third neural network model, and determine the anatomical axis of the medullary cavity;

Identify the grayscale image based on the fourth neural network model, and determine the central axis of the femoral neck;

Determine the femoral neck shaft angle according to the anatomical axis of the medullary cavity and the central axis of the femoral neck;

The size of the femoral stem prosthesis is determined according to the femoral neck shaft angle, the area of the medullary canal determined during the process of determining the anatomical axis of the medullary canal, and the center of rotation of the femoral head.
The method for processing a total hip joint image according to claim 1, wherein the osteotomy line is determined according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition of the hip joint Locations include:

Coincide the rotation center of the femoral stem prosthesis with the rotation center of the acetabular cup to determine the actual position of the femoral stem prosthesis;

Determine the position of the osteotomy line along the coating position of the femoral stem component.
The method for processing a total hip joint image according to claim 4, wherein the identifying the grayscale image based on the third neural network model, and determining the anatomical axis of the medullary cavity comprises:

Predict each pixel value of the grayscale image based on the third neural network model, and determine the femoral head area and the bone cortex area;

Determine the medullary cavity area according to the femoral head area and the bone cortex area;

The anatomical axis of the medullary canal was determined by linear fitting to the coordinates of multiple center points in the medullary canal region.
The method for processing a total hip joint image according to claim 4, wherein the identifying the grayscale image based on the fourth neural network model, and determining the central axis of the femoral neck comprises:

Predict each pixel value of the grayscale image based on the fourth neural network model, and determine the femoral head area and the femoral neck base area;

Calculate the femoral head center coordinates and the femoral neck base center coordinates corresponding to the femoral head area and the femoral neck base area according to the centroid formula of the plane image;

The central axis of the femoral neck is determined according to the central coordinates of the femoral head and the central coordinates of the base of the femoral neck.
The method for processing a total hip joint image according to claim 1, further comprising calculating the postoperative leg length difference and offset distance according to the position of the osteotomy line.
A method for processing a total hip joint image, comprising:

Input the original image of the target hip X-ray with the information to be determined into the pre-trained first neural network model, and identify at least one teardrop key point position and at least one lesser trochanter in the target hip X-ray image. key point location;

Determine the leg length difference corresponding to the target hip X-ray image based on the connection line determined by the position of the at least one key point of the teardrop and the straight line determined by the position of the at least one key point of the lesser trochanter;

Wherein, the training process of the first neural network model includes: inputting the original X-ray image of the hip joint, the marked teardrop key point and the key point position of the femoral lesser trochanter as a sample set into the convolutional neural network, and the input said The original image is fitted with the Gaussian distribution function of the feature points, and the first neural network model is obtained by performing convolution pooling sampling and iterative learning and training, wherein the original image is the gray corresponding to the unlabeled X-ray sample image of the hip joint. Degree Chart.
A method for processing a total hip joint image, comprising:

Input the original X-ray image of the target hip joint of the information to be determined into the pre-trained second neural network model, and identify the position of the femoral head;

Calculate the center of rotation of the femoral head based on the centroid formula of the plane image; determine the diameter of the acetabular cup according to the diameter of the femoral head; determine the position of the acetabular cup according to the center of rotation of the femoral head and the diameter of the acetabular cup;

Wherein, the training process of the second neural network model includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, and performing convolution pooling sampling until iterative learning and training to obtain the second neural network. A neural network model, wherein the pixel attribute values include a value of 0 representing a background pixel and a value of 1 representing a pixel of the femoral head.
A method for processing a total hip joint image, comprising:

Input the original X-ray image of the target hip joint with the information to be determined into the pre-trained neural network model, and identify the femoral head area and the bone cortex area;

Determine the medullary cavity area according to the femoral head area and the bone cortex area; perform linear fitting on the coordinates of multiple center points of the medullary cavity area to determine the anatomical axis of the medullary cavity;

Input the original X-ray image of the target hip joint with the information to be determined into the pre-trained neural network model, and identify the femoral head area and the femoral neck base area;

Determine the femoral head center coordinate and the femoral neck base center coordinate based on the femoral head area and the femoral neck base area; and determine the femoral neck center axis according to the femoral head center coordinate and the femoral neck base center coordinate;

Determine the femoral neck shaft angle based on the anatomical axis of the medullary cavity and the central axis of the femoral neck; and determine the femoral stem prosthesis model according to the femoral neck shaft angle;

Wherein, the training process of the neural network model includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, performing convolutional pooling sampling until iterative learning and training to obtain a third neural network model, wherein the pixel attribute value includes a value of 0 representing a background pixel, a value of 1 representing a pixel of the femoral head, and a value of 2 representing a pixel of the cortical bone;

The training process of the neural network model further includes: inputting the original X-ray image of the hip joint and the marked pixel attribute value into the convolutional neural network, and performing convolutional pooling sampling and iterative learning and training to obtain a fourth neural network. model, wherein the pixel attribute value includes a value of 0 representing the background pixel, a value of 1 representing the pixel of the femoral head, and a value of 2 representing the pixel of the base of the femoral neck.
A processing device for a total hip joint image, comprising:

The scale calibration unit is configured to restore the real size of the X-ray image of the hip joint according to the ratio of the image size of the reference object and its actual size;

The leg length difference determining unit is configured to recognize the restored X-ray image of the hip joint based on the first neural network model, and determine the leg length difference;

an acetabular cup position determination unit, configured to identify the X-ray image of the restored hip joint based on the second neural network model, and determine the position of the acetabular cup;

The femoral stem prosthesis specification determination unit is configured to recognize the restored X-ray image of the hip joint based on the third neural network model and the fourth neural network model, and determine the femoral stem prosthesis specification;

The osteotomy line determination unit is configured to determine the position of the osteotomy line according to the rotation center of the femoral stem prosthesis and the rotation center of the acetabular cup determined during the X-ray image recognition of the hip joint.
A computer-readable storage medium storing computer instructions, the computer instructions being configured to cause the computer to execute the method for processing a total hip joint image according to any one of claims 1 to 8 , or claim 9 , or claim 10 , or the method for processing a total hip joint image according to claim 11 .