WO2023024883A1 - 基于深度强化学习的股骨柄放置方法、装置及电子设备 - Google Patents

基于深度强化学习的股骨柄放置方法、装置及电子设备 Download PDF

Info

Publication number
WO2023024883A1
WO2023024883A1 PCT/CN2022/110973 CN2022110973W WO2023024883A1 WO 2023024883 A1 WO2023024883 A1 WO 2023024883A1 CN 2022110973 W CN2022110973 W CN 2022110973W WO 2023024883 A1 WO2023024883 A1 WO 2023024883A1
Authority
WO
WIPO (PCT)
Prior art keywords
femoral
femoral stem
network
segmentation
femur
Prior art date
Application number
PCT/CN2022/110973
Other languages
English (en)
French (fr)
Inventor
张逸凌
刘星宇
Original Assignee
北京长木谷医疗科技有限公司
张逸凌
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京长木谷医疗科技有限公司, 张逸凌 filed Critical 北京长木谷医疗科技有限公司
Publication of WO2023024883A1 publication Critical patent/WO2023024883A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the present application relates to the field of medical technology, in particular to a deep reinforcement learning-based femoral stem placement method, device and electronic equipment.
  • the femoral stem as shown in Figure 1 is often used as a biological prosthesis and used in conjunction with hip joint components.
  • points A and The height difference at point B is the leg length difference.
  • the relative position of the femoral stem and the femur is mainly determined by manual experience. This method has low accuracy and may cause loosening of the femoral stem or hip joint pain of the patient due to improper relative position.
  • the main purpose of this application is to provide a femoral stem placement method and device based on deep reinforcement learning to solve the problem of femoral stem loosening or hip joint pain in patients caused by improper relative position of femoral stem and femur.
  • the first aspect of the present application provides a femoral stem placement method based on deep reinforcement learning, including:
  • the dual network architecture DDQN with deep reinforcement learning is used to output adjustment information according to the relative position of the femoral stem and the femur;
  • the femoral stem is adjusted according to the adjustment information, so that the femoral stem matches the femur.
  • the segmentation and extraction of the femoral medullary cavity cortex region through the segmentation neural network model based on the pelvis and femur image data includes:
  • the segmentation neural network model includes a cascaded first segmentation neural network and a second segmentation neural network
  • the associated parameters of the first segmentation neural network and the second segmentation neural network are determined by training and testing image data in a pre-stored medical image database.
  • the first segmentation neural network is used as a backbone network to roughly segment the pelvis and femur image data;
  • the second segmentation neural network is used for precise segmentation based on the rough segmentation
  • the first segmentation neural network is a full convolutional network FCN, a semantic segmentation network SegNet, a deep learning segmentation network Unet, a 3D-deep learning segmentation network 3D-Unet, an instance segmentation network Mask-RCNN, a hole convolution, and a semantic segmentation neural network ENet, Semantic Segmentation Network
  • CRFasRNN scene analysis network PSPNet, end-to-end semantic segmentation network ParseNet, image semantic segmentation network RefineNet, image segmentation model ReSeg, semantic segmentation network LSTM-CF, instance segmentation network DeepMask, semantic segmentation model DeepLabV1, semantic segmentation model DeepLabV2, semantic segmentation At least one of the models DeepLabV3;
  • the second segmentation neural network is at least one of EfficientDet, SimCLR, and PointRend.
  • the determining the femoral medullary canal axis of the femoral medullary canal cortex region, and placing the axis of the femoral stem along the axis of the femoral medullary canal for placing the femoral stem includes:
  • the axis of any type of femoral stem is placed at any position in the femur along the axis of the femoral medullary canal.
  • the dual network architecture DDQN using deep reinforcement learning outputs adjustment information according to the relative position of the femoral stem and the femur, including:
  • the method further includes:
  • the reward is determined according to the reward function, and the reward is transformed and fed back to the value network for learning and iteration, so that the reward reaches the maximum value;
  • the reward r is determined according to the following reward function:
  • the fitting degree is the average depth of the femoral stem embedded in the femoral medullary canal cortex
  • the leg length difference is the absolute value of the height difference of the upper edge of the bilateral lesser trochanter
  • a is the average embedding depth of all layers of the femoral medullary canal cortex that fits the femoral stem value
  • b is the set cutting depth.
  • the adjustment information includes at least one of femoral stem model adjustment information and femoral stem position adjustment information output by the classifier, and adjusting the femoral stem according to the adjustment information includes:
  • model adjustment information of the femoral handle is that the model of the femoral handle is increased, then the model of the femoral handle is increased by a corresponding size;
  • model adjustment information of the femoral stem is that the model of the femoral stem is reduced, then the model of the femoral stem is reduced by a corresponding size;
  • the position adjustment information of the femoral stem is that the femoral stem moves upward, then move the position of the femoral stem to the end close to the pelvis by a corresponding distance;
  • the position adjustment information of the femoral stem is that the femoral stem moves down, the position of the femoral stem is moved to the end away from the pelvis by a corresponding distance.
  • the second aspect of the present application provides a femoral stem placement device based on deep reinforcement learning, comprising:
  • the segmentation unit is configured to obtain the image data of the patient's pelvis and femur, based on the image data of the pelvis and femur, segment and extract the cortical area of the femoral medullary cavity through a segmentation neural network model;
  • a determination unit configured to determine the femoral canal axis of the femoral medullary canal cortex region, and align the axis of the femoral stem along the axis of the femoral canal for placing the femoral stem;
  • the output unit is configured as a dual-network architecture DDQN using deep reinforcement learning to output adjustment information according to the relative position of the femoral stem and the femur;
  • the adjustment unit is configured to adjust the femoral stem according to the adjustment information, so that the femoral stem matches the femur.
  • the third aspect of the present application provides a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the deep reinforcement learning-based method provided by any one of the first aspect. Femoral stem placement method.
  • the fourth aspect of the present application provides an electronic device, the electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores information that can be used by the at least one processor A computer program executed by a processor, the computer program is executed by the at least one processor, so that the at least one processor executes the femoral stem placement method based on deep reinforcement learning provided by any one of the first aspect.
  • the axis of the femoral stem is placed along the axis of the femoral medullary cavity, and adjusted through the double network architecture DDQN of deep reinforcement learning, so that the femoral stem matches the femur, which improves the accuracy of femoral stem placement and solves the problem of
  • the relative position of the femoral stem and the femur is improper, resulting in loosening of the femoral stem or pain in the patient's hip joint.
  • Fig. 1 is the schematic diagram of femoral stem prosthesis
  • Figure 2 is a schematic diagram of the difference in leg length with the height difference between point A and point B on the upper edge of the lesser trochanter on both sides;
  • Fig. 3 is the schematic flow chart of the femoral stem placement method based on deep reinforcement learning provided by the embodiment of the present application;
  • Fig. 4 is a working principle diagram of the segmentation neural network model provided by the embodiment of the present application.
  • FIG. 5 is a schematic diagram of the axis of the femoral medullary canal and the central axis of the femoral stem prosthesis provided by the embodiment of the present application;
  • FIG. 6 is a dual network architecture DDQN provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the contour of the femoral stem prosthesis embedded in the contour of the femoral medullary cavity cortex provided by the embodiment of the present application;
  • Figure 8 is a block diagram of a femoral stem placement device based on deep reinforcement learning provided by an embodiment of the present application
  • FIG. 9 is a block diagram of an electronic device provided by an embodiment of the present application.
  • the femoral stem as shown in Figure 1 is often used as a biological prosthesis and used in conjunction with hip joint components.
  • points A and The height difference at point B is the leg length difference.
  • the relative position of the femoral stem and the femur is mainly determined by manual experience. This method has low accuracy and may cause loosening of the femoral stem or hip joint pain of the patient due to poor relative position.
  • the embodiment of the present application provides a femoral stem placement method based on deep reinforcement learning, as shown in Figure 3, the method includes the following steps S101 to S104:
  • Step S101 Obtain image data of the patient's pelvis and femur, and segment and extract the femoral medullary cavity cortex region through the segmentation neural network model based on the image data of the pelvis and femur; in order to place the femoral stem prosthesis into the femur, it is necessary to obtain
  • the image data of the patient's pelvis and femur are manually marked or automatically marked with the cortical area of the femoral medullary cavity, so as to extract the cortical area of the femoral medullary cavity through the segmentation neural network model.
  • the neural network model is used to segment and extract the femoral medullary cavity cortex region, including:
  • the segmentation neural network model includes a cascaded first segmentation neural network and a second segmentation neural network
  • the associated parameters of the first segmentation neural network and the second segmentation neural network are determined by training and testing image data in a pre-stored medical image database.
  • the image data in the pre-stored medical image database includes a CT medical image dataset of patients with hip joint disease.
  • the first segmentation neural network is used as a backbone network to roughly segment the pelvis and femur image data;
  • the second segmentation neural network is used for precise segmentation based on the rough segmentation
  • the first segmentation neural network is a full convolutional network FCN, a semantic segmentation network SegNet, a deep learning segmentation network Unet, a 3D-deep learning segmentation network 3D-Unet, an instance segmentation network Mask-RCNN, a hole convolution, and a semantic segmentation neural network ENet, semantic segmentation network CRFasRNN, scene analysis network PSPNet, end-to-end semantic segmentation network ParseNet, image semantic segmentation network RefineNet, image segmentation model ReSeg, semantic segmentation network LSTM-CF, instance segmentation network DeepMask, semantic segmentation model DeepLabV1, semantic segmentation At least one of the model DeepLabV2 and the semantic segmentation model DeepLabV3;
  • the second segmentation neural network is at least one of EfficientDet, SimCLR, and PointRend.
  • the pixel-level segmentation neural network can be used to segment the cortical region of the femoral medullary cavity, including the following steps:
  • the CT two-dimensional medical image data set contains several CT two-dimensional medical images, which are marked manually or automatically on the two-dimensional CT medical images. Mark the cortical region of the femoral medullary cavity and use it as a database.
  • Two groups of CT two-dimensional medical images in different formats were divided into training set and test set according to the ratio of 7:3, the DICOM data of CT two-dimensional medical images were converted into JPG format pictures, and the labeled CT two-dimensional medical images The image is converted into a picture in PNG format, and the last two sets of data are saved as the input of the neural network.
  • each downsampling The layer includes 2 convolutional layers and 1 pooling layer, the size of the convolution kernel is 3*3, the size of the convolution kernel in the pooling layer is 2*2, and the number of convolution kernels in each convolution layer The number is 128, 256, 512; each upsampling layer includes 1 upsampling layer and 2 convolution layers, where the convolution kernel size of the convolution layer is 3*2, and the convolution kernel size in the upsampling layer is 2*2, and the number of convolution kernels in each upsampling layer is 512, 256, and 128.
  • the training batch_size is 6, the learning rate is set to 1e -4 , the optimizer uses the Adam optimizer, and the loss function used is DICE loss. All the training sets are sent to the network for training. According to the loss function in the training process , adjust the size of the training batch, and finally get the rough segmentation results of each part.
  • bilinear interpolation is used to upsample the prediction results of the previous step, and then N most uncertain points are selected in this denser feature map, such as points with a probability close to 0.5. Then calculate the feature representation of these N points and predict their labels. This process is repeated until the required size is up-sampled.
  • the segmentation neural network model outputs the cortical region of the femoral medullary cavity.
  • Step S102 Determine the femoral medullary canal axis of the femoral medullary canal cortical area, and align the axis of the femoral stem along the femoral medullary canal axis for placing the femoral stem; after extracting the femoral medullary canal cortical area by segmentation , extract the axis of the femoral medullary canal, and coincide the axis of the placed femoral stem prosthesis with the axis of the femoral medullary canal to ensure the most appropriate and best relative position between the femoral stem and the femur, and avoid postoperative loosening of the femoral stem or patient Complications such as hip pain.
  • the step S102 includes:
  • the classification neural network the cortical area of the femoral medullary cavity is classified into levels, and the level of the femoral medullary cavity is separated; the classification neural network is obtained according to a large number of training samples in advance, and the training samples are images that have marked the type of the femoral medullary cavity.
  • the input of the classification neural network is a DICOM two-dimensional cross-sectional image, and the output is whether it is the level of the femoral medullary cavity;
  • the classification neural network is a convolutional neural network LeNet, a convolutional neural network AlexNet, a visual convolutional neural network ZF-Net, and a convolutional neural network.
  • the method before performing layer classification on the femoral medullary cavity cortex region according to the classification neural network, the method further includes: obtaining a training sample, the training sample including a two-dimensional cross-sectional image manually marked with the category to which the layer belongs;
  • the classification neural network is obtained by training according to the training samples.
  • the method further includes: performing image sharpening processing on the femoral medullary cavity level according to a high threshold processing method.
  • a clearing process specifically to perform image clearing processing on the distinguished medullary canal levels, for example, OpenCV high threshold can be used.
  • the processing method makes the shape of the medullary cavity clearer.
  • the thresholding of the image is to use the distribution rule of the image pixels, set the threshold to segment the pixels, and then obtain the binary image of the image.
  • the straight line fitting method may be any existing straight line fitting algorithm such as the least square method, gradient descent, Gauss-Newton, and Lemma algorithm.
  • the axis of any type of femoral stem is placed at any position in the femur along the axis of the femoral medullary canal. After the femoral medullary canal axis is determined, any type of femoral stem prosthesis is randomly selected, and the central axis of the femoral stem prosthesis is placed at a random position in the femur along the femoral medullary canal axis. The axis of the medullary canal coincides to ensure the best relative position between the femoral stem and the femur.
  • the axis of the femoral medullary canal is shown on the right in Figure 5, and the central axis of the femoral stem prosthesis shown on the left coincides with the axis of the femoral medullary canal .
  • Step S103 Using the dual network architecture DDQN based on deep reinforcement learning, output adjustment information according to the relative position of the femoral stem and the femur; according to the dual network architecture DDQN based on deep reinforcement learning, determine the size of the current femoral stem and the relationship between the femoral stem and the The direction of improvement of the relative positional relationship of the femur is to predict the output adjustment information through the classifier of the dual network architecture DDQN, so as to adjust the model size of the femoral stem and the relative position of the femoral stem and the femur to be the most suitable or optimal, so that the leg length of the two legs The difference is as small as possible to avoid postoperative complications such as hip joint pain.
  • the step S103 includes:
  • the encoder-decoder network structure is connected to the fully connected layer FC layer, as shown in Figure 6, where the encoder is the encoder, the decoder is the decoder, and the FC layer is fully connected Layer, as a classifier, outputs the classification of femoral stem size change and femoral stem displacement;
  • DDQN dual network includes exploration network and value network
  • the classifier receives the relative position of the femoral stem and the femur, and outputs the adjustment information of the femoral stem model and the position of the femoral stem. If the output is an increase in size, the size of the femoral stem will be increased by the corresponding size. If the output is a decrease in size, the size of the femoral stem will increase.
  • the size of the model is reduced accordingly; if the output is that the femoral stem moves upward, the femoral stem moves a corresponding distance (for example, 1 mm) toward the end close to the pelvis; if the output is that the femoral stem moves downward, the femoral stem moves toward the end away from the pelvis corresponding distance.
  • the size of the model of the femoral stem is relatively fixed, and it can be arranged from size0 (model 0) to size12 (model 12) in order of size from small to large, and the shape of different models is unchanged and the size increases in turn.
  • the method further includes:
  • the reward is determined according to the reward function, and the reward is transformed and fed back to the value network for learning and iteration, so that the reward reaches the maximum value;
  • the reward r is determined according to the following reward function:
  • the fitting degree is the average depth of the femoral stem embedded in the femoral medullary canal cortex
  • the leg length difference is the absolute value of the height difference of the upper edge of the bilateral lesser trochanter
  • a is the average embedding depth of all layers of the femoral medullary canal cortex that fits the femoral stem value
  • b is the set depth of cut, that is, the best cut angle artificially set.
  • the schematic diagram of the femoral stem prosthesis embedded in the cortical contour of the femoral medullary cavity is shown in Figure 7, where the smooth curve is the cortical contour of the femoral medullary cavity, the broken line is the femoral stem prosthesis contour, and the lines in the shaded area are the femoral stem prosthesis contours at various levels
  • the depth of embedding into the cortex of the femoral medullary cavity, b is generally set by the doctor as 2 mm, when a is 0, the fit is 0, when a is 2 mm, the fit is maximum, and the prosthesis is embedded when it exceeds 2 mm If the depth is too deep, the fitting degree decreases, and when it is lower than 2 mm, the fitting degree increases with the increase of a, and the model is more realistic.
  • the reward obtained at this time can be calculated according to the reward function.
  • the larger the reward the more suitable the size of the femoral stem and the better the relative position between the femoral stem and the femur , the higher the accuracy;
  • the DDQN dual-network value network is used to fit the value function.
  • the reward given by the reward function is transformed and fed back to the value network for learning and iteration, so that the reward reaches the maximum value; after the training is completed, the value network is the final model obtained.
  • the classifier outputs the values of different femoral stem models and different relative positions between the femoral stem and the femur in the current scene. When the value is the largest, it can be regarded as the best femoral stem model and the best relative position between the femoral stem and the femur. .
  • Step S104 Adjust the femoral stem according to the adjustment information, so that the femoral stem matches the femur.
  • the femoral stem matches the femur the type of the femoral stem matches the femur, the position of the femoral stem matches the femur, the relative position of the femoral stem and the femur is optimal, and the leg length difference between the two legs is the smallest when the femoral stem is placed in the femur .
  • the adjustment information includes at least one of the femoral stem model adjustment information and the femoral stem position adjustment information output by the classifier, and the adjustment according to the adjustment information includes:
  • model adjustment information of the femoral handle is that the model of the femoral handle is increased, then the model of the femoral handle is increased by a corresponding size;
  • model adjustment information of the femoral stem is that the model of the femoral stem is reduced, then the model of the femoral stem is reduced by a corresponding size;
  • the position adjustment information of the femoral stem is that the femoral stem moves upward, then move the position of the femoral stem to the end close to the pelvis by a corresponding distance;
  • the position adjustment information of the femoral stem is that the femoral stem moves down, the position of the femoral stem is moved to the end away from the pelvis by a corresponding distance.
  • the femoral stem size adjustment information is that the femoral stem is increased by 1 size
  • the size of the femoral stem is increased by 1 size
  • the femoral stem size adjustment information is that the femoral stem is decreased by 2 sizes
  • the femoral stem The size of the model is reduced by 2 models; if the femoral stem position adjustment information is that the femoral stem is moved up by 1 mm, then the position of the femoral stem is moved to the end close to the pelvis by 1 mm; if the femoral stem position adjustment information is that the femoral stem is moved down 2 mm, then move the position of the femoral stem 2 mm away from the end of the pelvis;
  • the size or position of the femoral stem is adjusted so that the reward is maximized, the femoral stem matches the femur, and the legs of both legs when the femoral stem is placed in the femur.
  • the length difference is the smallest to achieve the most suitable femoral stem model and the best relative position between the femoral stem and the femur.
  • This application adjusts through the dual network architecture DDQN of deep reinforcement learning, selects the appropriate femoral stem model and the best relative position between the femur and the femoral stem, and ensures that the leg length difference between the two legs is minimized when the femoral stem is of appropriate size.
  • the accuracy of placing the femoral stem is improved, and the problem in the prior art that the femoral stem is loose or the hip joint pain of the patient is solved due to the improper relative position of the femoral stem and the femur.
  • the reward can be maximized, the femoral stem matches the femur, and the leg length difference between the two legs is the smallest, so as to achieve the optimal femoral stem model and the femoral stem and femur.
  • Optimal relative position of the femur Optimal relative position of the femur.
  • the embodiment of the present application also provides a deep reinforcement learning-based femoral stem placement device for implementing the above-mentioned deep reinforcement learning-based femoral stem placement method, as shown in FIG. 8 , the device includes:
  • Segmentation unit 81 used to obtain the patient's pelvis and femur image data, based on the pelvis and femur image data, segment and extract the femoral medullary cavity cortex region through the segmentation neural network model;
  • a determining unit 82 configured to determine the axis of the femoral medullary canal in the cortical region of the femoral medullary canal, and align the axis of the femoral stem along the axis of the femoral medullary canal for placing the femoral stem;
  • the output unit 83 is used to adopt the dual network architecture DDQN of deep reinforcement learning to output adjustment information according to the relative position of the femoral stem and the femur;
  • the adjustment unit 84 is configured to adjust the femoral stem according to the adjustment information, so that the femoral stem matches the femur.
  • An embodiment of the present application also provides an electronic device.
  • the electronic device includes one or more processors 91 and a memory 92 , and one processor 91 is taken as an example in FIG. 9 .
  • the controller may also include: an input device 93 and an output device 94 .
  • the processor 91 , the memory 92 , the input device 93 and the output device 94 may be connected through a bus or in other ways. In FIG. 9 , connection through a bus is taken as an example.
  • the processor 91 can be a central processing unit (Central Processing Unit, referred to as CPU), and the processor 91 can also be other general processors, digital signal processors (Digital Signal Processor, referred to as DSP), application specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), field-programmable gate array (Field-Programmable Gate Array, referred to as FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips , the general purpose processor can be a microprocessor or any conventional processor.
  • CPU Central Processing Unit
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the general purpose processor can be a microprocessor or any conventional processor.
  • the memory 92 can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as program instructions/modules corresponding to the control method in the embodiment of the present application.
  • Processor 91 executes various functional applications and data processing of the server by running non-transitory software programs, instructions and modules stored in memory 92, that is, implements the femoral stem placement method based on deep reinforcement learning in the above method embodiment.
  • the memory 92 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function; the data storage area may store data created according to use of a processing device operated by the server, and the like.
  • the memory 92 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices.
  • the memory 92 may optionally include a memory located remotely from the processor 91, and these remote memories may be connected to a network connection device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the input device 93 can receive input numbers or character information, and generate key signal input related to user settings and function control of the processing device of the server.
  • the output device 94 may include a display device such as a display screen.
  • One or more modules are stored in the memory 92, and when executed by one or more processors 91, perform the method shown in FIG. 3 .
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, referred to as ROM), a random access memory (Random Access Memory, referred to as RAM), a flash memory (Flash Memory, referred to as FM), hard disk (Hard Disk Drive, referred to as HDD) or solid-state drive (Solid-State Drive, referred to as SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.
  • ROM Read-Only Memory
  • RAM random access memory
  • FM Flash Memory
  • HDD Hard Disk Drive
  • SSD solid-state drive

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种基于深度强化学习的股骨柄放置方法、装置及电子设备，其中方法包括：获取患者骨盆及股骨图像数据，通过分割神经网络模型分割提取股骨髓腔皮质区域；确定股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿股骨髓腔轴线，以用于放置股骨柄；采用双网络架构DDQN，根据股骨柄与股骨的相对位置，输出调节信息；根据调节信息对股骨柄进行调节，以使股骨柄与股骨相匹配。本申请将股骨柄的轴线沿股骨髓腔轴线放置，并通过双网络架构DDQN进行调节，提高了股骨柄放置时的精度，使得股骨柄与股骨相匹配，解决了现有技术中因股骨柄与股骨的相对位置不当，导致股骨柄松动或患者髋关节疼痛的问题。

Description

基于深度强化学习的股骨柄放置方法、装置及电子设备
相关申请的交叉引用
本申请要求于2021年08月24日提交的申请号为202110974127.3，名称为“基于深度强化学习的股骨柄放置方法、装置及电子设备”的中国专利申请的优先权，其通过引用方式全部并入本文。
技术领域
本申请涉及医学技术领域，具体涉及一种基于深度强化学习的股骨柄放置方法、装置及电子设备。
背景技术
在医学上，当患者的髋关节因病变导致行动不便时，通常会以手术的方式为病人置换人工髋关节，使患者恢复日常的活动能力。在髋关节置换手术中，往往将如图1所示的股骨柄作为生物型假体，与髋关节组件配合使用。在置换人工髋关节时，需要采用合适大小的股骨柄，并且确定股骨柄和股骨的相对位置以使置换后两腿的腿长差尽可能小，图2中双侧小转子上缘A点和B点的高度差即为腿长差。
然而，相关技术中股骨柄与股骨的相对位置主要依靠人工经验确定，这种方式精确度较低，容易因为相对位置不当导致股骨柄松动或患者髋关节疼痛。
针对相关技术中因股骨柄与股骨的相对位置不当，导致股骨柄松动或患者髋关节疼痛的问题，目前尚未提出有效的解决方案。
发明内容
本申请的主要目的在于提供一种基于深度强化学习的股骨柄放置方法及装置，以解决股骨柄与股骨的相对位置不当导致股骨柄松动或患者髋关节疼痛的问题。
为了实现上述目的，本申请的第一方面提供了一种基于深度强化学习的股骨柄放置方法，包括：
获取患者骨盆及股骨图像数据，基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域；
确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄；
采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息；
根据所述调节信息对所述股骨柄进行调节，以使所述股骨柄与股骨相匹配。
可选地，所述基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域，包括：
通过分割神经网络模型对所述骨盆及股骨图像数据进行图像分割；
其中，所述分割神经网络模型包括级联的第一分割神经网络和第二分割神经网络；
所述第一分割神经网络和第二分割神经网络的关联参数，是通过对预先存储的医学图像数据库中的图像数据进行训练和测试确定的。
可选地，所述第一分割神经网络，用于作为主干网络，对所述骨盆及股骨图像数据进行粗分割；
所述第二分割神经网络，用于基于所述粗分割进行精确分割；
所述第一分割神经网络为全卷积网络FCN、语义分割网络SegNet、深度学习分割网络Unet、3D-深度学习分割网络3D-Unet、实例分割网络Mask-RCNN、空洞卷积、语义分割神经网络ENet、语义分割网络
CRFasRNN、场景解析网络PSPNet、端到端语义分割网络ParseNet、图像语义分割网络RefineNet、图像分割模型ReSeg、语义分割网络LSTM-CF、实例分割网络DeepMask、语义分割模型DeepLabV1、语义分割模型DeepLabV2、语义分割模型DeepLabV3中的至少一种；
所述第二分割神经网络为EfficientDet、SimCLR、PointRend中的至少一种。
可选地，所述确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄，包括：
根据分类神经网络对所述股骨髓腔皮质区域进行层面分类，分出股骨髓腔层面；
确定所有股骨髓腔层面的中心点；
对所有股骨髓腔层面的中心点进行直线拟合，确定股骨髓腔皮质区域的股骨髓腔轴线；
将任一型号股骨柄的轴线沿所述股骨髓腔轴线放置到股骨内的任一位置。
可选地，所述采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息，包括：
基于双网络架构DDQN，搭建encoder-decoder网络结构；
获取所述股骨髓腔皮质区域的冠状面视图，并将所述冠状面视图输入到所述encoder-decoder网络结构；
通过DDQN双网络中的探索网络，根据所述股骨柄与股骨的相对位置，输出股骨柄型号、股骨柄位置的调节信息。
可选地，在输出股骨柄型号、股骨柄位置的调节信息之后，所述方法还包括：
通过DDQN双网络中的价值网络，根据奖励函数确定奖励，将所述奖励经过变换后反馈给所述价值网络进行学习和迭代，以使奖励达到最大值；
其中，根据下述奖励函数确定奖励r：
r＝贴合度/腿长差
贴合度＝-a 2+2ab
贴合度为股骨柄嵌入股骨髓腔皮质的平均深度，腿长差为双侧小转子 上缘高度差的绝对值，a为所有股骨髓腔皮质与股骨柄相贴合的层面嵌入深度的平均值，b为设定的切入深度。
可选地，所述调节信息包括分类器输出的股骨柄型号调节信息、股骨柄位置调节信息中的至少一种，所述根据所述调节信息对所述股骨柄进行调节包括：
如果所述股骨柄型号调节信息为股骨柄型号增大，则将股骨柄的型号增大相应的尺寸；
如果所述股骨柄型号调节信息为股骨柄型号减小，则将股骨柄的型号减小相应的尺寸；
如果所述股骨柄位置调节信息为股骨柄上移，则将股骨柄的位置向靠近骨盆的一端移动相应的距离；
如果所述股骨柄位置调节信息为股骨柄下移，则将股骨柄的位置向远离骨盆的一端移动相应的距离。
本申请的第二方面提供了一种基于深度强化学习的股骨柄放置装置，包括：
分割单元，被配置为获取患者骨盆及股骨图像数据，基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域；
确定单元，被配置为确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄；
输出单元，被配置为采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息；
调节单元，被配置为根据所述调节信息对所述股骨柄进行调节，以使所述股骨柄与股骨相匹配。
本申请的第三方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使计算机执行第一方面任意一项提供的基于深度强化学习的股骨柄放置方法。
本申请的第四方面提供了一种电子设备，所述电子设备包括：至少一 个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的计算机程序，所述计算机程序被所述至少一个处理器执行，以使所述至少一个处理器执行第一方面任意一项提供的基于深度强化学习的股骨柄放置方法。
本申请实施例将股骨柄的轴线沿所述股骨髓腔轴线放置，并通过深度强化学习的双网络架构DDQN进行调节，使得股骨柄与股骨相匹配，提高了股骨柄放置时的精度，解决了现有技术中因股骨柄与股骨的相对位置不当，导致股骨柄松动或患者髋关节疼痛的问题。
附图说明
为了更清楚地说明本申请具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为股骨柄假体的示意图；
图2为以双侧小转子上缘A点和B点的高度差为腿长差的示意图；
图3为本申请实施例提供的基于深度强化学习的股骨柄放置方法流程示意图；
图4为本申请实施例提供的分割神经网络模型的工作原理图；
图5为本申请实施例提供的股骨髓腔轴线和股骨柄假体中轴线的示意图；
图6为本申请实施例提供的双网络架构DDQN；
图7为本申请实施例提供的股骨柄假体轮廓嵌入股骨髓腔皮质轮廓的示意图；
图8为本申请实施例提供的基于深度强化学习的股骨柄放置装置框图；
图9为本申请实施例提供的电子设备框图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
在髋关节置换手术中，往往将如图1所示的股骨柄作为生物型假体，与髋关节组件配合使用。在置换人工髋关节时，需要采用合适大小的股骨柄，并且确定股骨柄和股骨的相对位置以保证置换后两腿的腿长差尽可能小，图2中双侧小转子上缘A点和B点的高度差即为腿长差。然而，相关技术中股骨柄与股骨的相对位置主要依靠人工经验确定，这种方式精确度较低，容易因为相对位置不佳导致股骨柄松动或患者髋关节疼痛。
为了解决上述问题，本申请实施例提供了一种基于深度强化学习的股骨柄放置方法，如图3所示，该方法包括如下的步骤S101至步骤S104：
步骤S101：获取患者骨盆及股骨图像数据，基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域；为了将股骨柄假体放置到股骨内，需要获取患有髋关节疾病患者的骨盆及股骨图像数据，手动标注或自动标注股骨髓腔皮质区域，以通过分割神经网络模型提取股 骨髓腔皮质区域。
具体的，所述步骤S101中的基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域，包括：
通过分割神经网络模型对所述骨盆及股骨图像数据进行图像分割；
其中，所述分割神经网络模型包括级联的第一分割神经网络和第二分割神经网络；
所述第一分割神经网络和第二分割神经网络的关联参数，是通过对预先存储的医学图像数据库中的图像数据进行训练和测试确定的。预先存储的医学图像数据库中的图像数据包括患有髋关节疾病患者的CT医学图像数据集。
可选地，所述第一分割神经网络，用于作为主干网络，对所述骨盆及股骨图像数据进行粗分割；
所述第二分割神经网络，用于基于所述粗分割进行精确分割；
所述第一分割神经网络为全卷积网络FCN、语义分割网络SegNet、深度学习分割网络Unet、3D-深度学习分割网络3D-Unet、实例分割网络Mask-RCNN、空洞卷积、语义分割神经网络ENet、语义分割网络CRFasRNN、场景解析网络PSPNet、端到端语义分割网络ParseNet、图像语义分割网络RefineNet、图像分割模型ReSeg、语义分割网络LSTM-CF、实例分割网络DeepMask、语义分割模型DeepLabV1、语义分割模型DeepLabV2、语义分割模型DeepLabV3中的至少一种；
所述第二分割神经网络为EfficientDet、SimCLR、PointRend中的至少一种。
例如，可以使用像素级分割神经网络分割出股骨髓腔皮质区域，包括下述步骤：
首先，获取患有髋关节疾病患者的CT二维医学图像数据集，CT二维医学图像数据集中包含若干个CT二维医学图像，在CT二维医学图像上采用手动标注、或自动标注的方式标注股骨髓腔皮质区域，将其作为数据库。将两组不同格式的CT二维医学图像分别按照7:3的比例划分为训练 集、测试集，将CT二维医学图像的DICOM数据转换成JPG格式的图片，将标注后的CT二维医学图像转换成PNG格式的图片，保存后两组数据作为神经网络的输入。
然后，建立分割神经网络模型pointrend+unet，第一分割神经网络为深度学习分割网络unet，第二分割神经网络为pointrend，本申请实施例提供的分割神经网络模型的工作原理图如图4所示；
其中，利用unet网络作为主干网络，对其进行粗分割，第一阶段使用4次下采样学习图像的深层特征，然后进行4次上采样以将特征图重新存储到图像中，其中每个下采样层中包括2个卷积层和1个池化层，卷积核大小为3*3，池化层中的卷积核大小为2*2，每个卷积层中的卷积核的个数为128、256、512；每个上采样层中包括1个上采样层和2个卷积层，其中卷积层的卷积核大小为3*2，上采样层中的卷积核大小为2*2，每个上采样层中的卷积核个数为512、256、128。最后一次上采样结束后设有一个的dropout层，droupout率设置为0.7。所有的卷积层后面都设有激活函数relu函数。然后使用pointrend精确分割结果，选择一组置信度为0.5的一组点，提取被选择出来的点的特征，这些点的特征通过双线性插值Bilinear计算，使用一个小型的分类器去判断这个点属于哪个类别。这其实是等价于用一个1*1的卷积来预测，但是对于置信度接近于1或者0的点并不计算。从而提高分割的精准度。
模型训练过程中，训练的batch_size为6，学习率设置为1e -4，优化器使用Adam优化器，使用的损失函数为DICE loss，将训练集全部送入网络进行训练，根据训练过程中损失函数的变化，调整训练批次的大小，最终得到各个部分的粗分割结果。进入pointrend模块后，先会使用双线性插值上采样前一步分割预测结果，然后在这个更密集的特征图中选择N个最不确定的点，比如概率接近0.5的点。然后计算这N个点的特征表示并且预测它们的labels，这个过程一直被重复，直到上采样到需要的大小。对于每个选定点的逐点特征表示，使用简单的多层感知器进行逐点预测，因为多层感知器MLP预测的是各点的分割label，所以可以使用unet粗分割任务中的loss来训练。最后，分割神经网络模型输出股骨髓腔皮质区域。
步骤S102：确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄；通过分割提取出股骨髓腔皮质区域后，提取股骨髓腔轴线，并将放置的股骨柄假体的轴线与股骨髓腔轴线重合，以保证股骨柄与股骨之间为最适当的最佳相对位置，避免术后出现股骨柄松动或患者髋关节疼痛等并发症。
具体的，所述步骤S102包括：
根据分类神经网络对所述股骨髓腔皮质区域进行层面分类，分出股骨髓腔层面；分类神经网络是提前根据大量的训练样本得到的，训练样本为已经标注好股骨髓腔层面类型的图像，分类神经网络的输入为DICOM二维横断面图像，输出为是否为股骨髓腔层面；分类神经网络为卷积神经网络LeNet、卷积神经网络AlexNet、可视化卷积神经网络ZF-Net、卷积神经网络GoogleNet、卷积神经网络VGG、卷积神经网络Inception、卷积神经网络ResNet、卷积神经网络DensNet、卷积神经网络Inception ResNet中的至少一种。
具体的，在根据分类神经网络对所述股骨髓腔皮质区域进行层面分类之前，所述方法还包括：获取训练样本，所述训练样本包括人工标注所在层面属于的类别的二维横断面图像；根据所述训练样本训练得到所述分类神经网络。
确定所有股骨髓腔层面的中心点；包括根据二维图像中心点计算公式确定每个髓腔层面的中心点，其中，二维图像中心点计算公式可以为平面图像的质心公式，即根据平面图像的质心公式确定髓腔层面的中心点。
具体的，在确定所有股骨髓腔层面的中心点之前，所述方法还包括：根据高阈值处理方法，对股骨髓腔层面进行图像清晰化处理。为了进一步提高股骨髓腔轴线的精度，在确定所有股骨髓腔层面的中心点之前，增加清晰化处理的过程，具体为对区分出的髓腔层面进行图像清晰化处理，比如可以使用OpenCV高阈值处理方法，使髓腔形态更加清晰。图像的阈值化就是利用图像像素点分布规律，设定阈值进行像素点分割，进而得到图像的二值图像。
对所有股骨髓腔层面的中心点进行直线拟合，确定股骨髓腔皮质区域 的股骨髓腔轴线；将股骨髓腔皮质区域的所有股骨髓腔层面的中心点进行直线拟合，得到股骨髓腔皮质区域对应的股骨髓腔轴线。其中，直线拟合的方式可以为最小二乘法、梯度下降、高斯牛顿、列-马算法等现有的任意直线拟合算法。
将任一型号股骨柄的轴线沿所述股骨髓腔轴线放置到股骨内的任一位置。确定股骨髓腔轴线后，随机选择任意一种型号的股骨柄假体，将股骨柄假体的中轴线沿股骨髓腔轴线放置到股骨内的一个随机位置，此时需要保证股骨柄轴线与股骨髓腔轴线重合，以确保股骨柄与股骨之间的最佳相对位置，如图5中右侧所示的股骨髓腔轴线，左侧所示的股骨柄假体中轴线与股骨髓腔轴线重合。
步骤S103：采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息；根据基于深度强化学习的双网络架构DDQN，判断当前股骨柄的型号大小以及股骨柄与股骨的相对位置关系的改善方向，通过双网络架构DDQN的分类器预测输出调节信息，以调节股骨柄的型号大小以及股骨柄与股骨的相对位置为最适合或最佳，使得两腿的腿长差尽可能最小，避免患者出现髋关节疼痛等术后并发症。
具体的，所述步骤S103包括：
基于双网络架构DDQN，搭建encoder-decoder网络结构；并且，encoder-decoder网络结构连接全连接层FC layer，如图6所示，其中，encoder为编码器，decoder为解码器，FC layer为全连接层，作为分类器，输出股骨柄型号变化和股骨柄位移的分类；
获取所述股骨髓腔皮质区域的冠状面视图，并将所述冠状面视图输入到所述encoder-decoder网络结构；
DDQN双网络包括探索网络和价值网络；
通过DDQN双网络中的探索网络，根据所述股骨柄与股骨的相对位置，输出股骨柄型号、股骨柄位置的调节信息。分类器接收股骨柄与股骨的相对位置，输出股骨柄型号、股骨柄位置的调节信息，如果输出是型号增大则股骨柄的型号增大相应的尺寸，如果输出是型号减小则股骨柄的型号减小相应的尺寸；如果输出是股骨柄上移则股骨柄向靠近骨盆的一端移 动相应的距离(例如，移动1毫米)，如果输出是股骨柄下移则股骨柄向远离骨盆的一端移动相应的距离。其中，股骨柄的型号的大小是相对固定的，按照尺寸从小到大排序可以从size0(型号0)一直排列至size12(型号12)，不同型号间形状不变尺寸依次增大。
可选地，在输出股骨柄型号、股骨柄位置的调节信息之后，所述方法还包括：
通过DDQN双网络中的价值网络，根据奖励函数确定奖励，将所述奖励经过变换后反馈给所述价值网络进行学习和迭代，以使奖励达到最大值；
其中，根据下述奖励函数确定奖励r：
r＝贴合度/腿长差
贴合度＝-a 2+2ab
贴合度为股骨柄嵌入股骨髓腔皮质的平均深度，腿长差为双侧小转子上缘高度差的绝对值，a为所有股骨髓腔皮质与股骨柄相贴合的层面嵌入深度的平均值，b为设定的切入深度，即人为设定的最佳的切入角度。
股骨柄假体轮廓嵌入股骨髓腔皮质轮廓的示意图如图7所示，其中，平滑曲线为股骨髓腔皮质轮廓，折线为股骨柄假体轮廓，阴影区域各线为各个层面股骨柄假体轮廓嵌入股骨髓腔皮质轮廓的深度，b一般由医生设定为2毫米，当a为0时，贴合度为0，当a为2毫米时，贴合度最大，超过2毫米时假体嵌入太深贴合度下降，低于2毫米时贴合度随着a变大而增大，模型较为符合现实。
在股骨柄的型号和股骨柄与股骨的相对位置确定后，可以根据奖励函数计算此时得到的奖励，奖励越大说明股骨柄的型号大小选择得越适合、股骨柄与股骨的相对位置越佳，精确度越高；
DDQN双网络价值网络用于拟合价值函数，奖励函数给出的奖励经过变换后反馈给价值网络再进行学习和迭代，以使奖励达到最大值；在训练完成后价值网络就是得到的最终模型，在预测时分类器输出当前场景下不同股骨柄的型号和股骨柄与股骨不同相对位置的价值，价值最大时对应的 即可视为最佳的股骨柄型号和股骨柄与股骨的最佳相对位置。
步骤S104：根据所述调节信息对所述股骨柄进行调节，以使所述股骨柄与股骨相匹配。当股骨柄与股骨相匹配时，股骨柄型号与股骨相匹配，股骨柄位置与股骨相匹配，股骨柄与股骨的相对位置达到最佳，股骨柄放置在股骨内时两腿的腿长差最小。
其中，所述调节信息包括分类器输出的股骨柄型号调节信息、股骨柄位置调节信息中的至少一种，所述根据所述调节信息进行调节包括：
如果所述股骨柄型号调节信息为股骨柄型号增大，则将股骨柄的型号增大相应的尺寸；
如果所述股骨柄型号调节信息为股骨柄型号减小，则将股骨柄的型号减小相应的尺寸；
如果所述股骨柄位置调节信息为股骨柄上移，则将股骨柄的位置向靠近骨盆的一端移动相应的距离；
如果所述股骨柄位置调节信息为股骨柄下移，则将股骨柄的位置向远离骨盆的一端移动相应的距离。
例如，如果股骨柄型号调节信息为股骨柄增大1个型号，则将股骨柄的型号增大1个型号的尺寸，如果股骨柄型号调节信息为股骨柄减小2个型号，则将股骨柄的型号减小2个型号的尺寸；如果股骨柄位置调节信息为股骨柄上移1毫米，则将股骨柄的位置向靠近骨盆的一端移动1毫米，如果股骨柄位置调节信息为股骨柄下移2毫米，则将股骨柄的位置向远离骨盆的一端移动2毫米；
通过分类器输出的股骨柄型号变化和股骨柄位移的分类，对股骨柄的型号或位置进行调节，以使奖励达到最大，股骨柄与股骨相匹配，股骨柄放置在股骨内时两腿的腿长差为最小，达到最适股骨柄型号和股骨柄与股骨的最佳相对位置。
从以上的描述中，可以看出，本申请实现了如下技术效果：
本申请通过深度强化学习的双网络架构DDQN进行调节，选择合适的股骨柄型号以及股骨和股骨柄的最佳相对位置，保证在股骨柄大小合适 的情况下使两腿的腿长差达到最小，提高了股骨柄放置时的精度，解决了现有技术中因股骨柄与股骨的相对位置不当，导致股骨柄松动或患者髋关节疼痛的问题。
通过DDQN双网络调节股骨柄型号大小和股骨柄与股骨的相对位置，可以使得奖励达到最大，股骨柄与股骨相匹配，两腿的腿长差为最小，达到最适股骨柄型号和股骨柄与股骨的最佳相对位置。
需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
本申请实施例还提供了一种用于实施上述基于深度强化学习的股骨柄放置方法的基于深度强化学习的股骨柄放置装置，如图8所示，该装置包括：
分割单元81，用于获取患者骨盆及股骨图像数据，基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域；
确定单元82，用于确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄；
输出单元83，用于采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息；
调节单元84，用于根据所述调节信息对所述股骨柄进行调节，以使所述股骨柄与股骨相匹配。
本申请实施例还提供了一种电子设备，如图9所示，该电子设备包括一个或多个处理器91以及存储器92，图9中以一个处理器91为例。
该控制器还可以包括：输入装置93和输出装置94。
处理器91、存储器92、输入装置93和输出装置94可以通过总线或者其他方式连接，图9中以通过总线连接为例。
处理器91可以为中央处理器(Central Processing Unit，简称为CPU)，处理器91还可以为其他通用处理器、数字信号处理器(Digital Signal Processor，简称为DSP)、专用集成电路(Application Specific Integrated  Circuit，简称为ASIC)、现场可编程门阵列(Field-Programmable Gate Array，简称为FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等芯片，或者上述各类芯片的组合，通用处理器可以是微处理器或者任何常规的处理器。
存储器92作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块，如本申请实施例中的控制方法对应的程序指令/模块。处理器91通过运行存储在存储器92中的非暂态软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例的基于深度强化学习的股骨柄放置方法。
存储器92可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据服务器操作的处理装置的使用所创建的数据等。此外，存储器92可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器92可选包括相对于处理器91远程设置的存储器，这些远程存储器可以通过网络连接至网络连接装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置93可接收输入的数字或字符信息，以及产生与服务器的处理装置的用户设置以及功能控制有关的键信号输入。输出装置94可包括显示屏等显示设备。
一个或者多个模块存储在存储器92中，当被一个或者多个处理器91执行时，执行如图3所示的方法。
本领域技术人员可以理解，实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成的，程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各电机控制方法的实施例的流程。其中，存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，简称为ROM)、随机存储记忆体(Random Access Memory，简称为RAM)、快闪存储器(Flash Memory，简称为FM)、硬盘(Hard Disk Drive，简称为HDD)或固态硬盘(Solid-State Drive，简 称为SSD)等；存储介质还可以包括上述种类的存储器的组合。
虽然结合附图描述了本申请的实施方式，但是本领域技术人员可以在不脱离本申请的精神和范围的情况下作出各种修改和变型，这样的修改和变型均落入由所附权利要求所限定的范围之内。

Claims (10)

  1. 一种基于深度强化学习的股骨柄放置方法，包括：
    获取患者骨盆及股骨图像数据，基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域；
    确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄；
    采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息；
    根据所述调节信息对所述股骨柄进行调节，以使所述股骨柄与股骨相匹配。
  2. 根据权利要求1所述的方法，其中，所述基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域，包括：
    通过分割神经网络模型对所述骨盆及股骨图像数据进行图像分割；
    其中，所述分割神经网络模型包括级联的第一分割神经网络和第二分割神经网络；
    所述第一分割神经网络和第二分割神经网络的关联参数，是通过对预先存储的医学图像数据库中的图像数据进行训练和测试确定的。
  3. 根据权利要求2所述的方法，其中，所述第一分割神经网络，用于作为主干网络，对所述骨盆及股骨图像数据进行粗分割；
    所述第二分割神经网络，用于基于所述粗分割进行精确分割；
    所述第一分割神经网络为全卷积网络FCN、语义分割网络SegNet、深度学习分割网络Unet、3D-深度学习分割网络3D-Unet、实例分割网络Mask-RCNN、空洞卷积、语义分割神经网络ENet、语义分割网络CRFasRNN、场景解析网络PSPNet、端到端语义分割网络ParseNet、图像语义分割网络RefineNet、图像分割模型ReSeg、语义分割网络LSTM-CF、实例分割网络DeepMask、语义分割模型DeepLabV1、语义分割模型DeepLabV2、语义分割模型DeepLabV3中的至少一种；
    所述第二分割神经网络为EfficientDet、SimCLR、PointRend中的至少一种。
  4. 根据权利要求1所述的方法，其中，所述确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述 股骨柄，包括：
    根据分类神经网络对所述股骨髓腔皮质区域进行层面分类，分出股骨髓腔层面；
    确定所有股骨髓腔层面的中心点；
    对所有股骨髓腔层面的中心点进行直线拟合，确定股骨髓腔皮质区域的股骨髓腔轴线；
    将任一型号股骨柄的轴线沿所述股骨髓腔轴线放置到股骨内的任一位置。
  5. 根据权利要求1所述的方法，其中，所述采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息，包括：
    基于双网络架构DDQN，搭建encoder-decoder网络结构；
    获取所述股骨髓腔皮质区域的冠状面视图，并将所述冠状面视图输入到所述encoder-decoder网络结构；
    通过DDQN双网络中的探索网络，根据所述股骨柄与股骨的相对位置，输出股骨柄型号、股骨柄位置的调节信息。
  6. 根据权利要求5所述的方法，在输出股骨柄型号、股骨柄位置的调节信息之后，所述方法还包括：
    通过DDQN双网络中的价值网络，根据奖励函数确定奖励，将所述奖励经过变换后反馈给所述价值网络进行学习和迭代，以使奖励达到最大值；
    其中，根据下述奖励函数确定奖励r：
    r＝贴合度/腿长差
    贴合度＝-a 2+2ab
    贴合度为股骨柄嵌入股骨髓腔皮质的平均深度，腿长差为双侧小转子上缘高度差的绝对值，a为所有股骨髓腔皮质与股骨柄相贴合的层面嵌入深度的平均值，b为设定的切入深度。
  7. 根据权利要求1所述的方法，其中，所述调节信息包括分类器输出的股骨柄型号调节信息、股骨柄位置调节信息中的至少一种，所述根据所述调节信息对所述股骨柄进行调节包括：
    如果所述股骨柄型号调节信息为股骨柄型号增大，则将股骨柄的型号增大相应的尺寸；
    如果所述股骨柄型号调节信息为股骨柄型号减小，则将股骨柄的型号减 小相应的尺寸；
    如果所述股骨柄位置调节信息为股骨柄上移，则将股骨柄的位置向靠近骨盆的一端移动相应的距离；
    如果所述股骨柄位置调节信息为股骨柄下移，则将股骨柄的位置向远离骨盆的一端移动相应的距离。
  8. 一种基于深度强化学习的股骨柄放置装置，包括：
    分割单元，被配置为获取患者骨盆及股骨图像数据，基于所述骨盆及股骨图像数据通过分割神经网络模型分割提取股骨髓腔皮质区域；
    确定单元，被配置为确定所述股骨髓腔皮质区域的股骨髓腔轴线，并将股骨柄的轴线沿所述股骨髓腔轴线，以用于放置所述股骨柄；
    输出单元，被配置为采用深度强化学习的双网络架构DDQN，根据所述股骨柄与股骨的相对位置，输出调节信息；
    调节单元，被配置为根据所述调节信息对所述股骨柄进行调节，以使所述股骨柄与股骨相匹配。
  9. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使计算机执行权利要求1-7任意一项所述的基于深度强化学习的股骨柄放置方法。
  10. 一种电子设备，所述电子设备包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的计算机程序，所述计算机程序被所述至少一个处理器执行，以使所述至少一个处理器执行权利要求1-7任意一项所述的基于深度强化学习的股骨柄放置方法。
PCT/CN2022/110973 2021-08-24 2022-08-08 基于深度强化学习的股骨柄放置方法、装置及电子设备 WO2023024883A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110974127.3A CN113744214B (zh) 2021-08-24 2021-08-24 基于深度强化学习的股骨柄放置装置及电子设备
CN202110974127.3 2021-08-24

Publications (1)

Publication Number Publication Date
WO2023024883A1 true WO2023024883A1 (zh) 2023-03-02

Family

ID=78732602

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/110973 WO2023024883A1 (zh) 2021-08-24 2022-08-08 基于深度强化学习的股骨柄放置方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN113744214B (zh)
WO (1) WO2023024883A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116597002A (zh) * 2023-05-12 2023-08-15 北京长木谷医疗科技股份有限公司 基于深度强化学习的股骨柄自动放置方法、装置及设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113744214B (zh) * 2021-08-24 2022-05-13 北京长木谷医疗科技有限公司 基于深度强化学习的股骨柄放置装置及电子设备
CN117455935B (zh) * 2023-12-22 2024-03-19 中国人民解放军总医院第一医学中心 基于腹部ct医学图像融合及器官分割方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130274633A1 (en) * 2012-04-12 2013-10-17 Avenir Medical, Inc. Computer-Assisted Joint Replacement Surgery and Navigation Systems
US20140093153A1 (en) * 2012-09-28 2014-04-03 Siemens Corporation Method and System for Bone Segmentation and Landmark Detection for Joint Replacement Surgery
CN108764241A (zh) * 2018-04-20 2018-11-06 平安科技（深圳）有限公司 分割股骨近端的方法、装置、计算机设备和存储介质
CN111888059A (zh) * 2020-07-06 2020-11-06 北京长木谷医疗科技有限公司 基于深度学习与x线的全髋关节置换术前规划方法及装置
CN112971981A (zh) * 2021-03-02 2021-06-18 北京长木谷医疗科技有限公司 基于深度学习的全髋关节置换翻修术前规划方法和设备
CN113744214A (zh) * 2021-08-24 2021-12-03 北京长木谷医疗科技有限公司 基于深度强化学习的股骨柄放置方法、装置及电子设备

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020033656A1 (en) * 2018-08-08 2020-02-13 Loyola University Chicago Methods of classifying and/or determining orientations of objects using two-dimensional images
US20220148739A1 (en) * 2019-02-05 2022-05-12 Smith & Nephew, Inc. Use of robotic surgical data for long term episode of care
CN111179350B (zh) * 2020-02-13 2022-04-08 张逸凌 髋关节图像处理系统
CN111292363B (zh) * 2020-02-13 2022-02-22 张逸凌 一种关节图像处理方法、装置和计算设备
CN111652888B (zh) * 2020-05-25 2021-04-02 北京长木谷医疗科技有限公司 基于深度学习的确定髓腔解剖轴线的方法及装置
CN113017829B (zh) * 2020-08-22 2023-08-29 张逸凌 一种基于深度学习的全膝关节置换术的术前规划方法、系统、介质和设备
CN112641511B (zh) * 2020-12-18 2021-09-10 北京长木谷医疗科技有限公司 关节置换手术导航系统及方法
CN112842529B (zh) * 2020-12-31 2022-02-08 北京长木谷医疗科技有限公司 全膝关节图像处理方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130274633A1 (en) * 2012-04-12 2013-10-17 Avenir Medical, Inc. Computer-Assisted Joint Replacement Surgery and Navigation Systems
US20140093153A1 (en) * 2012-09-28 2014-04-03 Siemens Corporation Method and System for Bone Segmentation and Landmark Detection for Joint Replacement Surgery
CN108764241A (zh) * 2018-04-20 2018-11-06 平安科技（深圳）有限公司 分割股骨近端的方法、装置、计算机设备和存储介质
CN111888059A (zh) * 2020-07-06 2020-11-06 北京长木谷医疗科技有限公司 基于深度学习与x线的全髋关节置换术前规划方法及装置
CN112971981A (zh) * 2021-03-02 2021-06-18 北京长木谷医疗科技有限公司 基于深度学习的全髋关节置换翻修术前规划方法和设备
CN113744214A (zh) * 2021-08-24 2021-12-03 北京长木谷医疗科技有限公司 基于深度强化学习的股骨柄放置方法、装置及电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116597002A (zh) * 2023-05-12 2023-08-15 北京长木谷医疗科技股份有限公司 基于深度强化学习的股骨柄自动放置方法、装置及设备
CN116597002B (zh) * 2023-05-12 2024-01-30 北京长木谷医疗科技股份有限公司 基于深度强化学习的股骨柄自动放置方法、装置及设备

Also Published As

Publication number Publication date
CN113744214B (zh) 2022-05-13
CN113744214A (zh) 2021-12-03

Similar Documents

Publication Publication Date Title
WO2023024883A1 (zh) 基于深度强化学习的股骨柄放置方法、装置及电子设备
US11455774B2 (en) Automated 3D root shape prediction using deep learning methods
CN113076987B (zh) 骨赘识别方法、装置、电子设备及存储介质
WO2022183719A1 (zh) 基于深度学习的全髋关节置换翻修术前规划方法和设备
WO2022257345A1 (zh) 医学图像融合方法及系统、模型训练方法及存储介质
WO2023024882A1 (zh) 基于深度学习的股骨髓腔形态识别方法、装置及存储介质
CN109727235B (zh) 一种基于深度学习的器官自动勾画算法
CN110008992B (zh) 一种用于前列腺癌辅助诊断的深度学习方法
WO2022247173A1 (zh) 图像识别及模型训练的方法、关节位置识别的方法
JP6853419B2 (ja) 情報処理装置、情報処理方法、コンピュータプログラム
CN104346799B (zh) 一种ct图像中脊髓的提取方法
CN112233777A (zh) 基于深度学习的胆结石自动识别及分割系统、计算机设备、存储介质
CN113962927B (zh) 基于强化学习的髋臼杯位置调整方法、装置及存储介质
KR102461343B1 (ko) 금속 인공 음영이 포함된 의료영상에서 치아 랜드마크 자동 검출 방법 및 시스템
KR101769808B1 (ko) 무릎 자기공명영상의 관절연골 분할 장치 및 그 분할 방법
CN106780491B (zh) Gvf法分割ct骨盆图像中采用的初始轮廓生成方法
CN110751179A (zh) 病灶信息获取方法、病灶预测模型的训练方法及超声设备
CN113658165A (zh) 杯盘比确定方法、装置、设备及存储介质
CN113012093A (zh) 青光眼图像特征提取的训练方法及训练系统
CN106780492B (zh) 一种ct骨盆图像的关键帧提取方法
Kwon et al. Multistage probabilistic approach for the localization of cephalometric landmarks
CN113241155A (zh) 一种头颅侧位片中标志点的获取方法及系统
CN115252233B (zh) 基于深度学习的全髋关节置换术中髋臼杯的自动化规划方法
Liu et al. Tracking-based deep learning method for temporomandibular joint segmentation
CN114612391A (zh) 基于深度学习的全髋关节术后腿长差的计算方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22860229

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE