TWI753412B - A method for generating a model for automatically locating an anchor point, a skeletal state analysis method, and an electronic system - Google Patents

Abstract

A method for generating a model for automatically locating an anchor point, comprising: an X-ray machine photographing a predetermined bone and an implant of a plurality of reference patients to generate multiple pieces of training image data; a processing unit for each training image data, generating a plurality of manual positioning data corresponding to the training image data according to a plurality of manual positioning instructions received through an input unit, each manual positioning data indicating the position of a positioning point in the training image data; and For each positioning point, the processing unit trains a convolutional neural network model according to the manual positioning data indicating the position of the positioning point and the training image data to generate an automatic positioning model. The automatic positioning model uses An automatic positioning data is generated according to a to-be-analyzed image data.

Description

A method for generating a model for automatically locating an anchor point, a skeletal state analysis method, and an electronic system

The present invention relates to a method for generating a model, in particular to a method for generating a model for automatically locating an anchor point. The present invention also relates to a bone state analysis method and an electronic system.

There are many types of hip joint diseases, such as femoral head necrosis caused by alcohol or inflammatory immune diseases, early degeneration of the hip joint caused by hip dysplasia common in Asian populations, and femoral bone common in the elderly population with low osteoporosis. Neck fracture. Artificial hip surgery will be required when a related disorder occurs that seriously affects daily life. Preoperative, intraoperative, postoperative, and prognosis tracking of artificial hip replacement requires precise positioning to avoid misalignment of implant correction and long-term observation of abnormalities. In the previous positioning method, physicians or technicians observed the X-ray images of the pelvis, and manually positioned important objects with computer software, so as to interpret the state of the artificial hip joint replacement. However, the aforementioned prior art highly relies on professional medical personnel to perform manual positioning, and errors or errors are inevitable in manual positioning.

Therefore, an object of the present invention is to provide a method for generating a model for automatically locating an anchor point.

Another object of the present invention is to provide a bone state analysis method.

Another object of the present invention is to provide an electronic system.

Therefore, the present invention generates a method for automatically locating a model for locating points, and is implemented by an electronic system, the electronic system includes an X-ray machine, an input unit and a processing unit, and the method includes: the X-ray machine shoots a predetermined bone of a plurality of reference patients and an implant mounted on the predetermined bone to generate a plurality of training image data respectively related to the reference patients and including the image of the predetermined bone and the image of the implant; For each training image data, the processing unit generates a plurality of manual positioning data corresponding to the training image data according to a plurality of manual positioning instructions received through the input unit, and each manual positioning data indicates that in the training image data and the processing unit for each positioning point, according to the manual positioning data and the training image data indicating the position of the positioning point, train a convolutional neural network model to generate an automatic A positioning model, the automatic positioning model is used to generate an automatic positioning data according to a to-be-analyzed image data generated by photographing the predetermined bone of a target patient and the implant by the X-ray machine, and the automatic positioning data indicates the to-be-analyzed image data. Analyze the location of the anchor point in the image data.

The bone state analysis method of the present invention is implemented by an electronic system, the electronic system includes an X-ray machine and a processing unit, and the method includes: the X-ray machine photographing a predetermined bone of a target patient and a device installed in the predetermined bone the implant of the bone to generate an image data to be analyzed including the image of the predetermined bone and the image of the implant; the processing unit uses the automatic positioning models to generate a plurality of separate indications according to the image data to be analyzed Obtaining automatic positioning data of the positions of the positioning points in the image data to be analyzed; and the processing unit generating a bone state parameter value related to distance or angle according to the automatic positioning data.

In some implementation aspects, the electronic system further includes a display unit, and after generating the automatic positioning data, the method further includes: the processing unit displays a display unit through the display unit according to the image data to be analyzed and the automatic positioning data The automatic marking image data includes the image of the predetermined bone, the image of the implant, and a plurality of marked images respectively indicating the positioning points.

In some implementation aspects, after generating the bone state parameter value, the method further includes: the processing unit displays the bone state parameter value via the display unit.

In some implementation aspects, after generating the bone state parameter value, the method further includes: the processing unit determining whether the bone state parameter value meets a warning condition; and when the processing unit determines that the bone state parameter value meets the warning condition , the processing unit displays a warning message indicating that the bone state parameter value meets the warning condition via the display unit.

The electronic system of the present invention includes an X-ray machine and a processing unit. The X-ray machine photographs a predetermined bone of a target patient and an implant mounted on the predetermined bone to generate an image data to be analyzed including an image of the predetermined bone and an image of the implant.

The processing unit uses the automatic positioning models to generate a plurality of automatic positioning data respectively indicating the positions of the positioning points in the image data to be analyzed according to the image data to be analyzed.

The processing unit generates a bone state parameter value related to distance or angle according to the automatic positioning data.

The effect of the present invention is: for each positioning point, the processing unit trains the convolutional neural network model according to the manual positioning data and the training image data indicating the position of the positioning point to generate the automatic positioning models, and using the automatic positioning models by the processing unit to generate a plurality of automatic positioning data respectively indicating the positions of the positioning points in the image data to be analyzed, and generating the related automatic positioning data according to the automatic positioning data The skeletal state parameter value of the distance or angle is improved, thereby improving the errors or errors that may occur in manual positioning in the prior art, and freeing professional medical personnel from spending time on performing manual positioning.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are designated by the same reference numerals.

Referring to FIG. 1 , an embodiment of an electronic system 100 of the present invention includes an X-ray machine 1 , an input unit 2 , a display unit 3 and a processing unit 4 .

The input unit 2 includes, for example (but not limited to) a keyboard and a mouse. The display unit 3 is, for example (but not limited to) a liquid crystal screen. The processing unit 4 is electrically connected to the X-ray machine 1, the input unit 2 and the display unit 3, and may include (but not limited to) a single-core processor, a multi-core processor, a single-core or multi-core mobile phone processor, for example , a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable logic gate array (FPGA), a special application integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), At least one of a graphics processing unit (GPU).

Referring to FIG. 1 , FIG. 2 and FIG. 3 , the steps of the method for generating a model for automatically locating an anchor point performed by the electronic system 100 are described below. First, as shown in step S01, the X-ray machine 1 photographs a predetermined bone 200 of a plurality of reference patients and an implant 300 installed in the predetermined bone 200 to generate a plurality of images respectively related to the reference patients and including The training image data of the image of the predetermined bone 200 and the image of the implant 300 . In this embodiment, the predetermined bone 200 is a pelvis (Pelvis), and the implant 300 is an artificial hip joint (Hip), but not limited thereto (for example, not limited to the patient's left hip or right hip) .

Next, as shown in step S02, for each training image data, the processing unit 4 generates a plurality of manual positioning data corresponding to the training image data according to a plurality of manual positioning instructions received through the input unit 2, each The manual positioning data indicates the position of a certain positioning point P in the training image data. In this embodiment, the processing unit 4 executes a manual labeling program to display a manual labeling GUI via the display unit 3, and the manual labeling GUI includes the training image data and is available for the user ( For example, professional medical personnel) generate the manual positioning data by operating the input unit 2 to mark the positioning points P on the training image data.

In this embodiment, the number of the positioning points P is 20 (but not limited thereto), that is to say, each training image data corresponds to 20 pieces of manual positioning data. The anchor point P01 is the lowest point of the first sacral iliac joint (SI joint), the anchor point P02 is the lowest point of the second sacral iliac joint, the anchor point P03 is the lowest point of the first side Teardrop, and the anchor point P04 It is the lowest point of the second side teardrop bone (Teardrop), the location point P05 is the lowest point of the first side obturator foramen, the location point P06 is the lowest point of the second side obturator foramen, the location point P07 is the first side ischium The lowest point of (Ischium), the locating point P08 is the lowest point of the second ischium, the locating point P09 is the lower edge of the lesser trochanter (Lesser trochanter) on the first side, the locating point P10 is the lower edge of the second side lesser trochanter, and the locating point P11 is the hip The outermost edge of the long axis of the cup (Cup), the anchor point P12 is the innermost edge of the long axis of the acetabular cup, the anchor point P13 is the lateral edge of the widest axis of the femoral head, and the anchor point P14 is the medial edge of the widest axis of the femoral head The positioning point P15 is the center point of the femoral head, the positioning point P16 is the short-axis edge of the acetabular cup, the positioning point P17 is the Greater trochanter tip, the positioning point P18 is the upper edge of the femoral stem (Stem), the positioning point P19 is the midpoint of the femoral canal adjacent to the upper edge of the lesser trochanter, and the location point P20 is the midpoint of the femoral canal adjacent to the end edge of the femoral stem.

Next, as shown in step S03, for each positioning point P, the processing unit 4 trains a convolutional neural network (Convolutional Neural Network) according to the manual positioning data indicating the position of the positioning point P and the training image data Neural Network, CNN) model to generate an automatic localization model. For example, if the number of the training image data is 300 pieces, the processing unit 4 for each positioning point P, according to the 300 pieces of training image data and corresponding to the 300 pieces of training image data respectively, and indicates 300 pieces of manual positioning data of the position of the positioning point P are obtained, and the convolutional neural network model is trained to generate the automatic positioning model. In this embodiment, the processing unit 4 generates 20 automatic positioning models corresponding to the 20 positioning points P respectively. In this embodiment, the convolutional neural network model is a model named UNet++, wherein the parameters are updated by back-propagation for each training result, but the convolutional neural network model does not use UNet++ as the limit.

Referring to FIG. 1 and FIG. 4 , the steps of the bone state analysis method executed by the electronic system 100 are described below. First, as shown in step S11, the X-ray machine 1 photographs the predetermined bone 200 of a target patient and the implant 300 installed in the predetermined bone 200 to generate an image including the predetermined bone 200 and the implant The to-be-analyzed image data of the image of the object 300.

Next, as shown in step S12, the processing unit 4 uses the automatic positioning models generated in step S03 to generate a plurality of positions respectively indicating the positioning points P in the image data to be analyzed according to the image data to be analyzed. of automatic location data. The number of the automatic positioning data is 20 in this embodiment.

Next, as shown in step S13, the processing unit 4 generates a plurality of bone state parameter values related to distance or angle according to the automatic positioning data. The definitions of these bone state parameter values are described below.

Referring to FIG. 5 , a bone state parameter value is the included angle between a straight line passing through the positioning points P11 and P12 and a straight line passing through the positioning points P03 and P04 . Alternatively, the angle between the straight line passing through the positioning point P11 and the positioning point P12 and the straight line passing through the positioning point P07 and the positioning point P08 may be used. The bone state parameter value can be used to assess whether the implant 300 is prone to dislocation or impingement.

Referring to FIG. 6 , a bone state parameter value is the longitudinal distance between the positioning point P11 and the positioning point P03 . One of the bone state parameters is the longitudinal distance between the positioning point P12 and the positioning point P03. One of the bone state parameters is the longitudinal distance between the positioning point P15 and the positioning point P03. The aforementioned three bone state parameter values are used to evaluate whether the implant 300 has moved upward. If the aforementioned three bone state parameter values continue to increase, it means that the implant 300 is loose.

Referring to Figure 7, a bone state parameter value is calculated by the following formula:

Wherein A in the above formula is the longitudinal distance between the midpoint of the line connecting the positioning point P01 to the positioning point P02 and the midpoint of the line connecting the positioning point P03 to the positioning point P04, and B is the connecting line from the positioning point P03 to the positioning point P04 The longitudinal distance between the midpoint of and the midpoint of the line connecting the positioning point P05 to the positioning point P06.

Referring to Fig. 3, the definitions of the remaining skeleton state parameter values are described as follows. One of the bone state parameter values is the included angle between the straight line passing through the positioning point P11 and the positioning point P12 and the straight line passing through the positioning point P12 and the positioning point P16. One of the skeleton state parameter values is (the shortest distance from the anchor point P09 to the straight line passing through the anchor point P07 and the anchor point P08) - (the shortest distance from the anchor point P10 to the straight line passing through the anchor point P07 and the anchor point P08). One of the bone state parameters is the minimum distance between the anchor point P15 and a straight line passing through the anchor points P19 and P20. One of the bone state parameters is the minimum distance between the positioning point P15 and the vertical line extending longitudinally through the positioning point P03. One of the bone state parameters is the shortest distance between the positioning point P13 and the positioning point P14. One of the bone state parameters is the shortest distance between the positioning point P11 and the positioning point P12. One of the bone state parameters is the longitudinal distance between the positioning point P17 and the positioning point P18. One of the bone state parameters is the ratio of the shortest distance between the positioning point P11 and the positioning point P15 to the shortest distance between the positioning point P15 and the positioning point P12.

Referring to FIG. 1 and FIG. 4, then, as shown in step S14, the processing unit 4 displays an automatically marked image data through the display unit 3 according to the to-be-analyzed image data and the automatic positioning data, and the automatically marked image data includes the The image of the predetermined bone 200, the image of the implant 300, and a plurality of marker images indicating the positioning points P, respectively. The automatically marked image data can be used by medical personnel for reference before and during subsequent operations.

Next, as shown in step S15 , the processing unit 4 displays the bone state parameter values via the display unit 3 for medical personnel to view and evaluate the state of the predetermined bone 200 and the implant 300 .

Next, as shown in step S16, the processing unit 4 determines whether each bone state parameter value meets a corresponding warning condition, and if so, executes step S17. In step S17 , the processing unit 4 displays a warning message indicating that the bone state parameter value meets the corresponding warning condition via the display unit 3 . Each warning condition is, for example, that the corresponding bone state parameter value does not fall within a normal value range.

To sum up, the electronic system 100 of the present invention uses the processing unit 4 for each positioning point P to train the convolutional neural network according to the manual positioning data indicating the position of the positioning point P and the training image data network model to generate the automatic positioning model, and the processing unit 4 uses the automatic positioning models to generate a plurality of automatic positioning data respectively indicating the positions of the positioning points P in the image data to be analyzed, and The bone state parameter values related to the distance or the angle are generated according to the automatic positioning data, so as to improve the errors or errors that may occur in manual positioning in the prior art, and save the professional medical personnel from spending time on performing manual positioning. Therefore, It can indeed achieve the purpose of the present invention.

However, the above are only examples of the present invention, and should not limit the scope of implementation of the present invention. Any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the patent specification are still included in the scope of the present invention. within the scope of the invention patent.

100・・・Electronic systems 1････････････････････････････ X-ray machine 2・・・・Input unit 3・・・・Display unit 4・・・・Processing unit 200... Predetermined Bones 300...Implants P, P01~P20・・・・Location point S01~S03... Steps S11~S17... Steps

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: 1 is a schematic diagram of a hardware connection relationship of an embodiment of the electronic system of the present invention; 2 is a flow chart of this embodiment illustrating a method performed by the electronic system to generate a model for automatically locating an anchor point; Fig. 3 is a schematic diagram of this embodiment, illustrating the positions of a plurality of anchor points; FIG. 4 is a flow chart of this embodiment, illustrating a method for analyzing a bone state performed by the electronic system; 5 is a schematic diagram of the embodiment, illustrating the definition of a bone state parameter value; FIG. 6 is a schematic diagram of this embodiment, illustrating the definition of another bone state parameter value; and FIG. 7 is a schematic diagram of this embodiment, illustrating the definition of yet another bone state parameter value.

S01~S03... Steps

Claims (9)

A method of generating a model for automatically locating an anchor point is implemented by an electronic system, the electronic system comprising an X-ray machine, an input unit and a processing unit, the method comprising: The X-ray machine captures a predetermined bone of a plurality of reference patients and an implant mounted on the predetermined bone to generate a plurality of images respectively related to the reference patients and including the predetermined bone and the implant. training image data; For each training image data, the processing unit generates a plurality of manual positioning data corresponding to the training image data according to a plurality of manual positioning instructions received through the input unit, and each manual positioning data indicates that in the training image data the location of an anchor point of ; and For each positioning point, the processing unit trains a convolutional neural network model according to the manual positioning data indicating the position of the positioning point and the training image data to generate an automatic positioning model. The automatic positioning model uses generating an automatic positioning data according to a to-be-analyzed image data generated by photographing the predetermined bone of a target patient and the implant by the X-ray machine, the automatic positioning data indicating the position of the positioning point in the to-be-analyzed image data Location. A bone state analysis method is implemented by an electronic system, the electronic system includes an X-ray machine and a processing unit, and the method includes: The X-ray machine photographs a predetermined bone of a target patient and an implant mounted on the predetermined bone to generate an image data to be analyzed including the image of the predetermined bone and the image of the implant; The processing unit uses the automatic positioning models as described in claim 1 according to the image data to be analyzed to generate a plurality of automatic positioning data respectively indicating the positions of the positioning points in the image data to be analyzed; and The processing unit generates a bone state parameter value related to distance or angle according to the automatic positioning data. The bone state analysis method according to claim 2, wherein the electronic system further includes a display unit, and after generating the automatic positioning data, the method further includes: The processing unit displays an automatically marked image data through the display unit according to the to-be-analyzed image data and the automatic positioning data, and the automatically marked image data includes the image of the predetermined bone, the image of the implant, and a plurality of respectively indicating Marker images for these anchor points. The bone state analysis method according to claim 2, wherein the electronic system further includes a display unit, and after generating the bone state parameter value, the method further includes: The processing unit displays the bone state parameter value via the display unit. The bone state analysis method according to claim 2, wherein the electronic system further includes a display unit, and after generating the bone state parameter value, the method further includes: The processing unit determines whether the bone state parameter value meets an alert condition; and When the processing unit determines that the bone state parameter value meets the warning condition, the processing unit displays a warning message through the display unit indicating that the bone state parameter value meets the warning condition. An electronic system comprising: an X-ray machine; and a processing unit; The X-ray machine photographs a predetermined bone of a target patient and an implant mounted on the predetermined bone to generate an image data to be analyzed including the image of the predetermined bone and the image of the implant; The processing unit uses the automatic positioning models as described in claim 1 according to the image data to be analyzed to generate a plurality of automatic positioning data respectively indicating the positions of the positioning points in the image data to be analyzed; The processing unit generates a bone state parameter value related to distance or angle according to the automatic positioning data. The electronic system according to claim 6, further comprising a display unit, wherein the processing unit displays an automatically marked image data according to the to-be-analyzed image data and the automatic positioning data, and the automatically marked image data includes the predetermined skeleton. An image, an image of the implant, and a plurality of marker images respectively indicating the locating points. The electronic system according to claim 6, further comprising a display unit, wherein the processing unit displays the bone state parameter value via the display unit. The electronic system according to claim 6, further comprising a display unit, wherein the processing unit determines whether the bone state parameter value meets a warning condition; When the processing unit determines that the bone state parameter value meets the warning condition, the processing unit displays a warning message through the display unit indicating that the bone state parameter value meets the warning condition.

Images