TWI758952B - A method of assisted interpretation of bone medical images with artificial intelligence and system thereof - Google Patents

Abstract

The present invention is provided a method of assisted interpretation of bone medical images with artificial intelligence. The method includes step S1, receiving a medical image related to a bone; step S2, using a deep learning algorithm to calculate the medical image to establish a distribution; step S3, selecting automatically one or more types of plural types according to the distribution; step S4, using a hotspot image algorithm to mark a designated part of the bone; and step S5, displaying the marked the designated part of the medical image and its corresponding type to provide a piece of auxiliary diagnosis information. The invention also provides a system of assisted interpretation of bone medical images with artificial intelligence.

Description

Artificial intelligence-assisted bone medical image interpretation method and system

The invention belongs to the technical field of intelligent medical treatment, in particular to an artificial intelligence-assisted bone medical image interpretation method and system for bone-assisted image recognition.

Traditionally, doctors need to judge the condition of bones through X-rays or MRIs; however, it is difficult to distinguish small damages on bones from X-rays or MRIs. pain points.

For example, the triangular fibrocartilage complex (TFCC) of the wrist is a structure between the distal ulna, the distal radius, and the ulnar metacarpal bone, which includes ligaments and cartilage tissue and plays a role in stabilizing the distal An important role of the distal radioulnar joint Injuries to the triangular fibrocartilage complex of the wrist occur in pronation and hyperextension movements of the wrist, and may also be associated with distal radius fractures. The more common ball sports injuries include tennis and badminton.

At present, the diagnosis of TFCC injury is mainly based on medical imaging. For example, X-rays can check the ulnar variance ("positive" ulnar variance, which may indicate that the space where the TFCC is located is compressed and can be used as a diagnosis of TFCC. In addition, magnetic resonance imaging (Magnetic Resonance Imaging) mainly observes whether there is rupture of the triangular fibrocartilage complex at the ulnar or radial attachment points. .

Due to the complex structure of the triangular fibrocartilage complex and the variety of potential lesions, sometimes MRI images cannot determine whether there is damage. Several existing research papers have been published on how to use imaging to improve the clinical interpretation of MRI lesions, which is enough to prove that the difficulty of interpreting TFCC injuries is a common problem for orthopaedic physicians.

In view of this, the present invention proposes an artificial intelligence-assisted bone medical image interpretation method and a system thereof, which are used to solve the deficiencies of the prior art.

The first objective of the present invention is to provide an artificial intelligence-assisted bone medical image interpretation method, which uses artificial intelligence image recognition technology to assist physicians in marking the position and severity of subtle injuries on bones.

The second object of the present invention is to predict and classify the damage of the triangular fibrocartilage complex of the wrist according to the above-mentioned artificial intelligence-assisted bone medical image interpretation method.

The third object of the present invention is to predict bone age based on the above-mentioned artificial intelligence-assisted bone medical image interpretation method.

The fourth object of the present invention is to use the existing medical image and the judged result as the medical sample image to train the deep learning algorithm according to the above-mentioned artificial intelligence-assisted bone medical image interpretation method.

The fifth object of the present invention is to establish a type by deep learning algorithm according to the above-mentioned artificial intelligence-assisted bone medical image interpretation method, so as to distinguish the type of bone into normal type and abnormal type, and the abnormal part can also be imported For example palmer classification.

The sixth object of the present invention is to provide a review confirmation form including the image number and the true/false confirmation column according to the above-mentioned artificial intelligence-assisted bone medical image interpretation method, so that doctors can record the medical image of the corresponding image number in the review confirmation form according to the image number. Review results of sample images to optimize deep learning algorithms.

The seventh object of the present invention is to provide an artificial intelligence-assisted bone medical image interpretation system, which is used to realize the artificial intelligence-assisted bone medical image interpretation method.

In order to achieve the above object and other objects, the present invention provides an artificial intelligence-assisted bone medical image interpretation method. The artificial intelligence-assisted bone medical image interpretation method includes step S1, receiving a medical image, wherein the medical image is related to a bone; step S2, executing a deep learn algorithm to calculate the medical image, so as to capture the medical image from the medical image. Take a feature point (feature point) to establish a distribution state; Step S3, execute the deep learning algorithm to select one or more types from the complex number type according to the distribution state; Step S4, use a hot spot image algorithm to calculate The medical image is used to mark a designated part of the bone in the medical image; and step S5 is to display the medical image of the marked designated part and its corresponding type, so as to provide an auxiliary diagnosis information.

In order to achieve the above object and other objects, the present invention provides an artificial intelligence-assisted bone medical image interpretation system, which includes an input unit, a processing unit and an output unit. The input unit receives a medical image, wherein the medical image is related to a bone. The processing unit is connected to the input unit. The processing unit executes a deep learning algorithm to calculate the medical image, and extracts a feature point from the medical image to establish a distribution state, and the deep learning algorithm selects one or more types from the complex type according to the distribution state, and another processing unit A hot spot image algorithm is executed to calculate the medical image to mark a specified part of the bone in the medical image. The output unit is connected to the processing unit. The output unit outputs the medical image of the designated part of the marked bone and the selected type or the like.

Compared with the conventional technology, the artificial intelligence-assisted bone medical image interpretation method and system provided by the present invention can perform image judgment, injury classification, and injury focus marking functions for minor injuries on the bone, so as to assist doctors in medical diagnosis. In addition, the doctor can also give feedback on the content of the present invention to enhance the judgment ability of the artificial intelligence.

In order to fully understand the purpose, features and effects of the present invention, the present invention is described in detail by the following specific embodiments and the accompanying drawings. The description is as follows.

In the present disclosure, the use of "a" or "an" is used to describe the elements, elements and components described herein. This is done only for convenience of description and to provide a general sense of the scope of the invention. Thus, unless it is clear that it is meant otherwise, such descriptions should be read to include one, at least one, and the singular also includes the plural.

In the present invention, the terms "comprising", "including", "having", "containing" or any other similar terms are intended to encompass non-exclusive inclusions. For example, an element, structure, article or device containing a plurality of elements is not limited to those elements listed herein, but may include not explicitly listed but generally inherent to the element, structure, article or device other requirements. Otherwise, unless expressly stated to the contrary, the term "or" refers to an inclusive "or" and not an exclusive "or".

In the present invention, the description of the steps is not limited by the order of the preceding and subsequent descriptions. The steps disclosed in the present invention are mainly for convenience of illustrating the embodiment. In other embodiments, the smoothness of the execution of the steps can be adjusted or changed.

Please refer to FIG. 1 , which is a schematic flowchart of an artificial intelligence-assisted bone medical image interpretation method according to a first embodiment of the present invention. In FIG. 1 , the AI-assisted bone medical image interpretation method starts at step S11 , which is to receive a medical image, for example, the medical image is related to a bone, such as a hand bone, a skull bone, a foot bone, a hip bone, and the like. In addition, the medical image can be generated from an X-ray machine, Computed Tomography, Magnetic Resonance Imaging, or from a medical image database. For example, in this embodiment, the bone is taken as an example of a hand bone, and the X-ray medical image of the hand bone in FIG. 2( a ) can be referred to together. In Fig. 2(a), a schematic X-ray view of the bone of Fig. 1 of the present invention is illustrated.

Before performing step S11, the step of training the deep learning algorithm may be performed first. Referring to FIG. 3 together, it is a schematic flowchart illustrating the training of the deep learning algorithm of the present invention. In FIG. 3, the step of training the deep learning algorithm starts at S31, which is to provide a plurality of medical sample images. Wherein, the medical sample images are related to bones, which are the image types corresponding to step S11.

Step S32 is to classify the medical sample images to classify the medical sample images into plural types. Referring to FIG. 4 , it is a schematic structural diagram illustrating the training of the deep learning algorithm in FIG. 3 of the present invention. In FIG. 4, the types are 1A, 1B, 1C and 1D defined as non-normal types and normal types.

In step S33 , a deep learning algorithm is used to calculate the medical sample images corresponding to each of the types, and feature points are extracted from the medical sample images of each of the types.

In step S34, the corresponding distribution states generated by the feature points are recorded, so that the abnormal types of Type 1A, Type 1B, Type 1C and Type 1D and the normal types have their corresponding distribution states. Wherein, in this embodiment, the number of abnormal types is described by taking 4 as an example. In other embodiments, the number of abnormal types may be less than 4 or more than 4, and is not limited to 4 kind. In yet another embodiment, the abnormal type can be classified by Palmer. Palmer classification can be further divided into trauma (traumatic injury) and degenerative injury (degenerative injury) and other types.

Step S35 , marking the designated part of the bone on the medical sample images, for example, the designated part may be the triangular fibrocartilage complex of the wrist. Also refer to the marked hand bone X-ray medical image in Figure 2(b). Figure 2(b) is a schematic X-ray diagram illustrating the marked bone of Figure 2(a) of the present invention. In Figure 2(b), the triangular fibrocartilage of the wrist is calculated by, for example, a hot spot image algorithm, and the triangular fibrocartilage can be marked in a frame on the X-ray medical image of the hand bone. Among them, the hot spot image algorithm can be performed using gradient-based sensitivity analysis and other algorithms that can explain AI.

It is worth noting that, in another embodiment, the deep learning algorithm can further import impact factors for calculation, for example, the impact factors can be age, gender, BMI, disease history, surgery history, and the image factor is related to the subject of medical images. irradiator.

Returning to FIG. 3 , in step S36 , a predetermined number of the medical sample images are randomly selected for review by the doctor to generate a review result of the medical sample images. Wherein, these medical sample images are marked with designated parts. Referring to FIG. 4 , it is shown that the review result provides the image numbers of the medical sample images and their corresponding true (T)/false (F) confirmation columns for doctors to confirm that the medical sample images are calculated by the deep learning algorithm. Is it trustworthy. For example, in this embodiment, if the medical sample image is calculated by the deep learning algorithm to show the correct type, the doctor will check the medical sample image classification as true (T) in the true (T)/false (F) confirmation column ; On the contrary, the doctor will check the medical sample image classification as false (F) in the True (T)/False (F) confirmation column. Also in this embodiment, the predetermined number of the medical sample images may be set to be at least 100, and there are at least 50 normal type of the medical sample images in the predetermined number.

In another embodiment, the training of the deep learning algorithm may further include step S37 of optimizing the deep learning algorithm using the review result.

Returning to FIG. 1 , step S12 is continued, in which a deep learning algorithm is executed to calculate the medical image, so as to extract feature points from the medical image to establish a distribution state. The feature points are calculated by the deep learning algorithm to display the probability distribution according to the type in the distribution state. The deep learning algorithm is a convolutional neural network. After the deep learning algorithm calculates the medical image, it is predicted that the medical image belongs to one or more of these types.

In step S13, a deep learning algorithm is executed to select one or more types from the complex type according to the distribution state, that is, to classify the medical images into one or more types. Referring to FIG. 5 together, it shows the probability distribution of the feature points in the distribution state of a certain medical image after the deep learning algorithm is applied. Taking Figure 5 as an example, the values of the probability distribution are distributed in various types of states, but the values in Type 1C and Type 1D are higher, so the deep learning algorithm predicts that the default medical image of the medical image belongs to Type 1C. Or maybe Type 1C and Type 1D. In other words, in one embodiment, when the value of the probability distribution exceeds a predetermined threshold, the medical images corresponding to the feature points are classified into one or more of these types.

Returning to FIG. 1 , in step S14 , the hot spot image algorithm is used to calculate the medical image, so as to mark a specified part of the bone in the medical image, as shown in FIG. 2( b ).

In step S15, the marked medical image of the designated part and its corresponding type are displayed, so as to provide an auxiliary diagnosis information.

Please refer to FIG. 6 , which is a block diagram of an artificial intelligence-assisted bone medical image interpretation system according to a second embodiment of the present invention. In FIG. 6 , the artificial intelligence-assisted bone medical image interpretation system 10 includes an input unit 12 , a processing unit 14 and an output unit 16 .

The input unit 12 receives a medical image MIMG. Among them, the medical image MIMG is related to a bone 2 .

The processing unit 14 is connected to the input unit 12 . The processing unit 14 executes a deep learning algorithm DLA to calculate the medical image MIMG, extracts a feature point from the medical image MIMG to establish a distribution state DS, and the deep learning algorithm DLA selects one from the complex type CLS according to the distribution state DS or multiple types of CLS, the processing unit 14 executes a hot spot image algorithm HPA to calculate the medical image MIMG, so as to mark a specified part 22 of the bone 2 in the medical image MIMG.

The output unit 16 is connected to the processing unit 14 . The output unit 16 outputs the marked medical image MIMG' of the designated part 22 of the bone 2 and the selected type CLS or the same type CLS.

Please refer to FIG. 7 , which is a block diagram of an artificial intelligence-assisted bone medical image interpretation system according to a third embodiment of the present invention. In FIG. 7 , the artificial intelligence-assisted bone medical image interpretation system 10' includes the input unit 12, the processing unit 14, and the output unit 16 of the second embodiment, and further includes a feedback module 18.

The input unit 12 , the processing unit 14 and the output unit 16 are as described above, and will not be repeated here.

The feedback module 18 is connected to the processing unit 14 . The feedback module 18 transmits the medical images and/or the selected type or types of feedback information of the designated part of the marked bone to optimize the deep learning algorithm.

It is worth noting that, in the foregoing embodiment, the doctors and medical images can be implemented in, for example, the same medical field, and can also be implemented across medical fields through the cloud. For example, doctors from all over the world can use the results of their feedback. To strengthen and optimize deep learning algorithms.

The present invention has been disclosed above with preferred embodiments, but those skilled in the art should understand that the embodiments are only used to describe the present invention, and should not be construed as limiting the scope of the present invention. It should be noted that all changes and substitutions equivalent to the embodiments should be set to be included within the scope of the present invention. Therefore, the protection scope of the present invention should be defined by the scope of the patent application.

S11-S15: Method steps S31-S37: Method steps 2: bones 22: Designated parts 10, 10': Artificial intelligence-assisted bone medical image interpretation system 12: Input unit 14: Processing unit 16: Output unit 18: Feedback Module MIMG, MIMG': Medical Imaging DLA: Deep Learning Algorithms cls:type HPA: Hotspot Image Algorithm

FIG. 1 is a schematic flowchart of a method for artificial intelligence-assisted bone medical image interpretation according to a first embodiment of the present invention. Figure 2(a) is a schematic X-ray diagram illustrating the bone of Figure 1 of the present invention. FIG. 2(b) illustrates the labeled hand bone X-ray medical image of FIG. 3 of the present invention. FIG. 3 is a schematic flowchart illustrating the training of the deep learning algorithm of the present invention. FIG. 4 is a schematic diagram illustrating the structure of training the deep learning algorithm of FIG. 3 of the present invention. FIG. 5 is a schematic diagram illustrating the state of the probability distribution of FIG. 1 in each type of the present invention. FIG. 6 is a block diagram of an artificial intelligence-assisted bone medical image interpretation system according to a second embodiment of the present invention. FIG. 7 is a block diagram of an artificial intelligence-assisted bone medical image interpretation system according to a third embodiment of the present invention.

S11-S15: Method steps

Claims (15)

An artificial intelligence-assisted bone medical image interpretation method is executed by an artificial intelligence-assisted bone medical image interpretation system. The artificial intelligence-assisted bone medical image interpretation system includes an input unit, a processing unit and an output unit, and the method includes: S1: Receive a medical image with the input unit, wherein the medical image is related to bone; S2: Execute a deep learning algorithm with the processing unit to calculate the medical image and extract features from the medical image point (feature point) to establish a distribution state, to display a probability distribution conforming to one or more types in the distribution state; S3: execute the deep learning algorithm with the processing unit to determine the probability distribution according to the value of the probability distribution The medical image corresponding to the feature point is classified into the one or more types, and the one or more types are selected from the plural types; S4: the processing unit uses the hot spot image algorithm to calculate the medical image, so that the medical image is Mark the designated part of the bone; and S5: output the marked medical image of the designated part and its corresponding type or types for providing auxiliary diagnosis information, wherein the types are divided into a A normal type and an abnormal type, and the number of the abnormal type is one or plural. The artificial intelligence-assisted bone medical image interpretation method according to claim 1, wherein in step S1, the medical image is generated from an X-ray machine, an X-ray tomography (Computed Tomography) and a Magnetic Resonance Imaging (Magnetic Resonance Imaging). at least one. The artificial intelligence-assisted bone medical image interpretation method according to claim 1, wherein the designated part is the triangular fibrocartilage complex of the wrist. The artificial intelligence-assisted bone medical image interpretation method according to claim 3, wherein the value of the probability distribution exceeds a predetermined threshold, and the medical image corresponding to the feature point is classified into one or more of these types. The artificial intelligence-assisted bone medical image interpretation method according to claim 1, wherein the deep learning algorithm executed by the processing unit is a convolutional neural network, and the deep learning algorithm calculates the medical After imaging, the medical image is predicted to belong to one or more of these categories. The artificial intelligence-assisted bone medical image interpretation method according to claim 1, wherein the hot spot image algorithm is gradient-based sensitivity analysis. The artificial intelligence-assisted bone medical image interpretation method according to claim 1, wherein the deep learning algorithm further introduces an impact factor for calculation, wherein the impact factor is at least one of age, gender, BMI, disease history and surgery history. The artificial intelligence-assisted bone medical image interpretation method as claimed in claim 1, wherein before performing step S1 with the processing unit, further comprises training the deep learning algorithm: S6: providing a plurality of medical sample images, wherein the medical sample images related to the bone; S7: classify the medical sample images to distinguish the medical sample images into the types; S8: use the deep learning algorithm to calculate the medical sample images corresponding to each of the types, in Extracting feature points from the medical sample images of each of the types; S9: recording the corresponding distribution state generated by the feature points; S10: marking the designated part of the bone in the medical sample images; and S11: Randomly select a predetermined number of the medical sample images for review by the doctor to generate a review result of the medical sample images, wherein the medical sample images have been marked with the designated part. The artificial intelligence-assisted bone medical image interpretation method as described in claim 8 further comprises step S12, using the review result to optimize the deep learning algorithm. The artificial intelligence-assisted bone medical image interpretation method according to claim 8, wherein in step S11, the review result provides the image numbers of the medical sample images and their corresponding true/false confirmation fields. The artificial intelligence-assisted bone medical image interpretation method according to claim 10, wherein in step S11, a review confirmation form including the image number and the true/false confirmation column is provided for the doctor to perform the review according to the image number. The confirmation form records the review result of the medical sample images corresponding to the image number. The artificial intelligence-assisted bone medical image interpretation method according to claim 8, wherein the predetermined number of the medical sample images is at least 100 and there are at least 50 normal type of the medical sample images in the predetermined number. The artificial intelligence-assisted bone medical image interpretation method as claimed in claim 8, wherein the types adopt Palmer classification, wherein the Palmer classification is divided into trauma (traumatic injury) and degenerative injury (degenerative injury) ). An artificial intelligence-assisted bone medical image interpretation system, comprising: an input unit, which receives medical images, wherein the medical images are related to bones; a processing unit, which is connected to the input unit, and the processing unit executes a deep learning algorithm to calculate the medical image, and extracting feature points from the medical image to establish a distribution state, so as to The distribution state shows a probability distribution that conforms to one or more types, and the deep learning algorithm determines that the medical image corresponding to the feature point is classified into the one or more types according to the value of the probability distribution, with a complex number of The type selects the one or more types, and the processing unit executes a hot spot image algorithm to calculate the medical image, so as to mark the designated part of the bone in the medical image; and an output unit is connected to the processing unit, and the output unit outputs The marked medical image of the designated part of the bone and the type or types selected for providing auxiliary diagnostic information, wherein the types are distinguished into a normal type and an abnormal type, the abnormal type of Quantity is one or plural. The artificial intelligence-assisted bone medical image interpretation system according to claim 14, further comprising a feedback module, which is connected to the processing unit, and the feedback module transmits the marked medical image of the designated part of the bone and the selected Feedback of the type or at least one of the types to optimize the deep learning algorithm.

Images