WO2022139246A1 - Fracture detecting method and device using same - Google Patents

Fracture detecting method and device using same Download PDF

Info

Publication number
WO2022139246A1
WO2022139246A1 PCT/KR2021/018314 KR2021018314W WO2022139246A1 WO 2022139246 A1 WO2022139246 A1 WO 2022139246A1 KR 2021018314 W KR2021018314 W KR 2021018314W WO 2022139246 A1 WO2022139246 A1 WO 2022139246A1
Authority
WO
WIPO (PCT)
Prior art keywords
fracture
bone
region
image
suspected
Prior art date
Application number
PCT/KR2021/018314
Other languages
French (fr)
Korean (ko)
Inventor
김광기
이기택
김영재
정태석
Original Assignee
가천대학교 산학협력단
의료법인 길의료재단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 가천대학교 산학협력단, 의료법인 길의료재단 filed Critical 가천대학교 산학협력단
Publication of WO2022139246A1 publication Critical patent/WO2022139246A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/45—For evaluating or diagnosing the musculoskeletal system or teeth
    • A61B5/4504—Bones
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
    • A61B6/505—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of bone
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
    • A61B8/0875—Detecting organic movements or changes, e.g. tumours, cysts, swellings for diagnosis of bone
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5223—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for extracting a diagnostic or physiological parameter from medical diagnostic data
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the present invention relates to a method for detecting a fracture and a device using the same.
  • a fracture refers to a state in which the continuity of a bone, epiphyseal plate, or articular surface is completely or incompletely lost, and is usually caused by external forces. At this time, fractures are often accompanied by damage to soft tissues or organs in the vicinity of the bone. Depending on the location of the fracture, fractures can be broadly divided into limb fractures, vertebral fractures, and other fractures such as ribs, skulls, orbits.
  • diagnosis of a fracture may be performed through X-ray imaging.
  • diagnosis of fractures based on X-ray images may require additional special examinations such as computed tomography and magnetic resonance imaging because the results are ambiguous or it is difficult to accurately confirm the fracture pattern.
  • the proposed fracture detection system still has a limitation in that it is difficult to apply to actual clinical practice due to high false positives for the detection of fractures, especially for the detection of fractures of the skull in which blood vessels and skull junctions exist in a form similar to a fracture. there may be
  • a new fracture detection system that minimizes the occurrence of false positives by applying a deep learning-based prediction model configured to automatically predict by learning a fracture bone medical image. wanted to develop.
  • the inventors of the present invention designed a new fracture detection system to detect a fracture suspected region using a predictive model and to finally classify a fracture by removing a similar fracture region in a shape similar to a fracture among the fracture suspected regions.
  • the inventors of the present invention were able to set the blood vessels and the bone junction line having characteristics different from those of a fracture in which the thickness of the line is constant and the size and location are not determined as the similar fracture region.
  • the inventors of the present invention found that, in the case of blood vessels, the line thickness is irregular and is located near the eyeball and temple, and in the case of the bone tangent line, the line shape is irregular, the line thickness is thicker than the fracture line, and it exists at a constant location for each individual.
  • the inventors of the present invention were able to design the final fracture determination by filtering the classification result of the predictive model for the fracture detection system so that similar fracture regions such as blood vessels or bone junctions are removed.
  • the inventors of the present invention believe that by providing a fracture detection system based on a predictive model, the sensitivity of fracture diagnosis will be increased even if the medical staff does not perform additional bone medical imaging by providing information on parts not identified with the naked eye. could be expected
  • the inventors of the present invention tried to apply a learnable multi view algorithm to classify fractures by considering all of the bone medical images taken from a plurality of directions with respect to the predictive model.
  • the inventors of the present invention were able to apply the training bone medical image to which the medical bone image taken from a plurality of angles is matched to the learning of the predictive model.
  • the problem to be solved by the present invention is a fracture detection method, configured to detect a fracture in a medical bone image using a deep learning-based prediction model, determine and remove a similar fracture area, and finally provide a fracture site, and To provide a device using this.
  • a fracture detection method is a fracture detection method implemented by a processor, comprising: receiving a medical bone image of an object; determining a fracture suspected region for a medical bone image using the .
  • the medical bone image may include a plurality of medical bone images taken from a plurality of angles with respect to a target site of an object.
  • the determining of the fracture suspected region includes determining the fracture suspect region for each of a plurality of medical bone images by using a prediction model, and the determining of the similar fracture region includes each of the plurality of medical medical images. It may include the step of determining a similar fracture area among the fracture suspected areas for the. Determining the fracture site may include determining a fracture area for each of a plurality of bone medical images.
  • the predictive model may be a plurality of predictive models configured to predict a fracture site for each of a plurality of bone medical images.
  • the determining of the fracture suspected region may include determining the fracture suspected region for each of the plurality of bone medical images by using each of the plurality of prediction models.
  • the predictive model may be a single predictive model configured to predict a fracture site with respect to a plurality of bone medical images.
  • the determining of the fracture suspected region may include determining the fracture suspected region for a plurality of bone medical images by using a single prediction model.
  • the plurality of medical images of bones may be at least two of a medical image of a frontal bone, a medical image of a posterior bone, a medical image of a right bone, and a medical image of a left bone.
  • the method includes, after determining the fracture site, comparing the fracture site of a set image selected from among a plurality of bone medical images, and finally determining the fracture site based on the comparison result It may further include the step of In this case, the set image may be a medical image of a frontal bone and a medical image of a posterior bone, or a medical image of a right bone and a medical image of a left bone.
  • the comparing the fracture site may include comparing the determined location of the fracture site with respect to the set image.
  • the fracture is a linear fracture
  • the determining of the similar fracture region may include determining the similar fracture region based on at least one of the thickness, shape, and location of the suspected fracture region.
  • the pseudo-fracture region is a region of at least one of a bone junction, a blood vessel, and a bone overlap.
  • the medical bone image may be one of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image.
  • the fracture suspected region is the skull, mandible, hyoid bone, cervical vertebrae, thoracic vertebrae, lumbar vertebrae, ribs, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunar, triangular, and cephalothorax.
  • crotch bone at least one of the crotch bone, scapula, papillary bone, papillary bone, metacarpal, finger bone, tibia, femur, patella, tibia, fibula, talus, calcaneus, scaphoid, cuboid, sphenoid, metatarsal and toe bones It may be a suspected fracture area.
  • the device includes a communication unit configured to receive a medical bone image of the subject, and a processor connected to the communication unit, wherein the processor uses a predictive model configured to output a fracture site by inputting the medical bone image as an input, It is configured to determine a fracture suspect region for the image, determine a similar fracture region among the suspected fracture regions, and filter the suspected fracture region to remove the similar fracture region to determine a fracture site.
  • a predictive model configured to output a fracture site by inputting the medical bone image as an input, It is configured to determine a fracture suspect region for the image, determine a similar fracture region among the suspected fracture regions, and filter the suspected fracture region to remove the similar fracture region to determine a fracture site.
  • the medical bone image may include a plurality of medical bone images taken from a plurality of angles with respect to a target site of an object.
  • the processor determines a fracture suspected region for each of the plurality of medical bone images by using the prediction model, determines a similar fracture region from among the fracture suspect regions for each of the plurality of medical images of bone, and determines a plurality of medical images and may be further configured to determine a fracture area for each.
  • the predictive model may be a plurality of predictive models configured to predict a fracture site for each of a plurality of bone medical images.
  • the processor may be configured to determine a fracture suspected region for each of a plurality of bone medical images by using each of a plurality of predictive models.
  • the predictive model may be a single predictive model configured to predict a fracture site with respect to a plurality of bone medical images.
  • the processor may be configured to determine a fracture suspected region for a plurality of bone medical images by using a single prediction model.
  • the plurality of medical images of bones may be at least two of a medical image of a frontal bone, a medical image of a posterior bone, a medical image of a right bone, and a medical image of a left bone.
  • the processor is further configured to compare a fracture site of a set image selected from among a plurality of bone medical images, and finally determine a fracture site based on the comparison result, wherein the set image is a frontal bone medical image It may be an image and a medical image of the posterior bone, or a medical image of a right bone and a medical image of the left bone.
  • the processor may be further configured to compare the determined location of the fracture site with respect to the set image.
  • the present invention may provide a fracture detection system configured to predict and detect a fracture site with respect to a bone medical image based on a deep learning algorithm.
  • the present invention has the effect of predicting a fracture with high accuracy by providing a deep learning-based fracture detection system.
  • the present invention provides a fracture detection system designed to detect a fracture suspected region using a predictive model, and to finally classify a fracture by removing a similar fracture region in a shape similar to a fracture among the fracture suspected region, thereby providing a fracture detection system such as a skull fracture. It can provide reliable diagnostic results for areas where accurate diagnosis is difficult.
  • the present invention provides a fracture detection system based on a predictive model, thereby providing information on parts not identified with the naked eye, thereby providing a high-sensitivity diagnosis result for a fracture without additional medical imaging of bone by a medical staff. have.
  • the present invention provides a fracture detection system using a predictive model based on a deep learning algorithm, thereby preventing erroneous interpretation by medical personnel and improving the workflow of medical personnel in actual clinical practice.
  • the effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.
  • FIG. 1A exemplarily shows a fracture detection system based on a device for fracture detection according to an embodiment of the present invention.
  • FIG. 1B exemplarily shows the configuration of a device for detecting a fracture according to an embodiment of the present invention.
  • FIG. 1C exemplarily illustrates the configuration of a medical staff device configured to receive a fracture-suspected region from the device for detecting a fracture according to an embodiment of the present invention.
  • 2A to 2C exemplarily show a procedure of a method for detecting a fracture according to an embodiment of the present invention.
  • 3 and 4 exemplarily show the structure of a prediction model used in various embodiments of the present invention.
  • 5A to 5E exemplarily show the structure of a prediction model used in various embodiments of the present invention.
  • expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.
  • expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together.
  • “A or B,” “at least one of A and B,” or “at least one of A or B” means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.
  • first may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.
  • first user equipment and the second user equipment may represent different user equipment regardless of order or importance.
  • the first component may be named as the second component, and similarly, the second component may also be renamed as the first component.
  • a component eg, a first component is "coupled with/to (operatively or communicatively)" to another component (eg, a second component);
  • another component eg, a second component
  • the certain element may be directly connected to the other element or may be connected through another element (eg, a third element).
  • a component eg, a first component
  • another component eg, a second component
  • a device configured to may mean that the device is “capable of” with another device or parts.
  • a processor configured (or configured to perform) A, B, and C refers to a dedicated processor (eg, an embedded processor) for performing the corresponding operations, or by executing one or more software programs stored in a memory device.
  • a generic-purpose processor eg, a CPU or an application processor
  • the term "fracture” refers to a state in which the continuity of a bone, epiphyseal plate, or articular surface is completely or incompletely lost, and may include comminuted fractures, segmental fractures, linear fractures, depression fractures, and basal fractures.
  • the fracture within the present specification may be a linear fracture, more preferably a cranial linear fracture, but is not limited thereto.
  • the suspected fracture site is the mandible, hyoid, cervical, thoracic, lumbar, rib, sternum, clavicle, scapula, humerus, radius, ulna, scaphoid, lunate, triangular bone, cephalothorax, occipital bone, scapula , papilla, papilla, metacarpal, finger bone, zygomatic bone, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid bone, metatarsal bone, and toe bone.
  • the term “subject” may refer to any target for which a fracture is to be detected.
  • the subject may be a subject suspected of having a skull fracture.
  • the mandible, hyoid bone, cervical spine, thoracic spine, lumbar spine, rib, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunate, triangular bone, cephalothorax, crotch, scapula, papillary bone , papillary, metacarpal, finger bone, tibia, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid, metatarsal, and toe bones have.
  • the subject disclosed in the present specification may be any mammal other than humans, but is not limited thereto.
  • the term "medical bone image” is a medical image of a bone including a suspected fracture site, and may be one of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image.
  • the medical bone image may be an X-ray image, but is not limited thereto.
  • a medical image of a bone may be used interchangeably with a medical image.
  • the medical bone image may be a plurality of medical images taken from a plurality of angles.
  • the plurality of medical images may be at least two medical images among a front medical image, a rear medical image, a right side medical image, and a left side medical image.
  • the bone medical image may be a moving picture consisting of a plurality of frames.
  • fracture prediction may be possible for each frame of a moving picture according to the fracture detection method according to an embodiment of the present invention. That is, since the fracture prediction is possible simultaneously with the reception of the medical bone image from the camera, the fracture may be detected.
  • the medical bone image may be a two-dimensional image or a three-dimensional image.
  • the term "suspicious fracture region" may be a region suspected of a fracture in a medical image.
  • the fracture suspected region may include a region for a site where the actual fracture occurred and a fracture-like region having a shape similar to that of a fracture.
  • the fracture-like region may include a region for a region similar to a fracture, such as a bone junction line, a blood vessel, and a bone overlap line.
  • the fracture-like region is not limited thereto.
  • predictive model may be a model configured to output a fracture suspected region, that is, a fracture suspected region by inputting a medical bone image as an input.
  • the predictive model may be configured to divide and output a region suspected of fracture by inputting a medical bone image as an input.
  • the prediction model may include a plurality of models configured to predict a fracture suspected region for each of the medical bone images taken from a plurality of angles.
  • the predictive model may be a model configured to reduce erroneous detection of fractures by using an image obtained by matching a plurality of medical images captured from a plurality of angles as learning data.
  • the predictive model uses a front-to-back medical image in which the front medical image and the posterior medical image are registered, or a left-right registered medical image in which the left medical image and the right medical image are registered, for training to determine the fracture suspected region.
  • a front-to-back medical image in which the front medical image and the posterior medical image are registered
  • a left-right registered medical image in which the left medical image and the right medical image are registered
  • the registration of the medical images may be performed so that, after the positions and sizes of the respective medical images are matched, the predetermined positions of the fracture regions for each medical image are matched.
  • the two images may be registered after left and right inversion of the left side medical image or the right side medical image is performed.
  • fracture prediction performance may be superior to that of other models.
  • the prediction model may be RetinaNet trained to output a fracture suspected region for each of a plurality of images taken from a plurality of angles, but is not limited thereto, and Resnet50, Resnet-v2, Resnet101, Inception-v3, or VGG At least one selected from net, R, DenseNet, and a deep neural network (DNN) such as FCN, SegNet, DeconvNet, DeepLAB V3+, and U-net having an encoder-decoder structure, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet It may be algorithm-based.
  • DNN deep neural network
  • a plurality of models trained to predict a suspected fracture region for each angle of a medical image may be used to predict a suspected fracture region for each of the plurality of images.
  • the prediction model is a Multi-view Convolutional Neural Network (MVCNN) or Multi-Planar CNN (MPCNN), which is trained to predict a fracture suspected region with respect to a multi-view image (eg, a 3D image, or a multi-angle image).
  • MVCNN Multi-view Convolutional Neural Network
  • MPCNN Multi-Planar CNN
  • a single model trained to predict a fracture suspected region with respect to a multi-view image taken from multiple angles may be used to predict a suspected fracture region for a plurality of images from multiple angles.
  • the predictive model may be an ensemble model based on at least two algorithm models among the aforementioned algorithms.
  • the predictive model may be trained to classify a fracture suspected region and a similar fracture region such as a bone junction line, a blood vessel, and a bone overlap line within a medical image.
  • the prediction model may consist of a region segmentation model for segmenting a region of interest (ROI), which is a region suspected of fracture in a medical image, and a classifier for classifying a fracture region or a similar fracture region with respect to the segmented ROI.
  • ROI region of interest
  • FIGS. 1A to 1C a fracture detection system based on a device for detecting a fracture according to an embodiment of the present invention will be described with reference to FIGS. 1A to 1C .
  • 1A exemplarily shows a fracture detection system based on a device for fracture detection according to an embodiment of the present invention.
  • 1B exemplarily shows the configuration of a device for detecting a fracture according to an embodiment of the present invention.
  • 1C exemplarily illustrates the configuration of a medical staff device configured to receive a fracture-suspected region from the device for detecting a fracture according to an embodiment of the present invention.
  • the fracture detection system 1000 may be a system configured to provide information related to a fracture based on a medical bone image of an individual.
  • the fracture detection system 1000 includes a device for fracture detection configured to predict a fracture and determine a fracture suspected region based on a medical bone image, a medical staff device 200 for receiving information on fracture detection, and
  • the device 300 for providing a medical image may be configured to provide an image of a suspected fracture site.
  • the device for detecting a fracture 100 is a general-purpose computer, laptop, and/or data server that performs various calculations to diagnose a fracture site based on a medical bone image of an individual provided from the device for providing a medical image 300 , etc. may include
  • the medical staff device 200 may be a device for accessing a web server providing a web page or a mobile web server providing a mobile web site, but is not limited thereto.
  • the device 100 for detecting a fracture may receive a medical bone image from the device 300 for providing a medical image, and may provide information related to a fracture site from the received medical image of the bone.
  • the device 300 for providing a medical image may predict a fracture suspected region in the medical bone image by using the prediction model.
  • the device 100 for detecting a fracture may provide data associated with a fracture site for the subject to the medical staff device 200 .
  • the data provided from the device 100 for detecting a fracture may be provided as a web page through a web browser installed in the medical staff device 200 , or may be provided in the form of an application or a program. In various embodiments, such data may be provided in a form included in the platform in a client-server environment.
  • the medical staff device 200 is an electronic device that requests the provision of information on the fracture site for the subject and provides a user interface for displaying the fracture site prediction result data, a smartphone, a tablet PC (Personal Computer), and a notebook computer. and/or may include at least one of a PC and the like.
  • the medical staff device 200 may receive a detection result regarding a fracture of an object from the device 100 for detecting a fracture, and display the received result through a display unit.
  • the device 300 for providing a medical image may be an X-ray imaging device, an ultrasound device, a magnetic resonance imaging device, or a CT imaging device, but is not limited thereto.
  • the device 300 for providing medical images may be a separate server that provides various medical image databases.
  • the device 100 for detecting a fracture includes a storage unit 110 , a communication unit 120 , and a processor 130 .
  • the storage unit 110 may store various data generated while diagnosing a fracture site for an individual.
  • the storage unit 110 may be configured to store a medical bone image received from the device 300 for providing a medical image through the communication unit 120 to be described later, and furthermore, various products in the classification process of the predictive model. have.
  • the storage unit 110 is a flash memory type, hard disk type, multimedia card micro type, card type memory (eg, SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory.
  • a magnetic disk, and an optical disk may include at least one type of storage medium.
  • the communication unit 120 connects the fracture detection device 100 to enable communication with an external device.
  • the communication unit 120 may be connected to the medical staff device 200 and furthermore the medical image providing device 300 using wired/wireless communication to transmit/receive various data.
  • the communication unit 120 may receive a medical image of the bone of an object from the device 300 for providing a medical image.
  • the communication unit 120 may receive an X-ray image from the device 300 for providing a medical image.
  • the communication unit 120 may transmit the detection result to the medical staff device 200 .
  • the processor 130 is operatively connected to the storage 110 and the communication unit 120 , and may perform various commands for analyzing a medical bone image of an object.
  • the processor 130 may be configured to detect a fracture suspected site based on the medical bone image received through the communication unit 120 , and finally determine the fracture site by removing the fracture-like region.
  • the processor 130 may be based on a prediction model configured to predict a fracture suspected region based on the medical bone image.
  • the device 100 for detecting a fracture is not limited to a hardware design.
  • the processor 130 of the device 100 for fracture detection may be implemented in software. Accordingly, the detection result of the fracture may be displayed through the display unit of the medical image providing device 300 to which the software is connected.
  • the medical staff device 200 includes a communication unit 210 , a display unit 220 , a storage unit 230 , and a processor 240 .
  • the communication unit 210 may be configured to enable the medical staff device 200 to communicate with an external device.
  • the communication unit 210 may be connected to the device 100 for detecting a fracture using wired/wireless communication to transmit various data related to diagnosis of a fracture.
  • the communication unit 210 may receive, from the device 100 for detecting a fracture, a suspected fracture site predicted by the prediction models, and furthermore, a treatment prognosis.
  • the display unit 220 may display various interface screens for displaying the detection result of the fracture of the object.
  • the display unit 220 may include a touch screen, for example, a touch, gesture, proximity, drag, swipe using an electronic pen or a part of the user's body. A swipe or hovering input may be received.
  • the storage 230 may store various data used to provide a user interface for displaying result data.
  • the storage unit 230 may include a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD or XD). memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.
  • the processor 240 is operatively connected to the communication unit 210 , the display unit 220 , and the storage unit 230 , and may perform various commands for providing a user interface for displaying detection result data.
  • FIGS. 2A, 2B, and 2C a method for detecting a fracture according to an embodiment of the present invention will be described in detail with reference to FIGS. 2A, 2B, and 2C.
  • 2A to 2C exemplarily show a procedure of a method for detecting a fracture according to an embodiment of the present invention.
  • a procedure for detecting a fracture is as follows. First, a medical bone image of an object is received (S210), and a fracture suspected region for the medical image is determined by the prediction model (S220). Next, a similar fracture region is determined among the fracture suspect regions (S230), and finally, the suspected fracture region is filtered to determine a fracture site (S240).
  • a plurality of medical images photographed from a plurality of angles with respect to a target site of an object are received.
  • one of a medical bone image of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image is received.
  • X-ray images taken from a plurality of angles may be received, but the present invention is not limited thereto.
  • the skull mandible, hyoid, cervical, thoracic, lumbar, rib, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunar , triangle, cephalothorax, scapula, papillary, papillary, metacarpal, finger, tibia, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid, metatarsal and A medical image of a region suspected of fracture of at least one of the toe bones is received.
  • a fracture suspected region for the medical bone image is determined by the predictive model (S220).
  • the fracture suspected region is determined by a plurality of prediction models configured to predict a fracture site for each of a plurality of medical images.
  • a plurality of models trained to predict the suspected fracture region for each angle of the medical image may be used to predict the suspected fracture region for each of the plurality of images.
  • the fracture suspected region is determined by a single prediction model configured to predict the fracture site with respect to a plurality of medical images.
  • a single model trained to predict the suspected fracture region with respect to the multi-view image taken from multiple angles predicts the suspected fracture region with respect to a plurality of images from multiple angles. can be used for
  • the received front medical image 412 , the rear medical image 414 , the right side medical image 416 , and the left side medical image Each of a plurality of medical images 410 taken from a multi-angle of 418 is a front prediction model 422 , a rear prediction model 424 , a right prediction model 426 , and a plurality of prediction models of the left prediction model 428 . (420) is input. As a result, fracture suspicious regions 432 , 434 , 436a , 436b and 438 within the plurality of medical images 410 are determined.
  • the similar determination region is determined, not the region for the actual fracture site, and the similar determination region is removed.
  • the fracture suspected region is filtered to finally determine the fracture site (S240).
  • the similar fracture region is determined based on at least one of the thickness, shape, and location of the suspected fracture region.
  • the fracture-suspected regions determined by the plurality of prediction models 420 include regions 432 and 436b for actual fractures, and similar fracture regions 434, 436a and 438) may be included.
  • the regions 432 and 436b for fractures (refer to (a) of FIG. 2C ), in particular, for linear fractures, have a constant line thickness, and the size and location are not determined.
  • the pseudo-fracture areas 434, 436a and 438, and the middle blood vessels see (b) of FIG. 2c
  • the bone junction line (refer to (c) of FIG. 2C) has an irregular line shape, exists at a certain location, and may have a thicker line than that of a blood vessel or fracture site.
  • the similar fracture region is determined. Regions 434, 436a and 438 are determined, and in step S240 where the fracture site is determined, the similar fracture regions 434, 436a and 438 are removed. As a result, in the front medical image 412 and the right medical image 416 , regions 432 and 436b for fractures may be determined as fracture sites.
  • the false fracture region 434 , 436a , and 438 which is an erroneous detection region, may be removed to increase the sensitivity of fracture detection.
  • the step of determining the similar fracture region ( S230 ) is to classify a fracture site, such as a fracture region, or a similar fracture region, such as a blood vessel or bone junction region, by inputting a fracture suspected region, that is, a region of interest (ROI) as an input.
  • the configured classifier may be operated and performed.
  • the fracture site of a set image selected from among a plurality of medical images is compared, and the step of finally determining the fracture site based on the comparison result is further performed.
  • the set image may be a front medical image and a rear medical image, or a right side medical image and a left side medical image.
  • the position of the fracture area determined with respect to the front medical image 412 and the rear medical image 414 is compared, and the right side medical image 416 .
  • the location of the region for the fracture determined with respect to the left side medical image 418 may be compared.
  • the false detection that is, the similar fracture region is removed once again, and the fracture regions 432 and 436b for the front medical image 412 and the right side medical image 416 may be determined as final fracture sites.
  • a fracture diagnosis medical image 440 in which regions 432 and 436b for fracture are displayed may be provided.
  • the fracture regions 432 and 436b may be regions of different angles with respect to the same fracture site, but is not limited thereto.
  • the fracture detection system based on the fracture detection method according to the above various embodiments can overcome the limitations of the conventional fracture detection system.
  • the system can overcome the limitations of the conventional fracture detection system in that it can provide reliable diagnostic results for areas suspected of fractures having difficulty in visual identification of fractures only with X-ray images, such as the skull and ribs.
  • the system can reduce the time required for detection and provide accurate fracture diagnosis results regardless of the skill level of medical staff.
  • FIGS. 3 and 4 exemplarily show the structure of a prediction model used in various embodiments of the present invention.
  • the bone medical image is limited to X-ray images taken from the front, side, and back of the subject's face, and the target site is limited to a fracture of the skull.
  • the predictive model is not limited thereto and may be applied to diagnosing a fracture within a medical image of various regions.
  • the predictive model may be a RetinaNet-based predictive model composed of a plurality of artificial neural networks trained to predict a suspected fracture region by inputting each of a plurality of bone medical images 510 as an input.
  • the RetinaNet-based prediction model may be a plurality of models trained to predict a fracture suspected region for each of a plurality of medical images taken from multiple angles.
  • the structure of one model among a plurality of models will be described for convenience of description.
  • the plurality of artificial neural networks may include a first artificial neural network 520 , a second artificial neural network 530 , and a third artificial neural network 540 .
  • the first artificial neural network 520 may be a Residual Network (ResNet) that generates intermediate feature data (intermediate feature map) for each layer by inputting each of the plurality of bone medical images 510 as an input,
  • ResNet Residual Network
  • intermediate feature map intermediate feature map
  • the second artificial neural network 530 is a feature pyramid network that generates feature data corresponding to each layer by merging the intermediate feature data of a specific layer and the intermediate feature data of the next layer in the order of the lower layer from the upper layer.
  • Pyramid Network, FPN Pyramid Network
  • the layer corresponding to the feature data may mean each layer of Resnet.
  • the third artificial neural network 540 may include a bounding box regression subnet that adjusts a bounding box to include a fracture suspected region.
  • the third artificial neural network 540 may include a classification subnet for predicting (or classifying) the type of the fracture suspected region included in the bounding box. These subnets can be configured in parallel.
  • the operation of adjusting the bounding box may be performed using an offset between a ground-truth bounding box (ie, correct answer) indicating a fracture suspected region in an actual bone medical image and a predicted bounding box.
  • the offset may mean a degree to which the predicted bounding box and the ground-truth bounding box corresponding to the correct answer overlap (or match) each other.
  • the prediction result data 550 output through the prediction model may include a bounding box corresponding to each of the fracture suspicious regions 522 , 554 , 556a , 556b and 558 .
  • the prediction result data 550 may further include a prediction result (eg, a fracture, a blood vessel, a bone junction line, a bone overlap line, etc.) for the fracture-suspicious region.
  • a prediction result eg, a fracture, a blood vessel, a bone junction line, a bone overlap line, etc.
  • the classification result for the fracture suspected region may be output by a separate classifier configured to output the type of the region by inputting the suspected fracture region as an input.
  • an epoch may be 300 and a batch size may be 6, but is not limited thereto.
  • a single model for predicting the suspected fracture region may be used with respect to a multi-view image taken from multiple angles.
  • the prediction model is a plurality of artificial neural networks trained to predict a region suspected of fracture by inputting a multi-view medical image 610 in which a front medical image, a rear medical image, a right side medical image, and a left side medical image are combined. It may be a single prediction model 620 based on a multi-view convolutional neural network (MVCNN) consisting of .
  • MVCNN multi-view convolutional neural network
  • the multi-view medical images 612 , 614 , 616 and 618 in the multi-view medical image 610 are input by a plurality of first CNNs 622 .
  • a feature map is output for each of the various medical images.
  • a plurality of feature maps for each of the multiple medical images are aggregated into one feature map by a view pooling layer 624 .
  • the suspected fracture region 632 in the multi-view medical image 610 is determined by the single second CNN 626, including fully-connected layers. That is, the prediction result data 630 in which the fracture suspected region 632 is determined may be provided.
  • the medical image input to the MVCNN-based single prediction model 620 is not limited thereto, and may be a 3D image of a suspected fracture site.
  • the single prediction model 620 may be a Multi-Planar CNN (MPCNN) based model.
  • the fracture site prediction evaluation of the skull for the prediction model based on the ResNet-152 algorithm was performed, but the type of model and the target site are not limited thereto.
  • the prediction model trained with 810 fracture X-ray images and 829 normal X-ray images was evaluated using 183 X-ray images and 206 normal X-ray images.
  • the prediction model trained with 280 fracture X-ray images and 280 normal X-ray images was evaluated using 119 X-ray images and 118 normal X-ray images.
  • 5A to 5E exemplarily show the structure of a prediction model used in various embodiments of the present invention.
  • the prediction model used in various embodiments of the present invention is a rear X-ray image, a left side X-ray image, and a front X-ray image. It appears that the fracture suspected area (red box) is predicted to include the correct area (green box) labeled in advance for the fracture site in the image.
  • a CAM image indicating a degree of interest is shown in the prediction of the fracture suspected region of the predictive model.
  • the prediction model appears to have a high degree of interest in the fracture site where the actual fracture appears in prediction of the fracture suspected region.
  • FIG. 5C a performance evaluation result of the predictive model is shown.
  • the recall value which is an indicator of how well the predictive model detected the suspected fracture region, is 0.77, indicating how accurate the predicted fracture region is (that is, it corresponds to the actual fracture site).
  • the index precision is 0.73. Referring to FIG. 5D together, the average precision is shown as 0.7131.
  • the IOU (see FIG. 5e ) defined as the overlapping area for the correct answer area and the entire area of the prediction area was set to 0.1 or more, and the probability threshold was set to 0.1 or more.
  • the predictive model used in various embodiments of the present invention appears to find the fracture site with high accuracy in the skull X-ray image taken from a plurality of angles.
  • the present invention provides a fracture detection system based on a predictive model, and by providing information on parts not identified with the naked eye, it is possible to provide a high-sensitivity diagnosis result for a fracture without additional medical imaging of bone by a medical staff. have.
  • the present invention provides a fracture detection system using a predictive model based on a deep learning algorithm, thereby preventing erroneous interpretation by medical personnel and improving the workflow of medical personnel in actual clinical practice.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Optics & Photonics (AREA)
  • Dentistry (AREA)
  • Rheumatology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The present invention relates to a fracture detecting method implemented by a processor and provides a fracture detecting method and a device for detecting a fracture by using same, the method comprising the steps of: receiving a bone medical image for a subject; determining a fracture-suspected region in the bone medical image by using a prediction model that is configured to input a bone medical image and output a fracture region; determining an equivalent fracture region in the fracture-suspected region; and determining a fracture region by filtering off the equivalent fracture region in the fracture-suspected region.

Description

골절 검출 방법 및 이를 이용한 디바이스Fracture detection method and device using same
본 발명은, 골절 검출 방법 및 이를 이용한 디바이스에 관한 것이다.The present invention relates to a method for detecting a fracture and a device using the same.
골절이란, 뼈나 골단판 또는 관절면의 연속성이 완전 혹은 불완전하게 소실된 상태를 말하며, 이는 대개의 경우 외부의 힘에 의하여 발생한다. 이때, 골절은 뼈의 주변에 있는 연부 조직이나 장기들의 손상도 흔히 동반된다. 골절이 발생하는 위치에 따라 크게 골절은 사지골절, 척추골절 그리고 늑골, 두개골, 안와 등과 같은 기타 골절로 나눌 수 있다. A fracture refers to a state in which the continuity of a bone, epiphyseal plate, or articular surface is completely or incompletely lost, and is usually caused by external forces. At this time, fractures are often accompanied by damage to soft tissues or organs in the vicinity of the bone. Depending on the location of the fracture, fractures can be broadly divided into limb fractures, vertebral fractures, and other fractures such as ribs, skulls, orbits.
한편, 골절의 진단은, X-레이 촬영을 통해 진행될 수 있다. 그러나, X-레이 영상에 기초한 골절의 진단은, 그 결과가 애매하거나 골절 양상에 대한 정확한 확인이 어려워, 컴퓨터 단층촬영, 자기공명영상 촬영과 같은 추가적인 특수 검사가 요구될 수 도 있다.Meanwhile, diagnosis of a fracture may be performed through X-ray imaging. However, diagnosis of fractures based on X-ray images may require additional special examinations such as computed tomography and magnetic resonance imaging because the results are ambiguous or it is difficult to accurately confirm the fracture pattern.
나아가, 골 의료 영상에 기초한 골절 진단의 경우, 의료진들이 직접 육안으로 확인하여 골절의 유무를 판별함에 따라, 의료진들의 실력 또는 경험에 따라 의견 차이가 발생할 수 있어 신뢰도 높은 진단 결과를 제공하는 것에 한계가 있을 수 있다. Furthermore, in the case of fracture diagnosis based on bone medical images, there is a limit to providing reliable diagnostic results because medical personnel directly check with the naked eye to determine the presence of a fracture, and differences of opinion may occur depending on the skill or experience of medical personnel. there may be
한편, 골절 진단의 실패는, 환자의 경과 악화 및 의료 비용의 증가를 야기할 수 있어, 새로운 골절 검출 시스템에 대한 개발이 지속적으로 요구되고 있는 실정이다. On the other hand, failure of fracture diagnosis may cause deterioration of the patient's progress and an increase in medical costs, so the development of a new fracture detection system is continuously required.
발명의 배경이 되는 기술은 본 발명에 대한 이해를 보다 용이하게 하기 위해 작성되었다. 발명의 배경이 되는 기술에 기재된 사항들이 선행기술로 존재한다고 인정하는 것으로 이해되어서는 안 된다.The description underlying the invention has been prepared to facilitate understanding of the invention. It should not be construed as an admission that the matters described in the background technology of the invention exist as prior art.
전술한 한계를 극복하기 위한 방안으로, 의료진을 보조하는 소프트웨어 기반의 컴퓨터 보조 진단 (computer aided diagnosis, CAD) 시스템이 제안되었다. As a way to overcome the above limitations, a software-based computer aided diagnosis (CAD) system that assists medical staff has been proposed.
그러나, 제안된 골절 검출 시스템은, 여전히 골절의 검출, 특히 골절과 유사한 형태의 혈관, 두개골의 접합선이 존재하는 두개골의 골절의 검출에 대하여 위양성 (false positive) 이 높아 실제 임상에 적용되기 어렵다는 한계가 있을 수 있다.However, the proposed fracture detection system still has a limitation in that it is difficult to apply to actual clinical practice due to high false positives for the detection of fractures, especially for the detection of fractures of the skull in which blood vessels and skull junctions exist in a form similar to a fracture. there may be
한편, 본 발명의 발명자들은, 전술한 과제를 해결하기 위한 방안으로, 골절 골 의료 영상을 학습하여 자동으로 예측하도록 구성된 딥 러닝 기반의 예측 모델을 적용하여 위양성의 발생을 최소화하는 새로운 골절 검출 시스템을 개발하고자 하였다.On the other hand, the inventors of the present invention, as a way to solve the above problem, a new fracture detection system that minimizes the occurrence of false positives by applying a deep learning-based prediction model configured to automatically predict by learning a fracture bone medical image. wanted to develop.
보다 구체적으로, 본 발명의 발명자들은 새로운 골절 검출 시스템에 대하여, 예측 모델을 이용해 골절 의심 영역을 검출하고, 골절 의심 영역 중 골절과 유사한 형태의 유사 골절 영역을 제거하여 최종적으로 골절을 분류하도록 설계할 수 있었다.More specifically, the inventors of the present invention designed a new fracture detection system to detect a fracture suspected region using a predictive model and to finally classify a fracture by removing a similar fracture region in a shape similar to a fracture among the fracture suspected regions. could
이때, 본 발명의 발명자들은, 선의 굵기가 일정하고, 크기 및 위치가 정해져 있지 않은 골절과 상이한 특징을 갖는 혈관, 및 골 접합선 등을 유사 골절 영역으로 설정할 수 있었다.In this case, the inventors of the present invention were able to set the blood vessels and the bone junction line having characteristics different from those of a fracture in which the thickness of the line is constant and the size and location are not determined as the similar fracture region.
보다 구체적으로, 본 발명의 발명자들은, 혈관의 경우 선 굵기가 불규칙하고 안구, 관자 놀이 부근에 위치하며, 골 접한선의 경우 선 모양이 불규칙하고 선 굵기가 골절 선보다 굵고 개체마다 일정한 위치에 존재하는 특징을 인지할 수 있었다.More specifically, the inventors of the present invention found that, in the case of blood vessels, the line thickness is irregular and is located near the eyeball and temple, and in the case of the bone tangent line, the line shape is irregular, the line thickness is thicker than the fracture line, and it exists at a constant location for each individual. could recognize
그 결과, 본 발명의 발명자들은, 골절 검출 시스템에 대하여 예측 모델의 분류 결과에 대하여, 혈관 또는 골 접합선과 같은 유사 골절 영역이 제거되도록 필터링하여 최종적으로 골절이 결정되도록 설계할 수 있었다. As a result, the inventors of the present invention were able to design the final fracture determination by filtering the classification result of the predictive model for the fracture detection system so that similar fracture regions such as blood vessels or bone junctions are removed.
이에, 예측 모델에 의해 골절로 의심되는 영역을 검출하고 유사 골절 영역을 제거 하도록 설계된 골절 검출 시스템의 구축이 가능하였고, 본 발명의 발명자들은 상기 골절 검출 시스템을 제공함으로써 골절 검출의 민감도를 높일 수 있음을 인지할 수 있었다.Accordingly, it was possible to construct a fracture detection system designed to detect a region suspected of a fracture by the predictive model and remove a similar fracture region, and the inventors of the present invention can increase the sensitivity of fracture detection by providing the fracture detection system. could recognize
특히, 본 발명의 발명자들은, 예측 모델에 기초한 골절 검출 시스템을 제공함으로써, 육안으로 식별되지 않은 부분들에 대한 정보 제공에 의해 의료진이 추가적인 골 의료 영상 진단을 수행하지 않아도 골절 진단의 민감도가 높아질 것을 기대할 수 있었다. In particular, the inventors of the present invention believe that by providing a fracture detection system based on a predictive model, the sensitivity of fracture diagnosis will be increased even if the medical staff does not perform additional bone medical imaging by providing information on parts not identified with the naked eye. could be expected
또한, 본 발명의 발명자들은, 예측 모델에 대하여 복수의 방향에서 촬영된 골 의료 영상을 모두 고려하여 골절을 분류하도록 학습 가능한 멀티뷰 (multi view) 알고리즘을 적용하고자 하였다.In addition, the inventors of the present invention tried to apply a learnable multi view algorithm to classify fractures by considering all of the bone medical images taken from a plurality of directions with respect to the predictive model.
보다 구체적으로, 본 발명의 발명자들은 복수의 각도에서 촬영된 골 의료 영상이 정합된 학습용 골 의료 영상을 예측 모델의 학습에 적용할 수 있었다. More specifically, the inventors of the present invention were able to apply the training bone medical image to which the medical bone image taken from a plurality of angles is matched to the learning of the predictive model.
즉, 본 발명의 발명자들은 이와 같은 딥 러닝 기반의 골절 검출 시스템을 제공함으로써, 숙련도에 관계 없이 골절을 높은 정확도로 예측할 수 있음을 기대할 수 있었다.That is, the inventors of the present invention could expect that by providing such a deep learning-based fracture detection system, fractures can be predicted with high accuracy regardless of skill level.
이에, 본 발명이 해결하고자 하는 과제는, 딥 러닝 기반 예측 모델을 이용하여 골 의료 영상에 대하여 골절을 감지하고, 유사 골절 영역을 결정하여 제거한 후 최종적으로 골절 부위를 제공하도록 구성된, 골절 검출 방법 및 이를 이용한 디바이스를 제공하는 것이다.Accordingly, the problem to be solved by the present invention is a fracture detection method, configured to detect a fracture in a medical bone image using a deep learning-based prediction model, determine and remove a similar fracture area, and finally provide a fracture site, and To provide a device using this.
본 발명의 과제들은 이상에서 언급한 과제들로 제한되지 않으며, 언급되지 않은 또 다른 과제들은 아래의 기재로부터 당업자에게 명확하게 이해될 수 있을 것이다.The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.
전술한 바와 같은 과제를 해결하기 위하여 본 발명의 일 실시예에 따른 골절 검출 방법이 제공된다. 본 발명의 일 실시예에 따른 골절 검출 방법은, 프로세서에 의해 구현되는 골절 검출 방법으로서, 개체에 대한 골 의료 영상을 수신하는 단계, 골 의료 영상을 입력으로 하여 골절 부위를 출력하도록 구성된 예측 모델을 이용하여, 골 의료 영상에 대한 골절 의심 영역을 결정하는 단계, 골절 의심 영역 중 유사 골절 영역을 결정하는 단계, 및 유사 골절 영역이 제거되도록 골절 의심 영역을 필터링하여 골절 부위를 결정하는 단계를 포함한다.In order to solve the problems as described above, there is provided a fracture detection method according to an embodiment of the present invention. A fracture detection method according to an embodiment of the present invention is a fracture detection method implemented by a processor, comprising: receiving a medical bone image of an object; determining a fracture suspected region for a medical bone image using the .
본 발명의 특징에 따르면, 골 의료 영상은, 개체의 목적 부위에 대하여 복수의 각도에서 촬영한 복수의 골 의료 영상을 포함할 수 있다. 이때, 골절 의심 영역을 결정하는 단계는, 예측 모델을 이용하여 복수의 골 의료 영상 각각에 대한 골절 의심 영역을 결정하는 단계를 포함하고, 유사 골절 영역을 결정하는 단계는, 복수의 골 의료 영상 각각에 대한 골절 의심 영역 중, 유사 골절 영역을 결정하는 단계를 포함할 수 있다. 골절 부위를 결정하는 단계는, 복수의 골 의료 영상 각각에 대하여 골절 영역을 결정하는 단계를 포함할 수 있다.According to a feature of the present invention, the medical bone image may include a plurality of medical bone images taken from a plurality of angles with respect to a target site of an object. In this case, the determining of the fracture suspected region includes determining the fracture suspect region for each of a plurality of medical bone images by using a prediction model, and the determining of the similar fracture region includes each of the plurality of medical medical images. It may include the step of determining a similar fracture area among the fracture suspected areas for the. Determining the fracture site may include determining a fracture area for each of a plurality of bone medical images.
본 발명의 다른 특징에 따르면, 예측 모델은, 복수의 골 의료 영상 각각에 대하여 골절 부위를 예측하도록 구성된 복수의 예측 모델일 수 있다. 이때, 골절 의심 영역을 결정하는 단계는, 복수의 예측 모델 각각을 이용하여, 복수의 골 의료 영상 각각에 대한 골절 의심 영역을 결정하는 단계를 포함할 수 있다.According to another feature of the present invention, the predictive model may be a plurality of predictive models configured to predict a fracture site for each of a plurality of bone medical images. In this case, the determining of the fracture suspected region may include determining the fracture suspected region for each of the plurality of bone medical images by using each of the plurality of prediction models.
본 발명의 또 다른 특징에 따르면, 예측 모델은, 복수의 골 의료 영상에 대하여 골절 부위를 예측하도록 구성된 단일의 예측 모델일 수 있다. 이때, 골절 의심 영역을 결정하는 단계는, 단일의 예측 모델을 이용하여, 복수의 골 의료 영상에 대한 골절 의심 영역을 결정하는 단계를 포함할 수 있다.According to another feature of the present invention, the predictive model may be a single predictive model configured to predict a fracture site with respect to a plurality of bone medical images. In this case, the determining of the fracture suspected region may include determining the fracture suspected region for a plurality of bone medical images by using a single prediction model.
본 발명의 또 다른 특징에 따르면, 복수의 골 의료 영상은, 정면 골 의료 영상, 후면 골 의료 영상, 우측 골 의료 영상 및 좌측 골 의료 영상 중 적어도 두 개일 수 있다.According to another feature of the present invention, the plurality of medical images of bones may be at least two of a medical image of a frontal bone, a medical image of a posterior bone, a medical image of a right bone, and a medical image of a left bone.
본 발명의 또 다른 특징에 따르면, 상기 방법은, 골절 부위를 결정하는 단계 이후에, 복수의 골 의료 영상 중 선택된 세트 영상의 골절 부위를 비교하는 단계, 및 비교 결과에 기초하여 골절 부위를 최종 결정하는 단계를 더 포함할 수 있다. 이때, 세트 영상은, 정면 골 의료 영상 및 후면 골 의료 영상, 또는 우측 골 의료 영상 및 좌측 골 의료 영상일 수 있다.According to another feature of the present invention, the method includes, after determining the fracture site, comparing the fracture site of a set image selected from among a plurality of bone medical images, and finally determining the fracture site based on the comparison result It may further include the step of In this case, the set image may be a medical image of a frontal bone and a medical image of a posterior bone, or a medical image of a right bone and a medical image of a left bone.
본 발명의 또 다른 특징에 따르면, 골절 부위를 비교하는 단계는, 세트 영상에 대하여 결정된 골절 부위의 위치를 비교하는 단계를 포함할 수 있다. According to another feature of the present invention, the comparing the fracture site may include comparing the determined location of the fracture site with respect to the set image.
본 발명의 또 다른 특징에 따르면, 골절은 선형 골절이고, 유사 골절 영역을 결정하는 단계는, 골절 의심 영역의 두께, 형태 및 위치 중 적어도 하나에 기초하여, 유사 골절 영역을 결정하는 단계를 포함할 수 있다.According to another feature of the present invention, the fracture is a linear fracture, and the determining of the similar fracture region may include determining the similar fracture region based on at least one of the thickness, shape, and location of the suspected fracture region. can
본 발명의 또 다른 특징에 따르면, 유사 골절 영역은, 골 접합선, 혈관 및 골 중첩 중 적어도 하나의 영역이다.According to another feature of the present invention, the pseudo-fracture region is a region of at least one of a bone junction, a blood vessel, and a bone overlap.
본 발명의 또 다른 특징에 따르면, 골 의료 영상은, X-레이 영상, 컴퓨터 단층 촬영 영상, 자기 공명 영상 및 초음파 영상 중 하나일 수 있다.According to another feature of the present invention, the medical bone image may be one of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image.
본 발명의 또 다른 특징에 따르면, 골절 의심 영역은, 두개골, 하악골, 설골, 경추, 흉추, 요추, 늑골, 흉골, 쇄골, 견갑골, 상완골, 요골, 척골, 주상골, 월상골, 삼각골, 두상골, 대능형골, 소농형골, 유두골, 유두골, 중수골, 손가락뼈, 관골, 대퇴골, 슬개골, 경골, 비골, 거골, 종골, 주상골, 입방골, 쐐기뼈, 증족골 및 발가락뼈 중 적어도 하나에 대한 골절 의심 영역일 수 있다.According to another feature of the present invention, the fracture suspected region is the skull, mandible, hyoid bone, cervical vertebrae, thoracic vertebrae, lumbar vertebrae, ribs, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunar, triangular, and cephalothorax. , at at least one of the crotch bone, scapula, papillary bone, papillary bone, metacarpal, finger bone, tibia, femur, patella, tibia, fibula, talus, calcaneus, scaphoid, cuboid, sphenoid, metatarsal and toe bones It may be a suspected fracture area.
전술한 바와 같은 과제를 해결하기 위하여 본 발명의 다른 실시예에 따른 디바이스가 제공된다. 상기 디바이스는, 개체에 대한 골 의료 영상을 수신하도록 구성된 통신부, 및 통신부와 통신하도록 연결된 프로세서를 포함하고, 프로세서는 골 의료 영상을 입력으로 하여 골절 부위를 출력하도록 구성된 예측 모델을 이용하여, 골 의료 영상에 대한 골절 의심 영역을 결정하고, 골절 의심 영역 중 유사 골절 영역을 결정하고, 유사 골절 영역이 제거되도록 골절 의심 영역을 필터링하여 골절 부위를 결정하도록 구성된다.In order to solve the above problems, a device according to another embodiment of the present invention is provided. The device includes a communication unit configured to receive a medical bone image of the subject, and a processor connected to the communication unit, wherein the processor uses a predictive model configured to output a fracture site by inputting the medical bone image as an input, It is configured to determine a fracture suspect region for the image, determine a similar fracture region among the suspected fracture regions, and filter the suspected fracture region to remove the similar fracture region to determine a fracture site.
본 발명의 특징에 따르면 골 의료 영상은, 개체의 목적 부위에 대하여 복수의 각도에서 촬영한 복수의 골 의료 영상을 포함할 수 있다. 이때, 프로세서는, 예측 모델을 이용하여 복수의 골 의료 영상 각각에 대한 골절 의심 영역을 결정하고, 복수의 골 의료 영상 각각에 대한 골절 의심 영역 중, 유사 골절 영역을 결정하고, 복수의 골 의료 영상 각각에 대하여 골절 영역을 결정하도록 더 구성될 수 있다.According to a feature of the present invention, the medical bone image may include a plurality of medical bone images taken from a plurality of angles with respect to a target site of an object. In this case, the processor determines a fracture suspected region for each of the plurality of medical bone images by using the prediction model, determines a similar fracture region from among the fracture suspect regions for each of the plurality of medical images of bone, and determines a plurality of medical images and may be further configured to determine a fracture area for each.
본 발명의 다른 특징에 따르면, 예측 모델은, 복수의 골 의료 영상 각각에 대하여 골절 부위를 예측하도록 구성된 복수의 예측 모델일 수 있다. 이때, 프로세서는, 복수의 예측 모델 각각을 이용하여, 복수의 골 의료 영상 각각에 대한 골절 의심 영역을 결정하도록 구성될 수 있다.According to another feature of the present invention, the predictive model may be a plurality of predictive models configured to predict a fracture site for each of a plurality of bone medical images. In this case, the processor may be configured to determine a fracture suspected region for each of a plurality of bone medical images by using each of a plurality of predictive models.
본 발명의 또 다른 특징에 따르면, 예측 모델은, 복수의 골 의료 영상에 대하여 골절 부위를 예측하도록 구성된 단일의 예측 모델일 수 있다. 이때, 프로세서는, 단일의 예측 모델을 이용하여, 복수의 골 의료 영상에 대한 골절 의심 영역을 결정하도록 구성될 수 있다.According to another feature of the present invention, the predictive model may be a single predictive model configured to predict a fracture site with respect to a plurality of bone medical images. In this case, the processor may be configured to determine a fracture suspected region for a plurality of bone medical images by using a single prediction model.
본 발명의 또 다른 특징에 따르면, 복수의 골 의료 영상은, 정면 골 의료 영상, 후면 골 의료 영상, 우측 골 의료 영상 및 좌측 골 의료 영상 중 적어도 두 개일 수 있다.According to another feature of the present invention, the plurality of medical images of bones may be at least two of a medical image of a frontal bone, a medical image of a posterior bone, a medical image of a right bone, and a medical image of a left bone.
본 발명의 또 다른 특징에 따르면, 프로세서는, 복수의 골 의료 영상 중 선택된 세트 영상의 골절 부위를 비교하고, 비교 결과에 기초하여 골절 부위를 최종 결정하도록 더 구성되고, 세트 영상은, 정면 골 의료 영상 및 후면 골 의료 영상, 또는 우측 골 의료 영상 및 좌측 골 의료 영상일 수 있다.According to another feature of the present invention, the processor is further configured to compare a fracture site of a set image selected from among a plurality of bone medical images, and finally determine a fracture site based on the comparison result, wherein the set image is a frontal bone medical image It may be an image and a medical image of the posterior bone, or a medical image of a right bone and a medical image of the left bone.
본 발명의 또 다른 특징에 따르면, 프로세서는, 세트 영상에 대하여 결정된 골절 부위의 위치를 비교하도록 더 구성될 수 있다. According to another feature of the present invention, the processor may be further configured to compare the determined location of the fracture site with respect to the set image.
기타 실시예의 구체적인 사항들은 상세한 설명 및 도면들에 포함되어 있다.Details of other embodiments are included in the detailed description and drawings.
본 발명은, 딥 러닝 알고리즘에 기초하여 골 의료 영상에 대하여 골절 부위를 예측하고 검출하도록 구성된, 골절 검출 시스템을 제공할 수 있다.The present invention may provide a fracture detection system configured to predict and detect a fracture site with respect to a bone medical image based on a deep learning algorithm.
보다 구체적으로, 본 발명은, 딥 러닝 기반의 골절 검출 시스템을 제공함으로써, 골절을 높은 정확도로 예측할 수 있는 효과가 있다.More specifically, the present invention has the effect of predicting a fracture with high accuracy by providing a deep learning-based fracture detection system.
특히, 본 발명은 예측 모델을 이용해 골절 의심 영역을 검출하고, 골절 의심 영역 중 골절과 유사한 형태의 유사 골절 영역을 제거하여 최종적으로 골절을 분류하도록 설계된 골절 검출 시스템을 제공함으로써, 두개골 골절과 같이 골절의 정확한 진단이 어려운 부위에 대한 신뢰도 높은 진단 결과를 제공할 수 있다. In particular, the present invention provides a fracture detection system designed to detect a fracture suspected region using a predictive model, and to finally classify a fracture by removing a similar fracture region in a shape similar to a fracture among the fracture suspected region, thereby providing a fracture detection system such as a skull fracture. It can provide reliable diagnostic results for areas where accurate diagnosis is difficult.
즉, 본 발명은 예측 모델 기반의 골절 검출 시스템을 제공함으로써, 육안으로 식별되지 않은 부분들에 대한 정보 제공에 의해 의료진의 추가적인 골 의료 영상 진단의 수행 없이 골절에 대한 민감도 높은 진단 결과를 제공할 수 있다. That is, the present invention provides a fracture detection system based on a predictive model, thereby providing information on parts not identified with the naked eye, thereby providing a high-sensitivity diagnosis result for a fracture without additional medical imaging of bone by a medical staff. have.
또한, 본 발명은 딥 러닝 알고리즘 기반의 예측 모델을 이용한 골절 검출 시스템을 제공함으로써, 의료진의 잘못된 해석을 방지하고, 실제 임상 실무에 있어서 의료진의 워크 플로우를 향상시킬 수 있다. In addition, the present invention provides a fracture detection system using a predictive model based on a deep learning algorithm, thereby preventing erroneous interpretation by medical personnel and improving the workflow of medical personnel in actual clinical practice.
본 발명에 따른 효과는 이상에서 예시된 내용에 의해 제한되지 않으며, 더욱 다양한 효과들이 본 명세서 내에 포함되어 있다.The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.
도 1a는 본 발명의 일 실시예에 따른 골절 검출용 디바이스에 기초한 골절 검출 시스템을 예시적으로 도시한 것이다. 1A exemplarily shows a fracture detection system based on a device for fracture detection according to an embodiment of the present invention.
도 1b는 본 발명의 일 실시예에 따른 골절 검출용 디바이스의 구성을 예시적으로 도시한 것이다. 1B exemplarily shows the configuration of a device for detecting a fracture according to an embodiment of the present invention.
도 1c는 본 발명의 일 실시예에 따른 골절 검출용 디바이스로부터 골절 의심 부위를 수신하도록 구성된 의료진 디바이스의 구성을 예시적으로 도시한 것이다. 1C exemplarily illustrates the configuration of a medical staff device configured to receive a fracture-suspected region from the device for detecting a fracture according to an embodiment of the present invention.
도 2a 내지 2c는 본 발명의 일 실시예에 따른 골절 검출 방법의 절차를 예시적으로 도시한 것이다. 2A to 2C exemplarily show a procedure of a method for detecting a fracture according to an embodiment of the present invention.
도 3 및 4는 본 발명의 다양한 실시예에 이용되는 예측 모델의 구조를 예시적으로 도시한 것이다.3 and 4 exemplarily show the structure of a prediction model used in various embodiments of the present invention.
도 5a 내지 5e는 본 발명의 다양한 실시예에 이용되는 예측 모델의 구조를 예시적으로 도시한 것이다.5A to 5E exemplarily show the structure of a prediction model used in various embodiments of the present invention.
본 발명의 이점 및 특징, 그리고 그것들을 달성하는 방법은 첨부되는 도면과 함께 상세하게 후술되어 있는 실시예들을 참조하면 명확해질 것이다. 그러나, 본 발명은 이하에서 개시되는 실시예들에 한정되는 것이 아니라 서로 다른 다양한 형태로 구현될 것이며, 단지 본 실시예들은 본 발명의 개시가 완전하도록 하며, 본 발명이 속하는 기술분야에서 통상의 지식을 가진 자에게 발명의 범주를 완전하게 알려주기 위해 제공되는 것이며, 본 발명은 청구항의 범주에 의해 정의될 뿐이다. 도면의 설명과 관련하여, 유사한 구성요소에 대해서는 유사한 참조부호가 사용될 수 있다.Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like components.
본 문서에서, "가진다," "가질 수 있다," "포함한다," 또는 "포함할 수 있다" 등의 표현은 해당 특징(예: 수치, 기능, 동작, 또는 부품 등의 구성요소)의 존재를 가리키며, 추가적인 특징의 존재를 배제하지 않는다.In this document, expressions such as "have," "may have," "includes," or "may include" refer to the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.
본 문서에서, "A 또는 B," "A 또는/및 B 중 적어도 하나," 또는 "A 또는/및 B 중 하나 또는 그 이상" 등의 표현은 함께 나열된 항목들의 모든 가능한 조합을 포함할 수 있다. 예를 들면, "A 또는 B," "A 및 B 중 적어도 하나," 또는 "A 또는 B 중 적어도 하나"는, (1) 적어도 하나의 A를 포함, (2) 적어도 하나의 B를 포함, 또는(3) 적어도 하나의 A 및 적어도 하나의 B 모두를 포함하는 경우를 모두 지칭할 수 있다.In this document, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.
본 문서에서 사용된 "제1," "제2," "첫째," 또는 "둘째," 등의 표현들은 다양한 구성요소들을, 순서 및/또는 중요도에 상관없이 수식할 수 있고, 한 구성요소를 다른 구성요소와 구분하기 위해 사용될 뿐 해당 구성요소들을 한정하지 않는다. 예를 들면, 제1 사용자 기기와 제2 사용자 기기는, 순서 또는 중요도와 무관하게, 서로 다른 사용자 기기를 나타낼 수 있다. 예를 들면, 본 문서에 기재된 권리범위를 벗어나지 않으면서 제1 구성요소는 제2 구성요소로 명명될 수 있고, 유사하게 제2 구성요소도 제1 구성요소로 바꾸어 명명될 수 있다.As used herein, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of rights described in this document, the first component may be named as the second component, and similarly, the second component may also be renamed as the first component.
어떤 구성요소(예: 제1 구성요소)가 다른 구성요소(예: 제2 구성요소)에 "(기능적으로 또는 통신적으로) 연결되어((operatively or communicatively) coupled with/to)" 있다거나 "접속되어(connected to)" 있다고 언급된 때에는, 상기 어떤 구성요소가 상기 다른 구성요소에 직접적으로 연결되거나, 다른 구성요소(예: 제3 구성요소)를 통하여 연결될 수 있다고 이해되어야 할 것이다. 반면에, 어떤 구성요소(예: 제1 구성요소)가 다른 구성요소(예: 제2 구성요소)에 "직접 연결되어" 있다거나 "직접 접속되어" 있다고 언급된 때에는, 상기 어떤 구성요소와 상기 다른 구성요소 사이에 다른 구성요소(예: 제3 구성요소)가 존재하지 않는 것으로 이해될 수 있다.A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component); When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.
본 문서에서 사용된 표현 "~하도록 구성된(또는 설정된)(configured to)"은 상황에 따라, 예를 들면, "~에 적합한(suitable for)," "~하는 능력을 가지는(having the capacity to)," "~하도록 설계된(designed to)," "~하도록 변경된(adapted to)," "~하도록 만들어진(made to)," 또는 "~ 를 할 수 있는(capable of)"과 바꾸어 사용될 수 있다. 용어 "~하도록 구성된(또는 설정된)"은 하드웨어적으로 "특별히 설계된(specifically designed to)" 것만을 반드시 의미하지 않을 수 있다. 대신, 어떤 상황에서는, "~하도록 구성된 디바이스"라는 표현은, 그 디바이스가 다른 디바이스 또는 부품들과 함께 "~할 수 있는" 것을 의미할 수 있다. 예를 들면, 문구 "A, B, 및 C를 수행하도록 구성된(또는 설정된)프로세서"는 해당 동작을 수행하기 위한 전용 프로세서(예: 임베디드 프로세서), 또는 메모리 디바이스에 저장된 하나 이상의 소프트웨어 프로그램들을 실행함으로써, 해당 동작들을 수행할 수 있는 범용 프로세서(generic-purpose processor)(예: CPU 또는 application processor)를 의미할 수 있다.The expression "configured to (or configured to)" as used in this document, depending on the context, for example, "suitable for," "having the capacity to ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with another device or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the corresponding operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.
본 문서에서 사용된 용어들은 단지 특정한 실시 예를 설명하기 위해 사용된 것으로, 다른 실시예의 범위를 한정하려는 의도가 아닐 수 있다. 단수의 표현은 문맥상 명백하게 다르게 뜻하지 않는 한, 복수의 표현을 포함할 수 있다. 기술적이거나 과학적인 용어를 포함해서 여기서 사용되는 용어들은 본 문서에 기재된 기술분야에서 통상의 지식을 가진 자에 의해 일반적으로 이해되는 것과 동일한 의미를 가질 수 있다. 본 문서에 사용된 용어들 중 일반적인 사전에 정의된 용어들은, 관련 기술의 문맥상 가지는 의미와 동일 또는 유사한 의미로 해석될 수 있으며, 본 문서에서 명백하게 정의되지 않는 한, 이상적이거나 과도하게 형식적인 의미로 해석되지 않는다. 경우에 따라서, 본 문서에서 정의된 용어일지라도 본 문서의 실시 예들을 배제하도록 해석될 수 없다.Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, ideal or excessively formal meanings is not interpreted as In some cases, even terms defined in this document cannot be construed to exclude embodiments of this document.
본 발명의 여러 실시예들의 각각 특징들이 부분적으로 또는 전체적으로 서로 결합 또는 조합 가능하며, 당업자가 충분히 이해할 수 있듯이 기술적으로 다양한 연동 및 구동이 가능하며, 각 실시예들이 서로에 대하여 독립적으로 실시 가능할 수도 있고 연관 관계로 함께 실시 가능할 수도 있다. Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and as those skilled in the art will fully understand, technically various interlocking and driving are possible, and each embodiment may be implemented independently of each other, It may be possible to implement together in a related relationship.
본 명세서의 해석의 명확함을 위해, 이하에서는 본 명세서에서 사용되는 용어들을 정의하기로 한다. For clarity of interpretation of the present specification, terms used herein will be defined below.
본 명세서에서 사용되는 용어, "골절"은 뼈나 골단판 또는 관절면의 연속성이 완전 혹은 불완전하게 소실된 상태로서, 분쇄 골절, 분절성 골절, 선형 골절, 함몰 골절, 기저 골절 등을 아우를 수 있다. 바람직하게, 본원 명세서 내에서 골절은 선형 골절일 수 있고, 보다 바람직하게 두개골 선형 골절일 수 있으나, 이에 제한되는 것은 아니다. As used herein, the term "fracture" refers to a state in which the continuity of a bone, epiphyseal plate, or articular surface is completely or incompletely lost, and may include comminuted fractures, segmental fractures, linear fractures, depression fractures, and basal fractures. Preferably, the fracture within the present specification may be a linear fracture, more preferably a cranial linear fracture, but is not limited thereto.
예를 들어, 골절 의심 부위는, 하악골, 설골, 경추, 흉추, 요추, 늑골, 흉골, 쇄골, 견갑골, 상완골, 요골, 척골, 주상골, 월상골, 삼각골, 두상골, 대능형골, 소농형골, 유두골, 유두골, 중수골, 손가락뼈, 관골, 대퇴골, 슬개골, 경골, 비골, 거골, 종골, 주상골, 입방골, 쐐기뼈, 증족골 및 발가락뼈 중 적어도 하나의 부위일 수 있다.For example, the suspected fracture site is the mandible, hyoid, cervical, thoracic, lumbar, rib, sternum, clavicle, scapula, humerus, radius, ulna, scaphoid, lunate, triangular bone, cephalothorax, occipital bone, scapula , papilla, papilla, metacarpal, finger bone, zygomatic bone, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid bone, metatarsal bone, and toe bone.
본 명세서에서 사용되는 용어, "개체"는 골절을 검출 하고자 하는 모든 대상을 의미할 수 있다. 예를 들어, 개체는, 두개골 골절이 의심되는 개체일 수 있다. 그러나 이에 제한되지 않고, 하악골, 설골, 경추, 흉추, 요추, 늑골, 흉골, 쇄골, 견갑골, 상완골, 요골, 척골, 주상골, 월상골, 삼각골, 두상골, 대능형골, 소농형골, 유두골, 유두골, 중수골, 손가락뼈, 관골, 대퇴골, 슬개골, 경골, 비골, 거골, 종골, 주상골, 입방골, 쐐기뼈, 증족골 및 발가락뼈 중 적어도 하나의 부위에 대한 골절이 의심되는 개체일 수 있다. 한편, 본 명세서 내에 개시된 개체는, 인간을 제외한 모든 포유 동물일 수 있으나, 이에 제한되는 것은 아니다. As used herein, the term “subject” may refer to any target for which a fracture is to be detected. For example, the subject may be a subject suspected of having a skull fracture. However, without being limited thereto, the mandible, hyoid bone, cervical spine, thoracic spine, lumbar spine, rib, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunate, triangular bone, cephalothorax, crotch, scapula, papillary bone , papillary, metacarpal, finger bone, tibia, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid, metatarsal, and toe bones have. Meanwhile, the subject disclosed in the present specification may be any mammal other than humans, but is not limited thereto.
본 명세서에서 사용되는 용어, "골 의료 영상"은, 골절 의심 부위를 포함하는 골 의료 영상으로, X-레이 영상, 컴퓨터 단층 촬영 영상, 자기 공명 영상 및 초음파 영상 중 하나일 수 있다. 바람직하게, 본원 명세서에서 골 의료 영상은, X-레이 영상일 수 있으나, 이에 제한되는 것은 아니다. 한편, 본원 명세서에서 골 의료 영상은, 의료 영상과 상호 교환적으로 이용될 수 있다.As used herein, the term "medical bone image" is a medical image of a bone including a suspected fracture site, and may be one of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image. Preferably, in the present specification, the medical bone image may be an X-ray image, but is not limited thereto. Meanwhile, in the present specification, a medical image of a bone may be used interchangeably with a medical image.
본 발명의 특징에 따르면, 골 의료 영상은 복수의 각도에서 촬영한 복수의 의료 영상일 수 있다.According to a feature of the present invention, the medical bone image may be a plurality of medical images taken from a plurality of angles.
예를 들어, 복수의 의료 영상은, 정면 의료 영상, 후면 의료 영상, 우측면 의료 영상 및 좌측면 의료 영상 중 적어도 두 개의 의료 영상일 수 있다.For example, the plurality of medical images may be at least two medical images among a front medical image, a rear medical image, a right side medical image, and a left side medical image.
한편, 골 의료 영상은 복수의 프레임으로 이루어진 동영상일 수도 있다. 예를 들어, 복수의 프레임으로 구성된 골 의료 영상은 본 발명의 일 실시예에 따른 골절 검출 방법에 따라 동영상의 프레임 각각에 대하여 골절 예측이 가능할 수 있다. 즉, 카메라로부터 골 의료 영상의 수신과 동시에 골절 예측이 가능하여 골절에 대한 검출이 가능할 수 있다. On the other hand, the bone medical image may be a moving picture consisting of a plurality of frames. For example, in a medical bone image composed of a plurality of frames, fracture prediction may be possible for each frame of a moving picture according to the fracture detection method according to an embodiment of the present invention. That is, since the fracture prediction is possible simultaneously with the reception of the medical bone image from the camera, the fracture may be detected.
본 발명의 특징에 따르면, 골 의료 영상은, 2차원 영상, 3차원 영상일 수도 있다. According to a feature of the present invention, the medical bone image may be a two-dimensional image or a three-dimensional image.
본 명세서에서 사용되는 용어, "골절 의심 영역"은 의료 영상 내에서 골절로 의심되는 영역일 수 있다. 이때, 골절 의심 영역은 실제 골절이 일어난 부위에 대한 영역과 골절과 유사한 형태를 갖는 골절 유사 영역으로 이루어질 수 있다. 이때, 골절 유사 영역은, 골 접합선, 혈관 및 골 중첩 선과 같은 골절과 유사한 부위에 대한 영역을 포함할 수 있다. 그러나, 골절 유사 영역은 이에 제한되는 것은 아니다. As used herein, the term "suspicious fracture region" may be a region suspected of a fracture in a medical image. In this case, the fracture suspected region may include a region for a site where the actual fracture occurred and a fracture-like region having a shape similar to that of a fracture. In this case, the fracture-like region may include a region for a region similar to a fracture, such as a bone junction line, a blood vessel, and a bone overlap line. However, the fracture-like region is not limited thereto.
본 명세서에서 사용되는 용어, "예측 모델"은 골 의료 영상을 입력으로 하여, 골절 의심 부위, 즉 골절 의심 영역을 출력하도록 구성된 모델일 수 있다.As used herein, the term “predictive model” may be a model configured to output a fracture suspected region, that is, a fracture suspected region by inputting a medical bone image as an input.
보다 구체적으로, 예측 모델은, 골 의료 영상을 입력으로 하여 골절 의심 영역을 분할하여 출력하도록 구성될 수 있다. More specifically, the predictive model may be configured to divide and output a region suspected of fracture by inputting a medical bone image as an input.
본 발명의 특징에 따르면, 예측 모델은, 복수의 각도에서 촬영된 골 의료 영상 각각에 대하여 골절 의심 영역을 예측하도록 구성된 복수의 모델을 포함할 수 있다. According to a feature of the present invention, the prediction model may include a plurality of models configured to predict a fracture suspected region for each of the medical bone images taken from a plurality of angles.
이때, 예측 모델은 복수의 각도에서 촬영된 복수의 의료 영상이 정합된 영상을 학습 데이터로 이용하여 골절의 오검출을 줄이도록 구성된 모델일 수 있다.In this case, the predictive model may be a model configured to reduce erroneous detection of fractures by using an image obtained by matching a plurality of medical images captured from a plurality of angles as learning data.
예를 들어, 예측 모델은 정면 의료 영상 및 후면 의료 영상이 정합된 정면-후면 정합 의료 영상, 또는 좌측면 의료 영상 및 우측면 의료 영상이 정합된 좌측-우측 정합 의료 영상을 학습에 이용하여 골절 의심 영역을 결정하도록 구성될 수 있다.For example, the predictive model uses a front-to-back medical image in which the front medical image and the posterior medical image are registered, or a left-right registered medical image in which the left medical image and the right medical image are registered, for training to determine the fracture suspected region. can be configured to determine
이때, 의료 영상의 정합은, 각각의 의료 영상의 위치와 크기가 맞춰진 이후, 각 의료 영상에 대하여 미리 결정된 골절 영역의 위치가 일치하도록 수행될 수 있다. 특히, 좌측면 의료 영상 및 우측면 의료 영상이 정합될 경우, 좌측면 의료 영상 또는 우측면 의료 영상의 좌우 반전이 수행된 후, 두 개의 영상이 정합될 수 있다. In this case, the registration of the medical images may be performed so that, after the positions and sizes of the respective medical images are matched, the predetermined positions of the fracture regions for each medical image are matched. In particular, when the left side medical image and the right side medical image are matched, the two images may be registered after left and right inversion of the left side medical image or the right side medical image is performed.
즉, 본 발명의 다양한 실시예에 이용되는 예측 모델은 동일 개체에 대한 다각도의 영상이 정합된 의료 영상에 대하여 골절 부위를 예측하도록 학습함에 따라, 골절의 예측 성능이 다른 모델보다 우수할 수 있다. That is, as the prediction model used in various embodiments of the present invention learns to predict a fracture site with respect to a medical image to which multi-angle images of the same object are matched, fracture prediction performance may be superior to that of other models.
한편, 예측 모델은, 복수의 각도에서 촬영된 복수의 영상 각각에 대하여 골절 의심 영역을 출력하도록 학습된 RetinaNet일 수 있으나, 이에 제한되지 않고, Resnet50, Resnet-v2, Resnet101, Inception-v3, 또는 VGG net, R, DenseNet 및, encoder-decoder structure를 갖는 FCN, SegNet, DeconvNet, DeepLAB V3+, U-net와 같은 DNN (deep neural network), SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet 중 선택된 적어도 하나의 알고리즘에 기초할 수도 있다. On the other hand, the prediction model may be RetinaNet trained to output a fracture suspected region for each of a plurality of images taken from a plurality of angles, but is not limited thereto, and Resnet50, Resnet-v2, Resnet101, Inception-v3, or VGG At least one selected from net, R, DenseNet, and a deep neural network (DNN) such as FCN, SegNet, DeconvNet, DeepLAB V3+, and U-net having an encoder-decoder structure, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet It may be algorithm-based.
즉, 본 발명의 다양한 실시예에서, 의료 영상의 각도 별로 골절 의심 영역을 예측하도록 학습된 복수의 모델이 복수의 영상 각각에 대한 골절 의심 영역 예측하기 위해 이용될 수 있다. That is, in various embodiments of the present disclosure, a plurality of models trained to predict a suspected fracture region for each angle of a medical image may be used to predict a suspected fracture region for each of the plurality of images.
나아가, 예측 모델은, 멀티뷰 영상 (예를 들어, 3D 이미지, 또는 다각도 영상) 에 대하여 골절 의심 영역을 예측하도록 학습된, MVCNN (Multi-view Convolutional Neural Network) 또는 MPCNN (Multi-Planar CNN) 일수도 있다. 즉, 다각도에서 촬영된 멀티뷰 영상에 대하여 골절 의심 영역을 예측하도록 학습된 단일의 모델이 다각도의 복수의 영상에 대한 골절 의심 영역을 예측하기 위해 이용될 수도 있다.Furthermore, the prediction model is a Multi-view Convolutional Neural Network (MVCNN) or Multi-Planar CNN (MPCNN), which is trained to predict a fracture suspected region with respect to a multi-view image (eg, a 3D image, or a multi-angle image). may be That is, a single model trained to predict a fracture suspected region with respect to a multi-view image taken from multiple angles may be used to predict a suspected fracture region for a plurality of images from multiple angles.
그러나, 이에 제한되는 것이 아니며 상기 예측 모델은 전술한 알고리즘 중 적어도 두 개의 알고리즘 모델에 기초한 앙상블 모델일 수 도 있다.However, the present invention is not limited thereto, and the predictive model may be an ensemble model based on at least two algorithm models among the aforementioned algorithms.
또한, 예측 모델은, 의료 영상 내에서 골절 의심 영역, 및 골 접합선, 혈관 및 골 중첩 선과 같은 유사 골절 영역을 분류하도록 학습될 수도 있다. 예를 들어, 예측 모델은 의료 영상 내에서 골절 의심 영역인 ROI (region of interest) 를 분할하는 영역 분할 모델과, 분할된 ROI에 대하여 골절 영역 또는 유사 골절 영역을 분류하는 분류기로 이루어질 수도 있다. In addition, the predictive model may be trained to classify a fracture suspected region and a similar fracture region such as a bone junction line, a blood vessel, and a bone overlap line within a medical image. For example, the prediction model may consist of a region segmentation model for segmenting a region of interest (ROI), which is a region suspected of fracture in a medical image, and a classifier for classifying a fracture region or a similar fracture region with respect to the segmented ROI.
이하에서는 도 1a 내지 1c를 참조하여, 본 발명의 일 실시예에 따른 골절 검출용 디바이스에 기초한 골절 검출 시스템을 설명한다. Hereinafter, a fracture detection system based on a device for detecting a fracture according to an embodiment of the present invention will be described with reference to FIGS. 1A to 1C .
도 1a는 본 발명의 일 실시예에 따른 골절 검출용 디바이스에 기초한 골절 검출 시스템을 예시적으로 도시한 것이다. 도 1b는 본 발명의 일 실시예에 따른 골절 검출용 디바이스의 구성을 예시적으로 도시한 것이다. 도 1c는 본 발명의 일 실시예에 따른 골절 검출용 디바이스로부터 골절 의심 부위를 수신하도록 구성된 의료진 디바이스의 구성을 예시적으로 도시한 것이다. 1A exemplarily shows a fracture detection system based on a device for fracture detection according to an embodiment of the present invention. 1B exemplarily shows the configuration of a device for detecting a fracture according to an embodiment of the present invention. 1C exemplarily illustrates the configuration of a medical staff device configured to receive a fracture-suspected region from the device for detecting a fracture according to an embodiment of the present invention.
먼저, 도 1a을 참조하면, 골절 검출 시스템 (1000) 은, 개체에 대한 골 의료 영상을 기초로 골절과 관련된 정보를 제공하도록 구성된 시스템일 수 있다. 이때, 골절 검출 시스템 (1000) 은, 골 의료 영상에 기초하여, 골절을 예측하고 골절 의심 영역을 결정하도록 구성된 골절 검출용 디바이스 (100), 골절 검출에 대한 정보를 수신하는 의료진 디바이스 (200) 및 골절 의심 부위에 대한 영상을 제공하는, 의료 영상 제공용 디바이스 (300) 로 구성될 수 있다. First, referring to FIG. 1A , the fracture detection system 1000 may be a system configured to provide information related to a fracture based on a medical bone image of an individual. At this time, the fracture detection system 1000 includes a device for fracture detection configured to predict a fracture and determine a fracture suspected region based on a medical bone image, a medical staff device 200 for receiving information on fracture detection, and The device 300 for providing a medical image may be configured to provide an image of a suspected fracture site.
먼저, 골절 검출용 디바이스 (100) 는 의료 영상 제공용 디바이스 (300) 로부터 제공된 개체의 골 의료 영상을 기초로 골절 부위를 진단하기 위해 다양한 연산을 수행하는 범용 컴퓨터, 랩탑, 및/또는 데이터 서버 등을 포함할 수 있다. 이때, 의료진 디바이스 (200) 는 웹 페이지를 제공하는 웹 서버 (web server) 또는 모바일 웹 사이트를 제공하는 모바일 웹 서버 (mobile web server) 에 액세스하기 위한 디바이스일 수 있으나, 이에 한정되지 않는다. First, the device for detecting a fracture 100 is a general-purpose computer, laptop, and/or data server that performs various calculations to diagnose a fracture site based on a medical bone image of an individual provided from the device for providing a medical image 300 , etc. may include In this case, the medical staff device 200 may be a device for accessing a web server providing a web page or a mobile web server providing a mobile web site, but is not limited thereto.
보다 구체적으로, 골절 검출용 디바이스 (100) 는 의료 영상 제공용 디바이스 (300) 로부터 골 의료 영상을 수신하고, 수신된 골 의료 영상으로부터 골절 부위와 연관된 정보를 제공할 수 있다. 이때, 의료 영상 제공용 디바이스 (300) 는, 예측 모델을 이용하여 골 의료 영상에 대한 골절 의심 영역을 예측할 수 있다. 골절 검출용 디바이스 (100) 는 개체에 대한 골절 부위와 연관된 데이터를 의료진 디바이스 (200) 로 제공할 수 있다. More specifically, the device 100 for detecting a fracture may receive a medical bone image from the device 300 for providing a medical image, and may provide information related to a fracture site from the received medical image of the bone. In this case, the device 300 for providing a medical image may predict a fracture suspected region in the medical bone image by using the prediction model. The device 100 for detecting a fracture may provide data associated with a fracture site for the subject to the medical staff device 200 .
이와 같이 골절 검출용 디바이스 (100) 로부터 제공되는 데이터는 의료진 디바이스 (200) 에 설치된 웹 브라우저를 통해 웹 페이지로 제공되거나, 어플리케이션, 또는 프로그램 형태로 제공될 수 있다. 다양한 실시예에서 이러한 데이터는 클라이언트-서버 환경에서 플랫폼에 포함되는 형태로 제공될 수 있다.As such, the data provided from the device 100 for detecting a fracture may be provided as a web page through a web browser installed in the medical staff device 200 , or may be provided in the form of an application or a program. In various embodiments, such data may be provided in a form included in the platform in a client-server environment.
다음으로, 의료진 디바이스 (200) 는 개체에 대한 골절 부위에 대한 정보 제공을 요청하고 골절 부위 예측 결과 데이터를 나타내기 위한 사용자 인터페이스를 제공하는 전자 디바이스로서, 스마트폰, 태블릿 PC (Personal Computer), 노트북 및/또는 PC 등 중 적어도 하나를 포함할 수 있다.Next, the medical staff device 200 is an electronic device that requests the provision of information on the fracture site for the subject and provides a user interface for displaying the fracture site prediction result data, a smartphone, a tablet PC (Personal Computer), and a notebook computer. and/or may include at least one of a PC and the like.
의료진 디바이스 (200) 는 골절 검출용 디바이스 (100) 로부터 개체에 대한 골절에 관한 검출 결과를 수신하고, 수신된 결과를 표시부를 통해 표시할 수 있다. The medical staff device 200 may receive a detection result regarding a fracture of an object from the device 100 for detecting a fracture, and display the received result through a display unit.
의료 영상 제공용 디바이스 (300) 는 X-레이 촬영 디바이스, 초음파 디바이스, 자기공명영상 촬영 디바이스, CT 촬영 디바이스일 수 있으나, 이에 제한되는 것은 아니다. 예를 들어, 의료 영상 제공용 디바이스 (300) 는 다양한 의료 영상 데이터베이스를 제공하는 별도의 서버일 수도 있다.The device 300 for providing a medical image may be an X-ray imaging device, an ultrasound device, a magnetic resonance imaging device, or a CT imaging device, but is not limited thereto. For example, the device 300 for providing medical images may be a separate server that provides various medical image databases.
다음으로, 도 1b를 참조하여, 본 발명의 골절 검출용 디바이스 (100) 의 구성 요소에 대하여 구체적으로 설명한다. Next, with reference to FIG. 1B, the components of the device 100 for fracture detection of this invention are demonstrated in detail.
도 1b를 참조하면, 골절 검출용 디바이스 (100) 는 저장부 (110), 통신부 (120) 및 프로세서 (130) 를 포함한다. Referring to FIG. 1B , the device 100 for detecting a fracture includes a storage unit 110 , a communication unit 120 , and a processor 130 .
먼저, 저장부 (110) 는 개체에 대한 골절 부위를 진단하는 중에 생성된 다양한 데이터를 저장할 수 있다. 예를 들어, 저장부 (110) 는, 후술할 통신부 (120) 를 통해 의료 영상 제공용 디바이스 (300) 로부터 수신된 골 의료 영상, 나아가 예측 모델의 분류 과정에서의 다양한 산물들을 저장하도록 구성될 수 있다. 다양한 실시예에서 저장부 (110) 는 플래시 메모리 타입, 하드디스크 타입, 멀티미디어 카드 마이크로 타입, 카드 타입의 메모리 (예를 들어 SD 또는 XD 메모리 등), 램, SRAM, 롬, EEPROM, PROM, 자기 메모리, 자기 디스크, 광디스크 중 적어도 하나의 타입의 저장매체를 포함할 수 있다.First, the storage unit 110 may store various data generated while diagnosing a fracture site for an individual. For example, the storage unit 110 may be configured to store a medical bone image received from the device 300 for providing a medical image through the communication unit 120 to be described later, and furthermore, various products in the classification process of the predictive model. have. In various embodiments, the storage unit 110 is a flash memory type, hard disk type, multimedia card micro type, card type memory (eg, SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory. , a magnetic disk, and an optical disk may include at least one type of storage medium.
통신부 (120) 는 골절 검출용 디바이스 (100) 가 외부 디바이스와 통신이 가능하도록 연결한다. 통신부 (120) 는 유/무선 통신을 이용하여 의료진 디바이스 (200), 나아가 의료 영상 제공용 디바이스 (300) 와 연결되어 다양한 데이터를 송수신할 수 있다. 구체적으로, 통신부 (120) 는 의료 영상 제공용 디바이스 (300) 로부터 개체의 골 의료 영상을 수신할 수 있다. 예를 들어, 통신부 (120) 는 의료 영상 제공용 디바이스 (300) 로부터, X-레이 영상을 수신할 수 있다. 나아가, 통신부 (120) 는 의료진 디바이스 (200) 로 검출 결과를 전달할 수 있다.The communication unit 120 connects the fracture detection device 100 to enable communication with an external device. The communication unit 120 may be connected to the medical staff device 200 and furthermore the medical image providing device 300 using wired/wireless communication to transmit/receive various data. In detail, the communication unit 120 may receive a medical image of the bone of an object from the device 300 for providing a medical image. For example, the communication unit 120 may receive an X-ray image from the device 300 for providing a medical image. Furthermore, the communication unit 120 may transmit the detection result to the medical staff device 200 .
프로세서 (130) 는 저장부 (110) 및 통신부 (120) 와 동작 가능하게 연결되며, 개체에 대한 골 의료 영상을 분석하기 위한 다양한 명령들을 수행할 수 있다. The processor 130 is operatively connected to the storage 110 and the communication unit 120 , and may perform various commands for analyzing a medical bone image of an object.
구체적으로, 프로세서 (130) 는 통신부 (120) 를 통해 수신된 골 의료 영상에 기초하여 골절 의심 부위를 감지하고, 골절 유사 영역을 제거하여 골절 부위를 최종적으로 결정하도록 구성될 수 있다. Specifically, the processor 130 may be configured to detect a fracture suspected site based on the medical bone image received through the communication unit 120 , and finally determine the fracture site by removing the fracture-like region.
이때, 프로세서 (130) 는 골 의료 영상에 기초하여, 골절 의심 영역을 예측하도록 구성된, 예측 모델에 기초할 수 있다. In this case, the processor 130 may be based on a prediction model configured to predict a fracture suspected region based on the medical bone image.
한편, 골절 검출용 디바이스 (100) 는 하드웨어 적으로 설계된 것으로 제한되는 것은 아니다. 예를 들어, 골절 검출용 디바이스 (100) 의 프로세서 (130) 는 소프트웨어로 구현될 수 있다. 이에, 골절에 대한 검출 결과는 상기 소프트웨어가 연결된 의료 영상 제공용 디바이스 (300) 의 표시부를 통해 표시될 수도 있다.On the other hand, the device 100 for detecting a fracture is not limited to a hardware design. For example, the processor 130 of the device 100 for fracture detection may be implemented in software. Accordingly, the detection result of the fracture may be displayed through the display unit of the medical image providing device 300 to which the software is connected.
한편, 도 1c를 함께 참조하면, 의료진 디바이스 (200) 는 통신부 (210), 표시부 (220), 저장부 (230) 및 프로세서 (240) 를 포함한다. Meanwhile, referring to FIG. 1C , the medical staff device 200 includes a communication unit 210 , a display unit 220 , a storage unit 230 , and a processor 240 .
통신부 (210) 는 의료진 디바이스 (200) 가 외부 디바이스와 통신 가능하도록 구성될 수 있다. 통신부 (210) 는 유/무선 통신을 이용하여 골절 검출용 디바이스 (100) 와 연결되어 골절의 진단과 연관된 다양한 데이터를 송신할 수 있다. 구체적으로, 통신부 (210) 는 골절 검출용 디바이스 (100) 로부터, 예측 모델들에 의해 예측된 골절 의심 부위, 나아가 치료 예후 등을 수신할 수 있다. The communication unit 210 may be configured to enable the medical staff device 200 to communicate with an external device. The communication unit 210 may be connected to the device 100 for detecting a fracture using wired/wireless communication to transmit various data related to diagnosis of a fracture. Specifically, the communication unit 210 may receive, from the device 100 for detecting a fracture, a suspected fracture site predicted by the prediction models, and furthermore, a treatment prognosis.
표시부 (220) 는 개체의 골절의 검출 결과를 나타내기 위한 다양한 인터페이스 화면을 표시할 수 있다. The display unit 220 may display various interface screens for displaying the detection result of the fracture of the object.
다양한 실시예에서 표시부 (220) 는 터치스크린을 포함할 수 있으며, 예를 들면, 전자 펜 또는 사용자의 신체의 일부를 이용한 터치 (touch), 제스처 (gesture), 근접, 드래그 (drag), 스와이프 (swipe) 또는 호버링 (hovering) 입력 등을 수신할 수 있다. In various embodiments, the display unit 220 may include a touch screen, for example, a touch, gesture, proximity, drag, swipe using an electronic pen or a part of the user's body. A swipe or hovering input may be received.
저장부 (230) 는 결과 데이터를 나타내기 위한 사용자 인터페이스를 제공하기 위해 사용되는 다양한 데이터를 저장할 수 있다. 다양한 실시예에서 저장부 (230) 는 플래시 메모리 타입 (flash memory type), 하드디스크 타입 (hard disk type), 멀티미디어 카드 마이크로 타입 (multimedia card micro type), 카드 타입의 메모리 (예를 들어 SD 또는 XD 메모리 등), 램 (Random Access Memory, RAM), SRAM (Static Random Access Memory), 롬 (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), 자기 메모리, 자기 디스크, 광디스크 중 적어도 하나의 타입의 저장매체를 포함할 수 있다. The storage 230 may store various data used to provide a user interface for displaying result data. In various embodiments, the storage unit 230 may include a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD or XD). memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.
프로세서 (240) 는 통신부 (210), 표시부 (220) 및 저장부 (230) 와 동작 가능하게 연결되며, 검출 결과 데이터를 나타내기 위한 사용자 인터페이스를 제공하기 위한 다양한 명령들을 수행할 수 있다. The processor 240 is operatively connected to the communication unit 210 , the display unit 220 , and the storage unit 230 , and may perform various commands for providing a user interface for displaying detection result data.
이하에서는, 도 2a, 2b, 및 2c를 참조하여, 본 발명의 일 실시예에 따른 골절 검출 방법을 구체적으로 설명한다.Hereinafter, a method for detecting a fracture according to an embodiment of the present invention will be described in detail with reference to FIGS. 2A, 2B, and 2C.
도 2a 내지 2c는 본 발명의 일 실시예에 따른 골절 검출 방법의 절차를 예시적으로 도시한 것이다. 2A to 2C exemplarily show a procedure of a method for detecting a fracture according to an embodiment of the present invention.
먼저, 도 2a를 참조하면, 본 발명의 일 실시예에 따른 골절 검출의 절차는 다음과 같다. 먼저, 개체에 대한 골 의료 영상이 수신되고 (S210), 예측 모델에 의해, 골 의료 영상에 대한 골절 의심 영역이 결정된다 (S220). 그 다음, 골절 의심 영역 중 유사 골절 영역이 결정되고 (S230), 마지막으로 골절 의심 영역이 필터링되어 골절 부위가 결정된다 (S240).First, referring to FIG. 2A , a procedure for detecting a fracture according to an embodiment of the present invention is as follows. First, a medical bone image of an object is received (S210), and a fracture suspected region for the medical image is determined by the prediction model (S220). Next, a similar fracture region is determined among the fracture suspect regions (S230), and finally, the suspected fracture region is filtered to determine a fracture site (S240).
본 발명의 특징에 따르면, 골 의료 영상이 수신되는 단계 (S210) 에서, 개체의 목적 부위에 대하여 복수의 각도에서 촬영된 복수의 의료 영상이 수신된다.According to a feature of the present invention, in the step of receiving the bone medical image ( S210 ), a plurality of medical images photographed from a plurality of angles with respect to a target site of an object are received.
본 발명의 특징에 따르면, 골 의료 영상이 수신되는 단계 (S210) 에서, X-레이 영상, 컴퓨터 단층 촬영 영상, 자기 공명 영상 및 초음파 영상 중 하나의 골 의료 영상이 수신된다. According to a feature of the present invention, in the step of receiving the medical bone image ( S210 ), one of a medical bone image of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image is received.
바람직하게, 골 의료 영상이 수신되는 단계 (S210) 에서, 복수의 각도에서 촬영한 X-레이 영상이 수신될 수 있으나, 이에 제한되는 것은 아니다.Preferably, in the step (S210) of receiving the bone medical image, X-ray images taken from a plurality of angles may be received, but the present invention is not limited thereto.
본 발명의 다른 특징에 따르면, 골 의료 영상이 수신되는 단계 (S210) 에서, 두개골, 하악골, 설골, 경추, 흉추, 요추, 늑골, 흉골, 쇄골, 견갑골, 상완골, 요골, 척골, 주상골, 월상골, 삼각골, 두상골, 대능형골, 소농형골, 유두골, 유두골, 중수골, 손가락뼈, 관골, 대퇴골, 슬개골, 경골, 비골, 거골, 종골, 주상골, 입방골, 쐐기뼈, 증족골 및 발가락뼈 중 적어도 하나에 대한 골절 의심 부위에 대한 의료 영상이 수신된다.According to another feature of the present invention, in the step (S210) of receiving the bone medical image, the skull, mandible, hyoid, cervical, thoracic, lumbar, rib, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunar , triangle, cephalothorax, scapula, papillary, papillary, metacarpal, finger, tibia, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid, metatarsal and A medical image of a region suspected of fracture of at least one of the toe bones is received.
다음으로, 예측 모델에 의해, 골 의료 영상에 대한 골절 의심 영역이 결정된다 (S220). Next, a fracture suspected region for the medical bone image is determined by the predictive model (S220).
본 발명의 특징에 따르면, 골절 의심 영역이 결정되는 단계 (S220) 에서, 복수의 의료 영상 각각에 대하여 골절 부위를 예측하도록 구성된 복수의 예측 모델에 의해 골절 의심 영역이 각각 결정된다.According to a feature of the present invention, in the step of determining the fracture suspected region ( S220 ), the fracture suspected region is determined by a plurality of prediction models configured to predict a fracture site for each of a plurality of medical images.
보다 구체적으로, 골절 의심 영역이 결정되는 단계 (S220) 에서, 의료 영상의 각도 별로 골절 의심 영역을 예측하도록 학습된 복수의 모델이 복수의 영상 각각에 대한 골절 의심 영역 예측하기 위해 이용될 수 있다. More specifically, in the step of determining the suspected fracture region ( S220 ), a plurality of models trained to predict the suspected fracture region for each angle of the medical image may be used to predict the suspected fracture region for each of the plurality of images.
본 발명의 다른 특징에 따르면, 골절 의심 영역이 결정되는 단계 (S220) 에서, 복수의 의료 영상에 대하여 골절 부위를 예측하도록 구성된 단일의 예측 모델에 의해 골절 의심 영역이 결정된다.According to another feature of the present invention, in the step of determining the fracture suspected region ( S220 ), the fracture suspected region is determined by a single prediction model configured to predict the fracture site with respect to a plurality of medical images.
보다 구체적으로, 골절 의심 영역이 결정되는 단계 (S220) 에서, 다각도에서 촬영된 멀티뷰 영상에 대하여 골절 의심 영역을 예측하도록 학습된 단일의 모델이 다각도의 복수의 영상에 대하여 골절 의심 영역을 예측하기 위해 이용될 수 있다.More specifically, in the step of determining the fracture suspected region ( S220 ), a single model trained to predict the suspected fracture region with respect to the multi-view image taken from multiple angles predicts the suspected fracture region with respect to a plurality of images from multiple angles. can be used for
예를 들어, 도 2b를 함께 참조하면, 골절 의심 영역이 결정되는 단계 (S220) 에서, 수신된 정면 의료 영상 (412), 후면 의료 영상 (414), 우측면 의료 영상 (416) 및 좌측면 의료 영상 (418) 의 다각도에서 촬영된 복수의 의료 영상 (410) 각각이 정면 예측 모델 (422), 후면 예측 모델 (424), 우측면 예측 모델 (426) 및 좌측면 예측 모델 (428) 의 복수의 예측 모델 (420) 에 입력된다. 그 결과, 복수의 의료 영상 (410) 내에서 골절 의심 영역 (432, 434, 436a, 436b 및 438) 이 결정된다. For example, referring to FIG. 2B together, in the step S220 in which the fracture suspected region is determined, the received front medical image 412 , the rear medical image 414 , the right side medical image 416 , and the left side medical image Each of a plurality of medical images 410 taken from a multi-angle of 418 is a front prediction model 422 , a rear prediction model 424 , a right prediction model 426 , and a plurality of prediction models of the left prediction model 428 . (420) is input. As a result, fracture suspicious regions 432 , 434 , 436a , 436b and 438 within the plurality of medical images 410 are determined.
다시 도 2a를 참조하면, 유사 골절 영역이 결정되는 단계 (S230) 에서, 예측 모델에 의해 결정된 골절 의심 영역 중, 실제 골절 부위에 대한 영역이 아닌 유사 결정 영역이 결정되고, 유사 결정 영역이 제거되도록 골절 의심 영역이 필터링되어 최종적으로 골절 부위가 결정된다 (S240).Referring back to FIG. 2A , in the step of determining the similar fracture region ( S230 ), among the fracture suspected regions determined by the predictive model, the similar determination region is determined, not the region for the actual fracture site, and the similar determination region is removed. The fracture suspected region is filtered to finally determine the fracture site (S240).
본 발명의 특징에 따르면, 유사 골절 영역이 결정되는 단계 (S230) 에서, 골절 의심 영역의 두께, 형태 및 위치 중 적어도 하나에 기초하여, 유사 골절 영역이 결정된다.According to a feature of the present invention, in the step S230 in which the similar fracture region is determined, the similar fracture region is determined based on at least one of the thickness, shape, and location of the suspected fracture region.
예를 들어, 다시 도 2b를 참조하면, 복수의 예측 모델 (420) 에 의해 결정된, 골절 의심 영역은 실제 골절에 대한 영역 (432 및 436b) 과 골절과 형태가 유사한 유사 골절 영역 (434, 436a 및 438) 을 포함할 수 있다. For example, referring again to FIG. 2B , the fracture-suspected regions determined by the plurality of prediction models 420 include regions 432 and 436b for actual fractures, and similar fracture regions 434, 436a and 438) may be included.
보다 구체적으로, 도 2c를 함께 참조하면, 골절에 대한 영역 (432 및 436b) (도 2c의 (a) 참조), 특히 선형 골절에 대한 영역은 선의 굵기가 일정하고, 크기 및 위치가 정해져 있지 않다. 반면에, 유사 골절 영역 (434, 436a 및 438), 중 혈관 (도 2c의 (b) 참조) 은 여러 개의 선이 존재하며, 뻗어 나갈수록 선의 굵기가 얇아지고 안와부, 관자놀이 주변에 위치할 수 있다. 골 접합선 (도 2c의 (c) 참조) 은 선 모양이 불규칙하며, 일정한 위치에 존재하고 혈관 또는 골절 부위에 비하여 선의 굵기가 굵을 수 있다. More specifically, referring to FIG. 2C together, the regions 432 and 436b for fractures (refer to (a) of FIG. 2C ), in particular, for linear fractures, have a constant line thickness, and the size and location are not determined. . On the other hand, in the pseudo-fracture areas 434, 436a and 438, and the middle blood vessels (see (b) of FIG. 2c), several lines exist, and the line becomes thinner as it extends, and it can be located around the orbit and temple. have. The bone junction line (refer to (c) of FIG. 2C) has an irregular line shape, exists at a certain location, and may have a thicker line than that of a blood vessel or fracture site.
즉, 다시 도 2b를 참조하면, 유사 골절 영역이 결정되는 단계 (S230) 에서, 골절 의심 영역 (432, 434, 436a, 436b 및 438) 의 두께, 형태 및 위치 중 적어도 하나에 기초하여, 유사 골절 영역 (434, 436a 및 438) 이 결정되고, 골절 부위가 결정되는 단계 (S240) 에서 유사 골절 영역 (434, 436a 및 438) 이 제거된다. 그 결과, 정면 의료 영상 (412) 및 우측면 의료 영상 (416) 에 각각에 대한 골절에 대한 영역 (432 및 436b) 이 골절 부위로 결정될 수 있다. That is, referring back to FIG. 2B , in the step S230 in which the similar fracture region is determined, based on at least one of the thickness, shape, and location of the fracture suspicious regions 432 , 434 , 436a , 436b and 438 , the similar fracture region is determined. Regions 434, 436a and 438 are determined, and in step S240 where the fracture site is determined, the similar fracture regions 434, 436a and 438 are removed. As a result, in the front medical image 412 and the right medical image 416 , regions 432 and 436b for fractures may be determined as fracture sites.
즉, 유사 골절 영역이 결정되는 단계 (S230) 에서, 오검출 영역인 유사 골절 영역 (434, 436a 및 438) 이 제거되어 골절 검출의 민감도가 증가할 수 있다.That is, in the step S230 in which the similar fracture region is determined, the false fracture region 434 , 436a , and 438 , which is an erroneous detection region, may be removed to increase the sensitivity of fracture detection.
본 발명의 특징에 따르면, 유사 골절 영역이 결정되는 단계 (S230) 는, 골절 의심 영역, 즉 ROI (region of interest) 을 입력으로 하여 골절, 또는 혈관, 골 접합 영역과 같은 유사 골절 부위를 분류하도록 구성된 분류기가 동작하여 수행될 수도 있다.According to a feature of the present invention, the step of determining the similar fracture region ( S230 ) is to classify a fracture site, such as a fracture region, or a similar fracture region, such as a blood vessel or bone junction region, by inputting a fracture suspected region, that is, a region of interest (ROI) as an input. The configured classifier may be operated and performed.
본 발명의 다른 특징에 따르면, 골절 부위가 결정되는 단계 (S240) 이후에, 복수의 의료 영상 중 선택된 세트 영상의 골절 부위가 비교되고, 비교 결과에 기초하여 골절 부위가 최종 결정되는 단계가 더 수행된다. 이때, 세트 영상은, 정면 의료 영상 및 후면 의료 영상, 또는 우측면 의료 영상 및 좌측면 의료 영상일 수 있다. According to another feature of the present invention, after the step of determining the fracture site ( S240 ), the fracture site of a set image selected from among a plurality of medical images is compared, and the step of finally determining the fracture site based on the comparison result is further performed. do. In this case, the set image may be a front medical image and a rear medical image, or a right side medical image and a left side medical image.
예를 들어, 도 2b를 참조하면, 골절 부위가 최종 결정되는 단계에서, 정면 의료 영상 (412) 및 후면 의료 영상 (414) 에 대하여 결정된 골절에 대한 영역의 위치가 비교되고, 우측면 의료 영상 (416) 및 좌측면 의료 영상 (418) 에 대하여 결정된 골절에 대한 영역의 위치가 비교될 수 있다. 이를 통해 오검출, 즉 유사 골절 영역이 다시 한 번 제거되고, 정면 의료 영상 (412) 및 우측면 의료 영상 (416) 에 대한 골절에 대한 영역 (432 및 436b) 이 최종 골절 부위로 결정될 수 있다. 나아가, 골절에 대한 영역 (432 및 436b) 이 표시된 골절 진단 의료 영상 (440) 이 제공될 수 있다.For example, referring to FIG. 2B , in the stage in which the fracture site is finally determined, the position of the fracture area determined with respect to the front medical image 412 and the rear medical image 414 is compared, and the right side medical image 416 . ) and the location of the region for the fracture determined with respect to the left side medical image 418 may be compared. Through this, the false detection, that is, the similar fracture region is removed once again, and the fracture regions 432 and 436b for the front medical image 412 and the right side medical image 416 may be determined as final fracture sites. Furthermore, a fracture diagnosis medical image 440 in which regions 432 and 436b for fracture are displayed may be provided.
이때, 골절에 대한 영역 (432 및 436b) 은 동일한 골절 부위에 대한 다른 각도의 영역일 수도 있으나, 이에 제한되는 것은 아니다. In this case, the fracture regions 432 and 436b may be regions of different angles with respect to the same fracture site, but is not limited thereto.
이상의 다양한 실시예에 따른 골절 검출 방법에 기초한 골절 검출 시스템은, 종래의 골절 검출 시스템의 한계를 극복할 수 있다. 특히, 상기 시스템은, 두개골, 늑골과 같이 X-레이 영상만으로 골절의 육안 식별의 어려움이 있는 골절 의심 부위에 대하여 신뢰도 높은 진단 결과를 제공할 수 있다는 점에서 종래의 골절 검출 시스템의 한계를 극복할 수 있다. 나아가, 상기 시스템은 검출 소요 시간을 줄일 수 있으며, 의료진의 숙련도에 관계 없이 정확한 골절 진단 결과를 제공할 수도 있다.The fracture detection system based on the fracture detection method according to the above various embodiments can overcome the limitations of the conventional fracture detection system. In particular, the system can overcome the limitations of the conventional fracture detection system in that it can provide reliable diagnostic results for areas suspected of fractures having difficulty in visual identification of fractures only with X-ray images, such as the skull and ribs. can Furthermore, the system can reduce the time required for detection and provide accurate fracture diagnosis results regardless of the skill level of medical staff.
이하에서는, 도 3 및 도 4를 참조하여, 본 발명의 다양한 실시예에 따른 예측 모델의 구조를 설명한다. 도 3 및 4는 본 발명의 다양한 실시예에 이용되는 예측 모델의 구조를 예시적으로 도시한 것이다.Hereinafter, a structure of a prediction model according to various embodiments of the present invention will be described with reference to FIGS. 3 and 4 . 3 and 4 exemplarily show the structure of a prediction model used in various embodiments of the present invention.
이때, 제시된 실시예에서는 골 의료 영상을 피검자 얼굴의 전면, 측면 및 후면을 촬영한 X-레이 영상인 것으로 한정하고, 목적 부위를 두개골의 골절인 것으로 한정하여 설명하도록 한다. 그러나, 예측 모델은 이에 제한되지 않고 다양한 부위에 대한 의료 영상 내에서 골절을 진단하는 것에 적용될 수 있다. At this time, in the presented embodiment, the bone medical image is limited to X-ray images taken from the front, side, and back of the subject's face, and the target site is limited to a fracture of the skull. However, the predictive model is not limited thereto and may be applied to diagnosing a fracture within a medical image of various regions.
도 3를 참조하면, 예측 모델은 복수의 골 의료 영상 (510) 각각을 입력으로 하여 골절 의심 영역을 예측하도록 학습된 복수의 인공신경망으로 이루어진 RetinaNet기반의 예측 모델일 수 있다. 이때, RetinaNet기반의 예측 모델은 다각도에서 촬영된 복수의 의료 영상 각각에 대하여 골절 의심 영역을 예측하도록 학습된 복수의 모델일 수 있다. 다만, 본 예시에서는 설명의 편의를 위해 복수의 모델 중 하나의 모델의 구조에 대하여 설명한다. Referring to FIG. 3 , the predictive model may be a RetinaNet-based predictive model composed of a plurality of artificial neural networks trained to predict a suspected fracture region by inputting each of a plurality of bone medical images 510 as an input. In this case, the RetinaNet-based prediction model may be a plurality of models trained to predict a fracture suspected region for each of a plurality of medical images taken from multiple angles. However, in this example, the structure of one model among a plurality of models will be described for convenience of description.
보다 구체적으로, 복수의 인공신경망은 제1 인공신경망 (520), 제2 인공신경망 (530) 및 제3 인공신경망 (540) 을 포함할 수 있다.More specifically, the plurality of artificial neural networks may include a first artificial neural network 520 , a second artificial neural network 530 , and a third artificial neural network 540 .
제1 인공신경망 (520) 은 복수의 골 의료 영상 (510) 각각을 입력으로 각 레이어 (layer) 에 대한 중간 특징 데이터 (intermediate feature map) 를 생성하는 레스넷 (Residual Network, ResNet) 일 수 있으나, 이에 한정되지 않는다. The first artificial neural network 520 may be a Residual Network (ResNet) that generates intermediate feature data (intermediate feature map) for each layer by inputting each of the plurality of bone medical images 510 as an input, However, the present invention is not limited thereto.
제2 인공신경망 (530) 은 상위 레이어에서 하위 레이어 순으로 특정 레이어의 중간 특징 데이터와, 다음 레이어의 중간 특징 데이터를 합 (merging) 하여 각 레이어에 대응하는 특징 데이터를 생성하는 피쳐 피라미드 네트워크 (Feature Pyramid Network, FPN) 일 수 있으나, 이에 한정되지 않는다. 여기서, 특징 데이터에 대응하는 레이어는 레스넷의 각 레이어를 의미할 수 있다. The second artificial neural network 530 is a feature pyramid network that generates feature data corresponding to each layer by merging the intermediate feature data of a specific layer and the intermediate feature data of the next layer in the order of the lower layer from the upper layer. Pyramid Network, FPN), but is not limited thereto. Here, the layer corresponding to the feature data may mean each layer of Resnet.
제3 인공신경망 (540) 은 골절 의심 영역을 포함하도록 바운딩 박스 (bounding box) 를 조정하는 바운딩 박스 회귀 (Regression) 서브넷을 포함할 수 있다. 선택적으로, 제3 인공신경망 (540) 은 바운딩 박스에 포함된 골절 의심 영역의 종류를 예측 (또는 분류) 하는 분류 서브넷 (classification subnet) 을 포함할 수도 있다. 이러한 서브넷들은 병렬적으로 구성될 수 있다. The third artificial neural network 540 may include a bounding box regression subnet that adjusts a bounding box to include a fracture suspected region. Optionally, the third artificial neural network 540 may include a classification subnet for predicting (or classifying) the type of the fracture suspected region included in the bounding box. These subnets can be configured in parallel.
바운딩 박스를 조정하는 동작은 실제 골 의료 영상에서 골절 의심 영역을 나타내는 그라운드-트루스 (ground-truth) 바운딩 박스 (즉, 정답) 와 예측된 바운딩 박스 간의 오프셋 (offset) 을 이용하여 수행될 수 있다. 여기서, 오프셋은 예측된 바운딩 박스 및 정답에 해당하는 그라운드-트루스 바운딩 박스가 서로 중첩 (또는 일치) 되는 정도를 의미할 수 있다. 이를 통해서 예측 모델의 성능이 평가될 수 있다.The operation of adjusting the bounding box may be performed using an offset between a ground-truth bounding box (ie, correct answer) indicating a fracture suspected region in an actual bone medical image and a predicted bounding box. Here, the offset may mean a degree to which the predicted bounding box and the ground-truth bounding box corresponding to the correct answer overlap (or match) each other. Through this, the performance of the predictive model can be evaluated.
이러한 예측 모델을 통해서 출력된 예측 결과 데이터 (550) 는 골절 의심 영역 (522, 554, 556a, 556b 및 558) 각각에 대응하는 바운딩 박스를 포함할 수 있다. 다양한 실시예에서 예측 결과 데이터 (550) 는, 골절 의심 영역에 대한 예측 결과 (예를 들어, 골절, 혈관, 골 접합선, 골 중첩 선 등) 를 더 포함할 수도 있다. 그러나, 이에 제한되는 것은 아니다. 예를 들어, 골절 의심 영역에 대한 분류 결과는 골절 의심 영역을 입력으로 하여 영역의 종류를 출력하도록 구성된 별도의 분류기에 의해 출력될 수 있다. The prediction result data 550 output through the prediction model may include a bounding box corresponding to each of the fracture suspicious regions 522 , 554 , 556a , 556b and 558 . In various embodiments, the prediction result data 550 may further include a prediction result (eg, a fracture, a blood vessel, a bone junction line, a bone overlap line, etc.) for the fracture-suspicious region. However, the present invention is not limited thereto. For example, the classification result for the fracture suspected region may be output by a separate classifier configured to output the type of the region by inputting the suspected fracture region as an input.
이때, RetinaNet기반의 예측 모델 구축을 위한 파라미터로서, 에폭 (Epoch) 은 300, 배치 사이즈 (Batch size) 는 6일 수 있으나, 이에 제한되는 것은 아니다. In this case, as parameters for constructing a RetinaNet-based prediction model, an epoch may be 300 and a batch size may be 6, but is not limited thereto.
한편, 다양한 실시예에서 다각도에서 촬영된 멀티뷰 영상에 대하여 골절 의심 영역을 예측하는 단일 모델이 이용될 수 있다.Meanwhile, in various embodiments, a single model for predicting the suspected fracture region may be used with respect to a multi-view image taken from multiple angles.
도 4를 참조하면, 예측 모델은 정면 의료 영상, 후면 의료 영상, 우측면 의료 영상 및 좌측면 의료 영상이 합쳐진 멀티뷰 의료 영상 (610) 을 입력으로 하여 골절 의심 영역을 예측하도록 학습된 복수의 인공신경망으로 이루어진 MVCNN (Multi-view Convolutional Neural Network) 기반의 단일 예측 모델 (620) 일 수 있다. Referring to FIG. 4 , the prediction model is a plurality of artificial neural networks trained to predict a region suspected of fracture by inputting a multi-view medical image 610 in which a front medical image, a rear medical image, a right side medical image, and a left side medical image are combined. It may be a single prediction model 620 based on a multi-view convolutional neural network (MVCNN) consisting of .
보다 구체적으로, 멀티뷰 의료 영상 (610) 이 입력되면, 멀티뷰 의료 영상 (610) 내의 다각의 의료 영상 (612, 614, 616 및 618) 을 입력으로 하는 복수의 제1 CNN (622) 에 의해 다각의 의료 영상 각각에 대하여 특징맵 (feature map) 이 출력된다. 그 다음, 다각의 의료 영상 각각에 대한 복수의 특징맵은 뷰 풀링 레이어 (View pooling layer)(624) 에 의해 하나의 특징 맵으로 통합 (aggregation) 된다. 그 다음 완벽-연결된 레이어들 (fully-connected layers) 를 포함하는, 단일의 제2 CNN (626) 의 에 의해, 멀티뷰 의료 영상 (610) 내의 골절 의심 영역 (632) 이 결정된다. 즉, 골절 의심 영역 (632) 이 결정된 예측 결과 데이터 (630) 가 제공될 수 있다. More specifically, when the multi-view medical image 610 is input, the multi-view medical images 612 , 614 , 616 and 618 in the multi-view medical image 610 are input by a plurality of first CNNs 622 . A feature map is output for each of the various medical images. Then, a plurality of feature maps for each of the multiple medical images are aggregated into one feature map by a view pooling layer 624 . Then, the suspected fracture region 632 in the multi-view medical image 610 is determined by the single second CNN 626, including fully-connected layers. That is, the prediction result data 630 in which the fracture suspected region 632 is determined may be provided.
한편, MVCNN 기반의 단일 예측 모델 (620) 에 입력되는 의료 영상은 이에 제한되는 것이 아니며, 골절 의심 부위에 대한 3D 영상일 수도 있다. 나아가, 단일 예측 모델 (620) 은 MPCNN(Multi-Planar CNN) 기반의 모델일 수도 있다.Meanwhile, the medical image input to the MVCNN-based single prediction model 620 is not limited thereto, and may be a 3D image of a suspected fracture site. Furthermore, the single prediction model 620 may be a Multi-Planar CNN (MPCNN) based model.
평가: 본 발명의 다양한 실시예에 이용되는 예측 모델의 평가 결과Evaluation: Evaluation result of the predictive model used in various embodiments of the present invention
이하에서는, 도 5a 내지 5e를 참조하여, 본 발명의 다양한 실시예에 이용되는 예측 모델의 성능 평가 결과에 대하여 설명한다. Hereinafter, performance evaluation results of the prediction model used in various embodiments of the present invention will be described with reference to FIGS. 5A to 5E .
본 평가에서는, ResNet-152 알고리즘 기반의 예측 모델에 대한 두개골의 골절 부위 예측 평가가 수행되었으나, 모델의 종류 및 목적 부위는 이에 제한되는 것이 아니다. In this evaluation, the fracture site prediction evaluation of the skull for the prediction model based on the ResNet-152 algorithm was performed, but the type of model and the target site are not limited thereto.
이때, 810 장의 골절 X-레이 영상 및 829 장의 정상의 X-레이 영상으로 학습된 예측 모델에 대하여, 183 장의 X-레이 영상 및 206 장의 정상의 X-레이 영상을 이용한 평가가 수행되었다.At this time, the prediction model trained with 810 fracture X-ray images and 829 normal X-ray images was evaluated using 183 X-ray images and 206 normal X-ray images.
나아가, 280 장의 골절 X-레이 영상 및 280 장의 정상의 X-레이 영상으로 학습된 예측 모델에 대하여, 119 장의 X-레이 영상 및 118 장의 정상의 X-레이 영상을 이용한 평가가 수행되었다.Furthermore, the prediction model trained with 280 fracture X-ray images and 280 normal X-ray images was evaluated using 119 X-ray images and 118 normal X-ray images.
도 5a 내지 5e는 본 발명의 다양한 실시예에 이용되는 예측 모델의 구조를 예시적으로 도시한 것이다.5A to 5E exemplarily show the structure of a prediction model used in various embodiments of the present invention.
먼저, 도 5a의 (a), (b) 및 (c)를 참조하면, 본 발명의 다양한 실시예에 이용되는 예측 모델은, 후면 X-레이 영상, 좌측면 X-레이 영상 및 정면 X-레이 영상 내에서 골절 부위에 대하여 미리 라벨링된 정답 영역 (초록색 박스) 을 포함하도록 골절 의심 영역 (붉은색 박스) 을 예측한 것으로 나타난다.First, referring to (a), (b) and (c) of FIG. 5A , the prediction model used in various embodiments of the present invention is a rear X-ray image, a left side X-ray image, and a front X-ray image. It appears that the fracture suspected area (red box) is predicted to include the correct area (green box) labeled in advance for the fracture site in the image.
특히, 도 5b의 (a), (b) 및 (c)를 더욱 참조하면, 예측 모델의 골절 의심 영역의 예측에 있어서, 관심도를 나타내는 CAM 영상이 도시된다. In particular, referring further to (a), (b) and (c) of FIG. 5B , a CAM image indicating a degree of interest is shown in the prediction of the fracture suspected region of the predictive model.
구체적으로, 예측 모델은, 골절 의심 영역의 예측에 있어서, 실제 골절이 나타난 골절 부위에 높은 관심도를 갖는 것으로 나타난다. Specifically, the prediction model appears to have a high degree of interest in the fracture site where the actual fracture appears in prediction of the fracture suspected region.
도 5c를 참조하면, 예측 모델의 성능 평가 결과가 도시된다. Referring to FIG. 5C , a performance evaluation result of the predictive model is shown.
보다 구체적으로, 예측 모델이 골절 의심 영역을 얼마나 잘 검출 했는지를 나타내는 지표인 리콜 (recall) 값이 0.77로 나타나고, 예측된 골절 의심 영역이 얼마나 정확한지 (즉, 실제 골절 부위에 대응하는 지) 를 나타내는 지표인 정밀도는 0.73으로 나타난다. 도 5d를 함께 참조하면, 평균 정밀도는 0.7131로 나타난다.More specifically, the recall value, which is an indicator of how well the predictive model detected the suspected fracture region, is 0.77, indicating how accurate the predicted fracture region is (that is, it corresponds to the actual fracture site). The index precision is 0.73. Referring to FIG. 5D together, the average precision is shown as 0.7131.
이때, 예측 모델의 성능 평가를 위한, 정답 영역 및 예측 영역의 전체 영역에 대한 중첩 영역으로 정의되는 IOU (도 5e 참조) 는 0.1 이상으로 설정되었고, 확률 임계치는 0.1 이상으로 설정되었다. At this time, for the performance evaluation of the predictive model, the IOU (see FIG. 5e ) defined as the overlapping area for the correct answer area and the entire area of the prediction area was set to 0.1 or more, and the probability threshold was set to 0.1 or more.
즉, 본 평가 결과에 따르면, 본 발명의 다양한 실시예에 이용되는 예측 모델은 복수의 각도에서 촬영된 두개골 X-레이 영상 내에서 골절 부위를 높은 정확도로 찾는 것으로 나타난다.That is, according to the evaluation results, the predictive model used in various embodiments of the present invention appears to find the fracture site with high accuracy in the skull X-ray image taken from a plurality of angles.
이에, 본 발명은 예측 모델 기반의 골절 검출 시스템을 제공함으로써, 육안으로 식별되지 않은 부분들에 대한 정보 제공에 의해 의료진의 추가적인 골 의료 영상 진단의 수행 없이 골절에 대한 민감도 높은 진단 결과를 제공할 수 있다. Accordingly, the present invention provides a fracture detection system based on a predictive model, and by providing information on parts not identified with the naked eye, it is possible to provide a high-sensitivity diagnosis result for a fracture without additional medical imaging of bone by a medical staff. have.
또한, 본 발명은 딥 러닝 알고리즘 기반의 예측 모델을 이용한 골절 검출 시스템을 제공함으로써, 의료진의 잘못된 해석을 방지하고, 실제 임상 실무에 있어서 의료진의 워크 플로우를 향상시킬 수 있다. In addition, the present invention provides a fracture detection system using a predictive model based on a deep learning algorithm, thereby preventing erroneous interpretation by medical personnel and improving the workflow of medical personnel in actual clinical practice.
이상 첨부된 도면을 참조하여 본 발명의 실시 예들을 더욱 상세하게 설명하였으나, 본 발명은 반드시 이러한 실시 예로 국한되는 것은 아니고, 본 발명의 기술사상을 벗어나지 않는 범위 내에서 다양하게 변형 실시될 수 있다. 따라서, 본 발명에 개시된 실시 예들은 본 발명의 기술 사상을 한정하기 위한 것이 아니라 설명하기 위한 것이고, 이러한 실시 예에 의하여 본 발명의 기술 사상의 범위가 한정되는 것은 아니다. 그러므로, 이상에서 기술한 실시 예들은 모든 면에서 예시적인 것이며 한정적이 아닌 것으로 이해해야만 한다. 본 발명의 보호 범위는 아래의 청구범위에 의하여 해석되어야 하며, 그와 동등한 범위 내에 있는 모든 기술 사상은 본 발명의 권리범위에 포함되는 것으로 해석되어야 할 것이다.Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.
[부호의 설명] 100: 골절 검출용 디바이스, 110, 230: 저장부, 120, 210: 통신부, 130, 240: 프로세서, 200: 의료진 디바이스, 220: 표시부, 300: 의료 영상 제공용 디바이스[Description of symbols] 100: fracture detection device, 110, 230: storage unit, 120, 210: communication unit, 130, 240: processor, 200: medical staff device, 220: display unit, 300: medical image providing device
[이 발명을 지원한 국가연구개발사업] [National R&D project supporting this invention]
[과제고유번호] 1711103215, [과제번호] 2019-0-01750-002, [부처명] 과학기술정보통신부, [과제관리(전문)기관명] 정보통신기획평가원, [연구사업명] ICT 신시장창출 지원 사업, [연구과제명] (2단계 : 기술개발) 딥러닝 기법을 이용한 최적 사지압박 심혈관 치료기기 개발, [기여율] 1/2, [과제수행기관명] 주식회사 오스테오시스, [연구기간] 2019.07.01 ~ 2020.12.31[Project No.] 1711103215, [Project No.] 2019-0-01750-002, [Ministry] Ministry of Science and ICT, [Project management (specialized) institution name] Information and Communication Planning and Evaluation Institute, [Research project name] ICT new market creation support project, [Research project name] (Step 2: Technology development) Development of optimal limb compression cardiovascular treatment device using deep learning technique, [Contribution rate] 1/2, [Project name] Osteosys Co., Ltd., [Research period] 2019.07.01 ~ 2020.12 .31
[과제고유번호] 1711117058, [과제번호] 2020-0-00161-001, [부처명] 과학기술정보통신부, [과제관리(전문)기관명] 정보통신기획평가원, [연구사업명] 딥러닝 고도화 핵심기술개발, [연구과제명] 수술동영상 데이터 대상 오픈셋 학습 기반 능동기계학습 기술 개발, [기여율] 1/2, [과제수행기관명] 주식회사 엠티이지, [연구기간] 2020.04.01 ~ 2020.12.31[Project No.] 1711117058, [Project No.] 2020-0-00161-001, [Ministry] Ministry of Science, Technology and Information and Communication, [Name of project management (specialized) institution] Information and Communication Planning and Evaluation Institute, [Research project name] Deep learning advancement core technology development , [Research project name] Development of active machine learning technology based on open set learning for surgical video data, [Contribution rate] 1/2, [Project name of institution] MTG Co., Ltd., [Research period] 2020.04.01 ~ 2020.12.31

Claims (18)

  1. 프로세서에 의해 구현되는 골절 검출 방법으로,A fracture detection method implemented by a processor, comprising:
    개체에 대한 골 의료 영상을 수신하는 단계;Receiving a bone medical image of the subject;
    상기 골 의료 영상을 입력으로 하여 골절 부위를 출력하도록 구성된 예측 모델을 이용하여, 상기 골 의료 영상에 대한 골절 의심 영역을 결정하는 단계;determining a fracture suspected region for the medical bone image by using a prediction model configured to output a fracture site by receiving the medical bone image as an input;
    상기 골절 의심 영역 중 유사 골절 영역을 결정하는 단계, 및determining a similar fracture region among the fracture suspect regions; and
    상기 유사 골절 영역이 제거되도록 상기 골절 의심 영역을 필터링하여 골절 부위를 결정하는 단계를 포함하는, 골절 검출 방법.and determining a fracture site by filtering the suspected fracture region so that the similar fracture region is removed.
  2. 제1항에 있어서,According to claim 1,
    상기 골 의료 영상은,The bone medical image is
    상기 개체의 목적 부위에 대하여 복수의 각도에서 촬영한 복수의 골 의료 영상을 포함하고, Including a plurality of bone medical images taken from a plurality of angles with respect to the target site of the subject,
    상기 골절 의심 영역을 결정하는 단계는,Determining the fracture suspected region comprises:
    상기 예측 모델을 이용하여 상기 복수의 골 의료 영상 각각에 대한 상기 골절 의심 영역을 결정하는 단계를 포함하고,Comprising the step of determining the fracture suspected region for each of the plurality of bone medical images by using the prediction model,
    상기 유사 골절 영역을 결정하는 단계는,Determining the similar fracture region comprises:
    상기 복수의 골 의료 영상 각각에 대한 상기 골절 의심 영역 중, 상기 유사 골절 영역을 결정하는 단계를 포함하고,Determining the similar fracture region among the fracture suspected regions for each of the plurality of bone medical images,
    상기 골절 부위를 결정하는 단계는,Determining the fracture site comprises:
    상기 복수의 골 의료 영상 각각에 대하여 골절 영역을 결정하는 단계를 포함하는, 골절 검출 방법.Comprising the step of determining a fracture region for each of the plurality of bone medical images, fracture detection method.
  3. 제2항에 있어서,3. The method of claim 2,
    상기 예측 모델은,The predictive model is
    상기 복수의 골 의료 영상 각각에 대하여 골절 부위를 예측하도록 구성된 복수의 예측 모델이고,A plurality of prediction models configured to predict a fracture site for each of the plurality of bone medical images,
    상기 골절 의심 영역을 결정하는 단계는,Determining the fracture suspected region comprises:
    상기 복수의 예측 모델 각각을 이용하여, 상기 복수의 골 의료 영상 각각에 대한 상기 골절 의심 영역을 결정하는 단계를 포함하는, 골절 검출 방법. Using each of the plurality of predictive models, comprising the step of determining the fracture suspicious region for each of the plurality of bone medical images, fracture detection method.
  4. 제2항에 있어서,3. The method of claim 2,
    상기 예측 모델은,The predictive model is
    상기 복수의 골 의료 영상에 대하여 골절 부위를 예측하도록 구성된 단일의 예측 모델이고,It is a single prediction model configured to predict a fracture site with respect to the plurality of bone medical images,
    상기 골절 의심 영역을 결정하는 단계는,Determining the fracture suspected region comprises:
    상기 단일의 예측 모델을 이용하여, 상기 복수의 골 의료 영상에 대한 상기 골절 의심 영역을 결정하는 단계를 포함하는, 골절 검출 방법.Using the single prediction model, comprising the step of determining the fracture suspected region for the plurality of bone medical images, fracture detection method.
  5. 제2항에 있어서,3. The method of claim 2,
    상기 복수의 골 의료 영상은,The plurality of bone medical images,
    정면 골 의료 영상, 후면 골 의료 영상, 우측 골 의료 영상 및 좌측 골 의료 영상 중 적어도 두 개인, 골절 검출 방법.at least two of a frontal bone medical image, a posterior bone medical image, a right bone medical image and a left bone medical image, a fracture detection method.
  6. 제2항에 있어서,3. The method of claim 2,
    상기 골절 부위를 결정하는 단계 이후에,After determining the fracture site,
    상기 복수의 골 의료 영상 중 선택된 세트 영상의 골절 부위를 비교하는 단계, 및comparing a fracture site of a set image selected from among the plurality of bone medical images; and
    비교 결과에 기초하여 골절 부위를 최종 결정하는 단계를 더 포함하고,Further comprising the step of finally determining the fracture site based on the comparison result,
    상기 세트 영상은, 정면 골 의료 영상 및 후면 골 의료 영상, 또는 우측 골 의료 영상 및 좌측 골 의료 영상인, 골절 검출 방법.The set image is a frontal bone medical image and a posterior bone medical image, or a right bone medical image and a left bone medical image.
  7. 제6항에 있어서,7. The method of claim 6,
    상기 골절 부위를 비교하는 단계는,The step of comparing the fracture site is,
    상기 세트 영상에 대하여 결정된 골절 부위의 위치를 비교하는 단계를 포함하는, 골절 검출 방법.Comprising the step of comparing the determined location of the fracture site with respect to the set image, fracture detection method.
  8. 제1항에 있어서,According to claim 1,
    상기 골절은 선형 골절이고,the fracture is a linear fracture,
    상기 유사 골절 영역을 결정하는 단계는,Determining the similar fracture region comprises:
    상기 골절 의심 영역의 두께, 형태 및 위치 중 적어도 하나에 기초하여, 상기 유사 골절 영역을 결정하는 단계를 포함하는, 골절 검출 방법.Based on at least one of a thickness, a shape, and a location of the fracture suspected region, comprising the step of determining the similar fracture region, fracture detection method.
  9. 제1항에 있어서,According to claim 1,
    상기 유사 골절 영역은,The pseudo fracture region is
    골 접합선, 혈관 및 골 중첩 선 중 적어도 하나인, 골절 검출 방법.At least one of a bone junction line, a blood vessel, and a bone overlap line, a method for detecting a fracture.
  10. 제1항에 있어서,According to claim 1,
    상기 골 의료 영상은,The bone medical image is
    X-레이 영상, 컴퓨터 단층 촬영 영상, 자기 공명 영상 및 초음파 영상 중 하나인, 골절 검출 방법.A method for detecting a fracture, which is one of an X-ray image, a computed tomography image, a magnetic resonance image, and an ultrasound image.
  11. 제1항에 있어서,According to claim 1,
    상기 골절 의심 영역은, The fracture suspected area is
    두개골, 하악골, 설골, 경추, 흉추, 요추, 늑골, 흉골, 쇄골, 견갑골, 상완골, 요골, 척골, 주상골, 월상골, 삼각골, 두상골, 대능형골, 소농형골, 유두골, 유두골, 중수골, 손가락뼈, 관골, 대퇴골, 슬개골, 경골, 비골, 거골, 종골, 주상골, 입방골, 쐐기뼈, 증족골 및 발가락뼈 중 적어도 하나에 대한 골절 의심 영역인, 골절 검출 방법.skull, mandible, hyoid, cervical, thoracic, lumbar, rib, sternum, clavicle, scapula, humerus, radius, ulna, navicular, lunate, triangular bone, cephalothorax, croquette, scapula, papillary, papillary, A method for detecting a fracture, which is an area suspected of fracture of at least one of metacarpal, finger bone, zygomatic bone, femur, patella, tibia, fibula, talus, calcaneus, navicular, cuboid, sphenoid, metatarsal, and toe bones.
  12. 개체에 대한 골 의료 영상을 수신하도록 구성된 통신부, 및a communication unit configured to receive a bone medical image of the subject; and
    상기 통신부와 통신하도록 연결된 프로세서를 포함하고,a processor connected to communicate with the communication unit;
    상기 프로세서는, The processor is
    상기 골 의료 영상을 입력으로 하여 골절 부위를 출력하도록 구성된 예측 모델을 이용하여, 상기 골 의료 영상에 대한 골절 의심 영역을 결정하고,Using a predictive model configured to output a fracture site by inputting the bone medical image as an input, determine a fracture suspected region for the bone medical image,
    상기 골절 의심 영역 중 유사 골절 영역을 결정하고,determining a similar fracture area among the fracture suspected areas;
    상기 유사 골절 영역이 제거되도록 상기 골절 의심 영역을 필터링하여 골절 부위를 결정하도록 구성된, 골절 검출용 디바이스.and filter the suspected fracture region to determine a fracture site so that the similar fracture region is removed.
  13. 제12항에 있어서,13. The method of claim 12,
    상기 골 의료 영상은,The bone medical image is
    상기 개체의 목적 부위에 대하여 복수의 각도에서 촬영한 복수의 골 의료 영상을 포함하고, Including a plurality of bone medical images taken from a plurality of angles with respect to the target site of the subject,
    상기 프로세서는, The processor is
    상기 예측 모델을 이용하여 상기 복수의 골 의료 영상 각각에 대한 상기 골절 의심 영역을 결정하고,Determining the fracture suspected region for each of the plurality of bone medical images using the predictive model,
    상기 복수의 골 의료 영상 각각에 대한 상기 골절 의심 영역 중, 상기 유사 골절 영역을 결정하고,determining the similar fracture region among the fracture suspected regions for each of the plurality of bone medical images;
    상기 복수의 골 의료 영상 각각에 대하여 골절 영역을 결정하도록 더 구성된, 골절 검출용 디바이스.The device for detecting a fracture, further configured to determine a fracture region for each of the plurality of bone medical images.
  14. 제13항에 있어서,14. The method of claim 13,
    상기 예측 모델은,The predictive model is
    상기 복수의 골 의료 영상 각각에 대하여 골절 부위를 예측하도록 구성된 복수의 예측 모델이고,A plurality of prediction models configured to predict a fracture site for each of the plurality of bone medical images,
    상기 프로세서는, The processor is
    상기 복수의 예측 모델 각각을 이용하여, 상기 복수의 골 의료 영상 각각에 대한 상기 골절 의심 영역을 결정하도록 구성된, 골절 검출용 디바이스.A device for detecting a fracture, configured to determine the fracture suspected region for each of the plurality of bone medical images by using each of the plurality of predictive models.
  15. 제13항에 있어서,14. The method of claim 13,
    상기 예측 모델은,The predictive model is
    상기 복수의 골 의료 영상에 대하여 골절 부위를 예측하도록 구성된 단일의 예측 모델이고,It is a single prediction model configured to predict a fracture site with respect to the plurality of bone medical images,
    상기 프로세서는,The processor is
    상기 단일의 예측 모델을 이용하여, 상기 복수의 골 의료 영상에 대한 상기 골절 의심 영역을 결정하도록 구성된, 골절 검출용 디바이스.The device for detecting a fracture, configured to determine the fracture suspect region for the plurality of bone medical images by using the single predictive model.
  16. 제13항에 있어서,14. The method of claim 13,
    상기 복수의 골 의료 영상은,The plurality of bone medical images,
    정면 골 의료 영상, 후면 골 의료 영상, 우측 골 의료 영상 및 좌측 골 의료 영상 중 적어도 두 개인, 골절 검출용 디바이스.A device for detecting a fracture, comprising at least two of a frontal bone medical image, a posterior bone medical image, a right bone medical image, and a left bone medical image.
  17. 제13항에 있어서,14. The method of claim 13,
    상기 프로세서는, The processor is
    상기 복수의 골 의료 영상 중 선택된 세트 영상의 골절 부위를 비교하고,comparing the fracture site of a set image selected from among the plurality of bone medical images,
    비교 결과에 기초하여 골절 부위를 최종 결정하도록 더 구성되고,further configured to finally determine a fracture site based on the comparison result,
    상기 세트 영상은, 정면 골 의료 영상 및 후면 골 의료 영상, 또는 우측 골 의료 영상 및 좌측 골 의료 영상인, 골절 검출용 디바이스.The set image is a medical image of a frontal bone and a medical image of a posterior bone, or a medical image of a right bone and a medical image of a left bone, a device for detecting a fracture.
  18. 제17항에 있어서,18. The method of claim 17,
    상기 프로세서는,The processor is
    상기 세트 영상에 대하여 결정된 골절 부위의 위치를 비교하도록 더 구성된, 골절 검출용 디바이스.The device for detecting a fracture, further configured to compare the determined location of the fracture site with respect to the set image.
PCT/KR2021/018314 2020-12-22 2021-12-06 Fracture detecting method and device using same WO2022139246A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020200181117A KR102530016B1 (en) 2020-12-22 2020-12-22 Method for detecting fracture and device using the same
KR10-2020-0181117 2020-12-22

Publications (1)

Publication Number Publication Date
WO2022139246A1 true WO2022139246A1 (en) 2022-06-30

Family

ID=82159506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2021/018314 WO2022139246A1 (en) 2020-12-22 2021-12-06 Fracture detecting method and device using same

Country Status (2)

Country Link
KR (1) KR102530016B1 (en)
WO (1) WO2022139246A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117679160A (en) * 2023-12-26 2024-03-12 北京长木谷医疗科技股份有限公司 Method, device, equipment and readable storage medium for reducing wound fracture

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120143037A1 (en) * 2009-04-07 2012-06-07 Kayvan Najarian Accurate Pelvic Fracture Detection for X-Ray and CT Images
KR20180040287A (en) * 2016-10-12 2018-04-20 (주)헬스허브 System for interpreting medical images through machine learnings
KR101981202B1 (en) * 2018-12-11 2019-05-22 메디컬아이피 주식회사 Method and apparatus for reconstructing medical image
KR102119057B1 (en) * 2019-10-08 2020-06-29 (주)제이엘케이 Method for learning fracture diagnosis model and apparatus for the same
KR20200080626A (en) * 2018-12-27 2020-07-07 연세대학교 산학협력단 Method for providing information of lesion diagnosis and device for providing information of lesion diagnosis using the same

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108305248B (en) 2018-01-17 2020-05-29 慧影医疗科技（北京）有限公司 Construction method and application of fracture recognition model
LU101008B1 (en) * 2018-11-26 2020-05-26 Metamorphosis Gmbh Bone fracture detection and classification
KR102179090B1 (en) * 2020-01-30 2020-11-16 주식회사 투비코 Method for medical diagnosis by using neural network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120143037A1 (en) * 2009-04-07 2012-06-07 Kayvan Najarian Accurate Pelvic Fracture Detection for X-Ray and CT Images
KR20180040287A (en) * 2016-10-12 2018-04-20 (주)헬스허브 System for interpreting medical images through machine learnings
KR101981202B1 (en) * 2018-12-11 2019-05-22 메디컬아이피 주식회사 Method and apparatus for reconstructing medical image
KR20200080626A (en) * 2018-12-27 2020-07-07 연세대학교 산학협력단 Method for providing information of lesion diagnosis and device for providing information of lesion diagnosis using the same
KR102119057B1 (en) * 2019-10-08 2020-06-29 (주)제이엘케이 Method for learning fracture diagnosis model and apparatus for the same

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117679160A (en) * 2023-12-26 2024-03-12 北京长木谷医疗科技股份有限公司 Method, device, equipment and readable storage medium for reducing wound fracture

Also Published As

Publication number Publication date
KR102530016B1 (en) 2023-05-08
KR20220090172A (en) 2022-06-29

Similar Documents

Publication Publication Date Title
Thian et al. Convolutional neural networks for automated fracture detection and localization on wrist radiographs
WO2019132168A1 (en) System for learning surgical image data
WO2021049729A1 (en) Method for predicting likelihood of developing lung cancer by using artificial intelligence model, and analysis device therefor
TWI701680B (en) Method and system of analyzing medical images
WO2016126056A1 (en) Medical information providing apparatus and medical information providing method
WO2019054576A1 (en) Method and apparatus for fully automated segmentation of image of joint, based on patient-specific optimal thresholding method and watershed algorithm
WO2018080120A1 (en) Method and apparatus for follicular quantification in 3d ultrasound images
WO2017051944A1 (en) Method for increasing reading efficiency by using gaze information of user in medical image reading process and apparatus therefor
WO2007000940A1 (en) Abnormal shadow candidate detecting method, abnormal shadow candidate detecting device
WO2022131642A1 (en) Apparatus and method for determining disease severity on basis of medical images
WO2021261808A1 (en) Method for displaying lesion reading result
WO2022139246A1 (en) Fracture detecting method and device using same
JP2007151645A (en) Medical diagnostic imaging support system
WO2014200230A1 (en) Method and apparatus for image registration
WO2022124705A1 (en) Apparatus and method for providing medical image-based hologram
Mielnik et al. A novel method of synovitis stratification in ultrasound using machine learning algorithms: results from clinical validation of the MEDUSA project
WO2021206518A1 (en) Method and system for analyzing surgical procedure after surgery
Kumar et al. IOT enabled Intelligent featured imaging Bone Fractured Detection System
CN111061811B (en) Health big data management system based on block chain and cloud service
WO2020212962A2 (en) Foot type evaluation method and foot type evaluation device using same
WO2022050713A1 (en) Method for reading chest image
CN108710901B (en) Deep learning-based spine deformity screening system and method
KR20230013717A (en) Convolutional neural network based 3D detection device for bone fracture regions and its method
Singh et al. A real-time analytic face thermal recognition system integrated with email notification
CN109919935B (en) Head and neck blood vessel segmentation method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21911313

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21911313

Country of ref document: EP

Kind code of ref document: A1