KR102595012B1 - Apparatus and method for tracking hyoid bone - Google Patents

Abstract

A hyoid bone tracking device according to one aspect includes an image acquisition unit that acquires videofuoroscopic swallowing study (VFSS) images; and a processor that determines the relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images. may include.

Description

Apparatus and method for tracking hyoid bone}

Relates to a device and method for tracking the movement of the hyoid bone during swallowing.

Swallowing difficulty is a common major health problem with a prevalence of 15% to 22% in the elderly population. It can occur in patients with several neurological disorders, such as dementia, stroke, and Parkinson's disease, but it can also occur as a normal age-related change. Difficulty swallowing in older adults can lead to serious complications such as dehydration, malnutrition, and aspiration pneumonia, leading to increased hospitalization and mortality.

The standard test to evaluate swallowing function is the videofuoroscopic swallowing study (VFSS). VFSS allows visualization of swallowing-related anatomy and dynamics of the entire swallowing process, including the oral, pharyngeal, and esophageal stages. Quantitative kinematic analysis of swallowing difficulties is performed based on VFSS images, which contain information about the anatomical and dynamic characteristics of swallowing. However, kinematic analysis of swallowing requires a labor-intensive and time-consuming process for manual marking of swallowing structures, limiting its clinical utility and applicability.

Republic of Korea Patent Publication No. 10-2020-0058657

The purpose is to provide a hyoid bone tracking device and method.

A hyoid bone tracking device according to one aspect includes an image acquisition unit that acquires VFSS (videofuoroscopic swallowing study) images; and a processor that determines the relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images. may include.

The processor includes a hyoid bone tracking unit that extracts a region of interest including the hyoid bone from each VFSS image using a hyoid bone tracking model; a hyoid bone division unit that divides the hyoid bone in each extracted region of interest using a hyoid bone segmentation model; a specific point determination unit that determines a specific point of each of the divided hyoid bones; A cervical spine segmentation unit that segments the cervical spine in each VFSS image using a cervical spine segmentation model; a coordinate system generator that generates a coordinate system based on the cervical vertebrae based on the divided cervical vertebrae; and a trajectory determination unit that determines a relative trajectory of the hyoid bone with respect to the cervical spine based on the specific point of each hyoid bone and the generated coordinate system. may include.

The hyoid bone tracking unit may track the hyoid bone in the VFSS images using the hyoid bone tracking model and extract the region of interest from each VFSS image based on the hyoid bone tracking result.

The hyoid tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model may be generated in advance through machine learning or deep learning.

The specific point may include a center point, most anterior point, most posterior point, most superior point, and most inferior point.

The specific point determination unit may determine the specific point of each hyoid bone by shape matching each hyoid bone to a reference hyoid image showing the specific point of the hyoid bone.

The reference hyoid bone image may be an image of the hyoid bone divided from the first frame image among the VFSS images.

The coordinate system generator may determine two anatomical points in the divided cervical spine and generate the coordinate system including a first axis connecting the two anatomical points and a second axis perpendicular to the first axis. there is.

The trajectory determination unit may determine a relative trajectory of the hyoid bone with respect to the cervical vertebrae by displaying a specific point of each hyoid bone in the coordinate system.

The processor may include a model updating unit that updates at least one of the hyoid bone tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model; may further include.

A hyoid bone tracking method according to another aspect includes acquiring videofuoroscopic swallowing study (VFSS) images; and determining the relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images; may include.

The step of determining the relative trajectory of the hyoid bone with respect to the cervical spine includes extracting a region of interest including the hyoid bone from each VFSS image using a hyoid bone tracking model; segmenting the hyoid bone in each extracted region of interest using a hyoid bone segmentation model; determining a specific point of each segmented hyoid bone; Segmenting the cervical spine in each VFSS image using a cervical spine segmentation model; Creating a coordinate system based on the cervical vertebrae based on the divided cervical vertebrae; and determining a relative trajectory of the hyoid bone with respect to the cervical spine based on the specific point of each hyoid bone and the generated coordinate system. may include.

In the step of extracting the region of interest, the hyoid bone may be tracked in the VFSS images using the hyoid bone tracking model, and the region of interest may be extracted from each VFSS image based on the hyoid bone tracking result.

The hyoid tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model may be generated in advance through machine learning or deep learning.

The specific point may include a center point, most anterior point, most posterior point, most superior point, and most inferior point.

In the step of determining the specific point of each hyoid bone, the specific point of each hyoid bone can be determined by shape matching each hyoid bone to a reference hyoid image showing the specific point of the hyoid bone.

The reference hyoid bone image may be an image of the hyoid bone divided from the first frame image among the VFSS images.

The step of generating the coordinate system includes determining two anatomical points in the divided cervical spine, and creating the coordinate system including a first axis connecting the two anatomical points and a second axis perpendicular to the first axis. can be created.

In the step of determining the relative trajectory of the hyoid bone with respect to the cervical vertebrae, the relative trajectory of the hyoid bone with respect to the cervical vertebrae may be determined by displaying a specific point of each hyoid bone in the coordinate system.

The hyoid bone tracking method includes updating at least one of the hyoid bone tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model; may further include.

By determining the relative trajectory of the hyoid bone with respect to the cervical spine from VFSS images through machine learning or deep learning, the high-precision trajectory of the hyoid bone during swallowing can be automatically obtained.

Figure 1 is a diagram illustrating a hyoid bone tracking device according to an exemplary embodiment.
Fig. 2 is a diagram illustrating a processor according to an example embodiment.
Figure 3 is an example diagram for explaining the process of determining the hyoid bone trajectory.
Figure 4 is a diagram illustrating a hyoid bone tracking device according to an exemplary embodiment.
Figure 5 is a diagram illustrating a hyoid bone tracking method according to an exemplary embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Meanwhile, in each step, unless a specific order is clearly stated in the context, each step may occur in a different order from the specified order. That is, each step may be performed in the same order as specified, may be performed substantially simultaneously, or may be performed in the opposite order.

The terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. Terms are used only to distinguish one component from another. Singular expressions include plural expressions unless the context clearly indicates otherwise, and terms such as 'include' or 'have' refer to the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. It is intended to specify that something exists, but it should be understood as not precluding the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the division of components in this specification is merely a division according to the main function each component is responsible for. That is, two or more components may be combined into one component, or one component may be divided into two or more components for more detailed functions. In addition to the main functions that each component is responsible for, each component may additionally perform some or all of the functions that other components are responsible for, and some of the main functions that each component is responsible for may be performed by other components. It may also be carried out. Each component may be implemented as hardware or software, or as a combination of hardware and software.

FIG. 1 is a diagram illustrating a hyoid bone tracking device according to an exemplary embodiment, FIG. 2 is a diagram illustrating a processor according to an exemplary embodiment, and FIG. 3 is an exemplary diagram for explaining a process of determining a hyoid bone trajectory. .

The hyoid bone tracking device 100 according to an exemplary embodiment is a device that can track the hyoid bone of a subject and determine the trajectory of the hyoid bone from videofluoroscopic swallowing study (VFSS) images, and is mounted on an electronic device or It can be wrapped in a housing and formed as a separate device. Electronic devices include medical imaging devices, electronic medical devices, desktop computers, mobile phones, smartphones, tablets, laptops, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation devices, MP3 players, digital cameras, wearable devices, etc. It may include, but is not limited to this. Here, the VFSS image may be an image in which the subject's food swallowing process is captured using radiation.

Referring to FIGS. 1 to 3 , the hyoid bone tracking device 100 may include an image acquisition unit 110 and a processor 120.

The image acquisition unit 110 may acquire VFSS images of the subject.

For example, the image acquisition unit 110 includes a radiography device and can acquire VFSS images by directly taking VFSS images of the subject using the radiography device.

As another example, the image acquisition unit 110 may acquire VFSS images by receiving VFSS images of the subject from an external device (eg, a radiography device) that captures and/or stores VFSS images. At this time, the image acquisition unit 110 may use wired or wireless communication technology. Here, wireless communication technologies include Bluetooth communication, BLE (Bluetooth Low Energy) communication, Near Field Communication (NFC), WLAN communication, Zigbee communication, Infrared Data Association (IrDA) communication, and WFD. It may include (Wi-Fi Direct) communication, UWB (ultra-wideband) communication, Ant+ communication, WIFI communication, RFID (Radio Frequency Identification) communication, 3G communication, 4G communication, and 5G communication, but is not limited thereto.

The processor 120 may determine the relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images. For this purpose, the processor 120 includes a hyoid bone tracking unit 210, a hyoid bone division unit 220, a specific point determination unit 230, a cervical spine division unit 240, a coordinate system generation unit 250, and a trajectory determination unit 260. It can be included.

The hyoid bone tracking unit 210 can track the hyoid bone in the VFSS images 310 using the hyoid bone tracking model and extract a region of interest 320 including the hyoid bone from each VFSS image 310 based on the hyoid bone tracking result. there is. Here, the hyoid bone tracking model can be created in advance through machine learning or deep learning based on VFSS images for learning that display a region of interest including the hyoid bone. At this time, the machine learning model or deep learning model used for learning is SVM (Support Vector Machine), R-CNN (Region Based Convolutional Neural Networks), Fast R-CNN, Faster R-CNN, cascade R-CNN, and SSD (Single Shot MultiBox Detector), YOLO, AttentionNet, SPP-Net, RefineDet, Retina-Net, MDNet (Multi-Domain Networks), SDNet (Single-Domain Networks), Deformable convolutional networks, etc., but are not limited to these.

The hyoid bone segmentation unit 220 may segment the hyoid bone 330 in the region of interest 320 of each VFSS image using the hyoid bone segmentation model. Here, the hyoid bone segmentation model can be created in advance through machine learning or deep learning based on learning hyoid bone images into which the hyoid bone has been segmented. At this time, machine learning models or deep learning models used for learning may include, but are not limited to, FCN (Fully Convolutional Network), DeepLab, Attention U-Net, ReSeg, CRF (Conditional Random Field), etc.

The specific point determination unit 230 may determine the specific point 340 of each divided hyoid bone 330. For example, the specific point determination unit 230 may determine the specific point 340 of each hyoid bone 330 by shape matching each hyoid bone 330 to a reference hyoid image showing the specific point of the hyoid bone 330. At this time, the reference hyoid image may be a hyoid image segmented from the first frame image among VFSS images, and the specific point may be any point within the hyoid bone, for example, center point, most anterior point, most posterior point, most superior point, most inferior. It may be a point, etc.

The cervical spine segmentation unit 240 may segment the cervical spine 350 in each VFSS image 310 using a cervical spine segmentation model. Here, the cervical spine segmentation model can be created in advance through machine learning or deep learning based on VFSS images for learning in which the cervical spine has been segmented. At this time, machine learning models or deep learning models used for learning may include, but are not limited to, FCN (Fully Convolutional Network), DeepLab, Attention U-Net, ReSeg, CRF (Conditional Random Field), etc.

The coordinate system generator 250 may generate a coordinate system 360 based on the cervical vertebrae based on the divided cervical vertebrae 350. For example, the coordinate system generator 250 determines two anatomical points in the divided cervical spine 350 and includes a first axis connecting the two anatomical points and a second axis perpendicular to the first axis. A new coordinate system 360 can be created. Here, the anatomical points may include the second cervical vertebra (C2), the third cervical vertebra (C3), the fourth cervical vertebra (C4), and the fifth cervical vertebra (C5). And the origin of the coordinate system 360 may be either one of two anatomical points or an arbitrary point on the first axis.

The trajectory determination unit 260 is based on the specific point 340 of each hyoid bone determined by the specific point determination unit 230 and the coordinate system 360 generated by the coordinate system generator 250. 370) can be judged. For example, the trajectory determination unit 260 may display a specific point 340 of each hyoid bone on the coordinate system 360 to determine the relative trajectory 370 of the hyoid bone with respect to the cervical spine.

According to an exemplary embodiment, the processor 120 may further include a model update unit 270.

The model update unit 270 may update the hyoid bone tracking model, the hyoid bone split model, and the cervical spine split model.

For example, the model update unit 270 may update the hyoid bone tracking model in real time by using the hyoid bone tracking result of the hyoid bone tracking unit 210 as additional learning data. For example, the model update unit 270 may update the hyoid bone tracking model in real time by additionally learning the hyoid bone tracking result of the previous frame in the process of tracking the hyoid bone from the VFSS images 310 according to a set cycle. In this case, the hyoid bone tracking unit 210 can use the hyoid bone tracking model updated in real time through additional learning to track the hyoid bone in the next frame, thereby improving the accuracy of hyoid bone tracking.

For another example, the model update unit 270 periodically collects hyoid bone images for additional learning in which the hyoid bone is segmented or VFSS images for additional learning in which the cervical spine is segmented, and collects the collected hyoid bone images for additional learning or additional learning images in which the cervical spine is segmented. The hyoid bone segmentation model or cervical spine segmentation model can be updated based on VFSS images.

Figure 4 is a diagram illustrating a hyoid bone tracking device according to an exemplary embodiment.

Referring to FIG. 4, the hyoid bone tracking device 400 according to an exemplary embodiment includes an image acquisition unit 110, a processor 120, an input unit 410, a storage unit 420, a communication unit 430, and an output unit ( 440) may be included. Here, since the image acquisition unit 110 and the processor 120 are the same as described above with reference to FIGS. 1 to 3, their detailed description will be omitted.

The input unit 410 can receive various manipulation signals and information from the user. According to one embodiment, the input unit 410 includes a key pad, dome switch, touch pad, jog wheel, jog switch, and H/W button. It may include etc. In particular, when the touch pad forms a layered structure with the display, it can be called a touch screen.

The storage unit 420 may store programs or commands for operating the hyoid bone tracking device 400 and may store data input to the hyoid bone tracking device 400 and processed data. Additionally, the storage unit 420 may store a hyoid bone tracking model, a hyoid bone split model, a cervical vertebra split model, etc.

The storage unit 420 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and RAM. (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), PROM (Programmable Read Only Memory), magnetic memory, magnetic disk, optical disk. It may include at least one type of storage medium, etc. Additionally, the hyoid bone tracking device 400 may operate an external storage medium such as web storage that performs the storage function of the storage unit 420 on the Internet.

The communication unit 430 can communicate with an external device. For example, the communication unit 430 may transmit data input to the hyoid bone tracking device 400, stored data, and processed data to an external device, or may receive various data for tracking the hyoid bone from an external device.

At this time, the external device may be a medical device that uses data input to the hyoid bone tracking device 400, stored data, processed data, etc., or a printer or display device for outputting results. In addition, external devices may include, but are limited to, digital TVs, desktop computers, mobile phones, smart phones, tablets, laptops, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation devices, MP3 players, digital cameras, and wearable devices. It doesn't work.

The communication unit 430 can communicate with an external device using wired or wireless communication technology. At this time, wireless communication technologies include Bluetooth communication, BLE (Bluetooth Low Energy) communication, Near Field Communication (NFC), WLAN communication, Zigbee communication, Infrared Data Association (IrDA) communication, and WFD. (Wi-Fi Direct) communication, UWB (ultra-wideband) communication, Ant+ communication, WIFI communication, RFID (Radio Frequency Identification) communication, 3G communication, 4G communication, and 5G communication, etc., but this is only an example. , but is not limited to this.

The output unit 440 may output data input to the hyoid bone tracking device 400, stored data, processed data, etc. According to one embodiment, the output unit 440 may output the hyoid bone tracking result, hyoid bone segmentation result, cervical spine segmentation result, hyoid bone trajectory, etc. in at least one of an auditory method, a visual method, and a tactile method. To this end, the output unit 440 may include a display, speaker, vibrator, etc.

Figure 5 is a diagram illustrating a hyoid bone tracking method according to an exemplary embodiment.

The hyoid bone tracking method of FIG. 5 may be performed by the hyoid bone tracking devices 100 and 400 of FIG. 1 or FIG. 4 .

Referring to FIG. 5, the hyoid bone tracking device may acquire VFSS images of the subject (510).

For example, the hyoid bone tracking device includes a radiography device, and can acquire VFSS images by directly taking VFSS images of the subject using the radiography device. As another example, the hyoid bone tracking device may obtain VFSS images by receiving VFSS images of the subject from an external device (eg, a radiography device) that captures and/or stores VFSS images.

The hyoid bone tracking device may determine the relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images (520).

Specifically, the hyoid bone tracking device can track the hyoid bone in VFSS images using the hyoid bone tracking model and extract a region of interest including the hyoid bone from each VFSS image based on the hyoid bone tracking result (521). Here, the hyoid bone tracking model can be created in advance through machine learning or deep learning based on VFSS images for learning that display a region of interest including the hyoid bone.

The hyoid bone tracking device can segment the hyoid bone in the region of interest of each VFSS image using the hyoid bone segmentation model (522). Here, the hyoid bone segmentation model can be created in advance through machine learning or deep learning based on learning hyoid bone images into which the hyoid bone has been segmented.

The hyoid bone tracking device can determine a specific point of each divided hyoid bone (523). For example, the hyoid bone tracking device can determine the specific point of each hyoid bone by shape-matching each segmented hyoid bone to a reference hyoid image showing a specific point of the hyoid bone. At this time, the reference hyoid image may be a hyoid image segmented from the first frame image among VFSS images, and the specific point may be any point within the hyoid bone, for example, center point, most anterior point, most posterior point, most superior point, most inferior. It may be a point, etc.

The hyoid tracking device can segment the cervical spine in each VFSS image using the cervical spine segmentation model (524). Here, the cervical spine segmentation model can be created in advance through machine learning or deep learning based on VFSS images for learning in which the cervical spine has been segmented.

The hyoid bone tracking device can generate a coordinate system based on the cervical vertebrae based on the segmented cervical vertebrae (525). For example, a hyoid tracking device may determine two anatomical points in a segmented cervical spine and create a new coordinate system that includes a first axis connecting the two anatomical points and a second axis perpendicular to the first axis. You can. Here, the anatomical points may include the second cervical vertebra (C2), the third cervical vertebra (C3), the fourth cervical vertebra (C4), and the fifth cervical vertebra (C5). And the origin of the coordinate system may be either one of two anatomical points or an arbitrary point on the first axis.

The hyoid bone tracking device can determine the relative trajectory of the hyoid bone with respect to the cervical spine based on the determined specific point of each hyoid bone and the generated coordinate system (526). For example, the hyoid bone tracking device can determine the relative trajectory of the hyoid bone with respect to the cervical spine by displaying a specific point of each hyoid bone in a coordinate system.

According to an example embodiment, the hyoid bone tracking device may periodically update the hyoid bone tracking model, hyoid bone segmentation model, and cervical spine segmentation model.

For example, the hyoid bone tracking device may use the hyoid bone tracking result of step 521 as additional learning data to update the hyoid bone tracking model in real time. For example, the hyoid bone tracking device may update the hyoid bone tracking model in real time by additionally learning the hyoid bone tracking results of the previous frame according to a set cycle. In this case, the hyoid bone tracking device can use the hyoid bone tracking model updated in real time through additional learning to track the hyoid bone in the next frame, thereby improving the accuracy of hyoid bone tracking.

For another example, the hyoid bone tracking device periodically collects hyoid bone images for additional learning in which the hyoid bone is segmented or VFSS images for additional learning in which the cervical spine is segmented, and collects the collected hyoid bone images for additional learning or VFSS images for additional learning in which the cervical spine is segmented. Based on this, the hyoid bone segmentation model or cervical spine segmentation model can be updated.

The above-described embodiments may be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media may include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc. Additionally, the computer-readable recording medium may be distributed over network-connected computer systems and written and executed as computer-readable code in a distributed manner.

So far, the present invention has been examined focusing on its preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Accordingly, the scope of the present invention is not limited to the above-described embodiments, but should be interpreted to include various embodiments within the scope equivalent to the content described in the patent claims.

100, 400: Hyoid tracking device
110: Image acquisition unit
120: processor
210: Hyoid bone tracking unit
220: Hyoid bone division
230: Specific point determination unit
240: Cervical division
250: Coordinate system creation unit
260: Trajectory judgment unit
270: Model update unit
410: input unit
420: storage unit
430: Department of Communications
440: output unit

Claims (20)

An image acquisition unit that acquires VFSS (videofuoroscopic swallowing study) images; and
a processor that determines the relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images; Including,
The processor,
a hyoid bone tracking unit that extracts a region of interest including the hyoid bone from each VFSS image using a hyoid bone tracking model;
a hyoid bone division unit that divides the hyoid bone in each extracted region of interest using a hyoid bone segmentation model;
a specific point determination unit that determines a specific point of each of the divided hyoid bones;
A cervical spine segmentation unit that segments the cervical spine in each VFSS image using a cervical spine segmentation model;
a coordinate system generator that generates a coordinate system based on the cervical vertebrae based on the divided cervical vertebrae;
a trajectory determination unit that determines a relative trajectory of the hyoid bone with respect to the cervical spine based on the specific point of each hyoid bone and the generated coordinate system; and
a model updating unit that updates at least one of the hyoid bone tracking model, the hyoid bone division model, and the cervical spine division model; Including,
The hyoid bone tracking unit tracks the hyoid bone in the VFSS images using the hyoid bone tracking model and extracts the region of interest from each VFSS image based on the hyoid bone tracking result,
When updating the hyoid bone tracking model, the model update unit additionally learns the hyoid bone tracking result of the previous frame while the hyoid bone tracking unit tracks the hyoid bone in the VFSS images according to a set cycle, and updates the hyoid bone tracking model in real time. Update with
The hyoid bone tracking unit uses the hyoid bone tracking model updated in real time through the additional learning in the model update unit to track the hyoid bone in the next frame,
Hyoid tracking device.
delete delete According to paragraph 1,
The hyoid bone tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model are generated in advance through machine learning or deep learning,
Hyoid tracking device. According to paragraph 1,
The specific point includes the center point, most anterior point, most posterior point, most superior point, and most inferior point,
Hyoid tracking device. According to paragraph 1,
The specific point determination unit determines the specific point of each hyoid bone by shape matching each hyoid bone to a reference hyoid image showing the specific point of the hyoid bone,
Hyoid tracking device. According to clause 6,
The reference hyoid image is an image of the hyoid bone divided from the first frame image among the VFSS images,
Hyoid tracking device. According to paragraph 1,
The coordinate system generator determines two anatomical points in the divided cervical spine and generates the coordinate system including a first axis connecting the two anatomical points and a second axis perpendicular to the first axis.
Hyoid tracking device. According to paragraph 1,
The trajectory determination unit determines the relative trajectory of the hyoid bone with respect to the cervical vertebrae by displaying a specific point of each hyoid bone in the coordinate system.
Hyoid tracking device.
delete In the hyoid bone tracking method performed by a hyoid bone tracking device including an image acquisition unit and a processor,
Acquiring videofuoroscopic swallowing study (VFSS) images by the image acquisition unit; and
determining, by the processor, a relative trajectory of the hyoid bone with respect to the cervical spine from the acquired VFSS images; Including,
The step of determining, by the processor, the relative trajectory of the hyoid bone with respect to the cervical vertebrae,
Extracting a region of interest including the hyoid bone from each VFSS image using a hyoid bone tracking model;
segmenting the hyoid bone in each extracted region of interest using a hyoid bone segmentation model;
determining a specific point of each segmented hyoid bone;
Segmenting the cervical spine in each VFSS image using a cervical spine segmentation model;
Creating a coordinate system based on the cervical vertebrae based on the divided cervical vertebrae;
determining a relative trajectory of the hyoid bone with respect to the cervical spine based on the specific point of each hyoid bone and the generated coordinate system; and
Comprising updating at least one of the hyoid tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model,
In the step of extracting the region of interest, the hyoid bone is tracked in the VFSS images using the hyoid bone tracking model, and the region of interest is extracted from each VFSS image based on the hyoid bone tracking result,
In the updating step, when updating the hyoid bone tracking model, the hyoid bone tracking result of the previous frame is additionally learned in the process of tracking the hyoid bone in the VFSS images according to a set cycle, and the hyoid bone tracking model is updated in real time. ,
The step of extracting the region of interest is characterized in that the hyoid bone tracking model updated in real time through the additional learning is used to track the hyoid bone in the next frame.
How to track the hyoid bone.
delete delete According to clause 11,
The hyoid bone tracking model, the hyoid bone segmentation model, and the cervical spine segmentation model are generated in advance through machine learning or deep learning,
How to track the hyoid bone. According to clause 11,
The specific point includes the center point, most anterior point, most posterior point, most superior point, and most inferior point,
How to track the hyoid bone. According to clause 11,
The step of determining the specific point of each hyoid bone includes determining the specific point of each hyoid bone by shape matching each hyoid bone to a reference hyoid image showing the specific point of the hyoid bone.
How to track the hyoid bone. According to clause 16,
The reference hyoid image is an image of the hyoid bone divided from the first frame image among the VFSS images,
How to track the hyoid bone. According to clause 11,
The step of generating the coordinate system includes determining two anatomical points in the divided cervical spine, and creating the coordinate system including a first axis connecting the two anatomical points and a second axis perpendicular to the first axis. generating,
How to track the hyoid bone. According to clause 11,
The step of determining the relative trajectory of the hyoid bone with respect to the cervical vertebrae includes determining the relative trajectory of the hyoid bone with respect to the cervical vertebrae by displaying a specific point of each hyoid bone in the coordinate system.
How to track the hyoid bone.
delete