KR20230119527A - Device and method for needle injection guide for vocal fold treat - Google Patents

Abstract

Disclosed is a technique related to an injection needle insertion guide device and method for treating vocal cord diseases. An injection needle insertion guide device for treating vocal cord disease includes a vocal cord information input unit that receives 3D volume data composed of a neck CT image and vocal cord position information detected in the CT image, and a vocal cord information input unit directed toward the skin outside the neck centered on the vocal cord position. and a scanning path determination unit for determining a straight path passing through the smallest obstacle among radial straight paths. The device can prevent a patient from coughing and guide an optimal path for an injection needle to be inserted into the vocal cords quickly and accurately.

Description

Device and method for needle injection guide for vocal fold treat}

Data processing technology, in particular, a data processing technology related to a device for guiding an optimal injection needle insertion path in a region in which an injection needle can be inserted toward a vocal cord position in a CT image is disclosed.

It is necessary to inject drugs into the vocal folds to treat vocal cord diseases such as vocal cord paralysis. Conventionally, the drug was injected while checking with an endoscope through the airway. In this case, since the vocal cords have to be moved, the procedure is performed without anesthesia, but there is a problem such as the patient coughing after the endoscope is inserted.

Recently, a method of directly injecting into the vocal cords through the skin of the neck has been developed and used. In these cases, it is important to determine the optimal location and path of the needle. In general, the optimal needle injection path is determined manually by the doctor through CT images. Therefore, an automated device that suggests and guides the optimal injection route to the surgeon can help in successful surgery.

Registered Patent No. 10-1,705,199 published on February 9, 2017 relates to a simulation system for anterior cruciate ligament reconstruction surgery using medical images, and acquires images and 3D models of the subject's knee where ligament surgery will be performed A system for identifying the position of the reference point of the motion of the knee on a 3D model, tracking the change in the position of the reference point, and simulating the change in the length of a virtual ligament according to the change in the motion of the knee is disclosed.

Publication No. 10-2022-0,002,877 published on January 7, 2022 relates to a method for determining a surgical path based on image matching, building a 3D model based on medical image scans, and artificial intelligence engine It discloses a method for allocating each brain region as a cost function for an artificial intelligence engine to suggest to the surgeon one or more planned surgical routes with minimal physical damage to the patient.

However, a device that simply and accurately determines an optimal injection needle injection path for direct injection into vocal folds from a CT image has not yet been specifically disclosed.

An object of the proposed invention is to provide an injection needle insertion guide device and method for treating vocal cord diseases that determine the optimal route for an injection needle to be inserted into the vocal cord position.

Furthermore, an object of the proposed invention is to provide an injection needle insertion guide device and method for guiding injection needle insertion through a path that does not pass through an area where a patient coughs when the injection needle is inserted.

Furthermore, the proposed invention aims to provide an injection needle insertion guide device and method capable of accurately and quickly searching for an injection needle insertion path.

According to one aspect of the proposed invention, an injection needle insertion guide device for treating vocal cord diseases includes a vocal cord information input unit and an injection route determining unit. The vocal cord information input unit receives 3D volume data composed of a neck CT image and vocal cord position information detected from the CT image. The scanning path determination unit determines a straight line path passing through the least obstacle among radial straight paths toward the skin on the outside of the neck with the location of the vocal cords as the center.

According to a further aspect, the scanning path determining unit includes a maximum strength calculating unit and a minimum stiffness path selecting unit. The maximum intensity calculation unit obtains intensity values of CT images for each path on the radial straight path, and selects the maximum intensity value of each path as the stiffness value of the corresponding path. The minimum stiffness path selector selects a straight path having the minimum stiffness value from among the radial straight paths.

According to a further aspect, the scan path determining unit includes a needle injection range setting unit. The injection needle insertion range setting unit excludes a path out of a preset anatomically possible range among the radial straight paths.

According to a further aspect, the scan path determination unit includes an internal air area identification unit and an air passage path removal unit. The internal air region identifying unit identifies a region including air inside the neck in the CT image. The air passing path removing unit excludes a path passing through a region identified as a region containing air inside the neck from among the radial straight paths.

According to a further aspect, the internal air area identification unit includes an air area dividing unit, a GD conversion unit, and an internal air area determining unit. The air area segmentation unit classifies the air area using the HU threshold in the CT image. The GD conversion unit performs GD conversion on the CT image based on the outside air area of the CT image. The inner air area determining unit selects, as the inner air area, a portion of the air area that is disconnected from the outer air area identified through GD conversion.

According to another aspect of the proposed invention, the injection needle insertion guide method for vocal cord disease treatment performed by the injection needle insertion guide device for vocal cord disease treatment includes a vocal cord information input step and an injection route determination step. In the vocal cord information input step, 3D volume data composed of a neck CT image and vocal cord position information detected from the CT image are input. In the scanning path determination step, a straight line path passing through the minimum obstacle is determined among radial straight paths from the vocal cords to the external skin of the neck.

The scanning path determination step includes a maximum intensity calculation step and a minimum stiffness path selection step. In the maximum intensity calculation step, intensity values of CT images are obtained for each path on the radial straight path, and the maximum intensity value of each path is selected as the stiffness value of the corresponding path. In the minimum stiffness path selection step, a straight line path having the minimum stiffness value is selected from among the radial straight paths.

The step of determining the scanning path includes the step of setting the injection range of the injection needle. In the injection needle input range setting step, a path out of a preset anatomically possible range among the radial straight paths is excluded.

The scanning path determination step includes an internal air area identification step and an air passage path removal step. In the internal air region identification step, a region including air inside the neck is identified in the CT image. In the air passing path removing step, a path passing through an area identified as an area containing air inside the neck is excluded from the radial straight paths.

The internal air region identification step includes an air region classification step, a GD conversion step, and an internal air region determination step. In the air area classification step, the air area is classified using the HU threshold in the CT image. In the GD conversion step, GD conversion is performed on the CT image based on the outside air area of the CT image. In the internal air region determination step, a portion of the air region disconnected from the external air region identified through GD conversion is selected as the internal air region.

According to the proposed invention, the device and method for guiding injection needle insertion for the treatment of vocal cord diseases can guide an optimal path through which an injection needle is inserted into the vocal cord position using the intensity of a CT image of the vocal cord.

Furthermore, the proposed invention can provide an injection needle insertion guide device that does not cause coughing in the patient when the injection needle is inserted by excluding the path passing through the internal air area.

Furthermore, the proposed invention can quickly and accurately determine an injection needle insertion route by searching for an injection needle injection route within an anatomically possible range.

1 is a configuration diagram showing the configuration of an injection needle insertion guide device for treating vocal cord diseases according to an embodiment.
2 is a configuration diagram showing the configuration of a vocal cord positioning unit in an injection needle insertion guide device for treating vocal cord diseases according to an embodiment.
3 is a flowchart illustrating a method for guiding injection needle insertion for treating vocal cord diseases according to an embodiment.
4 is a flowchart showing in detail the re-detection of the position of the vocal cords in the injection needle insertion guide method for the treatment of vocal cord diseases according to an embodiment.
5 is a flowchart showing in detail the step of determining an injection path in the injection needle insertion guide method for treating vocal cord disease according to an embodiment.
6 is a photograph illustrating a method for guiding injection needle insertion for treating vocal cord disease according to an embodiment.
7 is a conceptual diagram illustrating a vocal cord position reinforcement learning step in a method for guiding injection needle insertion for vocal cord disease treatment according to an embodiment.
8 is a photograph and a graph showing the distance between the left and right vocal cords used in the step of verifying the position of the vocal cords in the injection needle insertion guide method for the treatment of vocal cord diseases according to an embodiment.
9 is a photograph illustrating an injection path determination step in a method for guiding injection needle insertion for treatment of vocal cord diseases according to an embodiment.
10 is a photograph showing examples of vocal cord detection results and injection needle projection results obtained using a needle injection guide method for treating vocal cord diseases according to an embodiment.
11 is a graph showing the accuracy of vocal cord detection results and injection needle projection results obtained using a needle insertion guide method for treating vocal cord diseases according to an embodiment.
12 is a photograph comparing a method using vocal cord position verification and a reinforcement learning method without verification in the injection needle insertion guide method for vocal cord disease treatment according to an embodiment.

The foregoing and additional aspects are embodied through embodiments described with reference to the accompanying drawings. It is understood that the elements of each embodiment can be combined in various ways within one embodiment or with elements of another embodiment without contradiction with each other or other references. Based on the principle that the inventor can properly define the concept of terms in order to explain his/her invention in the best way, the terms used in this specification and claims have meanings consistent with the description or proposed technical idea. and should be interpreted as a concept. In this specification, a module or part is a set of program instructions stored in a memory to be executable by a computer or processor, or a set of electronic components or circuits such as ASICs and FPGAs to execute these instructions. Can be implemented using. Also, the operation of each module or part may be performed by one or a plurality of processors or devices.

The larynx is a raised part in the neck of the human body, located in front of the esophagus and connected to the trachea. The laryngeal cavity has two pairs of corrugated inner walls, commonly called vocal folds. In general, a pair of folds located on the upper side are called false folds or vocal folds, and a pair of folds located on the lower side are called true vocal cords or vocal cords. Expressed as 'vocal cord'. The right vocal cord position means the injection position of the vocal cord located on the right side of a pair of vocal folds, and the left vocal cord position means the injection position of the vocal fold located on the left side of the pair of vocal folds.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram showing the configuration of an injection needle insertion guide device for treating vocal cord diseases according to an embodiment.

According to one aspect of the proposed invention, an injection needle insertion guide device for treating vocal cord diseases includes a vocal cord information input unit 110 and an injection route determination unit 160. The vocal cord information input unit 110 receives 3D volume data composed of a neck CT image and vocal cord position information detected from the CT image. The scanning path determining unit 160 determines a straight path passing through the least obstacle among radial straight paths toward the skin outside the neck with the vocal folds as the center.

According to an additional aspect, the scan path determination unit 160 includes a maximum intensity calculation unit 195 and a minimum stiffness path selection unit 197. The maximum intensity calculation unit 195 obtains the intensity value of the CT image for each path on the radial straight path, and selects the maximum intensity value of each path as the stiffness value of the corresponding path. The minimum stiffness path selection unit 197 selects a straight line path having the minimum stiffness value among the radial straight paths.

According to an additional aspect, the scan path determining unit 160 includes a needle insertion range setting unit 180 . The injection needle insertion range setting unit 180 excludes a path out of a predetermined anatomically possible range from among the radial straight paths.

According to an additional aspect, the scan path determination unit 160 includes an internal air area identification unit 170 and an air passage path removal unit 193 . The internal air region identification unit 170 identifies a region including air inside the neck in the CT image. The air passing path removing unit 193 excludes a path passing through a region identified as a region containing air inside the neck from among the radial straight paths.

According to an additional aspect, the internal air area identification unit 170 includes an air area classification unit, a GD conversion unit, and an internal air area determination unit. The air area segmentation unit classifies the air area using the HU threshold in the CT image. The GD conversion unit performs GD conversion on the CT image based on the outside air area of the CT image. The inner air area determining unit selects, as the inner air area, a portion of the air area that is disconnected from the outer air area identified through GD conversion.

2 is a configuration diagram showing the configuration of a vocal cord positioning unit in an injection needle insertion guide device for treating vocal cord diseases according to an embodiment.

An injection needle insertion guide device for vocal cord disease treatment according to an embodiment includes a reinforcement learning (RL) learning unit 205, a vocal cord positioning unit 210, and a scanning path determining unit 160.

The vocal cord position determining unit 210 includes a vocal cord position detection unit 220, a vocal cord position verification unit 230, and an initial position setting unit 245. The vocal cord position detector 220 includes a right vocal cord detection neural network 223 that detects the right vocal cord position and a left vocal cord detection neural network 227 that detects the left vocal cord position.

The reinforcement learning (RL) learning unit 205 reinforces the right vocal cord detection neural network 223 and the left vocal cord detection neural network 227 of the vocal cord position detection unit 220 using the 3D volume of the neck CT image. The 3D volume data of the neck CT image is used for reinforcement learning (RL) after an expert marks the location of the right and left vocal cords as ground truth (GT). The right vocal cord detection neural network 223 and the left vocal cord detection neural network 227 use the same structure. Since the positions of the right vocal cord and the left vocal cord are different from each other, the trained right vocal cord detection neural network 223 and the left vocal cord detection neural network 227 may have different parameters.

When 3D volume data of the neck CT image is input to the vocal cord position detection unit 220 after learning has been completed, the right vocal cord detection neural network 223 and the left vocal cord detection neural network 227 detect the right vocal cord position and the left vocal cord position, respectively. The detected vocal cord position data is transmitted to the vocal cord position verifying unit 230 .

The vocal cord position verifying unit 230 verifies whether the detected vocal cord position corresponds to a normal vocal cord position rather than an incorrect vocal cord position. The vocal cord position verifying unit 230 verifies whether the position corresponds to the vocal cord position by using the distance between the left and right vocal cord positions. The vocal cord position verifying unit 230 may output a verification result together with the left and right vocal cord positions.

The vocal cord position determining unit 210 checks the verification result of the vocal cord position verification unit 230, and if the verification fails, the initial position setting unit 245 sets a new initial position and inputs it to the vocal cord position detection unit 220. The right vocal cord detection neural network 223 and the left vocal cord detection neural network 227 each start to detect the position of the vocal cords at different initial positions. When a new left and right vocal cord position is detected by each neural network, it is once again put into the vocal cord position verifying unit 230 and verified. The vocal cord position determining unit 210 checks the verification result of the vocal cord position verifying unit 230 and transmits the left and right vocal cord positions to the scanning path determining unit 160 when the verification is successful.

The scanning path determination unit 160 includes an internal air area identification unit 170, a needle insertion range setting unit 180, and a needle insertion path determination unit 190. The internal air region identification unit 170 identifies an air region using a Hounsfield Unit (HU) threshold, and identifies an external air region and an internal air region using GD (Gray-Weighted Distance) conversion. The needle input range setting unit 180 sets the needle input range to an anatomically possible needle input range. Excluding the lines passing through the air area, the maximum intensity of each line is obtained, and the line with the lowest maximum intensity is determined as the straight needle insertion path.

3 is a flowchart illustrating a method for guiding injection needle insertion for treating vocal cord diseases according to an embodiment.

According to one aspect of the proposed invention, the injection needle insertion guide method performed by the injection needle insertion guide device for treating vocal cord disease includes a vocal cord positioning step (S310). The vocal cord position determination step (S310) is performed by the vocal cord position determining unit 210, comprising a vocal cord position detection step (S320), a vocal cord position verification step (S330), a vocal cord position re-detection step (S340), and a vocal fold position detection step (S340). A confirmation step (S350) is included.

According to an additional aspect, the injection needle insertion guide method further includes a vocal cord position reinforcement learning step (S305). In the vocal cord position reinforcement learning step (S305) performed by the reinforcement learning (RL) learning unit 205, the right vocal cord position detection neural network and the left vocal cord position detection neural network are used in neck CT with annotations displayed at the right and left vocal cord positions. Reinforcement learning is performed using the 3D volume data of the image.

According to an additional aspect, the injection needle insertion guide method further includes an injection path determining step (S360). The scanning path determining step (S360) is performed by the scanning path determining unit 160, and determines a straight line path of a needle that can be injected from the skin outside the neck toward the right and left vocal cords by passing through the minimum obstacles. do.

In the vocal cord position detection step (S320), the right vocal cord position and left vocal cord position are detected from the 3D volume of the neck CT image, respectively, using the right vocal cord position detection neural network and the left vocal cord position detection neural network. In the vocal cord position verification step (S330), it is verified whether the right vocal cord positions and the left vocal cord positions detected in the vocal cord position detection step (S320) correspond to the anatomically appropriate vocal cord positions. In the vocal cord position re-detection step (S340), if it is determined that the right vocal cord position and the left vocal cord position are anatomically inappropriate vocal cord positions in the vocal cord position verification step (S330), the right vocal cord position and The position of the left vocal fold is detected again.

According to an additional aspect, in the vocal cord position verification step (S330), if the distance between the right vocal cord position and the left vocal cord position detected by the vocal cord position verification unit 230 exceeds a preset threshold range, the vocal cord position is anatomically inappropriate. judged by

4 is a flowchart showing in detail the re-detection of the position of the vocal cords in the injection needle insertion guide method for the treatment of vocal cord diseases according to an embodiment.

According to an additional aspect, the vocal cord position re-detection step (S340) includes an initial position resetting step (S445) and a vocal cord position detection step (S320). In the initial position resetting step (S445), the vocal cord position determining unit 210 assigns a plurality of initial positions randomly selected from the sampling space reduced from the 3D volume of the neck CT image to the center to the right vocal cord position detection neural network and the left vocal cord position detection neural network, respectively. convey The vocal cord position detection step (S320) may be performed in the same manner as the method performed before the vocal cord position verification step (S330).

According to an additional aspect, in the initial position resetting step (S445), when the second vocal cord position verification fails, the initial position selected in the sampling space is increased by 8 times compared to the case where the first verification fails, and the selected initial positions are set to the right vocal cord. It is transmitted to the position detection neural network and the left vocal cord position detection neural network respectively.

5 is a flowchart showing in detail the step of determining an injection path in the injection needle insertion guide method for treating vocal cord disease according to an embodiment.

According to an additional aspect, the scanning path determining step (S360) performed by the scanning path determining unit 160 includes the internal air area identification step (S570), the needle inputting range setting step (S580) and the needle inputting path determining step. (S590). In the internal air region identification step (S570), an external air region outside the neck skin and an internal air region inside the neck are distinguished and identified. In the step of setting the needle insertion range (S580), a range of a straight path in which the needle can be inserted anatomically is set at the detected positions of the right and left vocal cords. In the needle insertion path determination step (S590), a path with the smallest obstacle among paths within the range of the possible straight paths is selected as the needle input path.

According to an additional aspect, the internal air region identification step (S570) includes the air region classification step (S573) of dividing the air region using the HU threshold in the neck CT image, and the external air region of the neck CT image as a standard. A GD conversion step (S575) of performing GD conversion on the neck CT image, and an internal air region determination step (S577) of identifying a portion disconnected from the external air region as an internal air region are included.

According to an additional aspect, in the step of setting the needle insertion range (S580), the ranges of anatomically possible needle insertion paths are set in the detected positions of the right and left vocal folds.

According to an additional aspect, the needle insertion path determining step (S590) includes the air passing path removing step (S593) of excluding a path passing through air from the range of straight paths where the needle can be inserted anatomically, and the linear path A maximum strength calculation step (S595) of obtaining the strength of the line and setting the maximum strength as the stiffness of the corresponding straight line path, and a minimum rigid line path of setting the straight line path with the smallest stiffness among the available straight line paths as the needle insertion path A decision step (S597) is included.

In order to obtain the optimal injection needle insertion path (L ^* ), first, the strength (I) of each straight path among the range of straight paths (L) in which the needle can be injected anatomically is obtained, and the sum of the strength values (f) for each path _sum ), the average value (f _mean ), and the maximum value (f _max ) are obtained by Equation 1 below, and a path in which each value is minimized may be proposed.

[Equation 1]

Here, I(x) is the intensity of position x on path L.

If the paths with the minimum total value, average value, and maximum value are different from each other, it is desirable to suggest the path with the minimum maximum value of each path as the optimal path so that the obstacles are minimized. At this time, the maximum value of each path may be set as the stiffness of each path.

6 is a photograph illustrating a method for guiding injection needle insertion for treating vocal cord disease according to an embodiment.

6(a) is a horizontal cross-sectional CT image of the neck in which the positions of the left and right vocal cords determined in the vocal cord positioning step (S310) are marked in red. The detected vocal cord positions marked in red in the picture indicate the right vocal cord position and the left vocal cord position, respectively. The position of the vocal cords was detected using a reinforcement learning (RL) agent, and the landmarks on the back of the vocal cords were detected because the characteristics of the back of the vocal cords were more distinct than the front of the vocal cords. The injection needle is similarly inserted near the posterior landmark location.

6(b) is a horizontal cross-sectional CT image of the neck in which the route of the injection needle determined in the step of determining the injection route (S360) is indicated in yellow. The area marked in red in front of the location of the vocal cords is a location on the skin of the neck that is in contact with the outside air, and an injection needle can be inserted into this area. The area marked in blue means the horizontal angle at which the injection needle can be inserted centered on the left and right vocal folds. 6(c) is a vertical cross-sectional CT image of the neck in which the path of the injection needle determined in the step of determining the injection path (S360) is marked in yellow. The area marked in blue indicates a vertical angular area where the injection needle can be inserted centered on the position of the vocal cords.

7 is a conceptual diagram illustrating a vocal cord position reinforcement learning step in a method for guiding injection needle insertion for vocal cord disease treatment according to an embodiment.

7(a) shows an environment in the vocal cord position reinforcement learning step (S305) according to an embodiment. The environment in which the detection agent interacts is a 3D volume composed of 3D CT images of the neck. A 3D volume can be represented using three Cartesian axes X, Y, and Z, which point in the directions right-to-left, front-to-back, and bottom-to-up. p _r and p _l represent the target positions of the right and left vocal folds, respectively, and p _fold represents one of the two vocal fold positions. The intensity function T can be obtained using the HU (Hounsfield Unit) of the CT volume for all locations.

Agent (Agent) can be composed of a plurality. For example, an agent detecting the position of the right vocal cord and an agent detecting the position of the left vocal cord may be configured separately and used as two agents. 7(b) shows a configuration of one of the agents for detecting the position of the vocal cords in the vocal cord position reinforcement learning step (S305) according to an embodiment. An agent detecting the position of the right vocal cord and an agent detecting the position of the left vocal cord may be configured in the same form. These agents have the same form but have different targets, so they independently detect the location of the target.

Referring to FIG. 7 , in order to detect a target vocal cord position (p _fold ) of any one of the left and right vocal cord positions, the agent starts from an arbitrary position inside the 3D volume and performs subsequent steps to gradually reach the target vocal fold position. carry out At every step t, the current state (s _t ) is observed with the current location (p _t ) as the center, and the action (a _t ) is determined to move to the adjacent location (p _t+1 ). At this time, as a unit action (a _t ), six different actions can be used in the right, left, forward, backward, downward, and upward directions. The policy function π(s _t , a _t ) is used to generate the probability of an optimal action (a _t ) for an observed state (s _t ) at every step. After episode T containing these steps, the agent eventually converges to the target vocal cord position p _fold . The policy function is modeled with a deep neural network (DNN). In the case of 3D image input as shown in FIG. 7 (b), a 3D CNN can be used.

During training, a reward signal is provided at each step to move the agent closer to the goal. The reward r _t (p _t , a _t , p _t+1 ) for taking an action (a _t ) to switch from an arbitrary position (p _{t )} to the next position (p _t+1 ) is function can be used.

[Equation 2]

from here | | represents the vector norm operation.

Positive rewards are given when the agent moves closer to the goal, and negative rewards are given when the agent moves away from the goal. τ denotes a small distance over which the agent consistently receives a positive reward, indicating a state of convergence.

Training aims to iteratively optimize policy π to maximize the expected cumulative reward. In every iteration, the agent first performs several localization episodes using the current policy and collects the transition experience. Each experience (s, a, s', r) has the current state s(=s _t ), the action taken a(=a _t ), the next state s'(= s _t+1 ), and the achieved reward r(= r _t ) is composed of four pieces of information. A mini-batch is sampled from these experience samples, and the policy can be optimized using stochastic gradient ascent.

Directly optimizing raw rewards is difficult because of the large variance in experienced rewards. Thus, we use the value function V(s) to evaluate the policy for a given state s, maximizing the comparative advantage of the current policy. For all transitions (s, a, s', r), the gain (A) can be calculated as in Equation 3 below.

[Equation 3]

Here, 0 ≤ γ < 1 is a discount factor.

In Equation 3, the first part of the right side is the approximate cumulative discount reward (r + γV(sl)) achieved by the current policy. V(s) tracks the discounted return on the previous policy. Thus, advantage can be used to evaluate policy improvements.

To maximize this advantage, proximity policy optimization (PPO) is used. Proximity Policy Optimization (PPO) uses the following objective of Equation 4 that utilizes the ratio of the new policy to the old policy.

[Equation 4]

where π _old is the policy used to collect experience for policy update and π is the policy to be updated during optimization. The policy ratio can be simply expressed as ρ(π).

The objective of Equation 4 indicates that the policy for action (a) increases when the gain (A) is positive, and the policy for action (a) decreases when the gain (A) is negative. Also, the above policy ratios clip to avoid large policy updates. Therefore, the final objective of PPO can be expressed as Equation 5 below.

[Equation 5]

In summary, the agent first collects transitional experience episodes at the start of each training iteration using the corresponding policy. Using this experience, the policy is updated to maximize the objective of Equation 5 above by stochastic gradient ascent over several epochs.

Before continuing with the next iteration, the value function is updated to approximate the cumulative return of the current policy. where E _{(s, a, s', r)} [(r + γ V(s')) - V(s)] ² is minimized as the square of the difference between the present value and the expected discount reward of the current policy To do this, stochastic gradient descent is performed. Meanwhile, as shown in FIG. 7(b), a value function is also modeled by a deep neural network (DNN).

8 is a photograph and a graph showing the distance between the left and right vocal cords used in the step of verifying the position of the vocal cords in the injection needle insertion guide method for the treatment of vocal cord diseases according to an embodiment.

After the vocal cord position reinforcement learning step (S305), the vocal cord position detection step (S320) is performed. In the vocal cord position detection step (S320), the vocal cord position is detected by inputting the 3D CT image to the neural network that has undergone reinforcement learning. The detected result may include false positives that converge to the wrong location. In order to further detect such false positives and improve model reliability, a vocal cord location verification step (S330) is performed.

There are cases in which a two-step model using a second model is used to reduce false positives of prediction in the first model, but this two-step model is mainly intended to reduce undetected false positives. Therefore, in order to reduce false positive convergence to the wrong position, the vocal cord position verification step (S330) compares the relative position of the two vocal cords to verify whether the detected vocal cord position is an anatomically possible position. The two vocal cord detection agents self-verify the result by comparing the relative position of the vocal cords with the position of the vocal cords. That is, the agent may provide a failure signal when the detected vocal cord position is unreliable.

In FIG. 8(a), a distance vector d, which is a relative position between two vocal cords marked in red, is indicated by a blue arrow. 8(b) shows the relative position between the two vocal folds as coordinates. The two vocal cords have a strong anatomical relationship and show high consistency across many patients. Therefore, this relative relationship can be approximated with distance vectors d to N(μ, σ ² ) using a Gaussian model. Here, d (= p _l - p _r ) represents the distance vector from the position of the right vocal cord to the position of the left vocal cord. Mean μ = (μ _x , μ _y , μ _z ) and variance σ ² = (σ _x ² , σ _y ² , σ _z ² ) can be preset using expert-specified distances of vocal cord positions in the training image volume. .

Based on the above distance model, the vocal cord position prediction can be verified by comparing the resultant distance obtained from the vocal fold position prediction with the empirically obtained distance model. If the predicted positions of the right and left vocal folds are denoted by p _l and p _r , the corresponding distances can be obtained as d = p _l - p _r = (d _x , d _y , d _z ). If the failure signal F obtained by Equation 6 is 1, it indicates that vocal cord position prediction has failed, and if it is 0, it indicates that verification has been successfully performed.

[Equation 6]

Here, the value of the critical constant α = (α _x , α _y , α _z ) can be preset by cross-validating the training set.

Therefore, the vocal cord position detection result has three components ( _pr , p _l , F), that is, the right and left vocal cord positions and the detection failure signal. In this search-based positioning method, detection performance is highly dependent on the initial position. Therefore, when the failure signal F is 1, the vocal cord position re-detection step (S340) is performed to successfully predict the vocal fold position. The vocal cord position re-detection step (S340) includes the initial position resetting step (S445) and the vocal cord position detection step (S320), and the vocal cord position is searched again for various initial positions throughout the test volume. The vocal cord position detection step (S320) may be performed in the same manner as the vocal cord position detection step (S320) prior to the vocal cord position verification step (S330).

When a failure signal is output in the vocal cord position verification step (S330), the initial position resetting step (S445) is performed. In the initial location resetting step (S445), the initial location is reset and location detection is attempted using the same agent at another initial location. First, we reduce the sample space by excluding regions where vocal cords are unlikely to be present. Vocal cords usually appear near the center of a neck CT, excluding most of the right and left, anterior and posterior regions. For example, the size of the reduced sample space can be set to 200 * 200 * 75 voxels. In this sample space, n positions on each axis are first sampled. If n = 5, 5 * 5 * 5 = 125 initial positions can be sampled. The vocal cord position re-detection step (S340) is performed using these initial positions, and the vocal cord position verification step (S330) is performed using the predicted result. In step S350, the verified result may be determined and output to a storage device such as a memory or a disk device or a display device such as a monitor.

If the verification is not successful because all failure signals are output from the first sampling results, the second sampling is performed by doubling the axial sampling frequency. In the above example, 10 * 10 * 10 = 1000 initial positions are sampled, and 8 times more initial positions are set than the first sampling. For each initial position, the initial position resetting step (S445), vocal cord position redetection step (S340), and vocal fold position verification step (S330) are repeatedly performed.

This process of resetting the initial location and re-locating the location exponentially increases the calculation time. Therefore, parallelization of position detection episodes for different initial positions is required. The main bottleneck in computation during transition is GPU utilization when exploring 3D states via 3D CNNs to obtain policies. Therefore, sequentially executing episodes of rescanning by resetting the initial position wastes GPU calculations because only a single state is passed to the GPU at a time.

To take advantage of the GPU's parallel computation, the states of the worlds are batched and passed to the GPU in a single pass. To do this, we simultaneously perform agent switching in different worlds while maintaining different world copies for different episodes. At each step, we collect the current state from all worlds and pass it to the GPU to get the policy in a single pass (or multiple batches depending on GPU memory). Use that policy to update the other world's agent-location. According to this method, the efficiency can be improved many times.

9 is a photograph illustrating an injection path determination step in a method for guiding injection needle insertion for treatment of vocal cord diseases according to an embodiment.

If the vocal cord detection result is successfully verified, the position of the vocal cords is determined in step S350. The position of the vocal cords determined in the step of determining the position of the vocal cords (S350) may be transmitted to a storage device such as a memory or an output device such as a display. When the location of the vocal cords is determined in the vocal cord position determining step (S350), a scanning path determining step (S360) is performed. In the scanning path determining step (S360), a straight line path for inserting the injection needle is predicted using the detected vocal fold position. The scanning path determining step (S360) includes an internal air area identification step (S570), a needle input range setting step (S580), and a needle input path determining step (S590).

In the air area classification step of the internal air area identification step (S570), the air area of the CT image can be identified as the HU threshold. In order to identify the outer air area among the air areas in the GD conversion step of the inner air area identification step (S570), the outer air area of the right front corner indicated in red in FIG. 9 (a) is first seeded. select with Based on this seed, GD (Gray-Weighted Distance) conversion in which the distance is weighted by the intensity difference is performed to obtain the image of FIG. 9(b). Since the external air region is uniform and continuous and includes seeds, the GD conversion value of the entire external air region becomes 0, so that the external air region can be distinguished as shown in FIG. 9(c). In the internal air region determination step of the internal air region identification step (S570), the region physically separated from the external air region among the regions identified as the air region using the Hounsfield Unit (HU) threshold can be identified as the internal air region. there is.

In the step of setting a needle insertion range (S580), a range of a linear path in which the needle can be inserted anatomically is set at the detected positions of the right and left vocal folds. First, a midpoint m at which a center line perpendicular to a line connecting the left and right vocal folds indicated by a red line in FIG. 9(d) intersects the skin is found. The midpoint m can be found by finding the outermost point just before reaching the outer air region from the center line. It is preferable to insert the injection needle in the interval of 5 to 10 mm in the left and right directions based on the center line of the vocal cords. Therefore, in FIG. 9 (d), the needle insertion path can be searched in the area between the yellow lines that are spread up to 20 degrees (maximum θ = 20 degrees) from the left and right of the center line, which is the red line. In the case of the right vocal cord, the search space expands from the midpoint to the right, and in the case of the left vocal cord, the search space expands to the left. Considering the location of the thyroid gland and the cricoid thyroid gland, it is preferable to insert the injection needle in an area 60 degrees below the horizontal plane based on the location of the vocal cords.

In the air passage route removal step of the needle insertion route determining step (S590), a transcircular thyroid approach is used in which the injection needle moves along the submucosal route without penetrating the mucous membrane to minimize symptoms such as bleeding and coughing. To this end, a submucosal rectilinear path search is performed by excluding the mucosal transmucosal path through the internal air region.

The line (L ^* ), which is the optimal needle insertion straight path, is determined as the line that faces the least obstacle or stiffness in the path from the cortex to the vocal cords among the set of lines. In the maximum strength calculation step of the needle insertion path determining step (S590), the maximum strength is obtained for each straight path and set as the stiffness of the corresponding path. In the minimum stiffness linear path determination step of the needle insertion path determination step (S590), the optimal needle insertion line (L ^* ) is obtained by finding a line having the minimum stiffness in each path as shown in Equation 7 below.

[Equation 7]

Here, L is a line passing through the vocal cord position p belonging to lines Ω retrieved as submucosal lines, and I(p) is an intensity value on this line.

The injection needle insertion point can be obtained by obtaining the intersection point of this optimal line (L ^* ) and the skin.

10 is a photograph showing examples of vocal cord detection results and injection needle projection results obtained using a needle injection guide method for treating vocal cord diseases according to an embodiment.

To evaluate the method, a neck CT data set was collected at Seoul National University Bundang Hospital (SNUBH) in Seongnam, Korea. All ethical and experimental procedures and protocols were approved by the SNUH Internal Review Board. A total of 119 neck CT 3D volumes were obtained from 119 patients between October 2017 and August 2021. Vocal cord paralysis was suspected in all patients, but 34 patients were actually diagnosed with vocal cord paralysis. Among them, 27 patients had unilateral palsy and 7 patients had bilateral palsy. Table 1 summarizes the information of the data set used in the experiment.

data total male female age normal laterality bilateral training 50 27 23 62±14 36 11 3 test 66 31 35 61±16 46 16 4 total 116 58 58 - 82 27 7

Fifty volumes of the entire 3D volume were randomly sampled as a training set for training the detection agent. The remaining 69 volumes were used as test sets for evaluation. To train the vocal cord detection agent, experts in the same clinical site annotated the ground truth (GT) of the right and left posterior vocal folds in 50 training volumes. The final vocal cord detection and needle injection results for the unannotated test set were also presented to the same expert for evaluation by the expert identifying successes and failures. The results were displayed in DICOM, a medical digital imaging and communication standard, using maximum intensity burn-in annotation. Fig. 10(a) shows an example of vocal cord position detection results, and Fig. 10(b) shows an example of injection needle projection results.

11 is a graph showing the accuracy of vocal cord detection results and injection needle projection results obtained using a needle insertion guide method for treating vocal cord diseases according to an embodiment.

Since the needle projection calculation directly depends on vocal cord position, it is important to ensure good accuracy in vocal cord position detection. Five-fold cross validation was performed on the training set to obtain position detection error estimates for the vocal cords (i.e., the distance from the detected vocal cords to the expert annotation). Table 2 is a table showing position detection errors for various comparative examples and examples.

Right Vocal Cord (mm) Left Vocal Cord (mm) Comparative Example 1 coordinate regression 2.92±2.02 3.01±2.16 Comparative Example 2 heatmap regression 2.70±1.78 2.74±1.69 Comparative Example 3 reinforcement learning 2.29±1.31 2.45±1.38 Example 1 Reinforcement Learning + Position Verification 2.15±1.08 2.32±1.17

Comparative Example 1 and Comparative Example 2 used a coordinate regression method and a heatmap regression method, which are standard deep learning methods. At this time, all hyperparameter settings were kept the same as the basic model, and the number of convolutional blocks was adjusted according to the input size. Comparative Example 3 used a reinforcement learning (RL) method. Example 1 added a vocal cord position verification step to the reinforcement learning method.

Fig. 11(a) shows the vocal fold detection accuracy of each model as a graph. Vocal cord detection accuracy is the success rate of vocal cord detection evaluated by experts. Comparative Example 1 is indicated in blue, Comparative Example 2 in red, Comparative Example 3 in green, and Example 1 in orange. In the case of reinforcement learning (Comparative Example 3), vocal cord position detection performance was improved compared to standard deep learning methods (Comparative Example 1 and Comparative Example 2). However, reinforcement learning including vocal cord position verification (Example 1) shows better performance than reinforcement learning alone (Comparative Example 3). Reinforcement learning including vocal cord position verification (Example 1) self-verifies the detection result, and if it fails, position detection can be retried at a different initial position, so the probability of reaching the vocal cords through multiple paths increases.

12 is a photograph comparing a method using vocal cord position verification and a reinforcement learning method without verification in the injection needle insertion guide method for vocal cord disease treatment according to an embodiment.

12(a) shows a process in which a standard reinforcement learning (RL) agent detects vocal fold positions. The left vocal cord was found in the position of the vocal cord marked in red, but the right vocal cord converged to the wrong position.

12(b) shows an improved example of a method of including vocal cord position verification in reinforcement learning. In the initial trial, only the left vocal cord was successfully detected. During the vocal cord position verification process, detection failure due to an abnormal relative vocal cord position may be confirmed, and accordingly, several position detection episodes may be performed at different initial positions. It can be observed that the detection performance for this case varies for different initial positions. Therefore, as in the standard RL, an attempt to detect a position using a single initial position has a high probability of failure. Therefore, in one embodiment, after vocal cord position verification, vocal cord position detection was attempted through various paths through initial position resetting, and a successful vocal cord position detection result could be obtained.

The vocal cord position re-detection step (S340) for improvement is performed based on the vocal cord position verification step (S330). Therefore, the failure detection performance of the vocal cord position verification step (S330) was also evaluated. The relative position comparison method of the left and right vocal folds according to an embodiment was compared with a divergence-based determination method using the variance of the final agent position. In both cases, the optimal threshold for comparison was obtained after cross-validating the training examples.

Vocal cord position verification according to an embodiment had a sensitivity of 83.33%, specificity for detecting a failure case was 100%, and accuracy was 96.97%. On the other hand, the optimal sensitivity, specificity, and accuracy of the divergence-based determination were 66.67%, 70.37%, and 69.70%, respectively. The divergence measure provided a less accurate decision because the agent might converge to the wrong position, since the agent's trajectory is determined by its local state. Other soft tissue structures with the same characteristics as the focal soft tissue of the vocal cords may mislead the agent. In contrast, the relative position of the vocal cords is a relatively more reliable measurement because it has a high degree of consistency from patient to patient. The two agents independently localize the right and left vocal cords, and the probability that such two independent processes converge to that coherence by chance is low. Therefore, the detection result is likely to be true positive within the consistency range, and verification using the relative position of the vocal cords can extract failure cases with high accuracy.

Table 3 is a table showing the accuracy of the vocal cord detection process and the needle projection process used in the experiment.

Vocal cord detection needle projection right left right left Comparative Example 3 RL (reinforcement learning) 90.91% 86.36% 86.36% 80.20% Example 1 RL + position verification 95.45% 90.91% 90.91% 86.36%

We evaluated the needle projection accuracy using the minimum intensity line generated from the vocal cord position detected by the agent for the case where vocal cord position verification was used (Example 1) and the case where vocal cord position verification was not used (Comparative Example 3). . Needle projection evaluation was performed by an expert. In the case of general reinforcement learning (Comparative Example 3), the average needle projection accuracy was 84.07%, but in the case of vocal cord position verification (Example 1), the average needle projection accuracy was 89.39%, which was higher. Fig. 11(b) shows a graph of injection needle projection accuracy in two cases. It can be seen that Example 1 has higher vocal cord position detection accuracy than Comparative Example 1, and also has higher needle projection path determination accuracy. The vocal cord position detection accuracy was improved by performing the vocal cord position re-detection process for cases where vocal cord position detection failed through vocal cord position verification, so the accuracy of injection needle projection path determination was also improved. As a result, doctors who want to treat vocal cord diseases can be provided with a guide on how to insert a needle into the vocal cords.

Table 4 shows the calculation time required for each step of performing an embodiment.

step time (seconds)
no re-detection time (seconds)
sequential time (seconds)
parallel Vocal cord detection 0.63 1st position re-detection - 80.58 14.61 needle projection calculation 8.24 total 8.87

Calculation time was measured based on a 3.60 GHz CPU and RTX 3080 GPU. The average time for vocal cord detection was 0.63 seconds. The average time of the first position re-detection attempt with 125 episodes was 80.58 seconds. However, through parallelization, the required time could be reduced to 14.61 seconds. In three cases, a second position re-detection trial with 1000 episodes was performed. The needle projection calculation step took about 8.24 seconds. Therefore, the total required time was 8.87 seconds in the case of no position re-detection attempt, and 74.59 seconds in the case of a maximum of 2 position re-detection attempts. Averaged over all cases, the total calculation time was 18.83 seconds.

In the above, the present invention has been described through embodiments with reference to the accompanying drawings, but is not limited thereto, and should be interpreted to cover various modifications that can be obviously derived by those skilled in the art. The claims are intended to cover these variations.

110: vocal cord information input unit 160: scanning route determining unit
170: internal air area identification unit 180: injection needle insertion range setting unit
190: needle input line determination unit 193: air passage path removal unit
195: maximum strength calculation unit 197: minimum stiffness path selection unit
205: reinforcement learning (RL) learning unit 210: vocal cord positioning unit
220: vocal cord position detection unit 223: right vocal cord detection neural network
227: left vocal cord detection neural network 230: vocal cord position verification unit
245: initial position setting unit

Claims (10)

In the injection needle insertion guide device for the treatment of vocal cord diseases,
a vocal cord information input unit that receives 3D volume data composed of a neck CT image and vocal cord position information detected from the CT image; and
a scan path determining unit for determining a straight line path passing through the least obstacle among radial straight paths from the vocal cords to the skin on the outside of the neck;
Including, the injection needle insertion guide device. The method according to claim 1, wherein the scan path determining unit:
a maximum intensity calculation unit for obtaining an intensity value of a CT image for each path on the radial straight path and selecting a maximum intensity value of each path as a stiffness value of the corresponding path; and
a minimum stiffness path selection unit for selecting a straight line path having a minimum stiffness value among the radial straight paths;
Including, the injection needle insertion guide device. The method according to claim 1, wherein the scan path determining unit:
a needle injection range setting unit that excludes a path out of a preset anatomically possible range among the radial straight paths;
Including, the injection needle insertion guide device. The method according to claim 1, wherein the scan path determining unit:
an internal air region identification unit identifying a region containing air inside the neck in the CT image; and
an air passage path removing unit for excluding a path passing through an area identified as an area containing air inside the neck from among the radial straight paths;
Including, the injection needle insertion guide device. The method according to claim 4, wherein the internal air area identification unit:
an air area dividing unit dividing air areas using a HU threshold in the CT image;
a GD conversion unit that performs GD conversion on the CT image based on the outside air area of the CT image; and
an internal air region determining unit that selects, as an internal air region, a portion of the air region that is disconnected from the external air region identified by GD conversion;
Further comprising a, injection needle insertion guide device. In the injection needle insertion guide method for the treatment of vocal cord diseases,
a vocal cord information input step of receiving 3D volume data composed of a neck CT image and vocal cord position information detected from the CT image; and
a scanning path determination step of determining a straight line path passing through the minimum obstacle among radial straight paths from the vocal cords to the external skin of the neck;
Including, the injection needle insertion guide method. The method according to claim 6, wherein the scanning path determining step is:
a maximum intensity calculation step of obtaining an intensity value of a CT image for each path on the radial straight path and selecting a maximum intensity value of each path as a stiffness value of the corresponding path; and
a minimum stiffness path selection step of selecting a straight path having a minimum stiffness value among the radial straight paths;
Including, the injection needle insertion guide method. The method according to claim 6, wherein the scanning path determining step is:
a needle injection range setting step of excluding a path out of a predetermined anatomically possible range among the radial straight paths;
Including, the injection needle insertion guide method. The method according to claim 6, wherein the scanning path determining step is:
an internal air region identification step of identifying a region containing air inside the neck in the CT image; and
an air passage path removal step of excluding a path passing through an area identified as an area containing air inside the neck from among the radial straight paths;
Including, the injection needle insertion guide method. The method according to claim 9, wherein the internal air area identification step is:
an air area classification step of classifying an air area using a HU threshold in the CT image;
a GD conversion step of performing GD conversion on the CT image based on the outside air area of the CT image; and
an internal air region determination step of selecting, as an internal air region, a portion of the air region disconnected from the external air region identified by GD conversion;
Including, the injection needle insertion guide method.

Images