KR20190130310A - Catheter tracking system and controlling method thereof - Google Patents

Abstract

Disclosed are a catheter tracking system capable of accurately detecting the position of a catheter without additional devices such as markers and a control method thereof. The control method of the catheter tracking system comprises the steps of: photographing a catheter; dividing the first image frame photographed by the catheter into patches of a predetermined size; calculating the variance of pixel gradations for each of the divided patches; identifying the area of the catheter for patches with variance in a preset first threshold range, wherein the variance of the patches is calculated based on a convolutional neural network; and extracting position information of the catheter based on the identified area of the catheter.

Description

Catheter tracking system and control method {CATHETER TRACKING SYSTEM AND CONTROLLING METHOD THEREOF}

The present disclosure relates to a catheter tracking system and control method, and more particularly to a catheter tracking system and control method for tracking the catheter position using neural network learning.

Recently, according to the development of medical robot technology, a medical device for performing surgery using a small medical tool such as a catheter through minimal incision has been developed. In order to efficiently control a medical device performing surgery using a small medical tool such as a catheter, a technique for identifying a catheter position in real time in an image is required.

Medical devices, such as robotic catheter, must be precisely controlled to increase the success rate of the procedure, which requires techniques to track the catheter position in real time for precise control. Basic image processing techniques, such as binarization using thresholding and hessian filtering, are fast, but weak in image noise, and machine learning techniques based on general learning models require high computational throughput. It is not suitable for real-time tracking for catheter control.

In addition, a technique of acquiring position information of a catheter by attaching a marker to a body has been proposed. However, the catheter position calculation requires the surgeon to attach a plurality of markers to the patient's body surface prior to surgery. Attaching the marker to the patient's body causes inconvenience. In addition, there is a problem in that the accuracy of the image taken according to the marker attachment position, number, etc. is reduced, and an error occurs according to the marker attachment position and the movement of the patients.

Therefore, there is no need for additional devices such as markers for catheter position tracking or image correction, and for attaching additional devices, and there is a need for a technique capable of increasing the accuracy and detection speed of catheter position detection.

SUMMARY The present disclosure is directed to solving the above-described problems, and an object of the present disclosure is to provide a catheter tracking system and a control method capable of accurately detecting the position of a catheter without an additional device such as a marker.

According to an embodiment of the present disclosure for achieving the above object, the control method of the catheter tracking system comprises the steps of photographing the catheter, dividing the first image frame photographed by the catheter into a patch of a predetermined size, Calculating a variance of pixel gradations for each of the divided patches; identifying a region of the catheter for a patch whose calculated variance is within a preset first threshold range based on a convolutional neural network And extracting location information of the catheter based on the identified area of the catheter.

In addition, the convolutional neural network has a U-net structure, and the identifying may include: a pixel identified by the catheter based on a patch within the preset first threshold range, a foreground value, and the remaining pixels are a background. By generating a binary map that maps to the value of) can be identified the area of the catheter.

In addition, the U-net structure may include a convolutional layer and a deconvolutional layer, and each of the convolutional layer and the deconvolutional layer may perform a batch normalization process to reduce a learning time.

In addition, the deconvolution layer may perform a dropout process.

On the other hand, if the gray level of the pixel identified by the catheter is less than the second preset threshold value, the identifying may modify the mapping value of the pixel of the gray scale smaller than the second preset threshold value in the binary map as a background value. Can be.

The extracting may include generating the entire binary map of the first image frame size based on the binary map of the patch within the preset first threshold range and the binary map of the remaining patches in which all pixels are mapped to the background value. Can be.

The extracting may include extracting coordinate information of each pixel identified by the catheter based on the generated binary map, and extracting location information of the catheter by averaging the extracted coordinate information.

The control method of the catheter tracking system divides a next image frame of the first image frame into a patch having a predetermined size for a partial region based on the extracted position information of the catheter, and patches the divided partial region. The method may further include identifying an area of the catheter based on the method.

If the patch of the divided partial region does not include the catheter region, the control method of the catheter tracking system divides the entire next image frame into patches having a predetermined size, and pixels for each of the divided patches. The method may further include identifying the area of the catheter based on the variance of the gray level.

According to an embodiment of the present disclosure for achieving the above object, a catheter tracking system includes a processor for dividing a catheter, a camera for photographing the catheter, and a first image frame in which the catheter is photographed into patches of a predetermined size. Wherein the processor calculates a variance of pixel gradations for each of the divided patches and based on a convolutional neural network, the catheter for the patch for which the calculated variance is within a preset first threshold range. Identifying the area of the, and extracting the location information of the catheter based on the identified area of the catheter.

In addition, the convolutional neural network has a U-net structure, and the processor determines that a pixel identified as the catheter is a foreground value and the remaining pixels are a background value based on a patch within the preset first threshold range. By generating a binary map that is mapped to can identify the area of the catheter.

In addition, when the gray level of the pixel identified by the catheter is smaller than a second preset threshold value, the processor may modify a mapping value of a pixel of gray scale smaller than the second preset threshold value in the binary map as a background value. .

The processor may generate the entire binary map of the first image frame size based on the binary map of the patch within the preset first threshold range and the binary map of the remaining patches in which all pixels are mapped to the background value. .

The processor may extract coordinate information of each pixel identified by the catheter based on the generated whole binary map, and extract location information of the catheter by averaging the extracted coordinate information.

The processor may divide a next image frame of the first image frame into a patch having a predetermined size for a partial region based on the extracted location information of the catheter, and based on the patch of the divided partial region, The area of the catheter can be identified.

As described above, according to various embodiments of the present disclosure, the catheter tracking system and the control method are convenient because no additional device such as a marker is needed and no additional work such as an attaching operation is required.

In addition, the catheter tracking system and control method can perform a real-time processing of the image including the catheter region automatically because the performance is fast.

In addition, catheter tracking systems and control methods can achieve robust tracking performance for variables such as catheter's rapid position change and image noise.

In addition, catheter tracking systems and control methods can quickly track catheter and improve detection accuracy.

1 is a diagram illustrating a catheter tracking system according to an embodiment of the present disclosure.
2 is a diagram illustrating a framework of a catheter tracking system according to an embodiment of the present disclosure.
3 is a diagram illustrating a process of detecting a catheter from a frame after the first frame according to an embodiment of the present disclosure.
4 is a comparison result of the accuracy of the method according to the present disclosure and the existing method.
5 is a flowchart illustrating a catheter tracking system control method according to an exemplary embodiment.
6 is a flowchart illustrating a control method after a second frame according to an embodiment of the present disclosure.

Hereinafter, various embodiments will be described in more detail with reference to the accompanying drawings. Embodiments described herein may be variously modified. Specific embodiments are depicted in the drawings and may be described in detail in the detailed description. However, the specific embodiments disclosed in the accompanying drawings are only for easily understanding the various embodiments. Therefore, the technical spirit is not limited by the specific embodiments disclosed in the accompanying drawings, and it should be understood to include all equivalents or substitutes included in the spirit and scope of the invention.

 Terms including ordinal numbers such as first and second may be used to describe various components, but these components are not limited by the terms described above. The terms described above are used only for the purpose of distinguishing one component from another.

In this specification, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof. When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

On the other hand, "module" or "unit" for the components used in the present specification performs at least one function or operation. The module or unit may perform a function or an operation by hardware, software, or a combination of hardware and software. In addition, a plurality of "modules" or a plurality of "parts" other than a "module" or a "part" to be performed in specific hardware or performed by at least one control unit may be integrated into at least one module. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In addition, in describing the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be abbreviated or omitted. Meanwhile, although each embodiment may be independently implemented or operated, each embodiment may be implemented or operated in combination.

1 is a diagram illustrating a catheter tracking system according to an embodiment of the present disclosure.

Referring to FIG. 1, the catheter tracking system 100 includes a catheter 110, a camera 120, and a processor 130. The catheter 110 may be a medical instrument inserted into the patient's body to examine or treat the affected area of the patient 1. The camera 120 captures a video including the catheter 110. The camera 120 may be disposed at a position where the catheter 110 and the patient 1 may be photographed.

The processor 130 divides the first image frame of the video photographed by the catheter 110 into a patch having a predetermined size. For example, the resolution of the first image frame may be 960 × 960, and the processor 130 may divide the first image frame into patches of 160 × 160 size. That is, the processor 130 may divide the first image frame into 36 patches. The above-described resolution of the first image frame, the size of the divided patch, and the like are exemplary embodiments, and the catheter tracking system 100 may shoot at various resolutions and may split the patch into various sizes.

 The processor 130 calculates a variance of the gray scale for each of the divided patches. For example, the processor 130 calculates the variance of the pixel gray levels of the first patch based on the gray levels of 25600 pixels for the 160 × 160 first patch. In the same manner, the processor 130 may calculate the variance of the pixel gray levels for the 36 divided patches. The processor 130 identifies the area of the catheter for the patch whose variance calculated based on the convolutional neural network is within a preset threshold range. When the catheter 110 is included in the divided patch, the gray level of the catheter region in the divided patch may appear as a high value. On the other hand, the gradation of the remaining areas may appear to be relatively low. Therefore, the dispersion of the patch including the catheter region may be higher than the dispersion of the patch without the catheter region.

The preset threshold range may be set to various ranges based on the image photographed by the camera 120. For example, the range of the preset threshold may be 10 to 150. When the variance of the pixel gray levels of the first patch is less than 10 or greater than 150, the processor 130 does not perform the process of identifying the area of the catheter in the first patch. In other words, the processor 130 may skip the first patch. When the variance of pixel gray levels of the second patch is greater than 10 and less than 150, the processor 130 performs a process of identifying a region of the catheter in the second patch.

For example, the process of identifying the catheter region may be performed based on the convolutional neural network of the U-net structure. The process of identifying the catheter region can be learned by inputting the image frame including the catheter to the composite product neural network of the U-net structure. The multiplicative neural network of the U-net structure trained in identifying the catheter region may identify the catheter region from the new image frame. The processor 130 may generate a binary map by mapping a pixel identified as a catheter to a foreground value and mapping the remaining pixels to a background value in a patch whose dispersion is within a preset threshold range. For example, the foreground value may be 1 and the background value may be 0. A detailed process of identifying the area of the catheter will be described later.

The processor 130 extracts the location information of the catheter (or the location information of the catheter tip) based on the identified area of the catheter. As described above, the processor 130 may generate a binary map for a patch whose distribution is within a preset threshold range. In addition, the processor 130 may generate a binary map in which all pixels are mapped to a background value for a patch whose variance is out of a preset threshold range. The processor 130 may generate the entire binary map of the size of the first image frame based on the binary map of the patch in which the variance is within a preset threshold range and all the pixels mapped to the background value. The processor 130 may extract coordinate information of each pixel identified by the catheter based on the generated whole binary map, and extract catheter position information by averaging the extracted coordinate information. The above-described example is an embodiment, and the processor 130 may extract only the coordinate information of the outer pixel among the pixels identified as the catheter and extract the catheter position information, and extract only the coordinate information of a specific pixel among the pixels identified as the catheter. Catheter position information can be extracted. For example, the particular pixel may be set to the right end pixel of the catheter, 3 pixels from the right end, and so on.

The catheter tracking system may be implemented in the form of a system in which the catheter, camera and processor are separate devices, or may be a catheter device or a catheter robot implemented as one device.

2 is a diagram illustrating a framework of a catheter tracking system according to an embodiment of the present disclosure.

The catheter tracking system captures an image frame 11 containing the catheter. For example, the resolution of the image frame 11 in which the catheter is photographed may be 960 × 960. The catheter tracking system divides 12 the photographed first image frame 11 into a plurality of patches. For example, the size of the divided patch may be 160 × 160. The catheter tracking system may calculate a variance of all pixel gray levels included in the divided patch and determine whether the catheter tracking system is within a preset threshold range. For example, the preset threshold range may be set to 10 to 150 and may be set to 180 to 350. The range of the preset threshold may be set to an appropriate range based on characteristics of the image photographed by the camera 120.

The catheter tracking system may identify the area of the catheter for the patch 12-1 in which the variance of the pixel gray levels is within a preset threshold range. In one embodiment, the process of identifying the catheter region may be performed based on the convolutional neural network of the U-net structure. Synthetic multiplication neural network of U-net structure is a neural network that processes data in U shape, and includes a plurality of convolution layers, a plurality of deconvolution layers, a plurality of pooling layers, and a plurality of unpooling data. And an unpooling layer. In one embodiment, the multiplicative neural network of the U-net structure may include 10 convolutional layers, four pulling layers, four unpooling layers, and nine deconvolution layers. Convolutional and pooling layers reduce the size of the image and can extract features from low to high dimensions. The unpooling layer and the deconvolution layer may increase in size to predict the foreground area (eg, catheter area) and the background area. The multiplicative neural network of U-net structure can improve the image resilience compared to other techniques by using the output of the convolutional layer as input in the corresponding deconvolutional layer. The convolutional neural network of the U-net structure may be trained to identify an area of the catheter by inputting an image frame including a catheter. The multiplicative neural network of the U-net structure trained in identifying the catheter region may identify the catheter region from the new image frame.

The catheter tracking system may generate a binary map 13-1 in which the pixels identified as catheter in the patch are mapped to the foreground value and the remaining pixels are mapped to the background value. For example, the foreground value may be 1 and the background value may be 0. The catheter tracking system may generate a binary map corresponding to each patch through the above-described process based on all the patches whose distribution of pixel gray levels is within a preset threshold range. In addition, the catheter tracking system may generate a binary map in which all pixels of the patch whose variance of pixel gradations fall outside a preset threshold range are mapped to a background value. The catheter tracking system may reconstruct the generated binary map to produce a full binary map 13 of original size.

Occasionally, noise may be extracted at the boundary portion of the catheter region. Therefore, the noise removal process for the catheter region boundary portion may be performed. If the gray level of the pixel identified as the catheter is smaller than the preset foreground threshold value, the catheter tracking system may modify the mapping value of the pixel of the gray level smaller than the preset foreground threshold value in the generated binary map as the background value. For example, the catheter tracking system may have a foreground threshold set to 100. The two pixel gray levels of the boundary portion of the catheter region may be 180 and 80, respectively. Initially, the catheter tracking system can map both pixels to foreground values. However, during the noise reduction process, the catheter tracking system can modify the mapping value of 80 pixels smaller than the foreground threshold from the foreground value to the background value. The catheter tracking system can reduce noise in the catheter region through the above-described process.

The catheter tracking system may map the entire binary map 13 to the original image frame 14. The catheter tracking system may extract the location information of the catheter based on the entire binary map 13. The catheter tracking system may extract coordinate information of each pixel identified by the catheter based on the generated binary map, and extract catheter position information by averaging the extracted coordinate information. The extracted catheter position information may be relative position information. Also, as an example, since the camera is fixed, the catheter tracking system may identify absolute position information of the catheter based on camera calibration information and coordinate information of the catheter in the image frame.

Meanwhile, the convolutional layer and the deconvolutional layer may perform a batch normalization (BN) process to improve the learning speed (or convergence speed) in the learning process. The multiplicative neural network of U-net structure can improve the learning speed and become more robust against bad initial values In one embodiment, the batch normalization process is performed in the second convolutional layer and the second deconvolutional layer from the last. Can be performed.

In addition, the deconvolution layer of the convolutional neural network may perform a dropout process to prevent overfitting. Since the multiplicative neural network is trained based on limited data, an overfitting problem can be generated that shows good performance only for the learning model. Therefore, the convolutional neural network may exhibit good performance on newly input data by performing a dropout process on the deconvolution layer.

The catheter tracking system may be learned with a learning rate set to 0.001 for learning optimization. When the catheter tracking system is trained, a loss function is defined and the process of finding a parameter that minimizes the loss function using gradient descent may be performed. The learning rate refers to the degree of movement along the slope of the gradient descent method.

As described above, the catheter tracking system of the present disclosure performs the training and tracking of the composite product neural network only on a partial patch of the image frame without an external marker. Therefore, the catheter tracking system of the present disclosure does not require a marker attachment process through the above-described process, does not cause an error due to a variable of the marker, is robust to noise, enables real-time catheter tracking by fast data processing, and accurate Catheter tracking is possible.

So far, the process of extracting the catheter's position information from the first image frame has been described. The following describes the process of extracting the catheter's position information from the second and subsequent image frames.

3 is a diagram illustrating a process of detecting a catheter from a frame after the first frame according to an embodiment of the present disclosure.

Afterwards, the catheter tracking system can generate a 160 × 160 patch based on the center position of the catheter found in the previous frame, considering that the position change of the catheter between frames is less than a certain level.

Referring to FIG. 3, there is shown a second image frame 20 that includes a catheter 3-2. The catheter 3-1 indicated by a dotted line means a catheter included in the first image frame. That is, the catheter 3-1 indicated by the dotted line included in the first image frame is slightly moved so that the catheter 3-2 may be included in the second image frame 20 at a different position from the first image frame.

The catheter tracking system may divide the next image frame into a patch of a predetermined size for a partial region based on the location information of the catheter extracted from the first image frame. For example, the catheter tracking system may divide the patch into a patch having a predetermined size for a portion of the surrounding area based on the center position information of the catheter extracted from the first image frame. As an example, as shown in FIG. 3, the catheter tracking system extracts location information of the catheter 3-1 from the first image frame. Position information (eg, center position information) of the catheter 3-1 extracted from the first image frame may be included in the first patch 21-1. The catheter tracking system divides some peripheral areas into patches 21-2 and 25-1 around the first patch 21-1 including the position information of the catheter 3-1 extracted from the previous image frame. Can be. As described above, the catheter may be included in the second patch 25-1 of the patches of the partial peripheral area in successive image frames. Accordingly, the catheter tracking system can calculate the variance of pixel gray levels for the patches of the peripheral partial regions, and identify the area of the catheter for the patch whose calculated variance is within a preset threshold range. In the example of FIG. 3, the catheter tracking system can identify the catheter region only for the second patch 25-1 and extract the catheter's location information. The process of identifying and extracting the catheter region is the same as the process described above.

The catheter tracking system may identify the catheter region and extract the catheter's location information in the same manner as the above-described process for the second and subsequent image frames.

Meanwhile, the catheter may not be included in the divided peripheral patch due to the sudden change in the position of the catheter. Therefore, when the catheter is not included in the divided peripheral patch, the catheter tracking system may perform a process of extracting the area of the catheter from the above-described first image frame. The catheter tracking system may determine that the divided peripheral patches do not include the catheter when the variance of pixel gray level for the divided peripheral patches is not within the preset threshold range or when the U-net estimation results are all in the background. have.

4 is a comparison result of the accuracy of the method according to the present disclosure and the existing method.

For the evaluation of the present disclosure, 11 robot catheter videos obtained from the DGIST-ETH microrobot research center were used as experimental data. A semi-automatic particle filter-based tracking technique that requires user input to generate a catheter label for each image frame was applied to all the videos to create the end label.

Cross-validation was performed by dividing the data into two for verification, and the accuracy of the detection was determined based on the results obtained by the technique of the present disclosure and the dice score between the labels. In addition, a comparison result of detection accuracy and execution time is shown in FIG. 4 based on a detection result of applying a multiplicative neural network having a U-net structure to the entire image for comparison with an existing technique.

When the multiplicative neural network of the U-net structure was applied to the whole image, an average score of 88.1% was obtained, but the technique of the present disclosure showed 93.8% accuracy. In terms of execution time, the conventional technique took an average of 71 ms when using a PC equipped with an NVIDIA GTX 1080 graphics card, but the technique of the present disclosure took 5 ms. Thus, the technique of the present disclosure exhibits about 5% higher detection performance while operating about 14 times faster.

So far, various embodiments of the catheter tracking system have been described. In the following, a flowchart of a catheter tracking system control method is described.

5 is a flowchart illustrating a catheter tracking system control method according to an exemplary embodiment.

The catheter tracking system captures the catheter (S510). The catheter tracking system divides the first image frame in which the catheter is photographed into patches of a predetermined size (S520). For example, the resolution of the first image frame may be 960 × 960, and the size of the divided patch may be 160 × 160. The catheter tracking system calculates a variance of pixel gray levels for each of the divided patches (S530).

The catheter tracking system identifies the area of the catheter for the patch whose variance calculated based on the convolutional neural network is within a preset first threshold range (S540). For example, the convolutional neural network may be a U-net structure. The U-net structure may include a convolutional layer, a pooling layer, a deconvolutional layer, and an unpooling layer, and each of the convolutional layer and the deconvolutional layer may perform a batch normalization process to reduce learning time. . In addition, the deconvolution layer may perform a dropout process to prevent overfitting.

The catheter tracking system may generate a binary map in which pixels identified by the catheter are mapped to a foreground value and remaining pixels are mapped to a background value based on a patch within a preset first threshold range. In addition, the catheter tracking system may generate a binary map that maps all pixels of the patch not included in the preset first threshold range to the background value. Meanwhile, the catheter tracking system may further perform a noise canceling process. For example, when the gray level of the pixel identified as the catheter is smaller than the second preset threshold, the catheter tracking system may modify the mapping value of the pixel of the gray scale smaller than the second preset threshold value in the binary map as a background value. Can be.

The catheter tracking system extracts location information of the catheter based on the identified catheter's area (S550). The catheter tracking system may generate the entire binary map of the first image frame size based on the binary map of the patch within the preset first threshold range and the binary map of the remaining patches in which all pixels are mapped to the background value. The catheter tracking system may extract coordinate information of each pixel identified by the catheter based on the generated binary map, and extract the catheter position information by averaging the extracted coordinate information. For example, the location information of the catheter may be center location information of the catheter or may be location information of a specific area.

On the other hand, the catheter tracking system can identify the catheter region and extract the catheter's location information in a simpler manner for the second and subsequent image frames.

6 is a flowchart illustrating a control method after a second frame according to an embodiment of the present disclosure.

Referring to FIG. 6, the catheter tracking system captures the catheter (S610). The catheter tracking system divides the catheter into a patch having a predetermined size for a portion of the image frame after the catheter is photographed based on the extracted location information of the catheter (S620). In most cases, the movement of the catheter in successive image frames can be small. Therefore, the catheter tracking system may divide the patch into a patch having a predetermined size with respect to the area around the catheter extracted from the previous frame.

The catheter tracking system calculates a variance of pixel grayscales based on the patches of the divided partial regions (S630). The catheter tracking system determines whether an area of the catheter exists (S640). The catheter tracking system may determine that a region of the catheter exists when there is a patch whose distribution of pixel gray levels of the patch is within a preset threshold range. If the catheter tracking system determines that there is an area of the catheter, the catheter may be identified and the location information of the catheter may be extracted by performing the steps S540 and S550 described with reference to FIG. 5.

On the other hand, the catheter tracking system may determine that there is no catheter region in the patch of some regions when there is no patch whose pixel gradation is within a preset threshold range or when the U-net estimation results are all in the background. Can be. The catheter tracking system may identify the catheter and extract the catheter's position information by performing processes S520 to S550 on the entire image frame, as described with reference to FIG. 5.

The control method of the catheter tracking system according to the various embodiments described above may be provided as a computer program product. The computer program product may include a software program itself or a non-transitory computer readable medium in which the software program is stored.

The non-transitory readable medium refers to a medium that stores data semi-permanently and is readable by a device, not a medium storing data for a short time such as a register, a cache, a memory, and the like. Specifically, the various applications or programs described above may be stored and provided in a non-transitory readable medium such as a CD, a DVD, a hard disk, a Blu-ray disk, a USB, a memory card, a ROM, or the like.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the above-described specific embodiment, the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

100: catheter tracking system 110: catheter
120: camera 130: processor

Claims (15)

Photographing the catheter;
Dividing the first image frame photographed by the catheter into a patch having a predetermined size;
Calculating a variance of pixel gray levels for each of the divided patches;
Identifying a region of the catheter for a patch whose calculated variance is within a preset first threshold range based on a convolutional neural network; And
Extracting location information of the catheter based on the identified area of the catheter; and controlling the catheter tracking system. The method of claim 1,
The composite product neural network is a U-net structure,
The identifying step,
Identifying a region of the catheter by generating a binary map in which a pixel identified as the catheter is mapped to a foreground value and remaining pixels are mapped to a background value based on a patch within the preset first threshold range; Control method of catheter tracking system. The method of claim 2,
The U-net structure includes a convolutional layer and a deconvolutional layer, and each of the convolutional layer and the deconvolutional layer performs a batch normalization process to reduce a learning time. The method of claim 3,
The deconvolution layer performs a dropout process. The method of claim 2,
The identifying step,
If the gray level of the pixel identified as the catheter is smaller than a second preset threshold, controlling the catheter tracking system to modify the mapping value of the pixel of the gray scale smaller than the second preset threshold value in the binary map to a background value Way. The method of claim 2,
The extracting step,
And generating a full binary map of the first image frame size based on the binary map of the patch within the preset first threshold range and the remaining binary map of all the patches mapped to all background pixels. . The method of claim 6,
The extracting step,
And extracting coordinate information of each pixel identified by the catheter based on the generated whole binary map, and extracting location information of the catheter by averaging the extracted coordinate information. The method of claim 1,
The next image frame of the first image frame is divided into patches having the predetermined size for the partial region based on the extracted positional information of the catheter, and the region of the catheter is identified based on the divided partial region patches. And controlling the catheter tracking system. The method of claim 8,
When the patch of the divided partial region does not include the region of the catheter, the entire next image frame is divided into patches having a predetermined size, and the catheter of the catheter is based on the distribution of pixel gray levels for each of the divided patches. Identifying an area; further comprising a catheter tracking system. Catheter;
A camera for photographing the catheter; And
And dividing the first image frame photographed by the catheter into a patch having a predetermined size.
The processor,
Calculating a variance of pixel gradations for each of the divided patches, identifying a region of the catheter for a patch whose calculated variance is within a preset first threshold range based on a convolutional neural network, A catheter tracking system for extracting location information of the catheter based on the identified catheter region. The method of claim 10,
The composite product neural network is a U-net structure,
The processor,
Identifying a region of the catheter by generating a binary map in which a pixel identified as the catheter is mapped to a foreground value and remaining pixels are mapped to a background value based on a patch within the preset first threshold range; Catheter tracking system. The method of claim 11,
The processor,
And when the gray level of the pixel identified as the catheter is smaller than a second preset threshold value, modifying a mapping value of the pixel of gray scale smaller than the second preset threshold value in the binary map to a background value. The method of claim 11,
The processor,
And generating a full binary map of the first image frame size based on the binary map of the patch within the preset first threshold range and the rest of the patch's binary map that maps all pixels to a background value. The method of claim 13,
The processor,
And extracting coordinate information of each pixel identified by the catheter based on the generated whole binary map, and extracting the location information of the catheter by averaging the extracted coordinate information. The method of claim 10,
The processor,
The next image frame of the first image frame is divided into patches having the predetermined size for the partial region based on the extracted positional information of the catheter, and the region of the catheter is identified based on the divided partial region patches. Catheter tracking system.

Images