JP2015100712A - Surgical needle for surgical system with optical recognition function - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide improved image processing devices for use in image-guided surgery.SOLUTION: A system for image-guided surgery includes an image processing system, a medical tool comprising a shaft and a tip, and a display configured to display real-time representation information of the medical tool including position information of the tip regardless of whether the tip is covered or visible. The shaft includes an optically detectable feature that allows a position of the tip to be determined.

Description

(Refer to related applications)
This application claims priority based on US application Ser. No. 14 / 092,843 filed Nov. 27, 2013, the entire contents of which are incorporated herein by reference.

  The presently claimed technical field relating to embodiments of the present invention relates to a suture needle used with an image processing apparatus, and more particularly to an image processing apparatus including a sensor for monitoring and tracking the suture needle.

  In the field of image-guided surgical procedures, tracking and positioning of image processing apparatuses and medical instruments during medical practice is a particularly important technology, and the main fundamental technology in image-guided surgical systems (hereinafter referred to as “IGS”). It is believed that. The tracking techniques can be classified into the following four groups. 1) Machines including active robots (eg Da Vinci® Surgical Robot from Intuitive Surgical, USA) and passive programmable mechanical arms (eg FaroArm® from Faroe Technologies, UK) There are four types: type tracking, 2) optical tracking, 3) acoustic tracking, and 4) electromagnetic (hereinafter referred to as “EM”) tracking.

  On the other hand, ultrasound is known as a powerful image processing means in image guided surgical procedures such as ablation procedures, biological tissue examinations, radiotherapy and surgery. Therefore, by integrating such an optical tracking system or EM tracking system with an ultrasound (hereinafter referred to as “US”) image processing system, for example, tracking / guidance in liver resection or radiotherapy using external light Research into ultrasound-guided surgical procedures applicable to the field is ongoing. Such research is being carried out both in the literature and in field research in laboratories [E. M.M. Boctor, M.M. DeOliviera, M.M. Choti, R.A. Ghanem, R.A. H. Taylor, G.M. Hager, G. Fichinger, "Ultrasound Monitoring of Tissue Ablation via Deformation Model and Shape Priors, International Conferencing on Medical Image Competing in Attenuation." Rivaz, I.D. Fleming, L.M. Assumpcao, G.M. Fichtinger, U. Hamper, M.M. Choti, G. Hager, and E.M. Boctor, "Ablation monitoring with elastography: 2D in-vivo and 3D ex-vivo studies," International Conference on Medical Image Computing and Computer-Attenuator. Rivaz, P.M. Foroughhi, I .; Fleming, R.M. Zellars, E .; Boctor, and G.B. Hager, “Tracked Regulated Ultrasound Elastography for Targeting Breast Radiotherapy”, Medical Image Computing and Computer Assisted Intervention (MICCI9). Some systems on the market today integrate an EM tracking device into a high-end cart-based US system. Some small EM sensors are integrated into the ultrasound probe, while others are attached to and attached to the surgical instrument of interest.

  Whether for research or commercial purposes, the limitations of today's approach are attributed to the feasibility of currently available tracking technologies, system integration and system use in medical environments. For example, a mechanical tracking system is an expensive and intrusive solution, requires a large installation space and restricts user movement. On the other hand, the acoustic tracking system does not provide sufficient guidance accuracy. The optical tracking system technology requires intrusive equipment such as a base camera, and the EM tracking system technology also requires intrusive equipment such as a reference EM transmitter. Furthermore, the optical rigid body and the EM sensor must be attached to the image processing apparatus and all necessary surgical instruments, which requires offline measurement adjustment work and disinfection sterilization work. Thus, there remains a need for improved image processing devices for image guided surgery.

  Some aspects of the present invention include systems, instruments and methods. In one embodiment, an image guided surgical system is provided. The system is a medical device having a shaft portion and a tip portion, provided that the shaft portion of the medical device is optically detectable and can determine the position of the tip portion. A medical device having a non-uniform non-repeating pattern, an optical image processing system for detecting and tracking the position of the medical device regardless of which segments of the non-uniform non-repeating pattern are visible, and the optical image A display device for displaying real-time representation information of the medical device including position information of the tip based on data received from a processing system, and the non-uniform non-repeating pattern includes non-uniform non-repetitive white space and width. An image invitation characterized in that it comprises at least three partial arrays, and the pattern of each partial array appears only once. It is surgery system.

  In another embodiment, an instrument for use with an image guided surgical system is shown. The instrument includes a distal end portion and a shaft portion that is provided integrally with the distal end portion or attached to the distal end portion, and is used with an image guiding medical system, wherein the shaft portion is optically Having a detectable non-uniform non-repeating pattern, wherein the non-uniform non-repeating pattern is either imprinted, printed, etched, or applied to the shank, and the non-uniform non-repeating pattern is A non-uniform non-repeating blank and width fill array, the array having at least three partial arrays, and each partial array pattern appearing only once, and the non-uniform non-repeating pattern is: Whether the image-guided medical system is obscured by the corresponding image regardless of which segments of the non-uniform non-repeating pattern are visible Regardless, an instrument, characterized in that suitable for use in the image-guided medical system Upon determining the position of the tip in real time.

  In yet another embodiment, an image processing method of an image guided medical system is shown. The method includes receiving an input including light reflected from an instrument shaft from one or more optical devices by one or more processor devices, and detecting the position of the instrument by the one or more processor devices. An image processing method comprising: tracking; and displaying, by the one or more processor devices, an insertion depth of a tip of the instrument and one or more candidate positions of the tip. The shank has an optically detectable non-uniform non-repeating pattern, the non-uniform non-repeating pattern comprising a non-uniform non-repeating blank and width array of fills, the array comprising at least three sub-arrays And the pattern of each subsequence appears only once, and for the tracking, the one or more processor units can detect which segments of the non-uniform non-repetitive pattern Regardless of whether the visible, an image processing method characterized by it involves calculating the one or more candidate positions of the tip portion in the three-dimensional space using the non-uniform non-repeating pattern.

Further objects and advantages of the invention will become apparent upon consideration of the detailed description, drawings and embodiments of the invention.
FIG. 1 shows an image processing component of an image processing system according to an embodiment of the present invention. FIG. 2 shows an image processing system according to an embodiment of the present invention. FIG. 3 illustrates a sample tracking pattern according to an embodiment of the present invention. FIG. 4 shows an example of a workflow according to an embodiment of the present invention. FIG. 5 shows an exemplary embodiment of a computer and system construction for performing the method according to the invention. FIG. 6 shows a screen copy of the instrument tip according to one embodiment of the present invention.

  Several embodiments are described in detail below. In describing embodiments, specific terminology is used for the sake of clarity. However, the present invention is not limited to the specific terms so selected. Those skilled in the relevant art will recognize that other ways in which other equivalent components can be used have been developed without departing from the broad concepts of the present invention. All references cited anywhere in this specification are incorporated into this application by reference as if each were individually incorporated.

  As used herein, the terms “needle” and “medical instrument” refer to a long medical instrument such as a needle, pointer, biopsy instrument, laparoscope, resection device, surgical instrument, or long tool. Used as

  Some embodiments of the present invention describe "base technology" that enables IGI (image guided surgical procedures) beyond today's relatively narrow system of image guidance and tracking. The “foundation technology” aims to overcome the limitations of tracking, visualization and guidance at the same time. Specifically, techniques such as utilization and integration of technologies related to identification and tracking of medical devices using three-dimensional computer images and pattern projections, and tracking of image processing devices using local sensing techniques. As an example of IGI, US patent application No. 13 / US patent application entitled “Low-cost image-guided navigation and intervention systems of cooperating sensors” published in US 2013/0016185 / No. It is done.

  The present invention shares a tightly integrated common core of components and methods used for general imaging, projection, vision, and local sensing, covering a wide range of different embodiments.

    Some embodiments of the present invention provide a local sensing approach that implements medical imaging device tracking technology, for example, the potential to significantly reduce errors and increase positive patient outcome (medical outcome) Is a combination of complementary technologies with This approach can provide a foundational technology for tracking an image processing device such as an ultrasound probe, guiding treatment intervention and visualizing information according to embodiments of the present invention. Combining ultrasonic image processing with an image analysis algorithm according to an embodiment of the present invention, a photosensitive device equipped with a probe, and an independent optical inertial sensor, the latest movement of surgical instruments, other instruments or objects can be incrementally increased. By tracking, the position and trajectory can be reconstructed.

  In some embodiments of the present invention, medical devices and other devices can be segmented, tracked, and guided (using visual, ultrasound, and / or other imaging and localization methods). .

  Such an apparatus can allow an imaging procedure with improved sensitivity and specificity compared to current state of the art. This can be useful for applications that previously required harmful x-ray / CT or expensive MRI imaging, external tracking, expensive, inaccurate and time consuming and impractical hardware setup or accuracy and success I was very worried about the fatal lack of warranty: biopsy, RF / HIFU removal, etc .: 2D or 3D ultrasound based medical tool guidance, brachytherapy can be allowed: 3D ultrasound acquisition and medical instruments Open up a range of applications for guidance, precise brachytherapy species placement, tracked imaging, and other applications that rely on tracked instruments.

  Some embodiments of the present invention can provide several advantages over existing technologies. Specifically, low cost tracking, an ideal tracking system that combines local, compact and non-intrusive solutions, low cost tracking for handheld and compact ultrasound systems, local compact And non-intrusive solution / ideal tracking system is mainly used to intervene and track under visual tracking method in setting of other surgical procedures as well as used in point-of-care clinical suite Can provide several advantages over existing technologies such as a combination of simple tools.

  For example, some embodiments of the invention relate to an apparatus and method for tracking ultrasound probes and other image processing devices. In accordance with embodiments of the present invention, the position and trajectory of surgical instruments and other objects can be tracked by incrementally tracking the current motion by combining image analysis algorithms, probe-mounted photosensitive devices, and ultrasound imaging. It is possible to reconfigure. This allows application to some scenes that conventionally required expensive, inaccurate or unrealistic hardware setup. As an example, guidance of a medical device by three-dimensional ultrasound can be mentioned.

  Today's ultrasound examinations mainly use portable 2D ultrasound (US) probes that return a planar image slice through a scanned three-dimensional region ("region of interest" (ROI)). Today, in percutaneous surgical procedures that require guidance of a medical instrument, trajectory prediction of the medical instrument relies on tracking by a sensor attached to the distal (external) end of the medical instrument, and the operator's experience. The current situation is based on the ideal estimation of the trajectory. Systems that integrate 3D ultrasound, medical instrument tracking, medical instrument trajectory prediction, and interactive user guidance are very beneficial.

  FIG. 1 shows an embodiment of an image processing component 100 of an image processing system according to an embodiment of the present invention. The image processing component 100 includes an image processing device 110 and a bracket 120 configured to be detachable from the image processing device 110. In the example of FIG. 1, the image processing instrument 110 is an ultrasonic probe, and the bracket 120 is attached to the probe handle of the ultrasonic probe. The ultrasonic probe can be, for example, Ultrasonix # c5-2 manufactured by Analogic of Boston, Massachusetts. However, the broad concept of the present invention is not limited to this example. The bracket 120 can be configured to attach to other handheld instruments for image guided surgery, such as orthopedic power tools, stand-alone portable brackets, and the like. In other embodiments, the bracket 120 may be configured to be removable from, for example, a C-arm of an X-ray system or an MRI system.

Image processing component 100 may include a top shell 180 and a bottom shell 130 that are coupled together to form a head shell. Top shell 180 and bottom shell 130 may be securely coupled to stabilization assembly 170 (eg, a stabilization bar). The headshell can house the stabilization assembly 170 and other components that make up the image processing component 100. Screw 190 may be used to connect components of image processing component 100.

  The image processing component 100 may include one or more photosensitive devices 150 that are securely coupled to the stabilization assembly 170. In some embodiments of the present invention, the one or more photosensitive devices 150 are comprised of a visible light camera, an infrared camera, a time-of-flight camera, a PSD (position sensitive device) or reflective laser sensing. In the embodiment of FIG. 1, one or more photosensitive devices 150 can be arranged to observe the surface area during operation of the image processing component 100. In the embodiment shown in FIG. 1, the one or more photosensitive devices 150 may be arranged so as to be suitable for stereoscopic observation of the region of interest.

  The image processing component 100 may include a printed circuit board 140 that may include one or more microprocessors, one or more light emitting devices, and a memory device. The light emitting device can emit light in the visible spectrum, infrared, ultraviolet, or other spectrum. The light emitting device may include at least one of an LED, a light bulb, a CFL, and a laser. The printed circuit board 140 may be connected to one or more photosensitive devices 150 and securely coupled to the stabilization assembly 170.

  The image processing component 100 may further include a lens 160 that provides a screen for one or more photosensitive devices 150. In one embodiment, the lens 160 may be made of ultra-rigid gorilla glass having a thickness of 0.031 mm. The lens 160 may be entirely or partially frosted glass in order to diverge light emitted from one or more light emitting devices. The light emitted from the light emitting device may be diffused by a frosted glass surface, mesh, translucent tape, or other means.

  FIG. 2 shows an embodiment of an image processing system 200 according to an embodiment of the present invention. Image processing system 200 includes an image processing component 100 that is controlled by a user. Moreover, the user has inserted the medical instrument. The image processing system 200 includes an image display device 210. The image display device 210 displays an ultrasonic inspection image output from the image processing instrument 110. The image processing system 200 further includes an extended display device 220. The extended display device 220 is a touch panel that can be input by a user. The extended display device 220 displays the tracking information superimposed on the ultrasonic inspection image output from the image processing instrument 110. The tracking information includes the current tracking status, current position, current insertion depth of medical instruments, surgical instruments, and other instruments inserted by the user. The position of the tip of the medical instrument and the distance above the distance of the medical instrument tip to the selected target are also displayed superimposed.

  In FIGS. 1 and 2, an ultrasound imaging system is shown as an image processing system, and the bracket 120 is configured to be attached to an image processing instrument 110 of an ultrasound probe. It is not limited to examples. The bracket may be configured to be detachable from other image processing systems such as, but not limited to, X-ray or magnetic resonance imaging systems.

FIG. 3 shows an example of a tracking pattern according to this embodiment of the present invention. The lengths of the medical devices 310, 320 and 330 are 381 mm, 201 mm and 150 m, respectively
The length of the tracking pattern is the same length. The medical device includes a shaft portion and a tip portion. The tracking pattern is imprinted, printed, etched, or applied to a medical instrument shaft or other linear instrument. The image processing system 200 analyzes a tracking pattern on a medical instrument shaft or other linear instrument. The tracking pattern can include sequences in which each subsequence of any minimum length appears only once. The tracking pattern can include a pseudo-random binary array (PRBS), a de-Blanc array, or a barcode. The tracking pattern can be found in E.I. M.M. It may contain sequences generated by the method described by Petriu, “Absolute Positioning Measurement Using Pseudo-Random Binary Encoding”. The tracking pattern may be a non-uniform non-repeating pattern consisting of a non-uniform and non-repeating blank and width filled array, the array having at least three sub-arrays, and The pattern may be an arrangement that appears only once.

In addition, the tracking pattern may include a unique pseudo-random binary array (PRBS) pattern or de-Blanc pattern applied to the ring on the medical device or to the shaft of the medical device. The image processing system 200 identifies a medical device using a part of the identifiable pattern and knows which part of the medical device is the position of the tip of the medical device in a unique or finite number. By grasping as a set of individual possibilities, the insertion depth of the medical device can be visually tracked. The grasped insertion depth can be displayed on the medical instrument guidance display device (for example, the extended display device 220) so as to overlap the candidate position of the medical device. The medical instrument guidance display device can display real-time expression information of a medical instrument including position information of the distal end regardless of whether the distal end is covered or visible. Even if a part of the pattern is not optically detectable by the image processing system or only a part of the minimum length is optically detectable by the image processing system, image processing is performed due to the characteristics of the pattern. The system can accurately calculate the insertion depth of the tip of the medical device. Specifically, the image processing system can take k types of values (colors or shades) that are different from each other in which the pattern can be distinguished, and the total length of the pattern is the length corresponding to the number k ^{n of the} respective components. in the case of the, at least n / k ⁿ portions of the pattern when it is optically detectable by the image processing system, it is possible to accurately calculate the insertion depth.

The tracking pattern is first generated algorithmically. Specifically, E.I. M.M. Petriu al., "Absolute Positioning Measurement Using Pseudo-Random Binary Encoding " (http: surasshusurasshuwww.csi.uottawa.casurasshu~petriusurasshuELG5161 / PRBS / encoding.pdf ) pseudorandom binary code length of ^{2 n-1} by the method described in Is generated. In this technique, first, arbitrary n binary values (bits) are generated. Next, an exclusive OR of two bits at a specific position is calculated, and the result of such calculation is added to the end of the array, resulting in an n + 1 bit array. This procedure is repeated for the last ⁿ bits of the array, until finally a 2 ^n-1 bit array is generated. The tracking pattern may be aperiodic and can provide a unique location on the medical device. Regardless of which part of the medical instrument is visible, the tracking pattern allows the image processing system 200 to identify the position on the medical instrument having the pattern and detect the insertion depth. All medical devices may have the same pattern, and each device may have its own pattern to remove the ambiguity of the device in the visible region.

  In another embodiment, a mounting component such as a clip-on ring is secured to the medical device. The distance can be used to indicate to the image processing system 200 to determine the distance of the tip of the medical device with the ring attached from the mounting piece or ring and to estimate the tip position. The ring can be circular, elliptical, cylindrical or handle shaped. The mounting component or clip-on ring may be reflective (eg, infrared reflective).

  The pattern on the medical device may have infrared reflectivity. Having infrared reflectivity facilitates the detection of medical devices and helps eliminate false positives. Image processing system 200 includes a photosensitive device with a filter that allows light of a specific wavelength to pass through, a light source that illuminates the medical device, and a coating on the medical device that, when illuminated, generates or reflects light of a wavelength that passes through the filter of the photosensitive device You may have material. The photosensitive device may include one or more photosensitive devices 150, one or more PSDs (position sensitive devices), or combinations thereof. Further, the medical device may be fitted with a visible light visible pattern and an infrared visible pattern, or a ring. This allows the operator to generally see the operator's eyes (for example, a centimeter scale is swayed) but not visible to the human eye to identify the position where the photosensitive device can be optically detected. Medical instruments that convey useful information can be used.

  In other embodiments, the pattern can be used to reject or minimize false positive detection of medical devices. That is, by detecting only objects having predetermined visual characteristics, it is possible to avoid detection and tracking of unimportant objects. The patterns are distinguished by different combinations of visible features, for example, and can be used for disambiguation between different instruments. The pattern can be used to determine the suitability of the tool for a desired or current use (eg, to reach a particular depth target). If the tool is not appropriate, a warning can be displayed to the operator. The pattern can be used to verify calibration between photosensitive devices observing the photosensitive device and the instrument with features. Such verification can be accomplished by comparing the apparent heights (eg, measured) of features in a data set from multiple photosensitive devices and rejecting the calibration if the heights are different. it can. The pattern can also be used to improve the detection performance of the instrument by processing only monitored objects that exhibit the required pattern.

  FIG. 4 shows an example workflow 400 according to an embodiment of the present invention. In step 410, at least one photosensitive device 150 receives light. Such received light includes light reflected from the shaft of the medical device. The shaft or other device of the medical device has one or more visual features. The one or more visual features include a detectable pattern that was initially created using a pseudo-random binary code and is stamped on the shaft of the medical device. Either printed, etched, or fitted.

  In other embodiments, the visual feature includes a ring attached to the shaft of the medical device. The ring reflects light, and the cylindrical shape has a handle shape. The ring includes a detectable pattern that is used to calculate the insertion depth of the tip of the medical instrument, and the detectable pattern is first created using a pseudo-random binary code. The image processing system 200 first calculates the distance from the ring to the tip of the medical instrument, and uses the calculated distance to adjust the image processing system 200 for tracking the medical device. The workflow then moves from step 410 to step 420.

  In step 420, the insertion depth of the distal end portion of the medical instrument is calculated. The insertion depth is calculated based on one or more visual features provided on the shaft of the medical device. Even if a portion of the one or more visual features is not optically detectable by the at least one photosensitive device due to the characteristics of the one or more visual features, The insertion depth can be calculated accurately. For example, if the visual feature includes a detectable pattern that was initially created using a pseudo-random binary code, such a pattern is a non-repetitive pattern, so only a small portion of the pattern is optically detected. If not possible, the image processing system 200 can calculate the insertion depth. The workflow then moves from step 420 to 430.

  In step 430, the position of the tip of at least one medical device (eg, candidate position of the tip) is calculated using the one or more visual features. The calculated position of the distal end portion may be a three-dimensional plane, and may be based on the insertion position, the calculated insertion depth, and the medical device entry angle. The workflow then moves from step 430 to 440.

  In step 440, the insertion depth of the distal end portion of the medical instrument and the candidate position of the distal end portion are displayed on the extended display device 220. The surgeon or other medical personnel may use the displayed information, for example, when performing an IGI (image guided surgical procedure). The workflow ends with 440.

  In one embodiment, medical instrument tracking may be performed through one or more visual features of the instrument. (The basic method of instrument tracking is described in the literature already published by the inventor of the present invention. Specifically, Stolka et al. “Navigation with local sensors in handheld 3D ultrasound: initial in-vivo. "experence," SPIE Medical Imaging 2011, Lake Buena Vista, FL, USA, pp. 79681J-79681J. International Society for Optics and Photonics. cal Imaging, San Diego, CA, USA, pp. 83160U-83160U. International Society for Optics and Photonics, 2012 is incorporated into this application by reference.) Visual features include detectable patterns; The pattern is first created using a pseudo-random binary array, more commonly a de-Blanc array, and is imprinted, printed, etched or applied to the instrument. The pattern is used to detect the depth of insertion of an instrument that has entered the human body or animal. Alternatively, the visual feature may be an attachment part such as a ring attached to the instrument. The ring reflects sound and light and has a cylindrical shape or a handle shape. The ring may include a detectable pattern used to calculate the insertion depth of the instrument tip, which can be initially created using a pseudo-random binary array. . The image processing system 200 can first calculate the distance from the tip of the instrument to the ring and use the calculated distance to adjust the imaging system 200 so that the instrument can be tracked.

  Information displayed to assist in positioning the medical device may include information about the length of intersection of the medical device and the non-infinitely thin ultrasound image plane by drawing a marker on the medical device line. The extent of the intersection is shown. That is, the medical instrument line represents a trajectory of the medical instrument, and a part of the medical instrument line may indicate a region crossing the ultrasonic image plane, and may be shaded differently.

  The insertion depth can be calculated based on one or more visual features on the instrument. Due to the characteristics of the visual feature, it is possible to accurately calculate the insertion depth of the tip of the instrument even if some of the one or more visual features are not optically detectable from the photosensitive device. It is. For example, if the visual feature is a detectable pattern created using a pseudo-random binary array, the pattern is aperiodic and unique in small segment units. Therefore, the image processing system 200 can calculate the insertion depth even when only a small part of the instrument is optically detectable. By using the one or more visual features, the position of the tip of the instrument (candidate position of the tip) can be calculated. The calculated position of the distal end portion may be a three-dimensional plane, and may be based on the insertion position, the calculated insertion depth, and the medical device entry angle. The insertion depth of the tip of the instrument and the candidate position of the tip are displayed on the extended display device 220. The surgeon or other medical personnel may use the displayed information, for example, when performing an IGI (image guided surgical procedure).

  Hereinafter, an example of a technique for specifying the position of the distal end portion of the medical device in the stereoscopic image using the pattern of the shaft portion of the medical device in one embodiment will be described. As a first procedure to specify the position of the tip from a state where a pair of stereoscopic images (left and right photosensitive device images) is obtained and adjustment of the photosensitive device (internal and external parameters of the photosensitive device) is completed First, the left and right images are corrected. Next, in both images, the medical device is detected as a line (straight line) centered on the center of the shaft portion. In order to determine the position of the tip of the medical instrument in the three-dimensional image, the medical instrument line is reconstructed in the three-dimensional space. Therefore, the medical instrument line is sampled with a certain difference to obtain a 3D point set, the 3D point is re-projected on the left and right images, and two corresponding to the two corrected left and right images respectively. Obtain a 2D point set. Therefore, the pixel intensity is calculated using an interpolation method. This results in two intensity vectors with regular sampling. The two intensity vectors are then associated with all possible subpatterns. The sub-pattern is a continuous minimum length portion of the entire pattern and can be specified as a unique one. The position where the correlation is maximized and the correlation value are recorded for each sub-pattern. For each of the left and right vectors, the sub-pattern showing the maximum correlation value is selected. Since the offset (relative position) with the tip of the sub-pattern is known, the three-dimensional position of the tip can be estimated. It should be noted here that the left and right images provide two almost independent estimates of the tip position. As a verification procedure, it is verified that the two estimated tip positions must be closer than a threshold value. The final tip position is determined as a weighted average of these two estimated tip positions.

  FIG. 6 shows a screen copy of the instrument tip according to one embodiment of the present invention. FIG. 6 shows a screen copy of the extended display device 220 on which the ultrasonic image 610 is displayed. The extended display device 220 further displays the current position of the medical instrument including the tip and the expression information of the medical instrument 620 representing the future trajectory 630. The medical instrument is displayed as a double line. Are displayed with a plurality of lines perpendicular to the medical device, and the future trajectory 630 is displayed with a single line.

  In another embodiment, the light waves may be filtered by one or more photosensitive devices, allowing only certain wavelengths of light to pass and blocking other wavelengths of light. The medical device may be coated so that it emits light only when it receives light of a specific wavelength. The covering material generates or reflects light having a specific wavelength. The generated or reflected light of a specific wavelength may be detected by a photosensitive device. The generated or reflected light of a specific wavelength may reduce the occurrence of false positives. Furthermore, the covering material may emit or generate only light of a specific wavelength that the detectable pattern appears. The candidate position and insertion depth of the tip of the instrument may be calculated based on the displayed detectable pattern expressed by light of a specific wavelength.

[Example Computer System]
FIG. 5 illustrates an example of a computer system used to implement an embodiment of the present invention. Specifically, FIG. 5 illustrates an example of a computer system 500 used in a computer apparatus such as, but not limited to, a stand-alone device, a client device, and a server device. FIG. 5 shows an embodiment of a computer system used as a client device or a server device. The invention (including portions thereof and one function thereof) may be implemented using hardware, software, firmware, or combinations thereof, and may be implemented in one or more computer systems or other processing systems. Indeed, in one exemplary embodiment, the invention is implemented in one or more computer systems capable of performing the functions described herein. An example of a computer system 500 is shown in FIG. 5, which depicts a block diagram of a computer system in which the present invention can be implemented. Specifically, FIG. 5 includes, but is not limited to, Windows (registered trademark) NT / 98/2000 / XP / Vista / Windows 7 / Windows 8 / of Microsoft Corporation (registered trademark) in Redmond, Washington, USA. , A personal computer (PC) running an operating system such as MAC (registered trademark) OS, OS X, and iOS of Apple Corporation (registered trademark) of Cupertino, California, USA, or Linux (registered trademark) or other UNIX derivatives An embodiment of the invention with a computer running is shown. However, the present invention is not limited to these platforms, and the present invention can be implemented on a suitable computer running a suitable operating system. In one embodiment, the present invention may be implemented in a computer system as discussed herein. FIG. 5 shows a computer 500 constituting such a computer system. For example, but not limited to, computing device, communication device, telephone, personal digital assistant (PDA), iPhone (iPhone), 3G / 4G wireless device, wireless device, personal computer (PC), handheld PC, laptop Computer, Smartphone, Mobile device, Netbook, Handheld device, Portable device, Interactive TV device (iTV), Digital video recorder (DVR), Client workstation, Thin client, Thick client, Fat client, Proxy server, Network communication server Remote access devices, client computers, server computers, peer-to-peer devices, routers, web servers, data, And breakfasts, audio, video, telephone, or the like streaming technology servers, other components of the present invention other than the computer 500 may be implemented using a computer such as shown in FIG. In such exemplary embodiments, for example, using an interactive television device (iTV) via a digital video recorder (DVR), video on demand (VOD), and other on-demand viewing systems. Services can be provided on demand. As described in FIGS. 1 and 2, the computer system 500 can be used to implement components such as the network and image processing component 100 or other image processing system 200.

  The computer system 500 may be one or more processors 504, such as, but not limited to, the processor 504. The one or more processors 504 can be connected to a communication infrastructure 506 (eg, but not limited to a communication bus, crossover bar, interconnect, or network). The processor 504 may be any type of processor, microprocessor, or processing logic (eg, field programmable gate array (FPGA)) that can interpret and execute instructions. The processor 504 may be a single device (eg, single core) or a group of multiple devices (eg, multi-core). The processor 504 can include logic configured to execute computer-executable instructions configured to implement one or more embodiments. The instructions can reside in main memory 508 or secondary memory 510. The processor 504 may also include multiple independent cores such as a dual core processor or a multi-core processor. The processor 504 may also be in the form of one or more dedicated graphics cards, integrated graphics solutions, and may include a graphics processing unit (GPU), and / or a hybrid graphics solution. Various exemplary software embodiments can be described in terms of this exemplary computer system. After reading this description, it will become apparent to one skilled in the relevant art how to implement the invention using other computer systems and / or architectures.

  The computer system 500 includes a display interface 502 for displaying graphics, text, and other data on the display unit 501 from the communication infrastructure 506 (or a frame buffer not shown), such as, but not limited to. be able to. The display unit 501 is a screen of a television, a computer monitor or a mobile phone, for example. The output may be provided as sound from a speaker.

  The computer system 500 can include a main memory 508, a random access memory (RAM), a secondary memory 510, and the like. Main memory 508, random access memory (RAM), secondary memory 510, etc., may be computer readable media that store instructions for implementing the embodiments, such as dynamic RAM (DRAM) devices, flash memory devices, static memory devices. It may be a RAM device such as a RAM (SRAM) device.

  The secondary memory 510 includes, for example, but not limited to, a hard disk drive 512 and a movable storage device 514. Examples of the movable storage device 514 include a floppy (registered trademark) disk, a magnetic tape device, an optical disk drive, a CD- There are ROM, flash memory, and the like. For example, the movable storage device 514 reads / writes data from / to the movable storage unit 518 by a well-known method. The movable storage unit 518 may be referred to as a program storage device or a computer program product, and includes, but is not limited to, a floppy disk, a magnetic tape, and an optical disk that read and write data from and to the movable storage device 514. As can be understood from the above, the movable storage unit 518 includes a storage medium usable by a computer storing a computer program and data.

  In another exemplary embodiment, secondary memory 510 may include other similar devices for loading computer programs or other instructions into computer system 500. For example, there is a movable storage unit 522 and an interface 520. Further, program cartridges, cartridge interfaces (eg, but not limited to those found in video game devices), removable memory chips (eg, but not limited to, read-only erasable programmable memory (EPROM)) , There is a read-only programmable memory (PROM) and associated socket, and other movable storage unit 522 and interface 520 that allow software and data to be transferred from the movable storage unit 522 to the computer system 500 by these devices. Or data is transferred.

  The computer system 500 may further include an input device 503. The input device 503 has a mechanism for inputting data to the computer system 500 from a user or the like, or a combination of such mechanisms. The input device 503 has logic for receiving data to be input to the computer system 500 from, for example, a user. Examples of the input device 503 include, but are not limited to, a mouse, a light pen type pointing device, another pointing device such as a digitizer, a touch panel type display device, a keyboard, and other data input devices (all illustrated. Not included). Examples of other input devices 503 include, but are not limited to, biometric input devices, video sources, audio sources, microphones, webcams, video cameras, photosensitive devices and other cameras.

  The computer system 500 includes an output device 515 that is any mechanism or combination of such mechanisms that can output information from the computer system 500. The output device 515 may include logic for outputting information from the computer system 500. Embodiments of output device 515 include, but are not limited to, display 501, display interface 502, and display, printer, speaker, cathode ray tube (CRT), plasma display, light emitting diode (LED) display, liquid crystal display ( LCD), printer, vacuum fluorescent display (VFD), surface conduction electron-emitting device display (SED), field emission display (FED), and the like. The computer system 500 may include input / output (I / O) devices such as, but not limited to, an input device 503, a communication interface 524, a cable 528, and a communication path 526. These devices include, for example, without limitation, network interface cards, modems, and the like.

  The communication interface 524 transfers software and data between the computer system 500 and an external device.

  In this specification, the terms “computer program medium” and “computer readable medium” are not limited to these, but include a movable storage device 514, a hard disk in a hard disk drive 512, a flash memory, a movable disk, a fixed disk, and the like. Widely point to the medium in general. In addition, various communications such as wireless communications, conductive wires (eg, but not limited to, twisted pair, CAT5, etc.) or optical media (eg, but not limited to optical fibers) etc. It should be noted that the electromagnetic radiation can be encoded to transmit computer-executable instructions and / or computer data. Such a computer program provides software to the computer system 500. A computer readable medium having computer-executable instructions executed on a processor may be configured to store a program implementing various embodiments of the present invention. References to “one embodiment”, “an embodiment”, “one embodiment”, and “various embodiments” may include certain features, structures, or characteristics of the invention so described. It may be shown that not every embodiment includes the particular feature, structure, or characteristic.

  Further, repeated use of the phrases “in one embodiment” or “in an exemplary embodiment” may refer to the same embodiment, but is not necessarily so. Various embodiments in this specification may be combined, and new embodiments may be formed by combining features of the embodiments.

  Unless otherwise specified, as will be apparent from the following discussion, these terms are used throughout the discussion in this specification using terms such as “processing”, “computing”, “calculation”, and “judgment”. For example, data representing a physical quantity such as an electronic quantity in a register or memory of a system is processed and transformed into different data that similarly represents a physical quantity, and the computer memory, register or other information storage device, communication device or It can be understood that it refers to the action or process of a computer, computer system or similar electrical computing device stored on a display device.

  Similarly, the term “processor” refers to any device or part of an apparatus that processes electronic data from a register and / or memory to convert that electronic data into other electronic data stored in a register. Sometimes. Reference to a “computing platform” can include one or more processors.

  Embodiments of the invention can include an apparatus for performing the operations described herein. One device may be individually constructed for a desired application, or a general-purpose device that can be selectively activated or reconfigured by a program stored in the device may be constructed.

  Embodiments can be embodied as software components in many different ways. For example, it may be a stand-alone software package or, for example, a scientific modeling product, which may be a software package that is embedded as part of a large software product as a “tool”. It may be downloadable from a network, for example a website, as a stand-alone product or as an add-in package for installation of existing software applications. Also available as a client-server software application or as a web-enabled software application. It may also be part of a system for detecting network coverage and responsiveness. A general purpose computer may be specialized by storing programming logic so that the techniques described herein and the steps of FIG. 4 can be performed by one or more processors.

  Embodiments of the invention can include an apparatus for performing the operations described herein. One device may be individually constructed for a desired application, or a general-purpose device that can be selectively activated or reconfigured by a program stored in the device may be constructed.

  The above-described embodiments can be embodied as software components in many different ways. For example, it may be a stand-alone software package or a software package incorporated as a “tool” of a larger software product. It may also be downloadable from a network, such as a website, as a stand-alone product or as an add-in package installed in an existing software application. Further, it may be available as a client / server type software application, or may be available as a web-compatible software application.

  Although various embodiments of the present invention have been described, it should be understood that these are presented by way of example and not limitation. Accordingly, the scope of the invention should not be limited by any of the above-described embodiments, but should instead be defined only by the following claims and their equivalents.

DESCRIPTION OF SYMBOLS 100 Image processing component 110 Image processing instrument 120 Bracket 130 Bottom shell 140 Printed circuit board 150 Photosensitive device 160 Lens 170 Stabilization assembly 180 Top shell 190 Screw 200 Image processing system 210 Image display device 220 Expanded display device 310 Medical device 320 Medical device 330 Medical instrument 610 Ultrasound image 620 Medical instrument 630 Future trajectory

Claims (30)

A medical device having a shaft portion and a tip portion, wherein the shaft portion of the medical device is optically detectable and makes it possible to determine the position of the tip portion. A medical device having a repeating pattern;
An optical image processing system for detecting and tracking the position of the medical device regardless of which segments of the non-uniform non-repeating pattern are visible;
A display device for displaying real-time expression information of the medical instrument including positional information of the tip based on data received from the optical image processing system;
The non-uniform non-repeating pattern comprises a non-uniform non-repeating blank and width fill array, the array has at least three sub-arrays, and each sub-array pattern appears only once A featured image guided surgical system.   The image-guided surgical system according to claim 1, wherein the medical instrument includes any one of a needle, a pointer, a biological tissue examination instrument, a laparoscope, a resecting device, a surgical instrument, and a long instrument. .   The image-guided surgical system according to claim 1, wherein the optical image processing system calculates an insertion depth of the medical instrument and a candidate position of at least one tip.     The non-uniform non-repeating pattern is detectable by the optical image processing system, and the optical image processing system is based on the non-uniform non-repeating pattern on the condition that the non-uniform non-repeating pattern is detected. Calculation of the insertion depth of the tip of the medical device and / or tracking of the tip of the medical device are performed, and the non-uniform non-repeating pattern is such that each partial sequence of any minimum length appears only once. The image-guided surgical system according to claim 1, wherein the image-guided surgical system is based on a non-arranged sequence. A non-repeating pattern for a medical device having a shaft portion and a tip portion, wherein the shaft portion of the medical device is optically detectable and allows the position of the tip portion to be determined. A medical device having
An optical image processing system that detects and tracks the position of the medical device regardless of which segments of the non-repeating pattern are visible;
A display device for displaying real-time expression information of the medical instrument including positional information of the tip based on data received from the optical image processing system;
The image-guided surgical system according to claim 1, wherein the non-repetitive pattern is composed of either a de-Blanc array or a pseudo-random binary array.   The image-guided surgery of claim 1, wherein the non-uniform non-repeating pattern is either stamped, printed, etched, or applied to the shaft of the medical device. system.   The optical image processing system accurately calculates the insertion depth when only a portion of the minimum length of the non-uniform non-repeating pattern is optically detectable by the optical image processing system. Item 8. The image-guided surgical operation system according to Item 1. In the optical image processing system, each distinguishable component of the non-uniform non-repeating pattern can take different k kinds of values, and the length of the non-uniform non-repetitive pattern is ^{equal to the} number of the component k ⁿ . in case of a length, wherein the insertion depth can be calculated accurately when the least n / k ⁿ portions of non-uniform non-repeating pattern is optically detectable by the optical imaging system The image-guided surgical system according to claim 3.   The image-guided surgical system according to claim 1, further comprising an attachment part connected to the medical instrument.   The distance from the attachment part to the tip part is calculated by the optical image processing system, and the optical image processing system is adjusted to an actual distance from the attachment part to the tip part. Item 10. The image-guided surgical system according to Item 9.   The attachment part is made of at least one of a clip-on ring and a reflective material, and the clip-on ring is one of a circular shape, an elliptical shape, a cylindrical shape, and a handle shape. Image guided surgical system.   The mounting part includes a pattern detectable by the optical image processing system, and the pattern is used by the optical image processing system for calculation and tracking of the insertion depth, and the pattern first has a periodic arrangement. The image-guided surgical system according to claim 9, wherein the periodic array is an array in which each partial array having an arbitrary minimum length appears only once.   The optical image processing system further includes one or more photosensitive devices that are sensitive to light of a specific wavelength and restrict light of other wavelengths, and one or more light-emitting devices that irradiate the covering material of the medical instrument, The image-guided surgical system according to claim 1, wherein the covering material emits or reflects light having the specific wavelength in accordance with the irradiation.   The image-guided surgical system according to claim 13, wherein the one or more photosensitive devices include one or more cameras.   The dressing exhibits an illuminated pattern, the pattern is detectable by the optical image processing system, and the pattern is used by the optical image processing system for calculating and tracking the insertion depth, The image-guided surgical procedure of claim 13, wherein the pattern is initially created using a periodic array, wherein the periodic array is an array in which each sub-sequence of any minimum length appears only once. system. The tip,
An instrument used with an image guiding medical system, comprising a shaft portion integrally provided with the tip portion or attached to the tip portion,
The shank has an optically detectable non-uniform non-repeating pattern, and the non-uniform non-repeating pattern is at least either imprinted, printed, etched, or applied to the shank. The non-uniform non-repeating pattern comprises a non-uniform non-repeating blank and width fill array, the array having at least three sub-arrays, and each sub-array pattern appears only once. Without
The non-uniform non-repeating pattern indicates whether the image-guided medical system is obscured by the corresponding image regardless of which segments of the non-uniform non-repeating pattern are visible. Regardless of this, the apparatus is suitable for use in the image guided medical system in determining the position of the tip in real time. Receiving an input including light reflected from the shaft of the instrument from one or more optical devices by one or more processor devices;
Detecting and tracking the position of the instrument by the one or more processor devices; and displaying the insertion depth of the tip of the instrument and one or more candidate positions of the tip by the one or more processor devices. An image processing method including a process,
The shank of the instrument has an optically detectable non-uniform non-repeating pattern, the non-uniform non-repeating pattern comprising an array of fills having non-uniform non-repeating blanks and widths, the array comprising at least It has three sub-arrays, and the pattern of each sub-array appears only once, and for the tracking, which segment of the non-uniform non-repeating pattern is visible by the one or more processor devices Regardless of the method, the method includes calculating one or more candidate positions of the tip in a three-dimensional space using the non-uniform non-repetitive pattern.   The image processing method of claim 17, wherein the non-uniform non-repeating pattern minimizes false positive detection.   The image processing method according to claim 17, further comprising determining an instrument type of the instrument based on the non-uniform non-repeating pattern.   The method of claim 1, further comprising: determining whether the determined instrument type is suitable for a current application; and displaying one or more warnings when an inappropriate instrument is used. 20. The image processing method according to 19.   The method further includes verifying an adjustment between a first optical device and a second optical device of the one or more optical devices, wherein the verification includes the first of the one or more optical devices. Comparing the measured height of the non-uniform non-repeating pattern of the data set obtained from the optical device and the second optical device, and rejecting the adjustment if the measured heights are different The image processing method according to claim 17, further comprising:   The image processing method according to claim 17, wherein only an object recognized to exhibit the non-uniform non-repetitive pattern is processed.   The non-uniform non-repeating pattern is created using a periodic array, the periodic array being an array in which each partial sequence of any minimum length appears only once, and the non-uniform non-repeating pattern is The image processing method according to claim 17, wherein the shaft portion is stamped, printed, etched, or fitted.   The image processing method according to claim 17, further comprising a step of calculating an insertion depth of the tip portion of the instrument using the non-uniform non-repeating pattern of the shaft portion of the instrument.   In the step of calculating the insertion depth of the tip of the instrument, even if a portion of the non-uniform non-repeating pattern is not optically detectable by the one or more optical devices, the tip of the instrument 25. The image processing method according to claim 24, wherein the insertion depth is accurately calculated.   The image processing method according to claim 17, wherein an attachment component is connected to the shaft portion of the instrument. Calculating the distance from the mounting part to the tip,
27. The image processing method according to claim 26, further comprising a step of adjusting an optical image processing system based on the calculated distance from the attachment part to the tip portion.   The mounting part is composed of at least one of a clip-on ring, a reflective material, and a detectable pattern, and the clip-on ring is circular, elliptical, cylindrical, or handle-shaped, and the detectable pattern is the instrument. Is used to calculate the insertion depth of the tip portion, and the detectable pattern is first created using a periodic array, where each periodic array of any minimum length appears only once. 27. The image processing method according to claim 26, wherein the image processing method is an array. Receiving light waves from the one or more optical devices that allow light of a specific wavelength and limiting light of other wavelengths;
Irradiating the covering of the device;
The image processing method according to claim 17, further comprising a step of emitting or reflecting the light having the specific wavelength by the covering material. Displaying the non-repetitive pattern irradiated by the dressing;
Calculating a depth of insertion of the tip of the instrument based on the displayed non-repeating pattern, wherein the non-repeating pattern is initially created using a periodic array, the periodic array being optional 30. The image processing method according to claim 29, wherein each of the partial sequences of the minimum length is an array that appears only once.