KR101635731B1 - Visualization system and method for visualizing inner objects of human - Google Patents

Abstract

The present invention relates to a visualization system and method for visualizing an image representing a human body structure on human skin.
The present invention provides a visualization system for visualizing a human body structure, comprising: a probe that scans at least a part of a body of a subject to acquire an ultrasound image of a body structure of the subject, a probe tracker for providing position information of the probe, A relative position of the body structure to the probe is calculated from the ultrasound image, and based on the positional information of the probe obtained through the photographing unit and the calculated relative coordinates, the three-dimensional coordinates And a projection unit for projecting an image of the body structure on the skin of the subject by referring to the three-dimensional coordinates of the body structure.

Description

TECHNICAL FIELD [0001] The present invention relates to a visualization system and method for visualizing a human body structure,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a visualization system and method for visualizing a human body structure, and more particularly, to a visualization system and method for visualizing an image representing a human body structure on human skin.

In order to help diagnose or treat human body structures, a variety of medical imaging devices have been studied that visualize the organs inside the human body using ultrasound. Examples of such devices include a device for visualizing the lower limb for the purpose of diagnosing a venous disease such as deep vein thrombosis (DVT), a device for visualizing the lower jugular vein (internal jugular vein) or an axillary vein , axillary vein), which are configured to display acquired ultrasound images to a user (e.g., a practitioner) through a screen.

In recent years, techniques for achieving augmented reality (AR) by projecting medical image information on the patient's skin rather than on the screen have been proposed in order to improve the accuracy and intuitiveness of the procedure. As an example of these technologies, the research team at the University of Tokyo and Kobe University in Japan proposed a technique for visualizing various medical image information on the patient's skin by fixing a high-intensity beam projector on the ceiling of the operating room. In 2011, A handheld device capable of projecting a medical image at a desired position and angle using a laser projector has been proposed. However, these devices have problems in that the displayed image is low in accuracy, and the immediacy of the display of the image information and the real-time property are degraded because it is based on a pre-processed computerized tomography (CT) or magnetic resonance imaging (MRI).

U.S. Patent Publication No. 8,199,189 (June 12, 2012) U.S. Patent Publication No. 7,841,751 (Nov. 30, 2010)

"Ultrasound Angioscopy: Real-time, Two-dimensional, Intraluminal Ultrasound Imaging of Blood Vessels", published in 1988, Vol. 62, No. 7, pages 493-494 of the American Journal of Cardiology,

SUMMARY OF THE INVENTION It is an object of the present invention to provide a visualization system and method for visualizing a body structure of a person, which improves the accuracy and intuitiveness by displaying the image of the body structure of the person to be examined more accurately on the skin.

It is another object of the present invention to provide a visualization system and method for visualizing a human body structure with improved instantness and real-time display of image information.

It is still another object of the present invention to provide a visualization system and method for visualizing a human body structure that can more clearly visualize a body structure of a subject by mediating ultrasonic waves instead of near-infrared rays or visible rays.

A visualization system for visualizing a human body structure according to embodiments of the present invention includes a probe for scanning at least a part of a body of a subject to acquire an ultrasound image of the body structure of the subject; A probe tracker for providing position information of the probe; A photographing unit photographing the probe tracker or tracking the position of the probe tracker; A central processing unit for calculating the relative coordinates of the body structure with respect to the probe from the ultrasound image and calculating the three-dimensional coordinates of the body structure based on the positional information of the probe obtained through the imaging unit and the calculated relative coordinates, ; And a projection unit for projecting the image of the body structure on the skin of the subject by referring to the three-dimensional coordinates of the body structure.

In an embodiment, the probe tracker is attached to the probe, and the probe may be further provided with a water pouch for facilitating acquisition of the ultrasound image.

In an embodiment, the probe tracker may be an optical tracker that provides position information of the probe in an optical tracking manner, or a magnetic tracker that provides position information of the probe in a magnetic tracking manner.

In an embodiment, the central processing unit may separate the region corresponding to the body structure from the ultrasound image, and calculate the relative coordinates of the body structure with reference to the separated region.

In an embodiment, the photographing unit further photographs the image of the internal structure projected on the skin of the subject by the projection unit, and the central processing unit monitors the image of the internal structure photographed by the photographing unit, The controller can control the projection unit so that the image of the body structure is not projected when the monitored image of the body structure moves or changes by a predetermined degree or more.

In an embodiment, the photographing unit, the central processing unit, and the projection unit may be embedded in one hardware device.

As an embodiment, there is provided a goggle for detecting a pupil position of a practitioner performing a procedure on the subject; And a goggle tracker for providing positional information of the goggles, wherein the central processing unit is configured to determine, based on the pupil position of the practitioner sensed through the goggles and the positional information of the goggles acquired using the goggle tracker, Dimensional coordinate of the body structure by referring to the visual information of the practitioner, and the projection unit adjusts the three-dimensional coordinate of the body structure according to the three-dimensional coordinate of the body structure , The image of the body structure can be projected onto the skin of the subject.

In one embodiment, the central processing unit separates a skin region corresponding to the skin of the subject from the ultrasound image, calculates three-dimensional coordinates of the skin by referring to the separated skin region and the position information of the probe, The three-dimensional coordinates of the body structure can be converted or adjusted with reference to the three-dimensional coordinates of the skin together with the sight line information of the operator.

In an embodiment, the photographing unit may photograph the goggle tracker or track the position of the goggle tracker to provide the position information of the goggles to the central processing unit.

In an embodiment, the body structure may include veins, organs or tumors of the subject.

According to embodiments of the present invention, there is provided a method of visualizing a human body structure, comprising: scanning at least a part of a body of a subject with a probe to obtain an ultrasound image of the body structure of the subject; Separating a region corresponding to the body structure from the obtained ultrasound image and calculating relative coordinates of the body structure with respect to the probe based on the separated region; Capturing a probe tracker attached to the probe or tracking the position of the probe tracker to obtain position information of the probe; Calculating three-dimensional coordinates of the body structure based on the obtained position information of the probe and the calculated relative coordinates; And projecting the image of the body structure on the skin of the subject by referring to the three-dimensional coordinates of the body structure.

According to the embodiments of the present invention, the practitioner can perform the operation while viewing the image of the body structure projected on the skin of the recipient, whereby the accuracy and intuitiveness of the operation can be greatly improved.

In addition, since the position of the body structure detected through the probe can be projected on the skin of the patient, in real time, the immediacy and real time of the display of the image information can be improved. Further, since ultrasound waves are used instead of near infrared rays or visible rays, It is possible to more clearly visualize the body structure of the body.

1 is a schematic diagram illustrating a detailed configuration of a visualization system and a method for calculating the position of a body structure of a subject by the method according to an embodiment of the present invention.
FIG. 2 is a schematic view showing an operation method when an image of an internal structure is projected onto a skin of a subject according to the position of the internal structure calculated in FIG. 1. FIG.
3 is a side view showing a detailed configuration of the probe shown in Fig.
4 is a perspective view showing a detailed configuration of a visualization system according to another embodiment of the present invention.
FIG. 5 is a side view showing a method of visualizing an image of a body structure of a subject by the visualization system shown in FIG. 4. FIG.
6 is a schematic diagram illustrating a detailed configuration of a visualization system and a method of calculating the position of a body structure of a subject by the visualization system according to another embodiment of the present invention.
FIG. 7 is a schematic view showing an operation method when an image of an internal structure is projected onto a skin of a client according to the position of an internal structure calculated in FIG. 6. FIG.
8 is a flowchart schematically showing a visualization method for visualizing a human body structure according to an embodiment of the present invention.

The following detailed description refers to the accompanying drawings, which illustrate, by way of example, specific embodiments in which the invention may be practiced. Embodiments of the detailed description are provided for those of ordinary skill in the art to disclose the detailed description for carrying out the invention described herein.

Each of the embodiments of the present invention can describe different cases, but it does not mean that the embodiments are mutually exclusive. For example, the specific shapes, structures, and characteristics described in connection with one embodiment of the detailed description may be implemented in other embodiments without departing from the spirit and scope of the present invention. It is also to be understood that the location or arrangement of the individual components of the embodiments disclosed herein may be varied without departing from the spirit and scope of the invention.

On the other hand, in various embodiments, the same or similar reference numerals refer to the same or similar components. In the accompanying drawings, the sizes of the respective components may be exaggerated for explanatory purposes and need not be equal to or similar to the actual applied size.

Among medical image information technologies, medical ultrasonography is a method of visualizing the internal structures such as joints, muscles, or blood vessels by ultrasound for diagnosis and treatment, and is widely used in various medical environments today. Compared with other medical image information technology, ultrasound imaging is cheap, and equipment is simple, and medical image can be obtained in real time. Especially, when vein is visualized, it is much better than near infrared ray or visible ray method A clear vein image can be obtained. However, in the conventional ultrasound imaging, only the obtained image information is displayed only on the screen, and thus the accuracy and intuitiveness of the procedure are severely restricted.

In the present invention, the vein is separated from the acquired ultrasound image to obtain relative coordinates between the ultrasonic probe and the vein, the tracker attached to the ultrasonic probe is tracked by the stereo camera to determine the three-dimensional position of the ultrasonic probe, Dimensional coordinate of the vein by integrating the coordinate and the three-dimensional position of the ultrasonic probe, and projecting the vein image to the position of the patient's skin corresponding to the calculated three-dimensional coordinates of the vein, Thereby achieving the effect of overcoming the disadvantages and improving the accuracy and intuitiveness of the practitioner.

In the following, the present invention will be described in conjunction with more specific embodiments with reference to the accompanying drawings.

 1 is a schematic diagram illustrating a detailed configuration of a visualization system and a method for calculating the position of a body structure of a subject by the method according to an embodiment of the present invention. Referring to FIG. 1, a visualization system 1000 according to the present invention includes a probe 100, a central processing unit 200, a photographing unit 300, and a projection unit 400.

The probe 100 is an ultrasonic probe for acquiring an ultrasound image of an organ in the body of the subject 10 such as the vein 11 in the direction of the scan of the probe 100 on the skin of the subject 10 101 to obtain an ultrasound image at the corresponding position. The ultrasound image obtained by the probe 100 is provided to the central processing unit 200 and used to calculate the relative coordinates of the vein indicating the relative position of the vein 11 with respect to the probe 100. [

Meanwhile, the probe 100 includes a probe tracker 110 for tracking the position of the probe. As an example, the probe 100 may further include a water pocket attached to the probe tip. The specific configuration of the probe 100 will be described later in more detail in Fig.

The photographing unit 300 photographs or tracks the probe tracker 110 of the probe 100 (301). The photographing unit 300 is installed at a position and an angle capable of clearly photographing or tracking the position of the probe tracker 110 and can photograph or track the probe tracker 110 in an optical tracking or magnetic tracking manner, (301). When the photographing unit 300 uses the optical tracking method, the probe tracker 110 will be an optical tracker. On the other hand, when the photographing unit 300 uses the magnetic tracking method, the probe tracker 110 will be a magnetic tracker. The result of the photographing unit 300 photographing or tracking the position of the probe tracker 110 is provided to the central processing unit 200 and used to calculate the three-dimensional position of the vein 11.

As an embodiment, the photographing unit 300 may include a stereo camera that photographs the probe tracker 110 to generate an image.

The central processing unit 200 calculates the three-dimensional coordinates of the vein 11 based on the image information or the tracking information obtained through the probe 100 and the photographing unit 300.

The central processing unit 200 separates the region corresponding to the vein 11 from the ultrasound image of the subject 20 acquired through the probe 100 and then detects the relative position of the vein 11 with respect to the probe 100 (I.e., the relative coordinates of the vein 11 indicating the relative position between each point constituting the vein 11 and the probe 100). The relative coordinates of the calculated vein are the relative coordinates of the vein 11. As an example, the relative coordinates of the calculated vein may be two-dimensional coordinates.

In addition, the central processing unit 200 may be configured to detect the three-dimensional position of the probe tracker 110 and the position of the ultrasonic probe (e.g., the probe) based on the information (e.g., image information) 100 are determined.

The central processing unit 200 calculates the three-dimensional coordinates of the vein 11 by combining the relative coordinates of the vein 11 calculated previously and the three-dimensional position of the ultrasonic probe 100 (calibration).

The projection unit 400 projects (401) the vein image to the skin area of the subject 10 corresponding to the three-dimensional coordinate of the vein 11 calculated by the central processing unit 200, thereby visualizing the vein 11 of the subject do. Then, the practitioner visually observes the projected (visualized) image and performs necessary procedures. The configuration and operation of the projection unit 400 will be described later in more detail in Fig.

Meanwhile, such a series of operations and functions can be continuously performed in real time in accordance with the movement of the probe 100. For example, when a practitioner scans the probe 100 in a certain direction 101 on the skin of the subject 20, the ultrasound image of the subject 20 is acquired along the direction in which the subject is scanned, The movement of the ultrasonic tracker 110 attached to the probe 100 is photographed or tracked (301). At this time, the central processing unit 200 separates the region corresponding to the vein 11 from the ultrasound image acquired through the probe 100, and calculates the relative coordinates of the vein 11 with respect to the probe 100. The central arithmetic unit 200 calculates the three-dimensional positions of the probe tracker 110 and the probe 100 based on the information photographed or tracked by the photographing unit 300, Dimensional coordinates of the vein 11 indicating the coordinates where the vein 11 is actually located is calculated by combining the relative coordinates of the previously calculated vein 11 (calabration). The projection unit 400 projects the vein image on the skin of the subject 10 in real time in accordance with the calculated three-dimensional coordinate (401), and visualizes the vein 11 of the subject.

Various methods can be used as the calibration method of calculating the three-dimensional coordinates of the vein 11 by combining the three-dimensional position of the probe 100 with the relative coordinates of the vein 11. [ For example, common calibration methods that may be applied here include cross-wire phantom, tree-wire phantom, There are various methods such as a single-wall phantom method and a cambridge phantom method. In consideration of the features of each calibration method, one of them may be selectively applied to the present invention . Details of each of the calibration methods are well known in the art, and detailed description thereof will be omitted here.

In FIG. 1, the present invention is illustrated by calculating a three-dimensional position of a vein and then visualizing a vein image based on the calculated three-dimensional position. However, the scope of the present invention is not limited thereto. For example, the present invention can be applied to visualizing a variety of internal structures or internal structures that can be recognized by ultrasound, such as other organs other than intravenous or internal organs.

According to the above-described configurations, the practitioner can perform the operation while viewing the image of the internal structure (for example, vein) projected on the skin of the recipient, and the accuracy and intuitiveness of the operation can be greatly improved.

In addition, since the position of the body structure detected through the probe is projected on the skin of the patient in real time in real time, the immediacy and real time of the display of the image information can be improved. Moreover, since ultrasound waves are used instead of near infrared rays or visible rays, Can be more clearly visualized.

 FIG. 2 is a schematic view showing an operation method when an image of an internal structure is projected onto a skin of a subject according to the position of the internal structure calculated in FIG. 1. FIG. Referring to FIG. 2, the vein image 12 projected by the projection unit 400 of the visualization system 1000 is displayed on the skin of the subject 10.

The projection unit 400 projects the vein image 12 to the skin region of the corresponding subject 10 based on the actual three-dimensional coordinates of the vein 11 calculated by the central processing unit 200 (401). The projection unit 400 may include a projector or laser show equipment capable of projecting an image. At this time, the projection unit 400 may further include a special goggle for providing a vein image in an augmented reality manner instead of a projector or a laser show.

The projection unit 400 may project the vein image 12 onto the skin of the subject 10 and then continue projecting the image so that the vein image 12 is continuously maintained unless there is a separate projection stop command. can do.

 At this time, the photographing unit 300 may be configured to photograph the projected vein image 12 after the vein image 12 is projected onto the skin of the subject 10 by the projecting unit 400 . For this purpose, the photographing unit 300 is installed at an angle and at a position capable of holding the vein image 12 projected onto the skin of the subject 10 to the skin region of the subject 10 . As an embodiment, the photographing unit 300 may be configured to have a plurality of photographing apparatuses, one of which photographs the probe tracker 110 and the other photographs the projected vein image 12. [

The central processing unit 200 monitors the vein image 12 projected on the skin of the subject 10 in real time through the photographing unit 300. When the vein image 12 moves or changes beyond a predetermined degree, It is determined that the subject 10 has moved excessively and a projection stop command is transmitted to the projection unit 400 so that the vein image 12 is not projected any more.

3 is a side view showing a detailed configuration of the probe shown in Fig. Referring to FIG. 3, the probe 100 includes a probe tracker 110, a water pocket 120, and a body 130.

The probe tracker 110 may be an optical tracker that operates optically as a tracker for tracking the position of the probe 100, or a magnetic tracker that operates magnetically. The probe tracker 110 includes at least one sensing unit 111 that is photographed or sensed by the photographing unit 300. In an embodiment, the probe tracker 110 may include at least three sensing units 111. The probe tracker 110 is attached to any point on the body 130 of the probe 100.

The water bag 120 is attached to the end of the probe 100 according to the purpose of use of the probe 100. The water bag 120 may be attached to the end of the probe 100 to facilitate the transmission of ultrasonic signals, for example, when the probe 100 scans an ultrasound image.

The body 130 is a structure for forming and supporting the structure of the probe 100. The probe tracker 110 and the water pocket 120 may be attached to specific points on the body 130. As an embodiment, the main body 130 may further include a control switch (not shown) for controlling the operation of the probe 100.

 4 is a perspective view showing a detailed configuration of a visualization system according to another embodiment of the present invention. In the visualization system 2000 of FIG. 4, the imaging unit 2300, the projection unit 2400, and the central processing unit 2200 may be integrated together in one hardware device. At this time, the probe 2100 may be included in the integration device or may be separated from the integration device.

The concrete functions and operations of the probe 2100, the central processing unit 2200, the photographing unit 2300 and the projection unit 2400 of the visualization system 2000 shown in FIG. 4 are the same as those of the visualization system 1000 shown in FIG. 1 The functions and operations of the probe 100, the central processing unit 200, the photographing unit 300, and the projection unit 400 are substantially the same. However, in FIG. 4, these configurations are different only in that they are constituted by one integrated device.

As an example, the visualization system 2000 of FIG. 4 includes a table 2510 for holding or supporting a body part of a subject to be scanned by the probe 2100 or to which a vein image is to be projected by the projection unit 2400, And a moving wheel 2520 for moving the wheel.

 FIG. 5 is a side view showing a method of visualizing an image of a body structure of a subject by the visualization system shown in FIG. 4. FIG.

5, when the subject 10 places his body part (e.g., upper arm) on the table 2510, the probe 2100 scans a part of the body placed on the table 2510 to produce ultrasound And the photographing section 2300 photographs or tracks the position of the probe 2100 or the probe tracker 2110 (2301). The central processing unit 2200 calculates the three-dimensional coordinates of the vein of the subject 10 based on the position of the ultrasound image and the captured / traced probe 2100. The projection unit 2400 calculates the three- (2401) the vein image of the subject 10 is projected by projecting the vein image onto a part of the body of the subject.

The projection unit 2400 is configured to project a part of the body 10 of the subject 10 that is put on the table 2510 to the perspective of the subject 10, Of the body is fully contained in the angle of its own projection.

Meanwhile, components, operations, and functions of the visualization system 2000 that are not described here are substantially the same as those of the configuration, operation, and function of the visualization system 1000 described above.

6 is a schematic diagram illustrating a detailed configuration of a visualization system and a method of calculating the position of a body structure of a subject by the visualization system according to another embodiment of the present invention. In FIG. 6, the visualization system 3000 according to the present invention is extended to allow various non-vein structures to be visualized.

6, the visualization system 3000 includes a probe 3100, a central processing unit 3200, a photographing unit 3300, and a projection unit 3400, similar to the case described above.

6, the visualization system 3000 acquires an ultrasound image of the body structure 13 of the subject 10 through the probe 3100 and acquires ultrasound images from the ultrasound image acquired through the central processing unit 3200 The relative coordinates of the body structure 13 with respect to the probe 3100 are calculated after the region corresponding to the structure 13 is separated. Then, the probe tracker 3100 is photographed or traced (3301) through the photographing unit 3300 to acquire the three-dimensional position of the probe 3100, and the three-dimensional position of the obtained probe 3100 is transmitted to the body structure 13 (Or calaboration) with the relative coordinates of the body structure 13 to calculate the three-dimensional coordinates of the body structure 13. [

Meanwhile, the visualization system 3000 of FIG. 6 further includes a specialist goggles 3500 for tracking the operator's eyes and providing information on the operator's gaze. The central processing unit 3200 separates the region corresponding to the skin of the subject 10 from the acquired ultrasound image of the subject 10 and then extracts the relative coordinates of the skin of the subject 10 with respect to the probe 3100 More. The central processing unit 3200 calculates the three-dimensional coordinates of the subject's skin 10 based on the relative coordinates of the subject's skin and the three-dimensional position of the probe 3100, as in the case of the body structure 13 .

For example, in the case of visualizing other intravenous structures such as internal organs or tumors, the internal structures to be visualized may be located deep within the body of the recipient, unlike veins. In this case, if the medical image is projected by considering the operator's gaze and the skin height of the recipient, the error due to the depth of the internal structure can be reduced, which is helpful for accurate visualization of the internal structure.

6, a goggle tracker 3510 for photographing or tracking (3302) the position of the operator's goggles 3500 is attached to the operator's goggles 3500, and is inserted into the operator's goggles 3500 during the procedure The gaze tracker 3520 traces the pupil of the practitioner 20 in real time to acquire the pupil position of the practitioner 20. At this time, the photographing unit 3300 is installed at a position and an angle so that the goggle tracker 3510 attached to the operator goggles 3500 can be contained within the viewing angle. As an embodiment, the photographing section 3300 may have a plurality of photographing apparatuses, one of which may photograph the probe tracker 3110 and the other photographs the specialist goggles 3500 for a practitioner.

The central processing unit 3200 captures or tracks the goggle tracker 3510 through the photographing unit 3300 to obtain the three-dimensional position of the goggle 3500 for the operator. The visual information of the practitioner 20 is calculated by referring to the pupil position of the practitioner 20 obtained through the visual tracker 3520 together with the obtained three-dimensional position of the operator's goggles 3500.

The central processing unit 3200 calculates the three-dimensional coordinates of the body structure 13 calculated in advance based on the visual information of the practitioner 20 and the three-dimensional coordinates of the skin of the subject 10, Converting or modifying it to fit.

FIG. 7 is a schematic view showing an operation method when an image of an internal structure is projected onto a skin of a client according to the position of an internal structure calculated in FIG. 6. FIG.

7, the projection unit 3400 is provided on the skin of the subject 10, based on the three-dimensional coordinates of the body structure 13 converted by the central processing unit 3200 in accordance with the sight 3501 of the practitioner 20 The image 14 of the body structure is projected (3401).

The projection unit 3400 may include a projector or laser show equipment capable of projecting an image. The projection unit 3400 may be configured to project the image 14 of the body structure onto the skin of the subject 10 so that the image 14 of the body structure is continuously maintained unless a separate projection stop command is issued The image can be continuously projected.

At this time, after the image 14 of the body structure is projected on the skin of the subject 10 by the projecting unit 3400, the photographing unit 3300 can take the projected image 14 (3303). For this purpose, the photographing unit 3300 is disposed at a position and an angle that allows the photographing unit 3300 to capture the image of the body structure 14 projected onto the skin of the subject 10 to the skin region of the subject 10 Respectively. As an embodiment, the photographing section 3300 may include a separate photographing means or a camera for photographing the image 14 of the projected body structure.

The central processing unit 3200 monitors the image 14 projected on the skin of the subject 10 in real time through the photographing unit 3300 and detects that the image 14 of the projected body structure is moving , It is determined that the subject 10 has moved excessively and a projection stop command is transmitted to the projection unit 3400 so that the image is no longer projected.

8 is a flowchart schematically showing a visualization method for visualizing a human body structure according to an embodiment of the present invention. Referring to FIG. 8, the visualization method of the visualization system includes steps S110 to S140.

In step S110, the visualization system 1000 (see FIG. 1) first scans the physician's body with a probe.

In step S120, the visualization system 1000 acquires an ultrasound image for the previously scanned area. The ultrasound image may include image information about an internal body structure including a vein, an organ, a tumor, and the like of the subject, and image information about the skin of the subject.

In operation S 130, the visualization system 1000 separates the region corresponding to the body structure of the subject to be visualized from the obtained ultrasound image, and calculates the relative coordinates of the body structure of the subject with respect to the probe from the separated region. Then, the position of the probe tracker is photographed or tracked to acquire the three-dimensional coordinates of the probe.

Then, the visualization system 1000 combines (coordinates) the relative coordinates of the body structure or the three-dimensional coordinates of the probe, which are calculated or obtained, and calculates the three-dimensional coordinates of the body structure.

By way of example, the body structure may be any one of the organs located within the body of the recipient, such as the vein, organs or tumor of the recipient.

In step S140, the visualization system 1000 projects an image of the body structure on the skin of the subject according to the calculated three-dimensional coordinates of the body structure.

As an embodiment, the visualization system 1000 calculates the operator's gaze through the operator's goggles as shown in FIGS. 7 and 8, and then calculates the three-dimensional coordinates of the body structure according to the gaze of the operator Transformed or modified, and the three-dimensional coordinates of the transformed or modified body structure can be projected onto the skin of the recipient.

According to the above-described configurations, the practitioner can perform the operation while viewing the image of the internal structure (for example, vein) projected on the skin of the recipient, and the accuracy and intuitiveness of the operation can be greatly improved.

In addition, since the position of the body structure detected through the probe is projected on the skin of the patient in real time in real time, the immediacy and real time of the display of the image information can be improved. Moreover, since ultrasound waves are used instead of near infrared rays or visible rays, Can be more clearly visualized.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.

In addition, although specific terms are used herein, they are used for the purpose of describing the present invention only and are not used to limit the scope of the present invention described in the claims or the claims. Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be determined as set forth in the appended claims.

1000, 2000, 3000: visualization system 100, 2100, 3100: probe
200, 2200, 3200: central processing unit 300, 2300, 3300:
400, 2400, 3400: projecting unit 110, 3110, 3510:
120: water pocket 130: probe body
2510: Table 2520: Mobile wheels
3500: Operator's Goggles 3510: Eye Detector
11, 13: internal structure 12, 14: projection image

Claims (11)

A probe which scans at least a part of the body of the subject and acquires an ultrasound image of the internal structure of the subject;
A probe tracker for providing position information of the probe;
A photographing unit photographing the probe tracker or tracking the position of the probe tracker;
A central processing unit for calculating the relative coordinates of the body structure with respect to the probe from the ultrasound image and calculating the three-dimensional coordinates of the body structure based on the positional information of the probe obtained through the imaging unit and the calculated relative coordinates, ; And
And a projection unit for projecting an image of the body structure on the skin of the subject by referring to the three-dimensional coordinates of the body structure.
The method according to claim 1,
Wherein the probe tracker is attached to the probe,
Wherein the probe is further provided with a water bladder for facilitating acquisition of the ultrasound image.
3. The method of claim 2,
Wherein the probe tracker is an optical tracker that provides position information of the probe in an optical tracking manner or a magnetic tracker that provides position information of the probe in a magnetic tracking manner.
The method according to claim 1,
Wherein the central processing unit separates a region corresponding to the body structure from the ultrasound image and calculates the relative coordinates of the body structure with reference to the separated region.
The method according to claim 1,
Wherein the photographing unit further photographs the image of the internal structure projected on the skin of the subject by the projection unit,
Wherein the central processing unit monitors an image of the body structure photographed by the photographing unit so that the image of the body structure is no longer projected when the monitored image of the body structure moves or changes by a predetermined degree or more And controls the projection unit to visualize a human body structure.
The method according to claim 1,
Wherein the imaging unit, the central processing unit, and the projection unit are built in one hardware device, the visualization system for visualizing a human body structure.
The method according to claim 1,
A goggle for sensing a pupil position of a practitioner performing a procedure on the subject; And
Further comprising a goggle tracker for providing position information of the goggles,
The central processing unit may calculate gaze information of the operator according to the pupil position of the operator sensed through the goggles and the position information of the goggles obtained using the goggle tracker, The three-dimensional coordinates of the body structure are converted or adjusted,
Wherein the projection unit projects an image of the internal structure on the skin of the subject according to the three-dimensional coordinates of the converted or adjusted body structure.
8. The method of claim 7,
The central processing unit separates a skin region corresponding to the skin of the subject from the ultrasound image, calculates three-dimensional coordinates of the skin by referring to the separated skin region and the position information of the probe, Dimensional coordinate of the body structure with reference to the three-dimensional coordinates of the skin together with the three-dimensional coordinates of the skin.
8. The method of claim 7,
Wherein the photographing unit photographs the goggle tracker or tracks the position of the goggle tracker to provide the position information of the goggle to the central processing unit.
The method according to claim 1,
Wherein the in-vivo structure includes a vein, an organ, or a tumor of the subject.
Scanning at least a part of the body of the subject with a probe to obtain an ultrasound image of the body structure of the subject;
Separating a region corresponding to the body structure from the obtained ultrasound image and calculating relative coordinates of the body structure with respect to the probe based on the separated region;
Capturing a probe tracker attached to the probe or tracking the position of the probe tracker to obtain position information of the probe;
Calculating three-dimensional coordinates of the body structure based on the obtained position information of the probe and the calculated relative coordinates; And
And projecting an image of the body structure onto the skin of the subject by referring to the three-dimensional coordinates of the body structure.

Images