KR101334007B1 - Surgical Robot Control System and Method therefor - Google Patents

Abstract

Disclosed are a surgical robot control system and a control method thereof. The surgical robot control system according to the present invention sets the coordinates of the lesion position on the basis of the photographed image, and designates a needling target of the robot arm according to the set coordinates. Robot arm driving unit for driving the robot arm along; A current image capturing unit configured to capture a current image corresponding to a lesion position in a current posture of the patient; Extracts a plurality of feature points from the captured image, searches for positions corresponding to the feature points of the captured image in the current image, and changes the coordinates set by the robot arm driver based on the positional change of the feature points from the captured image to the current image. A coordinate change calculator for calculating; And a driving control unit for correcting the needling target of the robot arm according to the change calculated by the coordinate change calculator, and controlling the driving of the robot arm driving unit according to the corrected needling target of the robot arm.

Description

Surgical Robot Control System and Method therefor}

The present invention relates to a surgical robot control system and a control method thereof, and more particularly, to the position of the patient at the time of taking an image of X-ray, CT, MRI, etc. in order to confirm the location of the lesion to be operated by the patient. The present invention relates to a surgical robot control system and a method of controlling the same, which can accurately locate a patient's lesion and perform the surgery even when the patient's posture is changed.

Medically, surgery refers to the act of cutting, slitting, or manipulating skin, mucous membranes, or other breakfasts using medical devices to heal a disease.

Since traditional surgical procedures are generally completely dependent on the doctor's medical skills, the same surgical results may vary depending on the doctor's experience, expertise, and the like.

In order to solve such a problem, recently, a surgical method for dissecting a surgical site of a patient using a robot and treating, shaping or removing an organ therein has been developed. In general, the surgical method using a robot takes X-rays, CT (Computed Tomography), and MRI (Magnetic Resonance Imaging) images of the patient's surgical site before starting the operation. Set the coordinates for and operate the robot according to the set coordinates.

By the way, in the surgical method using a robot according to the prior art, the posture when the patient takes an image of X-ray, CT, MRI, etc. and the posture when taking a posture to receive the operation may be different, accordingly There is a problem that an error may occur in a coordinate value set on the basis of an image. In particular, in the case of surgery on a region where nerves and blood vessels are concentrated, such as the spine, there is a problem that the nerve or blood vessel of the patient may be damaged during the operation due to an error of a coordinate value set based on the captured image.

The present invention has been made to solve the above-described problems, the position of the patient at the time of imaging of X-ray, CT, MRI, etc. to confirm the location of the lesion to be operated on the patient and the patient at the time of performing the surgery It is an object of the present invention to provide a surgical robot control system and a method of controlling the same, which can accurately locate a patient's lesion and perform surgery.

Surgical robot control system according to an embodiment of the present invention for achieving the above object, setting the coordinates of the lesion position based on the pre-recorded image, and the needling target of the robot arm according to the set coordinates A robot arm driver configured to specify a (Needling Target) and thereby driving the robot arm; A current image capturing unit configured to capture a current image corresponding to a lesion position in a current posture of the patient; Extracts a plurality of feature points from the captured image, searches for positions corresponding to the feature points of the captured image in the current image, and changes the coordinates set by the robot arm driver based on the positional change of the feature points from the captured image to the current image. A coordinate change calculator for calculating; And a driving control unit for correcting the needling target of the robot arm according to the change calculated by the coordinate change calculator, and controlling the driving of the robot arm driving unit according to the corrected needling target of the robot arm.

The coordinate change calculator comprises: a feature point distance calculator configured to extract each protrusion of the bone as a feature point from the photographed image and calculate a distance between the extracted feature points; And a corresponding position search unit for searching for positions corresponding to the feature points of the photographed image from the current image based on the distance ratio between the feature points calculated by the feature point distance calculator.

The above-described surgical robot control system generates a virtual needling image corresponding to the robot arm driver, and virtually displays the virtual image on the current image by correcting the needling target of the robot arm with respect to the generated virtual needling image. It further includes a display unit.

The coordinate change calculator may extract each protrusion of the bone as a feature point from the photographed image photographed using the X-ray.

The coordinate change calculator may calculate the coordinates of the lesion position in the pre-photographed captured image by using at least one of computed tomography (CT) and magnetic resonance imaging (MRI).

Surgical robot control method according to an embodiment of the present invention for achieving the above object, setting the coordinates of the lesion position on the basis of the pre-recorded image, and setting the needling target of the robot arm; Designating a needling target of the robot arm according to the set coordinates; Photographing a current image corresponding to a lesion position in a current posture of the patient; Extracting a plurality of feature points from the captured image and searching for positions corresponding to the feature points of the captured image in the current image; Calculating a change in coordinates set based on a change in position of feature points from the captured image to the current image; Correcting the needling target of the robot arm according to the change in the calculated coordinates; And driving control of the robot arm according to the needling target of the corrected robot arm.

The above-described surgical robot control method further comprises: extracting each protrusion of the bone as a feature point in the photographed image, and calculating a distance between the extracted feature points; and further comprising: calculating the distance ratio between the feature points. A position corresponding to the feature points of the captured image may be searched based on the current image.

The above-described surgical robot control method generates a virtual needling image corresponding to the needling target of the set robot arm, corrects the needling target of the robot arm with respect to the generated virtual needling image, and virtualizes the current image. Displaying as; may further include.

In the location retrieval step, each protrusion of the bone is extracted as a feature point from the pre-photographed captured image using X-ray.

In the location retrieval step, the coordinates of the lesion position are calculated in the photographed image by using at least one of the CT and the MRI.

According to the present invention, even if the posture of the patient at the time of taking the image of X-ray, CT, MRI, etc. and the posture of the patient at the time of performing the surgery to confirm the location of the patient to operate the lesion, the lesion of the patient accurately You will be able to locate and perform the surgery.

1 is a view schematically showing a surgical robot control system according to an embodiment of the present invention.
2 is a diagram illustrating an example of a robot arm.
3 is a diagram illustrating an example of a C-arm apparatus as an example of a current image capturing unit.
4 is a diagram illustrating an example of extracting feature points from an X-ray image.
5 is a diagram illustrating an example of searching for a position corresponding to a feature point in a current image.
6 is a diagram illustrating an example of extracting feature points from an MRI image.
7 is a diagram illustrating an example of displaying a virtual needling image on a current image.
8 is a flowchart illustrating a surgical robot control method according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail the surgical robot device and the surgical method according to an embodiment of the present invention. 1 is a view schematically showing a surgical robot control system according to an embodiment of the present invention.

Referring to FIG. 1, a surgical robot control system 100 according to an exemplary embodiment of the present invention includes a robot arm driver 110, a current image photographing unit 120, a coordinate change calculator 130, and a drive controller 140. And a virtual image display unit 150.

The robot arm driver 110 sets the coordinates of the lesion position based on the pre-recorded photographed image, and specifies a needling target of the robot arm according to the coordinates of the set lesion position. Drive the robot arm. That is, the robot arm driving unit 110 checks the position of the lesion from a captured image such as a pre-recorded X-ray image, a computed tomography (CT) image, a magnetic resonance imaging (MRI) image, and specifies a reference point in the captured image. And set the coordinates of the lesion position with respect to the designated reference point. At this time, the robot arm driving unit 110 is a needling target point of the robot arm from the patient's reference point corresponding to the reference point of the pre-recorded captured image spaced a predetermined distance according to the enlargement / reduction ratio of the pre-recorded captured image The robot arm is driven to be targeted to the designated needling target point.

Here, the robot arm is a device for performing an operation such as cutting or suturing tissue in vivo, Figure 2 shows an example of the robot arm. That is, the robot arm 200 includes a plurality of joints 210, and a needle 220 is installed at the tip of the robot arm 200 to penetrate into the skin to perform cutting and suturing of the tissue. At this time, the robot arm 200 can be rotated up and down and angle adjustment using the joint 210, as well as the left and right rotation, and also using the height adjusting unit 230 between the robot arm 200 and the needle 220. The height of the needle 220 may be adjusted. In this case, a surgical tool such as a laser beam device may be mounted at the tip of the needle 220, or drugs may be directly injected through the needle 220. The driving method for the robot arm 200 may follow various known methods, and a detailed description thereof will be omitted herein. In addition, the shape of the robot arm 200 is not limited to the shape shown in FIG. 2, and various known types of robot arms may be used.

The current image capturing unit 120 captures a current image corresponding to the lesion position in the current posture at the time of surgery of the patient. In this case, the current image capturing unit 120 may take an image of the current posture of the patient in the operating room using the C-arm device as shown in FIG.

In general, the C-cancer device is a device for photographing the operation history of a patient before or during surgery using X-ray during surgery in neurosurgery, orthopedics, urology, etc. It is an important device of the operating room to observe the state of the affected area in real time.

As shown in FIG. 3, in the C-arm apparatus, an X-ray generator 310 generating X-rays and an X-ray detector 320 detecting X-rays generated by the X-ray generator 310 have a 'C' shape. Is connected to the C-arm support 330 of the. In addition, the C-arm support 330 is connected to the rotary block 351 of the main body 350 through the rotary shaft 353, the main body 350 and the monitor 340 is connected via a cable. At this time, the rotation shaft 353 can be rotated at any angle from the reference position. Through such a configuration, in the C-cancer device, the X-ray generator 310 radiates X-rays to an object such as a patient or an animal body, and the X-ray detector 320 detects X-rays passing through the object. The main body 350 processes the video signal, and the monitor 340 displays the video signal processed by the main body 350 as an image.

In FIG. 3, the C-arm apparatus is illustrated and described as an example of the current image capturing unit 120, but the current image capturing unit 120 is not limited to the C-arm apparatus, and is used by an O-arm apparatus or another device. May be

In addition, although the robot arm 200 and the current image capturing unit 120 are illustrated and described in FIG. 2 and FIG. 3 as being implemented as separate and independent devices, the robot arm 200 and the current image capturing unit 120 are illustrated. May be integrally formed into one device.

The coordinate change calculation unit 130 extracts a plurality of feature points from the pre-recorded captured image by using X-ray, CT, MRI, etc., and corresponds to the feature points in the current image captured by the current image capture unit 120. Search for a location. At this time, the posture of the patient at the time of taking the image using X-ray, CT, MRI and the like and the posture of the patient at the time of surgery is different, the coordinate of the lesion position set by the robot arm driver 110 is present in the operating room The current image photographed by the image capturing unit 120 is changed to another value. To this end, the coordinate change calculator 130 may calculate a change in the coordinate of the lesion position set by the robot arm driver 110 based on the position change of the feature points from the captured image to the current image. At this time, the coordinate change calculation unit 130 is preferably extracted as a feature point of each protrusion of the bone that does not change its shape, such as spine, pelvis, etc. in the pre-recorded captured image. In this case, the coordinate change calculation unit 130, in order to increase the accuracy of the coordinate change calculation of the lesion position, as shown in Figure 4, (a) the spine image of the side, (b) the front of the same lesion of the patient Each feature point may be extracted for a plurality of captured images such as a spine image.

The coordinate change calculator 130 may include a feature point distance calculator 132 and a corresponding location searcher 134.

The feature point distance calculator 132 calculates the distance between each of the extracted feature points in each protrusion of the bone in the pre-recorded captured image. For example, as shown in (a) and (b) of FIG. 4, the photographed image is divided at predetermined intervals, the inflection point of the outline of the bone is extracted as a feature point, and the distance between each feature point. Can be calculated.

The corresponding location search unit 134 may store the feature points of the photographed image photographed from the current image photographed by the current image photographing unit 120 based on the distance ratio between the feature points calculated by the feature point distance calculator 132. Search for the location corresponding to.

For example, assuming that the location of the lesion is near the neck bone, the feature point distance calculating unit 132 extracts the feature points as shown in FIG. You can calculate the distance between them. For ease of explanation, it is shown that the point of inflection is greater than or equal to the set value from the pre-recorded photographed image as a feature point and the distance between the feature points is calculated.

In addition, assuming that the current image at the time of surgery by the current image capturing unit 120 is the same as that of FIG. The position corresponding to each feature point can be retrieved from. That is, even when the position of the patient is changed, the shape of each bone node is kept constant, so that the corresponding position search unit 134 may search for positions corresponding to each feature point from the current image according to the ratio between the feature points.

At this time, the location of the lesion between the bone node and the bone node, as shown in Figure 5, the position in the photographed image and the position during surgery is changed as the patient's posture is changed. In this case, the coordinate change calculator 130 determines the position of the lesion in the current image photographed by the current image capturing unit 120 according to the coordinate of the lesion position and the distance ratio between the respective feature points in the photographed image. The change may be calculated by comparing the coordinates of the detected lesion position with the coordinates of the lesion position set by the robot arm driver 110.

In this case, the X-ray image may be used to extract each protrusion of the bone as a feature point, and the CT image or the MRI image may be used to calculate the location of the lesion near the bone and the distance ratio of each feature point. . In this case, the location ratio of the location of each lesion and the distance ratio of each feature point may be calculated as a feature point far from the feature point closest to the location of the lesion, and only a number set sequentially.

Surgical robot control system according to an embodiment of the present invention, as shown in Figure 6 (a) MRI image around the neck bone, (b) MRI image around the hip bone, as well as X-ray image, CT image, etc. The feature point can be extracted by using the A, and the change in the position of the lesion can be accurately calculated in comparison with the current image photographed by the current image capturing unit 120 during surgery. In addition, the position of the affected part is not limited to the neck or the waist shown in FIG. 6, and feature points for various parts can be extracted.

The driving controller 140 corrects the needling target point of the robot arm 200 according to the change of the coordinate calculated by the coordinate change calculator 130, and the robot arm driver 110 according to the corrected needling target point. Drive control.

The virtual image display unit 150 generates a virtual needling image corresponding to the robot arm driver 110 and corrects the needling target point of the robot arm 200 with respect to the generated virtual needling image to the current image. It can be displayed virtually. That is, as illustrated in FIG. 7, the virtual image display unit 150 needles virtual needles that need to be coordinated by the robot arm driver 110 with respect to the current image photographed by the current image photographing unit 120. By displaying an image and displaying a virtual needling image corresponding to the robot arm driver 110 corrected by the driving controller 150, it is possible to confirm whether the correction is normal before attempting the surgery.

8 is a flowchart illustrating a surgical robot control method according to an embodiment of the present invention.

1 to 8, the robot arm driving unit 110 checks the position of the lesion from a photographed image such as a pre-recorded X-ray image, a CT image, an MRI image, and designates a reference point in the photographed image. Set the coordinates of the lesion position with respect to the designated reference point (S802). At this time, the robot arm driving unit 110 is a needling target point of the robot arm from the patient's reference point corresponding to the reference point of the pre-recorded captured image spaced a predetermined distance according to the enlargement / reduction ratio of the pre-recorded captured image It is specified as (S804).

The current image capturing unit 120 captures the current image corresponding to the lesion position in the current posture at the time of surgery of the patient (S806).

The coordinate change calculator 130 extracts a plurality of feature points from the pre-photographed captured image using X-ray, CT, MRI, and the like (S808). In this case, the coordinate change calculator 130 extracts each protrusion of the bone as a feature point from the pre-recorded captured image, and the feature point distance calculator 132 may calculate the distance between the extracted feature points (S810). .

The corresponding location search unit 134 may store the feature points of the photographed image photographed from the current image photographed by the current image photographing unit 120 based on the distance ratio between the feature points calculated by the feature point distance calculator 132. Search for a location corresponding to the (S812).

The coordinate change calculator 130 may search for the location of the lesion in the current image captured by the current image capturing unit 120 according to the coordinates of the location of the lesion in the pre-recorded image and the distance ratio between the respective feature points. And, the coordinates of the retrieved lesion position and the coordinates of the lesion position set by the robot arm driver 110 are compared to calculate the change (S814).

The driving controller 140 corrects the needling target point of the robot arm 200 according to the change of the coordinate calculated by the coordinate change calculator 130 (S816).

The virtual image display unit 150 generates a virtual needling image corresponding to the robot arm driver 110 and corrects the needling target point of the robot arm 200 with respect to the generated virtual needling image to the current image. It can be displayed virtually (S818).

When the virtual needling image displayed by the virtual image display unit 150 is correctly needled as the target of the lesion position (S820), the driving controller 140 controls the robot arm driver 110 according to the corrected needling target point. Operation can be started by driving control (S822).

As a result, before the operation is performed by driving the robot arm 200, it is possible to confirm whether the correction is made correctly through the virtual needling image, and thus correct the lesion position even if the posture of the patient is changed. You can then run

100: surgical robot control system
110: robot arm drive unit
120: current video recording unit
130: coordinate change calculation unit
132: feature point distance calculating unit
134: corresponding position search unit
140: drive control unit
150: virtual image display unit

Claims (10)

In the robot control system for surgery,
The robot arm driver sets coordinates of the lesion position based on the photographed image, and designates a needling target of the robot arm according to the set coordinates, and drives the robot arm accordingly. ;
A current image photographing unit configured to capture a current image corresponding to the lesion position in a current posture of a patient;
Extracting a plurality of feature points from the captured image, searching for a position corresponding to the feature points of the captured image in the current image, and based on the positional change of the feature points from the captured image to the current image, the robot arm driving unit. A coordinate change calculator for calculating a change of coordinates set by the coordinate change; And
A driving control unit correcting a needling target of the robot arm according to the change calculated by the coordinate change calculator, and controlling the driving of the robot arm driving unit according to the corrected needling target of the robot arm;
Surgical robot control system comprising a. The method of claim 1, wherein the coordinate change calculation unit,
A feature point distance calculator for extracting each protrusion of the bone from the photographed image as a feature point and calculating a distance between the extracted feature points; And
A corresponding position search unit for searching for positions corresponding to the feature points of the photographed image from the current image based on a distance ratio between the feature points calculated by the feature point distance calculator;
Surgical robot control system comprising a. 3. The method of claim 2,
A virtual image display unit for generating a virtual needling image corresponding to the robot arm driver and correcting a needling target of the robot arm with respect to the generated virtual needling image and displaying the virtual image on the current image;
Surgical robot control system further comprising. The method of claim 2, wherein the coordinate change calculation unit,
Surgical robot control system, characterized in that for extracting each protrusion of the bone as a feature point from the photographed image taken by using X-ray. The method of claim 2, wherein the coordinate change calculation unit,
Surgical robot control system, characterized in that for calculating the coordinates of the lesion position in the pre-recorded image using at least one of CT (Computed Tomography) and MRI (Magnetic Resonance Imaging). In the surgical robot control method,
Setting coordinates of the lesion position based on the photographed image and setting a needling target of the robot arm;
Designating a needling target of the robot arm according to the set coordinates;
Photographing a current image corresponding to the lesion position in a current posture of a patient;
Extracting a plurality of feature points from the captured image and searching for positions corresponding to the feature points of the captured image in the current image;
Calculating a change in the coordinate set based on a change in position of feature points from the captured image to the current image;
Correcting a needling target of the robot arm according to the calculated change in the coordinates; And
Driving control of the robot arm according to the corrected needling target of the robot arm;
Surgical robot control method comprising a. The method according to claim 6,
Extracting each protrusion of the bone as a feature point from the photographed image, and calculating a distance between the extracted feature points;
More,
And a position corresponding to the feature points of the photographed image is searched from the current image based on the calculated distance ratio between the feature points. The method of claim 7, wherein
Generating a virtual needling image corresponding to the set needling target of the robot arm, and correcting the needling target of the robot arm on the generated virtual needling image and displaying the virtual needling image on the current image;
Surgical robot control method comprising a further. The method of claim 7, wherein the location search step,
Surgical robot control method, characterized in that for extracting each protrusion of the bone as a feature point from the pre-image taken by using the X-ray. The method of claim 7, wherein the location search step,
Surgical robot control method characterized in that for calculating the coordinates of the lesion position in the pre-photographed image using at least one of the CT and MRI.

Images